u 2- c.

 

......u............n.r.?
.

3h;
»
#353”
.1...
aha-WW»: .7
v, i... .
v 3.5- ”$2.33 .I-§$ I
{VI .11: 33
.. ..
2....
.
‘3. ll
9 .fi... hem.
a
‘21; .
P5311;

2...».
“3.... $2.:

3&1le
nil." \3.

W318

 

2
20061 .LlBRARY
Michigan State
university

 

 

 

This is to certify that the
dissertation entitled

INTERLEAVED SOURCE CODING FOR PACKET VIDEO
OVER ERASURE CHANNELS

presented by

JIN YOUNG LEE

has been accepted towards fulfillment
of the requirements for the

Ph.D. degree in Electrical Enweering

 

 

MM

 

Major Professor’s Signature

06;, Hf. 20%;
Date

MSU is an Afiirmative Action/Equal Opportunity Employer

--I-I-O-l-C-O-l-I-l-I-I-3-.-O-I-I-l-D-I-O-l-l-l-I-O-I-I-O-D-l-I-O-I-O-l-0-0-O-O-I-O-.-I-I—.-

PLACE IN RETURN BOX to remove this checkout from your record.
TO AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE DATE DUE DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5/08 KzlProj/AccaPrelelRC/DateDue indd

INTERLEAVED SOURCE CODING FOR PACKET VIDEO
OVER ERASURE CHANNELS

By

Jin Young Lee

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY
ELECTRICAL ENGINEERING

2008

 

ABSTRACT

INTERLEAVED SOURCE CODING FOR PACKET VIDEO
OVER ERASURE CHANNELS

By

Jin Young Lee
In this dissertation, a new source coding framework, Interleaved Source Coding
(ISC), is introduced and investigated. The basic idea of ISO is to code a single
video sequence into multi sub-sequences using a predictive video coder while
taking into consideration a given singe packet-erasure channel model and the
video frames’ temporal correlations to reduce the frequency and impact of the
cascaded effect of packet erasures and related propagation of decoding errors
resulting from the predictive nature of coded video. The ISC framework provides
optimum solution for different erasure channel models such that the impact of
losses are limited to a minimum number of video frames while reducing quality
degradation from video frame replacements during error concealments.
Initially, we focus on generating optimum ISC streams by partitioning a predictive
video sequence into two sub-sequences of coded video. The design of ISC em-

ploys Dynamic Programming based on a reward process. Memoryless Binary

Erasure Channel (BEC) model cases (lSC-BEC) with various erasure rates are
examined to validate the initial design of the proposed framework. The initial ISC
scheme is then extended and validated using a Markov Reward Process (MRP)
and a Markov Decision Process (MDP) that are mapped into a packet-erasure
channel with memory (lSD-MDP). Furthermore, two sub-stream canonical ISC
scheme is extended to multi-stream ISC and firm benefits of interleaving has
been observed with respect to channel condition and encoding characteristics.
Finally, rate-distortion optimized Forward Error Correction (FEC) is adopted into
ISC to maximize the performance of the proposed ISC framework. Unlike ISC-
BEC and lSD-MDP, ISC-F EC employs rate-distortion optimization in the optimal
interleaving-set selection process so that it can benefit from both FEC and ISC.
The performance improvement using rate-optimized ISC-FEC is analyzed,

evaluated, and compared with lSC-MDP.

To my father for his etema/ lo ve
And
To my mother for her endless sacrifice.

iv

 

TABLE OF CONTENTS

 

 

 

 

 

LIST OF TABLES ................................ ........ viii
LIST OF FIGURES ...... - .............................. ix
Chapter 1 ........................................................................................................... 1
Introduction ........................................................................................................ 1
1.1 Research Problems ..................................................................................................... 2

1.2 Contribution of Research .............................................................................................. 3

1.3 Organization ................................................................................................................ 6
Chapter 2 ........................................................................................................... 8
Background Information ..................................................................................... 8
2.1 Network lmpainnents ................................................................................................. 10

2.2 Variation in throughput ............................................................................................... 10

2.3 Packet Loss ............................................................................................................... 11

2.4 Channel Modeling ...................................................................................................... 12

2.5 Video Compression and Predictive \frdeo Coding ........................................................ 13
2.5.1 Video Compression ................................................................................................. 13
2.5.2 Predictive Video Coding .......................................................................................... 15

2.6 Error Resilient Video Coding ...................................................................................... 16
2.6.1 Scalable Video Coding (SVC) .................................................................................. 16
2.6.2 Multiple Description Coding (MDC) .......................................................................... 17
2.6.3 Mum-hypothesis Coding .......................................................................................... 18

2.7 Forward Error Correction (FEC) .................................................................................. 19
Chapter 3 ......................................................................................................... 20
Interleaved Source Coding over Binary Erasure Channel .............................. 20
3.1 General lnterleaving ...... ............................................................................................ 22

3.2 Interleaved Source Coding over Binary Erasure Channel (ISC-BEG) ........................... 27
3.2.1 Binary Erasure Channel ........................ - .................................................................. 2 7
3.2.2 Optimal lnterleaving with Reward based Decision Process (RDP) ............................. 28
3.2.3 ISC over BEC with Frame Correlation ...................................................................... 33
3.2.4 Generalization of the Temporal Correlation Measurement ........................................ 37

V

3.3 Evaluation and Analysis ............................................................................................. 39

3.3.1 Simulation Setup ..................................................................................................... 39
3.3.2 Simulation Results .................................................................................................. 41
3.3.3 Analysis .................................................................................................................. 61
Chapter 4 ......................................................................................................... 63
Intedeaved Source Coding over Channel with Memory .................................. 63
4.1 Interleaved Source Coding over Channel with Memory ................................................ 63
4.1.1 Gilbert Channel Model ............................................................................................. 63
4.1.2 lnterieaved Source Coding with Markov Decision Process (lSC-MDP) ...................... 65
4.1.3 lSC—MDP with Frame Correlation ............................................................................. 70
4.2 lSC-MDP Evaluation and Analysis .............................................................................. 7 0
4.2.1 Simulation Setup ..................................................................................................... 70
4.2.2 Simulation Results .................................................................................................. 72
4.2.3 Analysis .................................................................................................................. 79
4.2.3.1 Bitrate and GOV size variation effects ................................................................... 79
4.2.3.2 Correlation Gain Improvements ............................................................................ 80
4.2.3.3 Variation of Gilbert Model Parameter Pairs ............................................................ 80
4.2.3.4 Evaluation Summary ............................................................................................ 81
Chapter 5 ......................................................................................................... 83
Mum-Stream Interleaved Source Coding ........................................................ 83
5.1 Multi-Stream Interleaved Source Coding ..................................................................... 83
5.2 Multi-Stream ISC Evaluation and Analysis .................................................................. 86
5.2.1 Simulation Setup ..................................................................................................... 86
5.2.2 Simulation Results .................................................................................................. 88
5.2.3 Analysis ................................................................................................................ 104
5.2.3.1 Channel condition and GOV size variation effects ................................................ 104
5.2.3.2 Multi-stream interleaving effects .......................................................................... 106
5.2.3.3 Evaluation Summary .......................................................................................... 106
Chapter 6 ....................................................................................................... 108
Forward Error Correcfion for Interleaved Source Coding .............................. 108
6.1 Forward Error Correction .......................................................................................... 108
6.2 Forward Error Correction for Interleaved Source Coding (ISC-FEC) ........................... 109

vi

 

Ch;

Ref

6.2.1 Rate-Distortion Optimized ISC-FEC ....................................................................... 112

6.2.2 ISC-FEC ............................................................................................................... 1 15

6.3 ISC-FEC Evaluation and Analysis ............................................................................. 116
6.3.1 Simulation Setup ................................................................................................... 116
6.3.2 Simulation Results ................................................................................................ 117
6.3.3 Analysis ................................................................................................................ 138
6.3.3.1 Channel condition and GOV size variation effects ................................................ 138
6.3.3.2 Distortion Variation ............................................................................................. 138
6.3.3.3 Evaluation Summary .......................................................................................... 139
Chapter 7 ....................................................................................................... 141
Conclusions and Future Work ...................................................................... 141
7.1 Conclusion .............................................................................................................. 141

7.2 Future work ............................................................................................................. 142
References ..................................................................................................... 143

vii

 

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Table 7

Table 8

Table 9

LIST OF TABLES

Number of Possible lnterleaving Set. K , from (1) for M = 2 .................... 26

Average Frame Replacement Distance with a Single Lost Packet in a GOV .. 34

Distance Matrix DOC) for 80¢) shown in Figure 7 -(b) ................................... 36
MMSE Coefficients, {a, b, c} for given test sequences ................................... 42
Two state Markov transition matrix, P ........................................................... 66
Properties of MDP for Multimedia Stream lnterleaving .................................... 67
PSNR Differences (dB): @ 500kbps - @ 250kbps ........................................... 74
Number of Possible lnterleaving Set, K ........................................................ 84
Number of Possible lnterleaving Set, K h ....................................................... 85
viii

 

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

LIST OF FIGURES

Realtime Streaming Video Service System ............................................................. 8
lnterleaving of Predictive Video Coding ................................................................. 23
Traditional vs. ISC Video Coding .......................................................................... 25
Binary Erasure Channel’s Channel Model ............................................................. 27
View Altemation of Stages .................................................................................... 28
Frame Replacement Illustrations .......................................................................... 34
Sequence Shifting Illustration for Temporal Correlation Measurement .................... 35
Temporal Correlation of the Evaluation Sequences ............................................... 42
PSNR Differences between lSC-BEC-NC and Traditional @ 100KBPS .................. 44
PSNR Differences between lSC-BEC-NC and Traditional @ 500KBPS .................. 46
PSNR Differences between lSC-BEC-NC and Traditional @ 1MBPS ..................... 48
PSNR Differences between lSC—BEC-SC and Traditional @ foo/QPS .................. 50
PSNR Differences between ISC-BEC-SC and Traditional @ 500/QPS .................. 52
PSNR Differences between lSC—BEC—SC and Traditional @ 1MBPS ..................... 54
PSNR Differences between lSC—BEC—GC and Traditional @ 100KBP$ .................. 56
PSNR Differences between lSC-BEC-GC and Traditional @ 500KBPS .................. 58

ix

an

3L

Air

£3.
1‘31.

r:
'9'.r

V361

l PM

:U"

Figure 17 PSNR Differences between ISC-BEC-GC and Traditional @ 1MBPS ......

Figure 18 Two state Markov model with rewards 1;- ..............................................

Figure 19 Average PSNR (GOV Size vs. PSNR(dB)) @ 250kbps ...........................

Figure 20 Average PSNR (GOV Size vs. PSNR(dB)) @ 500kbps ...........................

Figure 21 PSNR Differences: Sequence Specific ISC and Non-ISC @250KBPS

Figure 22 PSNR Differences: Generic ISC vs. Non-ISC (a? 250KBP5‘. ....................

Figure 23 PSNR Differences: Sequence Specific ISC vs. Non-ISC @ 500kbps .......

Figure 24 PSNR Differences: Generic ISC vs. Non-ISC @ 500kbps ....................

Figure 25 Average PSNR over ten packet loss traces: two sub-stream ISC vs.
Akiyo: Number of frames per GOV based performance evaluation ........................

Figure 26 Average PSNR over ten packet loss traces: three sub-stream ISC vs.
Akiyo: Number of frames per GOV based performance evaluation ........................

Figure 27 Average PSNR over ten packet loss traces: four sub-stream ISC vs.
Akiyo: Number of frames per GOV based performance evaluation ........................

Figure 28 Average PSNR over ten packet loss traces: two sub-stream ISC vs.
Coastguard: Number of frames per GOV based performance evaluation ...............

Figure 29 Average PSNR over ten packet loss traces: three sub-stream ISC vs.
Coastguard: Number of frames per GOV based performance evaluation ...............

Figure 30 Average PSNR over ten packet loss traces: four sub-stream ISC vs.
Coastguard: Number of frames per GOV based performance evaluation ...............

Figure 31 Average PSNR over ten packet loss traces: two sub-stream ISC vs.

X

............... 6O

............... 63

............... 72

............... 73

............... 75

............... 76

............... 77

............... 78

Non-ISC for

............... 88

Non-ISC for

............... 89

Non-ISC for

............... 9O

Non-ISC for

............... 91

Non-ISC for

............... 92

Non-ISC for

............... 93

Non-ISC for

Foreman: Number of frames per GOV based performance evaluation .................................. 94

Figure 32 Average PSNR over ten packet loss traces: three sub-stream ISC vs. Non-ISC for
Foreman: Number of frames per GOV based performance evaluation .................................. 95

Figure 33 Average PSNR over ten packet loss traces: four sub-stream ISC vs. Non-ISC for
Foreman: Number of frames per GOV based performance evaluation .................................. 96

Figure 34 Average PSNR over ten packet loss traces: two sub-stream ISC vs. Non-ISC for
Mobile: Number of frames per GOV based performance evaluation ...................................... 97

Figure 35 Average PSNR over ten packet loss traces: three sub-stream ISC vs. Non-ISC for
Mobile: Number of frames per GOV based performance evaluation ...................................... 98

Figure 36 Average PSNR over ten packet loss traces: four sub-stream ISC vs. Non-ISC for
Mobile: Number of frames per GOV based performance evaluation ...................................... 99

Figure 37 Average PSNR over ten packet loss traces: two, three and four sub-stream for Akiyo:
Number of frames per GOV based performance evaluation .................................................. 100

Figure 38 Average PSNR over ten packet loss traces: two, three and four sub-stream for
Coastguard: Number of frames per GOV based performance evaluation ............................ 101

Figure 39 Average PSNR over ten packet loss traces: two, three and four sub-stream for
Foreman: Number of frames per GOV based performance evaluation ................................ 102

Figure 40 Average PSNR over ten packet loss traces: two, three and four sub-stream for

Mobile: Number of frames per GOV based performance evaluation .................................... 103
Figure 41 ISC-FEC illustration ............................................................................................ 108
Figure 42 Comparison of bitrate between non-FEC coding vs. FEC protected coding .......... 109

Figure 43 ISC-FEC design scheme and packet transmission illustration .............................. 110

xi

Figure 44
Figure 45
Figure 46
Figure 47
Figure 48
Figure 49
Figure 50
Figure 51
Figure 52
Figure 53
Figure 54
Figure 55
Figure 56
Figure 57
Figure 58
Figure 59

Figure 60

Rate-Distortion of the evaluation sequences ....................................................... 118

ISC-FEC for N<iyo, Maximum coding distortion = 1% .......................................... 118
ISC-FEC for Akiyo, Maximum coding distortion = 2% .......................................... 119
ISC-FEC for Akiyo, Maximum coding distortion = 3% .......................................... 120
ISC-FEC for Akiyo, Maximum coding distortion = 4% .......................................... 121
ISC-FEC for Akiyo, Maximum coding distortion = 5% .......................................... 122
ISC-FEC for Coastguard, Maximum coding distortion = 1% ................................. 123
ISC-FEC for Coastguard, Maximum coding distortion = 2% ................................. 124
ISC-FEC for Coastguard, Maximum coding distortion = 3% ................................. 125
ISC-FEC for Coastguard, Maximum coding distortion = 4% ................................. 126
ISC-FEC for Coastguard, Maximum coding distortion = 5% ................................. 127
ISC-FEC for Foreman, Maximum coding distortion = 1% ..................................... 128
ISC-FEC for Foreman, Maximum coding distortion = 2% ..................................... 129
ISC-FEC for Foreman, Maximum coding distortion = 3% ..................................... 130
ISC-FEC for Foreman, Maximum coding distortion = 4% ..................................... 131
ISC-FEC for Foreman, Maximum coding distortion = 5% ..................................... 132
ISC-FEC for Mobile, Maximum coding distortion = 1% ......................................... 133

xii

Flgu

Figu

Figure 61

Figure 62

Figure 63

Figure 64

ISC-FEC for Mobile, Maximum coding distortion = 2% ......................................... 134

ISC-FEC for Mobile, Maximum coding distortion = 3% ......................................... 135
ISC-FEC for Mobile, Maximum coding distortion = 4% ......................................... 136
ISC-FEC for Mobile, Maximum coding distortion = 5% ......................................... 137

xiii

Int:

The
worl
Sue
that
Cont
prov
Over
Sew
DUL
exce
00ml
me
0581‘,

VCed

Chapter 1

Introduction

The expansion of the underlying infrastructure of the Internet Protocol (IP) net-
work has lead to increased demand of realtime on-Iine streaming video services.
Such services are often used in multimedia content transmission such as video
chat, live news, video conferencing, video on demand (VOD), IPTV, user created
content (UCC) streaming, etc. However, despite of the growth and the im-
provements of the lntemet infrastructure, realtime streaming media transmitted
over best-effort IP network often faces difficulties in guaranteeing Quality-of-
Service (008). This is due to the network impairments, e.g., variation in through-
put, delay [1] and packet losses, which are often caused by the service requests
exceeding permitted limit of the networks [2]. The result is unreliable realtime
communication interactivity and degraded video quality during playback since the
streaming video is portioned into packets for delivery and played out simultane-
ously during video delivery. Detailed information on network impairments is pro-

vided in a letter section.

1.1

the

USE

lion.

her

911'

sire

tec

Te

mi

pie

ml

9’,

1.1 Research Problems

Most of the realtime streaming video services uses predictive video coders to in-
crease the network utilization. The predictive video coders use motions estima-
tion and compensation to reduce the redundancies among adjacent frames, and
hence reduce the number of bits required to represent the original video content.
Due to the nature of predictive video coders, the quality degradation of realtime
streaming video service is mainly caused by packet losses [3-17]. Therefore,
for playback quality improvement of realtime streaming video, special coding
techniques resilient to packet losses are required.

Techniques such as scalable coding [3, 7, 18-20], multi-hypothesis motion esti-
mation and compensation [21, 22], multi state video compression [23], and multi-
ple description coding (MDC) with path diversity [24-28] are few examples of
methods that are resilient to packet losses. Studies have shown that such cod-
ing schemes are resilient to packet losses, however, the coding complexities of
such coders are far greater than conventional predictive video coding, and hence,
their usages are limited. Detailed descriptions on the above coding schemes,

along with the pros and cons of them are given in Chapter 2.

ing

of!

the

mul

ton

1.2 Contribution of Research

In this dissertation, a new source coding framework, Interleaved Source Coding
(ISC), is introduced and investigated to provide packet loss resilient video-coding
for predictive video sequences. The basic idea of ISC is to code a single video
sequence into multiple sub-sequences using a predictive video coder while tak-
ing into consideration a given packet-erasure Channel model. ISC differs from
other multi-sequence coding, most notable Multiple Description Coding (MDC), in
the sense that ISC sub-streams are transmitted over a single channel rather than
multiple Channels. Therefore, ISC eliminates Channel selection, content distribu-
tion, and synchronization issues that are associated with MDC [24-28]. Therefore,
an ISC channel model is based on well-known parameters, such as packet era-
sure rate and packet-loss correlation, of a single erasure Channel. The objective
of the ISC coding framework is to reduce the frequency and impact of the cas-
caded effect of packet erasures and related propagation of decoding errors re-
sulted from the predictive nature of coded video. In other words, ISC provides op-
timum solution for different Channel models such that the impact of losses
caused by a given erasure Channel model (with memory [4, 12, 29, 30] or mem-

oryless [31-331) is limited to a minimum number of video frames. In addition, the

 

the

ti:

on

lnl

Dar

my

”10.

BEl

fTar

optimum ISC solution also reduces quality degradation resulting from video

frame replacements, which are used as low-complexity error concealments when

the decoder fails to recover a video frame due to packet losses. ISC achieves

this improvement by minimizing video frame replacement distances. Our pro-

posed design of ISC coded video employs Dynamic Programming [34-37] based

on a reward process. Furthermore, some coarse measure of video frames’

temporal correlations is also taken into consideration to reflect the video-frame

replacement/concealment process.

In this dissertation, ISC optimization is first examined for the canonical case of

partitioning a video sequence into two sub-sequences. This canonical case (two-

sequence ISC interleaving) is examined over various Channel models. The mem-

oryless Binary Erasure Channel (BEC) is known to be simplest erasure channel

model [31-33] Consequently, first, the memoryless EEC-based ISC model (ISC-

BEC) with various erasure rates are examined to validate the design of the ISC

framework. The ISC-BEC case is analyzed and studied for both non-correlation

and video frame correlation cases. In particular, the following cases are covered

for the lSC-BEC scenario; i) lSC-BEC without video frame correlation measure,

ii) lSC-BEC with sequence specific correlation measure, iii) lSC-BEC with ge-

ne

ch;

lse

Pr:

che

mc

WEI.

mo

als;

fen

ten.

Slit

neric frame correlation measure. Each scenario is examined with various BEC

Channel capacities. MPEG-4 video coder [38] is the Choice of predictive video

coder and results from each evaluation scenario are compared to non-

interleaving single layer coding to validate the performance improvement of ISC.

Building upon the initial ISC design over a BEC Channel model, the ISC scheme

is extended using a Markov Reward Process (MRP) and a Markov Decision

Process (MDP) ) [34, 35, 37, 39, 40] that are mapped into a packet-erasure

Channel with memory. We focus on the two-state Markov Model, a./r.a. Gilbert

model [15, 29, 30, 34, 38, 41-43], which is known to model network Channel

with packet losses more realistically when compared to the memoryless BEC

model. Similar to the BEC case, ISC over channels with memory (ISC-MDP) is

also evaluated for both non-correlation and correlation video cases. The per-

formance improvement using ISC over non-interleaving traditional predictive cod-

ing method is validated.

Upon the validation of the ISC design over Channel with memory, ISC-MDP is ex-

tended to multi-stream ISC to find relationships among the number of interleaving

sub-streams or the GOV size of an interleaving sub-stream and the performance

in terms video quality. Three and four sub-stream ISC cases are evaluated. The

9V

ha

ha

tic

Be

Fe

The
r69

U

00c

evaluations have shown that increasing the number of interleaving sub-streams
have no noticeable improvement on quality; however, firm benefits of interleaving
has been observed with respect to Channel condition and encoding characteris-
tics.

Based on the observations from the previous parts, rate-distortion optimized
Forward Error Correction (FEC) is adopted into ISC (ISC-F EC) to maximize the
performance of the proposed lnterleaving source coding framework. Unlike ISC-
BEC and lSD-MDP, ISC-FEC takes a slightly different approach in the optimal
interleaving-set selection process such that it can benefit from both FEC and ISC.
The performance improvement using rate-optimized ISC-F EC is analyzed,

evaluated, and compared with ISC-MDP.

1.3 Organization

The remainder of this dissertation is organized as follows: Background material
regarding network impairments, predictive coding, various packet loss resilient
coding methods, and Forward Error Correction (F EC) coding are given in Chap-
ter 2. In Chapter 3, the proposed ISC coding method is introduced with a gen-

eral description on interleaving, an analytical approach for identifying the opti-

mum interleaving set over the memoryless Binary Erasure Channel (BEC) using

Reward based Decision Process (RDP). Chapter 4 extends the ISC scheme for

Channel with memory using a Dynamic Programming algorithm in conjunction

with a Markov Reward Process (MRP) and a Markov Decision Process (MDP). In

Chapter 5, previously evaluated and validated the ISC design is extended to mul-

tiple sub-stream ISC to validate benefits of ISC as a function of the channel

model and encoding characteristics variation. Chapter 6 extends the ISC

framework to adopt rate-distortion optimized Fonrvard Error Correction (FEC),

ISC (ISC-FEC), which further improves the performance of the proposed ISC

framework. Finally, Chapter 7 concludes this dissertation and identifies future

directions.

 

Tl

re

Ar

Fir:

Ds'e

Chapter 2

Background Information

This Chapter gives an overview of the network impairments, predictive coding,
FonNard Error Correction (FEC) coding, and past and current work in packet loss

resilient coding method for realtime streaming video applications.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Camera r-F AID Converter —>L Encoder Network Adapter
Network Channel ‘
Network Adapter Decoder D/A Converter Monitor

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 1 Realtime Streaming Video Service System

A realtime streaming video service system can be represented as in Figure 1.
First, video signal is digitized and fed into the encoder. Then the encoder com-
presses the digitized video stream to reduce the bandwidth needed for transmis-
sion. Once compressed, the encoded signal is passed on to the network
adapter where it is broken into packets to be transmitted over the network. After
passing through various links and routers, packets reach the destination where

decoding and error handling of the transmitted packets are performed for presen-

bu

lSl

the

‘05;

Cfie

dict

f0f r

tation.

When measuring the Quality-of-Service (008) of the delivered media, degrada-

tion can be caused by any component in Figure 1. Coding perspective, signal

conversions, AID and D/A, affect the quality depending on the quantization val-

ues taken. In addition, compression scheme [38,41] plays major role in quality

degradation, however, improvement of coding schemes can overcome such

problem by providing high perceptual quality with high compression ratios.

Network perspective, the main cause is packet loss, and this is usually caused by

buffer overflow of packet transmitting network components. The buffer overflow

is usually observed when data arrives to the network components at a higher rate

than the buffer handling capacity of component. Most cases, such bottleneck

loss occurs at the packet switching routers in the network between servers and

Clients.

At any rate, while coding losses are controllable, network losses are often unpre-

dictable uncontrollable, method to reduce the impact of network losses is in need

for realtime streaming media service system.

HE

in

CL

9:".

we

2.1 Network lmpairrnents

Due to the realtime delivery constraints of realtime streaming video services of-
ten require reliable network that meets their quality of service (008) require-
ments, e.g., throughput, delay, and error resiliencies. However, the unpredictable
nature of intemet traffic results in network impairments such as variation in
throughput, delay, and packet losses, which in turn severely degrades the quality
of delivered video[1, 3-17, 32, 44-48]. The network impairments of the best-
effort IP network are associated with the main components, the routers and the

links interconnecting the routers, and often related to excess service requests.

2.2 Variation in throughput

Currently, the best-effort IP network does not provide end-to-end bandwidth res-
ervation mechanism. It is well known that the available bandwidth is unknown
due to fluctuations of the Internet traffic. As a result of a sudden increment of
Internet data transmission requests or a transmission rate of a media stream ex-
ceeding the available bandwidth of the connection, network congestion occurs

which in turn causes bursty packet losses or delays in media delivery.

10

2.3

Ba:

are

net

Wh

SlZi

low

her

It is

2.3 Packet Loss

Basic operations of the packet switching routers are as follows: Incoming packets
are stored or queued in the input buffers of the router to be processed by the
network processors. When processed, packets is fonrvarded to an appropriate
output port and stored in the output buffer to be released through the output link
connecting intermediate router.

When the incoming data rate is higher than the forwarding rate of the routers, the
size of processing queue increases, and unless the incoming rate decreases be-
low the forwarding rate, queue reaches the size of the buffer, bottleneck, and
hence, any new incoming packet is dropped automatically, bufi’er overflow.

It is well known that buffer overflow is the main cause of the packet losses in
best-effort packet switching network. It is possible to prevent buffer overflow by
increasing buffer size, however, it does not resolve packet loss problem due to
the expiration of the time-to-Iive (T'l' L) in the header of the IP packets, another
congestion control mechanism for the network.

It is seldom, however, link failures or unstable lossy links are another cause of

packet losses. In most cases, link related losses are observed in the wireless

ll

 

nel

los

link

par

2.4

The

W3

The

El;

Sio

orig

Th1

network environment. In the wireless environment, packets are lost when the
lossy link condition is observed due to Channel noise or signal strength fading, or
link failure from handoffs or intermittent loss of connection. In case of lossy link
condition, bit error rate (BER) increases and if the errors cannot be corrected, the

packet is dropped by network adapter.

2.4 Channel Modeling

The packet transmitting network Channel is usually modeled in two different
ways; memoryless channel and channel with memory.

The memoryless channel, a very simplistic channel model, is often called Binary
Erasure Channel (BEC) [31-33]. In this channel model, the packet’s transmis-
sion loss or success at time t is determined only by the packet transmission
originated at time t and does not have any relationship with prior or posterior
packet transmissions. From the statistical perspective, it can be said that
packet is lost with a certain probability p without any conditional probability.
Therefore, the memoryless channel shows independent and identical packet loss
distribution and hence, the packet losses observed in memoryless channel model

are mostly single isolated losses and the burst loss occurrences are very rare.

12

 

ter

QU

lac

Ch.

los

lol

2.5

2.5

Du,

Sire

However, studies on lntemet traffic have shown that majority of packet loss pat-
terns observed are bursty [4, 5, 11-15, 29,49, 50] due to the reasons described
in the previous section, hence the simple memoryless Channel model is not ade-
quate to model the lntemet channel realistically. Therefore, to supplement the
lack of adequacy, memory is added to the BEC model. By adding memory, the
Channel with memory model allows to define a conditional probability of a packet
lost based on the previous packet(s) transmission state, success or lost [29, 30,
38,41]. The most widely used model is the Gilbert model which is a two state
Markov model. The model consists of two states: the good (G) and bad (B)
channel state with two transition probabilities, p01 and p10. The parameters
for the model can be easily obtained from a packet loss trace and the parameters
for the model replication are set such that the model generates loss bursts Close

to the realistic packet trace.

2.5 Video Compression and Predictive Video Coding

2.5.1 Video Compression

Due to the bandwidth limitation of the underlying intemet infrastructure, video

streams are often compressed before transmission to meet the realtime delivery

13

 

cons
ofler
are i;
die is
spac.
capal
fits. fr
Coder
comp
Slorag
desire
Coder
entail:
”885K
clingy;

Video f:

Quality ,

constrains of the realtime streaming video service. Video compression methods,
often called video source coders, such as MPEG-2, MPEG-4, H.263, or H.264
are the ones that are widely used in today’s multimedia industry. When the me-.
dia is encoded using the source coder, it not only helps to reduce the storage
space, but also helps to maximize the utilization of bandwidth, limited by control
capability of underlying network infrastructure. However, aside from such bene-
fits, followings must be taken into the consideration before employing source
coders; computation complexities of encoder and decoder, and distortions from
compression.

Storage and bandwidth utilization perspective, high compression ratio is always
desired, however, the trade-offs are increased computation complexities of en-
coder and decoder of which the effect is severe enough to exceed realtime pres-
entation constraint, coding delays. In addition, distortions from high ratio com-
pression can also degrade presentation quality that will not meet desired quality
constraint set by the service or the viewer. Therefore, source coding of the
video for realtime application must be performed deliberately to meet delay and

quality constraints of the service.

14

2.5.
Vldl
algr
mos
cos
is u
dan

lion

ing

DGn
tau:
fine:
Codi

CUR]

 

2.5.2 Predictive Video Coding

Video compression standards [38,41] use some form of redundancy reduction
algorithm to improve the compression ratio with minimal quality degradation. In
most cases, for spatial redundancy reduction, intra-frame coding, the discrete
cosine transformation (DCT) in conjunction with the variable length coding (VLC)
is used where motion estimation and compensation are used for temporal redun-
dancy reduction, inter-frame coding. Since motion estimation and compensa-
tion depend on previously encoded frame to determine motion vectors, which
represent the transformation of current frame from previous one, such video cod-
ing schemes are also called predictive video coding.

The compression efficiency of the predictive video coding is far greater than that
still-image based non-predictive video coding, however, due to the temporal de-
pendent nature, the predictive video coding is more prone to error propagation
caused by packet losses than the other. While non-predictive video coding con-
fines reconstruction error from a single packet loss to one frame, predictive video
coding propagates reconstruction error to all future frames that depend on the
current frame from the motion estimation and compensation perspective, hence

creates similar reconstruction error patterns as with bursty packet losses.

15

pa

2.l
Fc

WT

CO:
Sc
ML

”‘9

2.6
See
can
the
a m

SCai

Therefore, it is important to reduce or eliminate the error propagation from the

packet loss in predictive video coding.

2.6 Error Resilient Video Coding

For realtime streaming video services over best-effort lP based packet network
where retransmission of lost packets are inadequate due to realtime delivery
constraint, to adequately minimize the error propagation effect in predictive video
coding, various coding techniques are adopted that are resilient to packet losses.
Scalable Video Coding (SVC) [3, 7, 18-20], Multiple Description Coding, and
Multi-hypothesis Coding schemes are few examples of error resilient video cod-

ing scheme that are available today.

2.6.1 Scalable Video Coding (SVC)

Scalable Video Coding (SVC) is the name of the H.264/MPEG4 AVC video
compression standard extension [41]. However, before the name was fixed to
the current video compression standard, scalable video coding In general meant
a multi-layered coding scheme that provides spatial, temporal, and SNR/quality

scalabilities[3, 7, 18-20].

16

has

her

date

Shel

Dacl

Codi

for It

The objective of the SVC‘, for both old and new terminology, is to provide multi-
ple scalabilities in one coded stream which can be transmitted at different bitrate
depending on the network condition. In order to achieve the goal, SVC encodes
video stream into one, base layer, or more layers, enhancement layers. The
base layer is necessary for the media stream to be decoded, whereas the en-
hancement layers are applied to improve stream quality, and yet the transmission
of the enhancement layers are optional depending on the Channel condition.
However, since each enhancement layer depends on either the base layer or its
subordinate layer, decoding of enhancement layers are interrupted whenever the
base layer and/or the subordinate layers are missing and, as a consequence, the
data of the respective enhancement layers is rendered useless. Studies have
shown that video streams coded with multiple scalable layers are resilient to
packet losses, however, despite of the improvement, due to the increased the
coding complexity compared to single layer coding, it is more feasible to be used

for the pre-encoded, stored media services.

2.6.2 Multiple Description Coding (MDC)

Multiple Description Coding (MDC) [24-28] is a coding scheme which encodes a

 

I Here, the acronym SVC is used for both old and new terminology.

l7

tic

pa

en

2.6.

Mul

98h:

(lard

mEr

0M:

such

single media stream into n independent sub streams (n >= 2), referred to as de-
scriptions, and transmits each sub streams over multiple paths. When transmit-
ted, any description Can be used for decoding and presentation, however, the
quality can be improved with the number of descriptions received in parallel.
Hence MDC resilient to the packet losses since an arbitrary subset of descrip-
tions can be used to decode the original stream, therefore, interruptions from
packet losses are minimal, they only degrades the quality of video. Despite the
error resilient property of MDC, the use of MDC is minimal due to the high coding
complexity, path selection problem for each description, and synchronization

complexity of received descriptions.

2.6.3 Multi-hypothesis Coding

Multi-hypothesis (MHC) coding [21, 22] uses multiple reference frames for motion
estimation and compensation. It differs from B-frame coding concept of stan-
dard predictive video coders since MHC uses only past frames for reference and
the number of references can be set arbitrary depending on the long-term mem-
ory capacity of the encoder and decoder. The error resilient property of MHC is

such that a frame can be decoded with marginal quality as long as there is a ref-

18

ere

plot

and

qua

mini

2.7

Fom

[BClS

Cone

”111m

”883 i

erence frame presents in the memory of the decoder. However, total group of
picture decoding failure problem still presents if the first frame in the group is lost,
and yet, the packet loss related error propagation of MHC is almost limited to
quality degradation, not total frame losses. Similar to MDC, the use of MHC is

minimal due to increased coding complexity.

2.7 Forward Error Correction (FEC)

Forward error correction (FEC) [6, 49-63] is an error correction system which cor-
rects errors at the receiving end of data transmission with the redundant error
correction codes that are added by the sender before transmission. The advan-
tage of F EC is that the receiver can avoid retransmission of data if error is ob-
served in the data. Therefore FEC is usually used when retransmission is not
adequate due to increase cost from retransmission or delay constraint of the data
retransmission. There are many types of FEC, but the most notable is Maxi-
mum Distance Separable (MDS) coding [49-51, 53-59] because of its compact-

ness and adaptation simplicity compared to other F EC schemes.

l9

Int

ch;

del

ll):

«at

To

Do:

lPtc

0V5

QUE

tiOr

Chapter 3

Interleaved Source Coding over Binary Erasure Channel

The purposed of this research is to propose a new packet loss resilient video-
coding approach for predictive video sequences with the following constraints:

1) It must take channel condition, in terms of loss probability. 2) The coding
complexity cannot be greater than any of the methods described in the previous
chapter. 3) Network transmission overhead must be minimized such that the
delay at the receiving end is confined by the minimum network transmission only.
In other words, neither frame synchronization delay for multiple path delivery, nor
retransmission delay is allowed.

To achieve the objectives stated above, Interleaved Source Coding (ISC) is pro-
posed and introduced in this deliverable. ISC codes a single video sequence
into multiple sub-sequences based on the network condition and transmits them
over a single erasure channel. The objective of ISC is to minimize the fre-
quency and impact of the cascaded effect of packet losses and related propaga-

tion of errors resulted from the predictive nature of predictive video coders. Par-

20

licu
imp
mer
cast
vide
The
(MC
sign
elim
kn0i
leve

less

Eras
VOm
large

Erie.

ticularly, the target is to design and optimum interleaving method such that the
impact of losses caused by a given erasure Channel model (with memory or
memoryless) is limited to a minimum number of video frames. In addition, in
case of decoder failed frame replacement, frozen frames, ISC presents smoother
video compared to the non-interleaving method.

The proposed ISC video coding differs from previous Multiple-Description-Coding
(MDC) based methods (e.g., ones proposed in [24-28]) since ISC is primarily de-
signed for transmission of encoded sequences over a single channel. This
eliminates Channel selection, content distribution, and synchronization issues
known to present with MDC [24-28] . In addition, interleaving could reduce the
level of coding inefficiency that normally characterizes MDC coding. Neverthe-
less, the proposed interleaved coding framework can be generalized for trans-
mission over multiple channels, and hence, it could include some form of MDC.
In this research, however, the main focus is on interleaved coding for the single
erasure-channel case. Furthermore, the proposed ISC framework is different
from other interleaving frameworks [16, 46, 52] since ISC is based on Frame in-
terleaving where others are based on packetor cal/interleaving. To find an in-

terleaving set, a Markov Decision Process (MDP) [34, 35, 37, 39, 40] and a Dy-

21

nar
mo:
con
side
to e
amc
defa

9056

3.1 l
Trac
num
that

GO)
The

diclii
9’3 i r

namic Programming algorithm [34-37] in association with a realistic packet loss
model are employed [4, 5]. In addition, some coarse measure of the temporal
correlation among pictures within a given video sequence is also taken into con-
sideration. This temporal correlation results in interleaving sets that are unique
to each video sequence. However, since measuring the temporal correlation
among video frames may not be always feasible for realtime applications due to
delay, complexity, and memory constraints, a generic correlation model is pro-

posed as well in case where the actual correlation cannot be computed.

3.1 General lnterleaving

Traditional predictive video coding partitions a single lengthy sequence into a
number of shorter length Group Of Video object planes (GOVs). It is well known
that this partitioning limits the impact of possible errors or losses into individual

' GOVs.

The proposed interleaved source coding (lSC)is a pre- and post-process of pre-
dictive source coders (Figure 2). It is possible to integrate lnterleavers and Merg-
ers into the predictive source coders and use a single encoder and decoder;

however, to simplify ISC adaptation, we employ ISC as a pre- and post-process

22

le.

Bf?

of the coders and leave the coders untouched. ISC reduces the impact of
losses within a given GOV and improves the overall quality of predictive video

over lossy packet networks.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Encoder 1
Input Sequence : Stream
Video lnterieaver Merger
Encoder M
Network Channel
Decoder 1
Stream : Sequence Output
lnterleaver Merger Video
Decoder M

 

 

Figure 2 lnterleaving of Predictive Video Coding
Illustration is given for two substream interieaving.

Brief description of the overall ISC process is the following: First, ISC separates
a single video sequence into M multiple sub-sequences using a Sequence ln-
ten'eaver, and the resulting sub-sequences are encoded using separate video
encoders. Then, a Stream Merger merges the encoded frames into a single
stream in the original-sequence frame order for transmission. In addition to the
ISC merged-stream, information regarding the interleaving pattern employed by
the encoder must be transmitted to the decoder prior to the ISC merged-stream

transmission. At the decoder side, the interleaving pattern is used by a corre-

23

sponding pair of Stream lnterieaverand Sequence Merger. Hence, the decoder
side’s Stream lnteneaverseparates the incoming frames or associated packets
into M sub-streams according to the transmitted interleaving pattern information.
The separated streams are decoded independent to each other and the Se-
quence Mergerfinalizes the process by merging the sub-sequences’ frames into
the proper order for playback.

When separating a single sequence into M sub-sequences, 3”), represented

by an index set, j = {1, 2,...M — 1, M} , interleaving set can be found with;

M . M . .
s={0 1 MxN—1}=Us(3), ns(J)=®, Vj,size(s(]))=N(l)
j=1 j=1

In addition, for true interleaving, we adopt the following ISC interleaving con-

straints;

2&0) (2) _ 3(1) (1),...,3(j) (N) _ 3(1) (N _ 1)} > N _1 (2)
where (M x N — 1) is the number of frames in the original non-interleaved se-
quences. In practice, (M x N — 1) could be the number of frames in a GOV,
and hence, the same interleaving is applied to all GOVs in the sequence or a
scene.

For example, in two sub-stream case, for a non-interleaved sequence with a

GOV size of 10, let S = {3(1),s(2)} be an interleaving sub-sequence set with

24

3(1):.{0 1 5 6 9} and 3(2)::{2 3 4 7 8} (Figure 3).

 

 

2)

(c) ISC Sub-sequence s(

Figure 3 Traditional vs. ISC Video Coding
Packet loss in the frame location of P4 in (a). The arrowed lines represent the coded frames
temporal dependencies in the predictive video coding. The dotted frames are the decoder failed
frames due to the loss. The shaded frames are belonged to the other sub-sequence in (b) and (C).

Here, the numbers in 3”) represent the frame locations in the non-interleaved
sequence and the coded stream’s frame transmission order. This interleaving in-
formation is required to be transmitted (e.g., as metadata) with the coded sub-
streams as stated previously. Once separated, the sub-sequences are encoded
as [11”111921P31P41 and 121912P22P32P42 for 3(1) and 3(2), respectively, and they are
transmitted in the following order: 111711121912P22P21P§P§Pij; in other words, the
merged coded sequence is transmitted in the same frame transmission order of

the non-interleaved traditional video coder.

25

During transmission, if a packet is lost that, for example, is a part of the 5th frame
(P4, in Figure 3-(a)), all 6 frames from P4 to P9 of the non-interleaved coding
are impacted severely and would not be decoded correctly. However, with in-
terleaving, all the frames in sub-sequence 3(1) are decoded successfully and
only three frames, P22,P§, and P42, from the sub-sequence 5(2) are not decoded.
Hence interleaving improves overall playback quality by limiting errors (due to
packet losses) to 5(2).

Since the formation of the optimal interleaving set could vary depending on the
channel model and the transmitting sequence, a problem rises here in Choosing
the optimal set from the set of all possible interleaved sequences. LetK be the
set of all possible interleaving sets for a given GOV size. The size of the set

K can be expressed as follows:

“—2 (MxN-z’xN)!

size(K)= 11;!) (M_i)(MxN—(i+1)><N)!N!

 

(3)

Table1 Number of Possible lnterleaving Set, K, from (1) for M = 2

 

GOV SIZE 10 12 14 16 18 20
SIZE K 126 462 1716 6435 24310 92378

 

 

 

 

 

 

 

 

 

 

As shown in Table 1 , the size of the set K could be quite large for any reason-

able GOV size (2N — 1). Hence, identifying the optimum interleaving set that

26

produces the best quality decoded video transmitted over a lossy network chan-
nel could be very computationally expensive task due to the vast size of K
(Table 1 ). Therefore, an efficient decision-based search algorithm is required
to choose and optimal interleaving set that gives the best quality video for a given

erasure-Channel model and a video sequence.

3.2 lnterieaved Source Coding over Binary Erasure Channel (ISC-BEG)

3.2.1 Binary Erasure Channel

 

 

1 — p
o e t 0
PC
Sender E Receiver
Fe
1 > 1
1 — [)8

Figure 4 Binary Erasure Channel's Channel Model

Binary Erasure Channel (BEC) model is one of the simplest communication
Channel modeling methods for data erasure channel (Figure 4). This model
characterizes the bit erasure phenomenon caused by channel noise or other data
erasure factors, with the erasure probability pe. The distribution of the binary

erasure Channel is known to be Independent and ldentically Distributed (i.i.d.),

27

hence data arrive in memoryless fashion and the channel capacity is known be
1 — pe .

Even though the modeling of EEO is based on the binary data bit transmission,
this can simply expanded to model packet erasure Channel as well. For packet
erasure channel, the bit erasure probability can be translated as the packet era-

sure probability, whereas the Channel capacity is same as that of the BEC.

3.2.2 Optimal lnterleaving with Reward based Decision Process (RDP)

For the transmission of a predictive coded (and packetized) sequence over a
packet erasure channel, the aggregated reward v(n — 1) can be defined as a
function of the number of transmitted packets. For any final time n — 1, in other
words, after 17. packet transmissions, define stage m as m time units before

final time, i.e., as time n — 1 — m in Figure 5.

 

 

0 1 2 n—3 n—2 n—1 Time
n—1 n—2 n—3 2 1 0 Stage
=>
1 2 n—3 n—2 n-1 Time
1 2 n—3 n—2 n—1 Stage

Figure 5 View Altemation of Stages

Hence, if the final time is time n - 1, the stage m corresponds to time n — 1.

28

For the aggregated reward v(n — 1) represents the performance of predictive
sequence transmission over a erasure Channel with a channel's packet erasure

rate p6.

]T (4)

T
p = [Good Erasure] = [1 — Pe Pe

To ease the matrix computation, p is transformed into the diagonal matrix P.

 

 

. 1 — p Oi
P = dzag(p) = 0 6 Pa (5)
'T
”(0) = 7' = [rGood rErasure‘ (6)
0(1) = r + Pt) (0) (6)

T
’U(’n — 1) = [vGood (n - 1) ”Erasure (n — 1)]
= r + Pv(n — 2)
(3)
= T '1' PT + P27‘ + + Pn_27‘ + Pn—lr

T'

 

 

n—l _
1+ 2 P'
i=1

For example, in EEG, if the instant rewards are {TGoodfl‘Emsure} = {1,0} , the re-
ward process is awarded with 1 for a successful packet and 0for a lost packet
during the transmission. In this case, after n packet transmissions, the aggre-
gated rewards, 1),-(n — 1) , represent the expected number of good packet trans-
missions with the initial packet transmission at statei, Good or Erasure.

To reflect ISC scheme into the above reward process, a decision process is em-

29

ployed to find an interleaving set that is most suitable for a given channel condi-
tion. In general, the objective is to maximize the number of frames (or associ-
ated packets) that can be decoded correctly. Hence, the following decision
process could guide us toward an “optimal” interleaving for a given erasure
channel model that achieves our objective; the interleaving set that provides the
highest sum of aggregated reward. In this decision process, a set of discount
factors, 70 is applied. The discount factors decide the amount of aggregated
reward to be propagated to the next stage. Incorporating equation (8) with the
discount factors gives an aggregated reward equation (9), where each coded

frame in a GOV is considered to be a single packet transmission iteration in EEG.

v(n — 1) = ra(n_1) + diag('ya(n_1))Pv(n — 2)

: a(n—1) + diag(7a(n—1))Pra(n—2) + (9)
n—l 2 n—l 1

+ H diag('ya(i)) Pn- Tau) + H diag('ya(,-)) Pn_ Tam)
i=2 i=1

 

 

 

 

In ISC, one of the two actions, Coding (C ), or Skip (S), is taken for each itera-
tion where a (n) denotes an action taken for the nth frame in a GOV. Let the
set of ISC sub-sequences in Figure 3 be the interleaving set Is , where k e K .

With respect tok, an ISC set is written as SO“) = {3(k’1),s(k’2)}. In ISC model,

each sub-sequence has its own lntra-coded I frame. It is possible to have a

30

single I framed shared among the interleaved sub-sequences, the main design
issue will be the interleaving of the predictive frames within the sequence GOVs.
Consequently, the frame numbers are rewritten so that each sub-sequence’s re-
ward computation starts from the time instance 0and backward.

3*(k’j)(n) = 3(k’j)(n) — 5(k’j) (O), for O _<_ n S N — 1 (10)
Hence, Smfrom Figure 3 are {3*(k’1)(0) 3*(k’1) (N — 1)} = {0 3 4 8 9} and
{3*(k’2) (O) 5*(k’2) (N — 1)} = {0 1 4 5 6}. For each sub-sequence, frames
are coded, or in other words, action 0 is performed at frame locations specified
in 3*(k’j). When the difference between two adjacent numbers in 3*(k’j) ex-
ceeds 1, which indicates the presence of skipped frames, action S is performed
for the frames in locationl .

N-l

l: U {50°07 (n) + 1,°--,s*(k’j) (n + 1) — 1}

n=0

(11)
V71 | 3*(k’j) (n + 1) — 3*(k’j) (n) > 1

This gives the action sets a(k’j) for the interleaving set SVC) from Figure 3 as
a(k’l)=[CSS c 05.95 c C] and a(k’2)=[C CSS 0 c C].
The instant reward For ISC over BEC, the discount factors for the coded frames,

action 0 , are '76 = {7000,},7Emsm} = {1, 0} , since packet Erasure forces the

decoder to stop and no further decoding is possible, hence aggregated reward is

31

not propagated unless the decoder is restarted.
Therefore, the proposed aggregated reward equations for single-packet-per-
frame are:
”(m (3*(kij) (0)) = 7‘0 (12)

HUM) (3*(k’j) (n — 1)) = TO + diag('yC)Pv(k’j) (3*(k’j) (n — 2)),Vn E 3*(k’j) (13)
This is valid since the reward aggregation for skipped frame sections can be ig-
nored due to memoryless nature of EEG, each frame arrived independently.
When coded sequences are packetized, the number of packets per frame varies
with the bitrate and frame rate of the encoder, and the packet size. In addition,
within a sequence, the number of packets per frame varies depending on the
coding type, (e.g., Intra-frame coding (l-frame) and Inter-frame coding (P-frame)),
and the motion of the sequence. Therefore, due to the unpredictability of the
variation of the number of packets per each coded frame, an average number of

packets per framen is used and the aggregated reward equations are as follows.

bitrate

 

(14)

 

 

framerate x packetsize

22W) (3%” (m) = m + diag('rc)P"”(k’j) (311m) (7‘ _ 2))’
(15)
for 1 S n S N — 1

The term P’7 is multiplied to the aggregated reward since a frame is decoded if

32

and only if all the packets in the coded frames are successfully transmitted. For
each interleaving setk , the sum of aggregated rewards gives corresponding ex-

pected number of successfully decoded frames.

M N—l _ * ,
'UUC) = Z Z ”(ksJ) (s U“) (71.)) (16)
j=1n=0

Hence, the set of aggregated rewards is expressed as:

Vac) = UV(k)(s(k’j))
j

where V(k)(s(k’j) (77.)) = v(k’j) (3*(k’j) (72.)), (17)

for O S n S N — 1
With the following equation, an interleaving set It is found such that our decision

criteria is satisfied, a set with the highest aggregated reward.

arg max um

k

(13)

 

 

3.2.3 ISC over BEC with Frame Correlation

In predictive video coding, when the decoder encounters a packet loss (or errors
in a transmitted packet), to continue the smooth video presentation (without blank
screen or distorted frames), a playback application often replaces the decoder
failed frames with the last successfully decoded frame until a successfully de-

coded frame arrives to restart the decoding process. Here, we refer to this last

33

imcav‘wr" -

(a) Traditional MPEG—4 Coding

    

 

 

(b) Interleaved MPEG-4 Coding

Figure 6 Frame Replacement Illustrations
Packet loss in the frame location of P4 in (a). The dotted arrowed lines represent the frame
replacement relationship for the decoder failed frames (dotted frames).

successfully decoded frame as the replacementframe. When the decoder failed
frames are replaced, the distances (in terms of number of pictures) between the
replacement frame and the replaced frames have effects on the smoothness of
the sequence flow and the overall quality of the playback sequence. This is due
to the fact that the shorter distance between the replacing frames indicates highly
correlated frame replacement in place of decoder failed frames. Figure 6 illus-

trates the frame replacement actions in case of decoder failure.

Table 2 Average Frame Replacement Distance with a Single Lost Packet in a GOV

 

 

 

 

 

 

GOV SIZE 10 12 14 16 18 20 '
NON-ISC 4.0000 4.6667 5.3333 6.0000 6.6667 7.3333
ISC 2.8265 2.9686 3.0740 3.1561 3.2230 3.2793

 

 

 

 

 

 

34

As shown in Table 2 , the average frame replacement distances due to a single
lost packet in a GOV is shorter for ISC than the traditional transmission method.
Hence it is expected that ISC produces smoother and higher quality video over
erasure Channels with decoder failed-frame replacements.

To incorporate the quality improvement from frame replacements, correlation
gain 90°) is added to equation (16) and a Dynamic Programming is used to find
an interleaving set that produces the highest RDP sum of the aggregated reward

with the correlation gain go“).

 

arg max v06) + 900)) (19)

k

The correlation gain g0“) is computed with the following steps. First, temporal
correlations are computed with average PSNR between original sequence and

temporally shifted sequences.

_ avg.PSNR(s,s + d)
avg.PSNR(s, s)

 

p(d)

 

(a) Shifted by 1 (b) Shifted by d

Figure 7 Sequence Shifting Illustration for Temporal Correlation Measurement

Figure 7 shows illustration on sequence shifting for the temporal correlation

35

measurement and the correlations are computed with equation(20) .
Second, a curve fitting method with the Minimum Mean Square Estimator
(MMSE) is used to obtain a function that represents temporal correlation of a

given sequence.

arg min[MSE {p(d),a x exp(—db) + c,‘v’d}) (21)
{a,b,c}

Table 3 Distance Matrix Dm for SUV) shown in Figure 7 -(b)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1 2 3 4 5 6 7 8 9 10
1 1 2 0 0 0 1 2 0 0 1
2 0 1 0 0 0 1 2 0 0 1
3 0 0 1 2 3 0 0 1 2 0
4 0 0 0 1 2 0 0 1 2 0
5 0 0 0 0 1 0 0 1 2 0
6 0 0 0 0 0 1 2 0 0 1
7 0 0 0 0 0 0 1 0 0 1
8 0 0 0 0 0 0 0 1 2 0
9 0 0 0 0 0 0 0 0 1 0
10 0 0 0 0 0 0 0 0 0 1

 

Third, a 2N by2N upper triangular distance matrix DU“) (Table 3) is generated
for each ISC set]: for single-packet-loss per GOV cases, since the main purpose
of interleaving method is to isolate decode failure to one sub-sequence. The

distance matrices’ diagonal indices indicate the first frame location in a GOV im-

36

pacted by a single packet loss. Hence, the non-zeros entries of the distance
matrix represent the distances from replacement frames to the replaced ones.
Finally, the correlation gain is computed with the following equations. W0“) is the
correlation weight matrix with respect to the distances from replacement frames
to the replaced ones. In case of replacements, the weight is multiplied by the
aggregated reward of the replacement frame and the discounted reward is given
to the replaced frame. C(k)is the correlation computed aggregated reward gain

matrix.

b
a x exp [— (DEC?) ) + c, VD)? :: 0

141(k) 2

w (22)

0, otherwise

V(k)* —_- VU‘) (M x N — 1) V0“) (0) V“) (M x N — 2) (23)

y 4:10

W)” x V(k)* (y —- D53), v09“)
GU“) — (24)

23,31 _
Va“) (y), otherwise
GOV SIZE
gas) = 2 of), Vx,y g GOV SIZE (25)
1:,y=1

3.2.4 Generalization of the Temporal Correlation Measurement
Measuring the temporal correlation among video frames within a complete GOV

may not be always feasible for realtime applications due to delay, complexity,

37

and memory constraints. Therefore, a more generic correlation model may be
required for the cases when the actual correlation cannot be computed. Below,
we present such a generic model. VI (k) is the set of the reward increments at

each sub-sequences’ reward calculation iteration.

V105) = UVI(k)(s(k’j))
j

where VI (k)(s(k’j ) (0)) = ”(’97) (3*(k’j) (0)),
(26)

VI(k)(s(kfj) (72)) = ”(kij) (5*(kij) (71)) _ v(kij) (3*(kij) (n _ 1)),

forlgngN—l

With respect to DU“) and VI“), the weight matrix WWI: is calculated with the

GOV SIZE

... T
following equation. Here, DU“) ><VI(k) )-:— Z Dgfyflx
y=1

 

is the average

 

 

reward increment of the successfully decoded frames in case of a single packet
loss in a GOV. Since the decoder failed frames are copied by the last success-
fully decoded frames, multiplying this value by the replacement frame's aggre-
gated reward estimates the correlation-based aggregated reward of the replaced
frame. Hence, the decrement is assumed to be exponential with respect to

temporal distances from the replacement frames to the replaced ones.

38

A

GOV SIZE
Dire)

W(k)* = [1309* x VI(k)T)-:— Z 009* Va:
y=1

3r?! ’

 

 

(27)
o, V0.92 :1: 0

1, otherwise

I! 1y

W)” x Vii” (y — 1993)), v09“) 2 0
000* = ’ (28)

V“) (y), otherwise
* GOV SIZE *
g0“) = Z 093, Vx,ySGOV SIZE (29)
z,y=l

The optimal interleaving set using the above generic correlation model can be

found using the following equation

 

argmax ”(k) + 900*) (30)

k

3.3 Evaluation and Analysis

3.3.1 Simulation Setup

For evaluation, CIF sequences of Akiyo, Foreman, Coastguard, and Mobile were

coded into an IPPP... GOV structure using an MPEG-4 encoder. GOV sizes (un-

interleaved size) of 10, 12, 14, 16, 18, and 20 were used to partition the evalua-

tion sequences. Frame rate of 15 frames per second, bitrate of 100kbps,

500kbps, and 1Mbps, and packet size of 572 Byte are used to represent emerg-

39

ing lntemet-access technologies (e.g., Modem, DSL/Cable and LAN connec-
tions). For Simulation, CIF sequences, Akiyo, Foreman, Coastguard, and Mobile,
are encoded using MPEG-4 encoder. Only Intra- (I) and lnter- (P) coded frames
are used to form GOV.

When the coded sequences are packetized, to limit the impact of a single packet
loss to a single frame, no packets are shared among two consecutive coded
frames. (In other words, each packet contains data that belongs to only one
video frame.) In addition, partial decoding is not employed for the frames with
losses and frozen frames for both ISC and traditional (non-ISC) cases. Three
ISC scenarios are simulated: (a) the non-correlation computation model (equa-
tion (18)), (b) sequence specific correlation gain computation model (equation
(19)), and (b) generic correlation gain computation model (equation (30)). We
refer to these scenarios as lSC-BEC-NC (non-correlation model), lSC-BEC-SC
(sequence specific correlation model), and lSC-BEC-GC (generic correlation
model), respectively. The ISC-BEC-NC scenario generates an optimal inter-
leaved pattern that is independent of the video sequence, and hence, it gener-
ates ISC pattern depending on the erasure-channel model only. It is important

to note that the lSC-BEC-GC case captures the correlation among frames in a

40

generic sense, and it does not measure correlation based on actual computation
of the correlation among the video frames. Hence, the lSC—BEC-GC scenario is
mainly dependent on the original GOV size of the video sequence being coded.
To simulate a statistically viable experiments and to capture a realistic network
loss patterns, ten packet loss traces were generated for each packet erasure rate
of 2%, 4%, 6%, 18%, and 20%. Each evaluation case is fitted into these
packet loss traces and the PSNR values are averaged to provide statistically sat-

isfying results for analysis.

3.3.2 Simulation Results

The simulation results are given in the following order: a) lSC-BEC-NC, b) ISC-
BEC-SC, and C) lSC-BEC-GC. The figures show the average PSNR difference
between lSC-BEC-XX and non-interleaving single layer MPEG-4 coding with
various BEC erasure rates and GOV sizes.

The sequence specific temporal correlations are computed following the illustra-
tion Figure 7 and the MMSE coefficients computed for the sequences in Figure 8

with equation (21) are given in Table 4 .

41

 

‘ * Akiyo
+ Coastguard
-><- Foreman
°-° ‘ -I- Mobile

 

 

 

VVVVVVVVVVAAAAAAAAAAAAA

Correlatlon

0.4 .

 

 

0.2
30

10 20
Temporal Distance d

Figure 8 Temporal Correlation of the Evaluation Sequences

Table 4 MMSE Coefficients, {a, b, c} for given test sequences

 

 

 

 

 

 

 

 

 

a b c
Akiyo 0.5148 0.2740 0.5633
Coastguard 0.6279 0.2567 0.3609
Foreman 0.6217 0.1251 0.3769
Mobile 0.7262 0.3718 0.2456

 

 

Akiyo @100kbps Coastguard @100kbps
.5 - ,.

 

 

 

 

 

 

    

 

 

 

 

 

 

 

 

 

 

 

 

 

    

 

 

 

 

 

 

a
3
Ti:
5 l
:E -3 . ‘ 0 .
E 10 15 20 10 15 20
j.—
5
8
O
3
In Foreman @100kbps Mobile @100kbps
m
C) 4.5 - 3.55
g L —E— 4% —B— 4%
C 4 + 6% 3. ‘9” 6%
g —e- 8% —e— 8%
m 35 _ —V— 10%
a
a 3 . 2.5 '
2.5 - 2‘.
i
2‘ 1.5 -
1.5 -
1‘»
1 n
E
0.5 ' ‘ 0.5 ' '
10 15 20 10 15 20

Prediction Refresh Rate (GOV size) : Frames

43

Akiyo @100kbps Coastguard @100kbps
5 -

 

 

 

 

 

 

 

 

 

 

 

 

 

Foreman @100kbps Mobile @100kbps

.0'

    

 

 

 

 

 

 

 

 

PSNR Gain of lSC-BEC-NC over Traditional (dB)

 

 

1 ' ‘ -2 ‘ 4
10 15 20 10 15 20
Prediction Refresh Rate (GOV size) : Frames

 

 

Figure 9 PSNR Differences between ISC-BEC-NC and Traditional @ 100KBPS

44

Akiyo @500kbps Coastguard @500kbps
25 - 9 - .

 

 

+296
-EI—4%
+696
+896

 

 

 

 

 

 

 

(d B)
O

 

 

 

 

 

 

 

 

 

 

 

 

 

  

 

 

 

To
5
33-5 I 0 l I
E 10 15 20 10 15 20
j—
5
5
0
2.
E Foreman @500kbps Mobile @500kbps
I 9 7 '
g —B— 4%
c: 6 _ —9— 6%
'6 —e— 8%
3 -v— 10%
5, n/
o. 5 ‘
4 .
<
3 .
l
2 L
4: 1 1 J
20 10 15 20

Prediction Refresh Rate (GOV size) : Frames

45

(d B)

_L
01

_,'8

_l _A

_I.

PSNR Gain of lSC-BEC-NC over Traditional
9

Akiyo @500kbps Coastguard @500kbps

 

 

 

 

 

  
  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Foreman @500kbps Mobile @500kbps
4- 6
—x— 12% i —x— 12%
2i -a— 14% —B— 14%
—e— 16% 5,? —e— 16%
—e— 18% < + 18%
—9— 20% —9— 20%
4.
3.
6 3'
< l
4) 2G-
2.
1 .
0 .
-4 ' -1 ' '
10 15 20 10 15 20

Prediction Refresh Rate (GOV size) : Frames
Figure 10 PSNR Differences between lSC-BEC-NC and Traditional @ 500KBPS

46

(<1l B)

33'

PSNR Gain of lSC-BEC-NC over Traditional

L.
_.o

25-

Akiyo @1Mbps Coastguard @1Mbps

 

 

-B— 14%

-—e—16%

—e—18%
+20%

 

 

 

 

 

Foreman @1Mbps

15

 

 
 
 
 
 
 

 

  
 
 

 

 

_4 .
20 10 15 20

Mobile @1Mbps

 

+ 12%
—a- 14%
2-5’ —e— 16%
—e— 18%
2. + 20%

 

 

kL

 

 

g

 

 

15

20 10 15 20
Prediction Refresh Rate (GOV size) : Frames

47

(<1l B)

PSNR Gain of ISC-BEC-NC over Traditional

20’

I
01

Akiyo @1Mbps

 

 

 

 

 

 

_I

 

_60

 

 

 

 

15 20

Foreman @1Mbps

 

15 20

Coastguard @1Mbps
15 -

 

+2%
+496
—e—G%
—e—8%

 

 

 

 

 

 

Mobile @1Mbps

 

+296
7.—B—4% e
—e—B%
—e—8%
6’+10%

 

 

 

 

 

-1 .
10 15 20

Prediction Refresh Rate (GOV size) : Frames

Figure 11 PSNR Differences between lSC-BEC-NC and Traditional @ 1MBPS

48

Akiyo @100kbps Coastguard @100kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

£6
E
E
g >
IE -3 ' 0
g 10 15 20 10 15 20
I'-
B
5
O
“9
0
% Foreman @100kbps Mobile @100kbps
I 4 ' 3.5 "
E2) —x— 2% —x— 2%
«5 -a— 4% '~' —8— 4%
.5 3.5 - —e— 6% 3<§ —e— 6%
8 —e— 8% ‘9‘ 8%
a: —v— 10%
z 3-
(D s
a. V .,
2.5» "
2‘- 7 M ”9
1.5- 3
< 9 g = /
< ° "
1~ I
5
0.5 1 ' 0 L g
10 15 20 10 15 20

Prediction Refresh Rate (GOV size) : Frames

49

Akiyo @100kbps Coastguard @100kbps
4.5 -

 

 

+12%
+14%
4- —e— 16%
—e— 18%

    

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

g 1.5-

Ti

8

:2 o L , 1 '
E 10 15 20 10 15 20
j—

5

8

o

"9

o

g Foreman @100ka Mobile @100kbps

. e- 7-

8 —-<— 12%

—5.5- —e— 14%

2 6* —e— 16%

3 5- +18%

% —V—

0,4.5-

a

 

 

 

 

   

1
Prediction Refresh Rate (GOV size) : Frames
Figure 12 PSNR Differences between ISC-BEC-SC and Traditional @ 100/09PS

50

(dB)
8

PSNR Gain of ISC-BEC-SC over Traditional

L.
_.01

Akiyo @500kbps

 

 

—e— 14%
—e— 16%
—e— 18%

 

 

 

 

_L
M

 

Foreman @500kbps

 

_L
O
u
'\

 

—x— 12%
—a— 14%
—e— 16%

'—e—18%

+20%

 
 

 

 

03,

 

 

 

Coastguard @500kbps

_s
O)

 

t +12%
—8— 14%
14' —e— 16%
~e— 18%
12, —v— 20%

 

 

 

 

 

 

Mobile @500kbps

 

    

 

 

2- 0_
o - v -1 -
1o 15 20 10 15 20

Prediction Refresh Rate (GOV size) : Frames

51

 

 

 

 

 

 

 

 

 

 

 

 

PSNR Gain of lSC-BEC-SC over Traditional (dB)

 

 

 

 

 

 

 

 

 

 

 

 

 

Akiyo @500kbps Coastguard @500kbps
25- 9.
+ 2% + 2%
—E+— 4% L —e- 4%
(K + 6% 8‘ —e— 6%
. —e— 8% —e— 8%
+ 10% 7.) -V— 10% ‘.
15-
C s ,
10 o
5‘ x . ° ’
I: = as o 9
, ‘19,”
r’ ,
0 . .
10 15 20 20
Foreman @500kbps Mobile @500kbps
10' a o 7'
+ 2% + 2%
-B— 4% )K + 4%
9’ —e— 6% 6_ -6— 6%
,‘y —e— 8% ,, —e— 896
8- —e— 10% —v— 10%
v 5. w 5
7. a A n
_. 4". "4A ‘ 0
= , 4‘:
< '. \V’ ‘
3- V . .,
i o
2.
3.
i V
4L 1 1 l J
10 15 20 10 15 20

Prediction Refresh Rate (GOV size) : Frames

Figure 13 PSNR Differences between ISC-BEC-SC and Traditional @ 500KBPS

52

it

_\

Akiyo @1Mbps Coastguard @1Mbps

 

 

  
    

 

 

 

 

 

 

 

 

 

_L

PSNR Gain of ISC-BEC-SC over Traditional (dB)

_L

 

  
 
 
   

 

 

—a—- 4% -El- 4%
_ —e— 6% 16' —e— 6%
—e— 8% —e— 8%
14_ -v— 10%
17'
‘ 12‘»
} 10c
E
8.
4.
l
0 ' ‘ 2 ' ‘
10 15 20 10 15 20
Foreman @1Mbps Mobile @1Mbps
4- 7-
—x— 2% “*7 2%
—B— 4%
2_ —e— 6%
—e— 8%
—v— 10%

 

 

 

 

 

 

SPA

 

 

 

I J

10 15 20 10 15 20
Prediction Refresh Rate (GOV size) : Frames

53

Akiyo @1Mbps Coastguard @1Mbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

r 8
+ 12% + 12%
—a— 14% I) —a— 14%
15_—e—16% 6_—6—16%
+18% +18%
+ 20% —v~ 20%
10- 4-
5 2-
c
0 0:
5-5 -2-
3
E
8
£10 ' ' -4 ‘
g 10 15 20 10 15
,:
53
6
0
8
lg Foreman @1Mbps Mobile @1Mbps
- 6- 3-
8 —x— 12% —x— 12%
% —El— 14% —E+— 14%
:5 +16% 25_+16%
g —e— ' —e— 18%
n: —v— —v— 20%
g l
[L 2/
l
1.5-

 

 

 

 

 

' u . -.‘ -:
1 0 1 5 20 1 0 1 5
Prediction Refresh Rate (GOV size) : Frames

Figure 14 PSNR Differences between ISC-BEC-SC and Traditional @ 1MBPS

54

PSNR Gain of lSC-BEC-GC over Traditional (dB)

Akiyo @100kbps Coastguard @100kbps
.5 ' A

 

 

    

12%
14%
16%
18%
20%

 
  
 
   
   
    

+++++

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

10 15 20
Foreman @100kbps Mobile @100kbps
6 r 7 .
+ 12% 12%
5.5 i —a— 14%
—e— 16% 6 '
5 . —e— 18%
+ 5::
4.5 -

 

 

1 ‘ -1 '
10 15 20 1 0 15 20
Prediction Refresh Rate (GOV size) : Frames

55

Akiyo @100kbps Coastguard @100kbps
.. 3

   
     
    
  

 

 

   
 
 
    

+ 2%
—a— 4%
—e— 6%
—e— 8%
+ 10%

2.5

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

6 1 :
8
a I
C
.g 0.5‘
:5
I!
l- 1
:5 -3 ' ' 0 '
5 10 15 20 10 15 20
0
(.9
O
8
8 Foreman @100kbps Mobile @100kbps
_. r 4.
.g3.5_ -B- 4% 3.5. -E— 4%
0 .9. 6% —6— 6%
tr -e— 8% 3 -e— 8%
5 3- + 10% K + 10%
n.
2.5-
2.5
21
21
15p
1.54- 1i
<
15' 0.5-
0.5 I I 0 l I
10 15 20 10 15 20

Prediction Refresh Rate (GOV size) : Frames
Figure 15 PSNR Differences between lSC-BEC-GC and Traditional @ mumps

56

Akiyo @500kbps . Coastguard @500kbps

 

  
 
  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

25- 10-
—x— 2% —x— 2%
WK —3— 4% 9' —B— 4% "' v
20- —9— 6% —e— 6%
—e— 8% 3' -e— 8%
—v— 10% 5 v
15- 7 ‘21 ~—:
< 6—
10- 5
4.
g 5 31 o
.g 0 fl .1
E 1'
“5’ 1o 15 20 1o 15 20
8
O
Lu
(I)
8 Foreman @500kbps Mobile @500ka
-10- 7-
"5 + 2% , —*- 2%
.g QT —B— 4% -a— 4%
(9 —e— 6% " 6- + 6%
I! 8' —9— 8% —e— 8%
5 —v— 10%
o- 7 5.
‘
6
4.
3

 

 

 

 

1 ' 1 ‘
10 15 20 10 15 20
' Prediction Refresh Rate (GOV size) : Frames

57

Akiyo @500kbps Coastguard @500kbps

 

 

 

 

 

  

 

 

 

 

 

 

 

 

 

   
   

 

 

 

 

 

 

 

 

 

 

2 16
i + 12% g —x— 12%
—a— 14% 14. -E*- 14%
203 —e— 16% + 16%
—e— 18% ‘9‘ 18%
15- + 20% 12‘ + 20%
10- 10,
5- 8-
E
0- 6-
-5l 4V
-10» 2’.
5‘
:115r 0-
(V
g 1
33-20 I J -2 I
g 10 15 20 10 15
p:
3
3
8
E Foreman @500kbps Mobile @500kbps
. 12r 5-
8 + 12% ’1 + 12%
‘10 —a— 14% —a— 14%
E —0—- 16% 4E! + 16%
'6 m —e— 18% —e— 18%
(D 8' ‘3» —v— 20%
g 3
g)- 6' ( v
< 2-
4
1L
2..
V 0»
0.
-2. '1.
_ —2-
.4 1
-5 ' - -3 -
10 15 20 10 15

Prediction Refresh Rate (GOV size) : Frames

Figure 16 PSNR Differences between lSC-BEC—GC and Traditional @ 500KBPS

58

Akiyo @1Mbps

 

 

+2%
+4%
+6%
—e—B%

    
 

 

 

 

0

(d B)

01

 

_L
_.o

_s
U"

  
 

Foreman @1Mbps

W

 

 

+2%
+4%
+6%
—e—8%

 

 

PSNR Gain of lSC-BEC-GC over Traditional

 

 

—v—.10%

 

 

15 20

Prediction Refresh Rate (GOV size) :

Coastguard @1Mbps

 

 

 

 

 

Mobile @1Mbps

6 +4%
|—e—-8%

 

 

 

 

PSNR Gain of lSC-BEC-GC over Traditional (dB)

Akiyo @1Mbps Coastguard @1Mbps

 

 

+12%
+14%

_—e—16%

—e— 18%
—v— 20%

 

 

c'n

 

 

 
 
    
  

 

 

 

 

 

 

 

 

 

 

 

 

 
 
  
   

 

 

 

 

 

 

10 ‘ ‘
10 15 20
Foreman @1Mbps Mobile @1Mbps
12' 5.
—x— 12% + 12%
—E+— 14% ‘9‘ 14%
10_ —e— 16% 4_ —e— 16%
—e— 18% —e- 18%
-—v- 20% —v— 20%
8- 3
6t- 2.
I
r
4f 1’
<
)
2- O<
V
O ' v -1 ‘ J
10 15 20 10 15 20

Prediction Refresh Rate (GOV size) : Frames

Figure 17 PSNR Differences between lSC-BEC-GC and Traditional @ 1MBPS

60

3.3.3 Analysis

Due to the nature of Binary Erasure Channel, memoryless, hence the packet

erasures are mostly isolated single erasures, it is difficult to generalize the per-

formance variance depending on channel erasure rate, GOV size, and correla-

tion implications. However, in most cases, the Interleaved Source Coding

framework has shown improvement over traditional non-interleaving single layer

coding. Noticeable observation is that when the packet erasure rate is low, i.e.,

2% ~ 10%, the lSC-BEC finds the optimum interleaving pattern that is very close

to dividing non-interleaving coding scheme’s GOV size in half. In fact, if there

was no ISC constraint, at least one skipped frame within a substream, ISC would

have found the half GOV size of the non-interleaving coding as the optimum in-

terleaving set. Overall observation indicates that under the given channel model,

even though the intra-frame coded I frames consumes more bandwidth than

the inter-frame coded Pframes, the frequent appearance of I frame improves

the overall quality of vided when channel is prone to packet losses. In addition,

when the different types of correlation models are applied to lSC-BEC, even

though the superiority is hard to distinguish, the generic correlation model shows

competitive results and yet the use of generic correlation model is acceptable

61

when actual correlation computation is not feasible.
In summary, the Interleaved Source Coding has shown its superiority compared
to non-interleaving coding, and ISC adaptation to the realistic channel model,

channel with memory, is permissible and feasible.

62

Chapter 4

Interleaved Source Coding over Channel with Memory

4.1 Interleaved Source Coding over Channel with Memory

4.1.1 Gilbert Channel Model

Previous efforts for the analysis and modeling of packet losses over the lntemet
(e.g.,[4, 5, 10-14, 16]) and wireless networks (e.g.,[29, 30]) have shown that

these losses exhibit Markovian properties.

1-P00

P00 0 0 P11

7‘0 71

1 — P11
Figure 18 Two state Markov model with rewards 1;

The two state Markov Model, a./r.a. Gilbe/T Model, is proven to replicate an ac-
ceptable erasure-channel model (Figure 18). It is possible to use higher order
Markov models; however, to reduce computational complexity, two state Markov

model is used throughout this research.

63

In the Gilbert channel model, the steady state probabilities in good state and bad

state is represented as following.

«(0) = —pL°—- and 7r(1) = fi— (31)
P01 + P10 P01 + P10

These values give coarse measure of a given channel’s packet transmission be-
havior. However, for statistical channel modeling, instead of the above prob-
abilities, the transition probabilities p01 and p10 (or p00 = 1— p01

and p11 2 1 — p10) could be used to characterize Gilbert channel model. Since it
is difficult to properly model a Gilbert channel with arbitrary transition probabilities,
p01 and p10, a more meaningful pair of parameters are the average loss rate, p1,
and the packet correlation, p; this pair can provide a practical and useful insight
of the channel while representing the state transition probabilities.

The average loss rate and the correlation between two consecutive packets can

be defined as follows:

_ P01

P1— , P=P01+p10-1 (32)
P01+P10

Hence, the transition probabilities represented by pland p are:

P00 =1’P1(1-P), P01 =P1(1-P)

P10 =(1-P1)(1—P): P11 =1—(1—p1)(1—p) (33)

In addition, the steady state probabilities are directly related to the loss rate pl;

«(0) = 1— pland 7r(1) = 171. Furthermore, the packet erasure correlation p pro-

64

vides an average measure of the correlation of two consecutive packets. In par-
ticular, when p = 0, P01 + p10 = 1, the loss process is memory-less, and the
above probability measures reduce to the special case of a memory-less Binary
Erasure Channel (BEC). In the sequel, we analyze the impact of the level of cor-
relation among consecutive packets, as represented by p, on ISC-based packet

video over a wide range of loss rate p1 values.

4.1.2 Interleaved Source Coding with Markov Decision Process (ISC-MDP)

For a Markov channel model, a Markov Reward Process (MRP) (e.g.,[34, 35])
can estimate the system’s performance using (a) the Markov channel’s transition
probabilities based on the packet transmission and (b) some model for the re-
wards that are associated with each system state. This reward-based MRP could
be used to measure the system’s performance after 11 packet transmissions,
and this, in turn, could guide the design of our ISC coding system (as explained
further below).

For the transmission of a predictive coded (and packetized) sequence over a
lossy Markov channel with a channel’s state transition matrixP , we define the

aggregated reward v(n — 1) as a function of the number of transmitted packets.

65

Table 5 Two state Markov transition matrix, P

 

 

 

 

 

 

Future
0 1
Current
0 p00 1 — p00
1 1 — p11 P11

 

 

After 11 packet transmissions, the aggregated reward v(n — 1) represents the
performance of predictive sequence transmission over a lossy channel with a
channel’s state transition matrixP. The reward equations (6)-(8) for BEC are
still valid for Gilbert Channel model, hence the variables are defined as following:
In a two state Markov channel model, if the instant rewards are {73, r1} = {1, 0},
the reward process is awarded with 1 for a successful packet and 0for a lost
packet during the transmission. In this case, after 71 packet transmissions, the
aggregated rewards, v,- (n — 1) , represent the expected number of good packet
transmissions with the initial packet transmission at statez'. To establish a MRP
for an erasure-channel model, Iet{0,1} be the corresponding state space to good
(0) and bad (1) packet transmissions. The instant rewardsn are assigned for
each state and they are awarded to the process whenever it reaches statez'
(Figure 18).

A Markov Decision Process (M DP) associates a Markov reward process with a

66

series of actions and decision criteria to find an “optimal” interleaving for a given

erasure channel model properties; packet loss rate and packet correlation value.

Similar to the decision process of EEO model, MDP finds the interleaving set that

provides the highest sum of MRP aggregated reward.

Different from BEC case, the reward aggregation for skipped frames cannot be

ignored any more due to the memory constraint, therefore, in MDP, a set of poli-

cies, mappings from states to actions, are associated with a new set of instant

rewards, a new set of discount factors and a modified set of state transition prob-

abilities (Table 6 ).

Table 6 Properties of MDP for Multimedia Stream lnterleaving

 

 

 

 

 

 

 

 

 

 

 

 

Poliabs I b tReward Di (Factor Tm .
I73 ’7 smun
. Probabilities
{Act10n,Current State} ra 7a
0 1
{0,0} 1 1 P P00 P01
TC '70 C
{0.1} 0 0 0 1
{3,0} 0 1 P00 P01
TS ’75 P S
{.5', 1} 0 1 P10 P11

 

 

 

 

 

 

 

 

 

 

In the instant reward perspective, instant reward of 1 is awarded for successful
transmission and decoding for the policy {0,0}. For all other policies, the in-
stant reward is set to 0, since no frames can be decoded, due to loss, or need to

67

be decoded, for skipped frames.

For the discount factor, the aggregated reward from the previous stage is fully
propagated to the current stage, hence the discount factor is set to 1 for all poli-
cies except policy {0, 1} . For the policy {0, 1}, since the decoder of predictive
coding is forced to stop when a lost packet is detected, the state 1 is considered
as a trapping state for action 0 , hence the discount factor is set to 0for the policy.
Furthermore, in ISC MDP model, once the decoder is stopped due to a lost
packet, it uses the last successfully decoded picture to replace the missing and
effected frames, and then it restarts when a successfully transmitted I frame of
a new GOV arrives to the decoder. Therefore, to reflect the trapping state, the
transition probabilities for the policy {0,1} are set to {1910,1911} = {0,1}. For all
other policies, the channel’s transition probabilities are used since the frame with
successfully transmitted packets or lost packets in skipped frames do not affect
the decoder. Therefore, the proposed MDP model’s aggregated reward equa-

tions for single-packet-per-frame are:

12(k’j) (3*(k’j) (0)) = TC (34)

68

“(’01) (3*(k,j) (n _ 1))

= 7c + diag(10)PcP3
,Vn E 3*(k’j)
This is valid since the aggregated reward for a skipped frame is:
11(k’j) (l) = 7‘3 + diag(7S)PSv(k’j) (l — 1)
_—_ [0 of + dz‘ag([1 1]T)Psv(k’j) (z - 1) (36)
= PSv(k’j) (l — 1)
For multiple packets per frame case, the reward equations are following:

00“” (3*(k’j) (0)) = r0 (37)

”(111) (31101) W)

nx(s*(k’j) (n)-s*(k’j) (n—1)—1)

= TO + diaghc )PCT’PS v(k’j) (3*(k’j) (n — 2)) (33)

forlSnSN—l

The term PC" is multiplied to the aggregated reward since a frame is decoded if

and only if all the packets in the coded frames are successfully transmitted.

77x 3*(k’j)(n)—s*(k’j)(n—l)—1
Similarly, P5 is multiplied to represent packets asso-

ciated with skipped frames. As in EEG case, for each interleaving setk , the
sum of aggregated rewards gives corresponding expected number of success-
fully decoded frames. Hence the equations (16)—(18) are still valid in finding an

optimal interleaving set I: that satisfies our decision criteria, a set with the high-

69

est MRP aggregated reward.

4.1.3 ISC-MDP with Frame Correlation

As described in section 3.3, frame correlation is also taken into consideration for
ISC-MDP to reflect the frame dependent nature of predictive video coding and
frame replacement process for decoder failed frames. Detailed description on
frame correlation calculations and associated reward computation equations are
provided in section 3.3. When incorporated with the ISC-MDP equations, (31)-
(38), the decision criteria and equations, (19)-(30), of lSC-BEC model are still

valid for ISC-MDP.

4.2 ISC-MDP Evaluation and Analysis

4.2.1 Simulation Setup

For evaluation, CIF sequences of Akiyo, Foreman, Coastguard, and Mobile were
coded into an IPPP... GOV structure using an MPEG—4 encoder. GOV sizes (un-
interleaved size) of 10, 12, 14, 16, 18, and 20 were used to partition the evalua-
tion sequences. Frame rate of 15 frames per second, bitrate of 250kbps and
500kbps, and packet size of 512 Byte are used to represent emerging lntemet-

access technologies (e.g., DSL/Cable and LAN connections). Only lntra- (I) and

70

Inter- (B coded frames are used to form GOV.

The evaluation scenarios are as same as the ones from lSC-BEC evaluation: (a)
no correlation (ISC-NC), sequence specific correlation (ISC-C), and generic cor-
relation (ISC-GC)

For the realistic network channel model simulation, packet-loss Markov transition
probabilities from [4,5], p00 = 0.9734, p11 = 0.7052 , are used to capture realistic
network loss patterns. In addition, 5%, 10%, and 15% packet loss rates were
used and packet correlation value of 0.3, 0.6, and 0.9 were used to represent low,
medium, and high correlation between the transmitted packets. For each net-
work Ioss probabilities, from [4, 5] or p1 — p pair, ten packet loss traces were
generated. Each evaluation case is fitted into these packet loss traces and the
PSNR values are averaged to provide statistically satisfying results for analysis.
The simulation results are given in the following order: (a) real network model

simulation, (b) variable p1 — p pair simulation.

71

4.2.2

415

40.5

39.5

38.5

33.5

32.5

31.5

30.5

4.2.2 Simulation Results
41.5 .

 

Akiyo

364
4015‘

 

 

 

Coastguard

 

 

 

 

 

 

 

72

35~
39:51 ‘ ‘.
c. . . .E]
3815 r l 1 1 Fr . 1 34
10 12 14 16 18 20 10 12 14 16
25 - .
Mobile
33:5-
24~
32151
23—
31.5 .
n 22 -
‘ ‘51
Q‘ ‘8
30.5 h 21
10 12 14 16 18 20 10 12 14 16 18 20
-B- ISC-C 250k+ ISC-GC 250k‘l- ISC-NC 250k-9' NO-ISC 250k
Figure 19 Average PSNR (GOV Size vs. PSNR(dB)) @250kbps

 

36£5~

 

 

 

Coastguard

 

 

 

42 .
41 _ 35.5 ~
40 -
34.5 1
39 - ‘.
A“ - - . A 33.5 -
38 1 ‘s
s‘ 'A
37 . , , T A’ . . 32.5
10 12 14 16 18 20
Foreman
34'* i:::§\% 2415-
A. . __
s ‘A\
33 4
TN 23.5 4
32 . ‘: . .
AQ . 22.5 '1
31 - 7‘,
‘. 21.5 ~
30 .
‘2' ' ’ 'A‘ 20 5
29 - 3A ‘
‘30
28 19.5

 

 

 

1O 12 14 16 18 20

Mobile

 

Q

~‘A"-‘A

 

10 12 14 16 18 20

10 12 14 16 18 20

<1:- ISC-C 500k + lSC-GC 500k: tr ISC-NC 500k-er NO-ISC 500k

Figure 20 Average PSNR (GOV Size vs. PSNR(dB)) @ 500kbps

73

Table 7 PSNR Differences (dB): @ 500kbps- @ 250kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1O 12 14 16 18 20
ISC-C 2.7269 -0.8161 0.6278 0.4076 -0.1 1 92 -0.01 85
ISC-GC .0 1 .1866 -0.9270 0.4635 0.0376 -0.0599 -0.0950
ISC-NC g 0.5363 1 .6376 0.2389 -0.4356 0.0310 -0.1814
NO-ISC -0.5449 -1 .1055 -2.0402 -1 .7833 -1 .4637 -0.9060
ISC-C '2 0.3728 -0.0300 -0.41 78 -0.7267 -0.9147 -0.8284
ISC-GC ‘3 0.1818 -0.0129 -O.3814 -0.7227 -1.1446 -0.9390
ISC-NC g 0.131 1 0.0701 -0.4260 -0.3865 -0.5083 -1 .0518
NO—ISC -0.4437 ~0.8419 -1.5190 -1.1299 -1.4293 -1.0905
ISC-C 0.4749 -0.2295 -0.7225 -O.8510 -1 .5208 -1 .6541
ISC-GO é 0.0937 -0.2174 -0.4077 -0.7553 -1 .4945 -1 .9041
ISC-NC E 0.0852 -0.0239 -0.1 164 -0.8747 -0.9077 -1 .3292
NO-ISC -1.2210 -1.4121 -2.7649 -1.9366 -2.6234 -1.4544
ISC-C 0.1777 -0.2298 -0.4303 -0.6446 -0.7273 -0.6882
ISC-GC 2% -0.0128 -0.2458 -0.5420 -0.5544 -0.8675 -0.9304
0
ISC-NC 2 0.0752 -0.3674 -0.7562 -0.61 21 -0.5850 -0.91 78
NO-ISC -1.0877 -1 .3254 -1 .9982 -2.2344 -2.4418 -1 .4970

 

 

74

 

Akiyo @250kbps

.5
A

 

- —1— 10% 0.6
+ 10% 0.3

_s
N

 

 

 

_s
O

 

PSNR Diferences: ISC-SC vs. Traditional (dB)

+ 10% 0.9 ,.

 

 

Foreman @250kbps

\l

 

PSNR Differences: ISC-SC vs. Traditional (dB)

 

 

I
4‘

0 15
GOV Size

Figure 21 PSNR Differences: Sequence Specific ISC and Non-ISC @250KBPS

20

 

 

 

 

 

 

Coastguard @250kbps
9 .
-a- 5% 0.9 4
8 ' + 5% 0.6
—e— 5% 0.3 4
7 .
6 . 4
5 .
4 .
<
3 ' 1
9’ I
2,..____ “_ ”
1 7" : _ .
0"" - = = a
-1 I . -
10 15 20

Mobile @250kbps

 

4?— 1596 0.9
4- + 15% 0.6
—<1— 15% 0.3

 

 

 

 

  
    

 

0 l
10 15
GOV Size

75

_s
A

_L _I
on O N

O)

PSNR Diferences: lSC-GC vs. Traditional (dB)

0| 0) \l

U?”

0)

)9

A
A

04

PSNR Differences: lSC-GC vs. Traditional (dB)

I
A
n

Akiyo @250kbps

 

-x- 10% 0.9 4
. —t— 10% 0.6
+ 10% 0.3

 

 

 

 

 

 

!\

 

 

—L
O

10 15 20
Foreman @250kbps
' : 2 . .7
15 20
GOV Size

Coastguard @250kbps

 

 

 

 

 

 

 

Mobile @250kbps
4.5

 

-‘6‘— 15% 0.9
4 +15%0.6
—e— 15% 0.3

 

 

 

3.5

 

 

 

10 15 20
GOV Size

Figure 22 PSNR Differences: Genen‘c lSC vs. Non-ISC @250KBP8.

PSNR Diferences: ISC~SC vs. Traditional (dB)

PSNR Differences: ISC-SC vs. Traditional (dB)

J

8

_L
‘ Ul

Akiyo @500kbps

 

 

+ 10% 0.9
—+— 10% 0.6
+ 10% 0.3

 

 

 

_l
o

M

Afie‘f‘AmT‘S‘P

.9)

 

.50“

 

Foreman @500kbps

 

1 5
GOV Size

Coastguard @500kbps

 

 

 

 

 

 

 

Mobile @500kbps

 

—v— 15% 0.9
—A— 15% 0.6
5 +15%0.3

 

 

 

 

 

Figure 23 PSNR Differences: Sequence Specific ISC vs. Non-ISC @ 500kbps.

77

Akiyo @500kbps Coastguard @500kbps

 

20 12)
+10%0.9

 

—B— 5% 0.9
—e— 5% 0.6
; —e— 5% 0.3

 

 

 

 

 

 

—|

_b

 

PSNR Diferences: lSC-GC vs. Traditional (dB)

 

 

   

Mobile @500kbps

 

 

 

 

 

 

 

 

 

a12— -
g —'6'— 15% 0.9
Tu 45' —e-— 15% 0.6
,510- 4‘ _<_ 15% 0.3
:E .
g s?
F 35’-
9’
3 3'
<

' 2.5 -
8 .

as 2'

8

5 1.5 -
5 1 -
D i
g 0.5
U) c
n. o - ,

10 15 20
GOV Size GOV Size

Figure 24 PSNR Differences: Genen‘c ISC vs. Non-ISC @ 500kbps

78

4.2.3 Analysis

4.2.3.1 Bitrate and GOV size variation effects

The non-ISC cases show linear downward trend with respect to the GOV size
and bitrate. This implies that such variations have negative impacts on the qual-
ity, since such changes increase the average number of packets per framen,
which in turn causes an increase in (a) the number of GOVs impacted by lost
packets, (b) the average number of replaced frames, and (c) the distance be-
tween the replacement frames.

For the ISC cases, with the GOV size increment, the average PSNR shows linear
trends similar to the non-ISC cases. However, the slope is rather flat when
compared to the non-ISC cases. This implies that the GOV size variation has
less negative impact on ISC method compared to the traditional non-ISC method.
When the sequences are coded using the same coding method at the same

GOV size, but with the different bitrates, e.g., 250kbps and 500kbps, Table 7 ,
shows that variation of bitrate has less impact on the PSNR values for the ISC
cases than the non-ISC cases; hence this shows that ISC reduces the negative

impact of increased 17, the average number of packets per frame, as stated pre-

79

viously.
In addition, as shown in, since the average PSNR gain of ISC cases over non-
ISC cases are higher, this implies that the ISC method performs better when

coded at higher bitrate.

4.2.3.2 Correlation Gain Improvements

The correlation-based models, both ISC-C and lSC-GC, provide improvements
over the non-correlation (ISC-NC) based scenario. In, the latter sets show im-
provements in PSNR gain for most of the evaluation cases, and hence demon-
strate the advantages of the correlation gain computation. When comparing the
two different correlation model sets, the generic correlation model shows com-
petitive results, and it is plausible to use the generic model in cases when the ac-

tual temporal correlation for a given sequence is not feasible to compute.

4.2.3.3 Variation of Gilbert Model Parameter Pairs

ISC shows improvements on most of the evaluation cases. It is clearly seen
that ISC advances the traditional method as the channel loss rate increases or
the packet correlation rate decreases. This is due to the fact that ISC reduces

impact of packet losses to the GOV by isolating losses to one of the two sub-

80

sequences and decreases frame replacement distances for decoder failed
frames. However, with the increment of the packet correlation constants, the
frequency of the long packet loss bursts increases, hence increases the chance

that both sub-sequences are impacted by the long packet loss bursts.

4.2.3.4 Evaluation Summary

Overall observation shows that the proposed ISC method improves over the tra-
ditional approach on most of the cases, especially for the sequences with high
motion or low temporal correlation (Figure 8). Up to 4 dB in average PSNR im-
provements is observed.

This represents a very significant improvement in quality for compressed video
applications. In particular, this demonstrates that ISC improves the quality of pre-
dictive coded sequences over an erasure channel by limiting losses to one of the
two sub-sequences, hence minimizing the cascaded effects of lost packets,
and/or decreasing the average frame replacement distance. In addition,
changes in bitrate or GOV size have less impact on ISC coded sequences. Fur-
thermore, when the non-correlation gain computed ISC (ISC-NC) sets are com-

pared to the correlation computed sets (ISC-C and lSC-GC), the latter sets show

81

some modest improvement in PSNR for most of the evaluation cases. Conse-

quently, it is feasible that significant improvements can be gained by taking into

consideration the channel model only, and hence, reducing the complexity for

identifying the optimum interleaving set. Once the optimum interleaving is identi-

fied for a given channel model, this interleaving can be applied to any video se-

quence (i.e., without taking into consideration the particular statistical properties

of the video sequence).

82

Chapter 5

Multi-Stream Interleaved Source Coding

From the lSC-BEC and ISC-MDP evaluations, under the same channel condition,
regardless of how large the bitrate consumption is (primarily by the intra-coded
frames), ISC-based coding have shown improvement in quality over traditional
non-interleaving coding. This part of the dissertation extends the canonical ISC
coding approach from two sub-streams to multi-stream coding. The main pur-
pose of the proposed extension is to investigate ISC flexibility under various
channel conditions with the variations of ISC-GOV refreshment rate and the

number of interleaving sub-streams.

5.1 Multi-Stream Interleaved Source Coding
For multi-stream ISC, two sub-stream set search approaches, extensive search

and greedy search approach, are taken. In sub-stream selection, the constraint
2H” (2) — 3U) (1),---,s(j) (N) — 5”) (N — 1)} > N — 1 from (2) is removed to

give more selection flexibility, and the new sub-stream selection algorithm is;

83

M . . F1 (39)
0 3(3) = O, Vj,size(s(3)) = N
j=1

For extensive search algorithm, the size of the set K of all possible interleaving

sets for a given GOV size is:

 

 

 

 

 

M —2 .
. M x N — x N I
szze(K) = H ( z. ) (40)
i=0 (M-Z)(MXN—(Z+1)XN)!N!
Table 8 Number of Possible lnterleaving Set, K
NON-ISC GOV SIZE 9 12 15 18 21
SIZEK . M = 3 280 5775 126126 2858856 66512160
NON-ISC GOV SIZE 8 12 16 20 24
SIZEK , M = 4 105 15400 2627625 48864376 96197645544

 

 

 

 

 

 

 

However, searching for an optimal interleaving set using extensive search ap-
proach is not always feasible due to the vast size of K .

The greedy search algorithm searches for an optimal multi-stream interleaving
set based on two sub-stream interleaving. First, the algorithm searches an op-
timal two sub-stream interleaving set SO“) = {3(k’1),s(k’2)} using (19) or (30).
Once the initial optimal interleaving set is found, for each sub-stream in S“), the
following pseudo code is used to find the final optimal interleaving set 30%) with

predefined hierarchical level index hmax.

84

 

 

 

for h = 1 to hmax
if Nh-l is even
Nh = Nh-l / 2
using (19) or (30), find
50920): {3861), 30912)}, j=1’2 8,3. S<kh’j)e 8(kh—1J)

else
stop
end

 

Note that the algorithm can terminate before reaching hmax since ISC constraint

(39) requires same number of frames in each sub-stream. Hence, the size of

the possible interleaving set K h can be calculated with the following equation

 

 

hmax
size(Kh) = (2N); +2 2 hx (Nh-l/h)! 2
2(N!) 4:1 2((Nh_1/2h)!)
where N0 = N, Nh_1/2h = int

Table 9 Number of Possible lnterleaving Set, K h

(41)

 

 

 

 

 

 

 

 

 

 

NON-ISC GOV SIZE 12 14 16 18 20 24
SIZE Kh,h = 0 462 6435 24310 92378 352716 1352078
SIZE Kh,h = 1 482 24380 352968 1353002
SIZE Kh,h = 2 24392 1353040

 

 

 

To find optimal interleaving solution over channel with memory, equations (31)-

(38) and (16)-(18) with the MDP variables from Table 6 are used for both ex-

tensive and greedy search algorithms.

85

 

5.2 Mum-Stream ISC Evaluation and Analysis

5.2.1 Simulation Setup

For evaluation, CIF sequences of Akiyo, Foreman, Coastguard, and Mobile were
coded into an IPPP... GOV structure using an MPEG-4 encoder. GOV sizes (un-
interleaved size) of 9, 12, 15, and 18 were used to partition the evaluation se-
quences into three sub-streams and un-interleaved GOV sizes of 12, 16, and 20
were used for four sub-stream interleaving. The extensive search algorithm was
used for three sub-stream cases and greedy algorithm was used for four sub-
stream cases. Frame rate of 15 frames per second, bitrate of 250kbps and
500kbps, and packet size of 512 Byte were used for encoding the sequences.
Only the generic correlation (lSC-GC) algorithm was used and the network condi-
tions were set to 5%, 10%, and 15% packet loss rates, p1, with varying packet
correlation value, p, of 0.3, 0.6, and 0.9. For each network condition

p1 - p pair, ten packet loss traces were generated. Each evaluation case is fit-
ted into these packet loss traces and the PSNR values are averaged to provide
statistically satisfying results for analysis.

The simulation results are given in the following order: (a) multi-stream interleav-

86

ing vs. non-interleaving performance evaluations with respect to number of

frames in a GOV, (b) performance evaluation between multi-stream interleaving

cases with respect to number of frames in a GOV.

87

5.2.2 Simulation Results

Akiyo @250kbps Akiyo @500kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Average PSNR over Ten Packet loss traces (dB)

 

 

 

\ ’ \

30’ \g 25 + 10% 0.9 \\ ,4
—I— 10960.6 \ - x’
+ 10960.3 *

25 . . 20 .

5 6 7 8 9 5 6 7 8 9
45-
_____ ;____

 

 

 

 

 

 

 

 

 

 

+ 15% 0.9 \\
15 —A— 15960.6 \4
—4— 15% 0.3 ‘~~--
. I . 10 . . t w
5 6 7 8 9 5 6 7 8 9

Number of frames per GOV, 2 stream ISC (Solid) vs. Traditional (Dashed)
Figure 25 Average PSNR over ten packet loss traces: two sub-stream ISC vs. Non-ISC for Aka/o:
Number of frames per GOV based performance evaluation

88

Akiyo @250kbps Akiyo @500kbps

 

 

 

 

 

 

 

 

 

 

    

 

 

 

 

 

 

 

 

 

a

3

(D

8 44 45'

g jflflf— :5 4?"
--—————+H_ ““““ ___ " ’f ““““ "‘

g 42 ' i

g W

5 4°

8 ~~~~~~ 35’ iiiiii T

g 38’ __________ + _______ + ~~~~~~~~ +

E 301‘.

36. \

6 6.. ‘‘‘‘‘‘‘ *x \ /*~-~.._

% 34- \\\ 25 + 10% 09 1” ‘~‘\*

(D \\\ —l— 10% 0.6

a. W ________ _* + 10% 0.3

8, 32 - 20 ‘

s 3 4 5 6 3 4 5 6

2

 

 

 

 

 

 

 

 

 

 

 

 

 

25. \\ i, aaaa + 15% 0.9 ,r’
4’ 15' —A— 15960.6 _\ x’
—4— 15960.3 \q/
20 . t . 10 . . .
3 4 5 6 3 4 5 6

Number of frames per GOV, 3 stream ISC (Solid) vs. Traditional (Dashed)

Figure 26 Average PSNR over ten packet loss traces: three sub-stream ISC vs. Non-ISC for
Air/ya Number of frames per GOV based performance evaluation

89

44.

c:- ------------ +3- ------------ +3

43

Average PSNR over Ten Packet loss traces (dB)
on
10

20
3

A
5

b
w '0

all

6;: 8 8,8

00

Akiyo @250kbps

l

 

 

 

‘7:

LI
1

 

 

 

 

 

 

 

 

J

5

 

3.5 4 4.5

Akiyo @500kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

20‘3‘~~~‘.
—v— 15% 0.9 fl~\\\
15» —e-— 15960.6 ‘\-\
—€-— 15% 0.3 7“«<]
10 I . A .
3 3.5 4 4.5 5

Number of frames per GOV, 4 stream ISC (Solid) vs. Traditional (Dashed)

Figure 27 Average PSNR over ten packet loss traces: four sub-stream ISC vs. Non-ISC for Ak/‘ya
Number of frames per GOV based performance evaluation

90

Coastguard @250kbps Coastguard @500kbps

38:5-— 4} ._. a

1
366m

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

< -----
34* 6 .... 0\
<> ..... A ‘0 ----- 9
32’ —a— 5% 0.9
—6— 59606 ’,-«9\
—e— 59603 ‘0" \‘0
32 30 *
6 7 8 9 5 6 7 8 9

 

 

 

 

 

 

 

 

 

5 6 7 8 9 5 6 7 8 9

Average PSNR over Ten Packet loss traces (dB)

 

 

 

 

 

 

 

 

 

 

 

 

 

\ 209 45- 15960.9 K [,4
24. ‘6‘“ —A— 15% 0.6 \ /
\4 —<1— 15960.3 \9/
22 A . . J 15 I . .
5 6 7 8 9 5 6 7 8 9

Number of frames per GOV, 2 stream ISC (Solid) vs. Traditional (Dashed)

Figure 28 Average PSNR over ten packet loss traces: two sub-stream ISC vs. Non-ISC for
Coastguard Number of frames per GOV based performance evaluation

91

Coastguard @250kbps Coastguard @500kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

E1?
3
637.
g 'W
3 361 “ --__,(
.9 'N
‘6 351’\_’\\*
x
o m‘. “““““
8 s41 4.
c ‘7‘“? -------- +
11’ 334
B 96. ~~~~~~~~
5 32' 7* ‘‘‘‘‘‘‘ *\\
I! \\ 30 —x—- 10% 0.9 \_,.w’
6 31 » \,,, i“ -—+—— 10960.6
0- + 1096 0.3 ~~~~~ ale ———————— 4..
330 . . . 28 . . J
g 3 4 5 6 3 4 5 6
>
<
38 40.
3W~a “x7 “\7
4%“: -------- 5,7
34%
30A- ------- A4
A ....... “x
32. ‘A\\\ ”,xA “we ________ A
\‘A”’
25-
30 ’,”4‘\\‘
<5 _______ 4‘ “ ”I 159609 \‘9
\\ 20 ‘9— . ““““
28 \\ /-’4 i + 1596 0.6 “3
‘qu” —<I— 15% 0.3
26 . . . 15 . . .
3 4 5 6 3 4 5 6

Number of frames per GOV, 3 stream ISC (Solid) vs. Traditional (Dashed)

Figure 29 Average PSNR over ten packet loss traces: three sub-stream ISC vs. Non-ISC for
Coastguard Number of frames per GOV based performance evaluation

92

Coastguard @250kbps Coastguard @500kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

37.5
:5» ___________
37» “Er ............ fl
36.5'?/e/E]
365v 4V“ —+>
{>— % —-o
35.5-
<>\
35» x.\
\\ 0 —8— 5960.9
3453 _______ ::.__O__,_,_,____:© 33' —6— 5960.6 _
\e ......... —e— 5% 0.3 0 ----------- o
34 . 32 9 . . -
3 35 4 45 5 3 35 4 45 5
37 33.
36’ ------------ -:‘: ————q _____________ i ~~~~~~~~~~~~ “

36\+\_+
-wf—f

N 3" D
«I...
~h~
_‘—
‘_~—~_+
‘
b ~
‘~
‘
‘ ~“~. Lh~
-4 Q
‘—
~-~.~_
‘

IJ'I
[-
6b

‘P

 

 

 

 

 

 

 

‘* 32’ 4.-
33» ‘\~-\
91- ______ 7“
------ 4.----- . + 10960.9 ‘\
32’ """" * 30" —+— 10960.6 7
+ 1096 0.3 ------------ 4.
31 A A L 1 28 A 1 . J
3 3.5 4 4.5 5 3 3.5 4 4.5 5

Average PSNR over Ten Packet loss traces (dB)
8

 

 

 

 

 

 

 

 

 

 

 

 

 

 

28. ‘\ 25’ —v— 15960.9 4\
\\ —A— 15960.6 ~~~-\
\ <1 —<I— 15960.3 ‘~~-
26 . . . 9 20 . . . *9
3 3.5 4 4.5 5 3 3.5 4 4.5 5

Number of frames per GOV, 4 stream ISC (Solid) vs. Traditional (Dashed)

Figure 30 Average PSNR over ten packet loss traces: four sub-stream ISC vs. Non-ISC for
Coastguard Number of frames per GOV based performance evaluation

93

Foreman @250kbps Foreman @500kbps

 

 

 

 

 

 

 

 

Oh 5555 0 ----- o\\\ 30 \\9— """ 0‘
30 ‘\ ‘0“ ~~~~~ ‘6.“
‘s\ 9 —B— 59609 \49
\‘0 28¢- —8— 59606
—e— 5960.3 'O~~ \
28 . 26 1 E fr—~—-__@
5 6 7 8 9 5 6 7 8 9

 

 

 

 

 

 

 

Average PSNR over Ten Packet loss traces (dB)

 

 

 

 

 

 

 

 

 

 

35W
302* ----- v- ----- W

<

 

25*
5.-“ // ‘flr ““““ A
20‘» “if
1.
15¢ +15%0.9 \
—A— 15960.6 ‘9 ~~~~~ <1
—6— 15% 0.3
. . . . 10 . . . J
5 6 7 8 9 5 6 7 8 9

Number of frames per GOV, 2 stream ISC (Solid) vs. Traditional (Dashed)

Figure 31 Average PSNR over ten packet loss traces: two sub-stream ISC vs. Non-ISC for Fore-
man: Number of frames per GOV based performance evaluation

94

Average PSNR over Ten Packet loss traces (dB)

Foreman @250kbps

~~
‘5‘
~
Q.
~

 

 

33“
\\
\\
32 ’ \\ ,f’e ‘‘‘‘‘
<> sssss \\3 rrrrr ‘0
-g13“_‘
31' “~~e ~~~~~
“‘0
30 . .
3 4 5 6

8

9h

 

N
0'!

35»

A [\AA

30-

25

 

20

3 4 5 6

 

Foreman @500kbps
38 .
15:17::2— ” a
36< ““““ B

 

 

 

 

 

 

32°“ ------- 94“‘
-“‘6\“‘
308x ‘8
—a— 596 0.9
28 —e— 596 0.6 ------ e ........ 0
—e— 596 0.3
26 A _1
3 4 5 6

 

 
   

 

 

 

 

 

 

.. + 1096 0.9 ‘+
—I— 1096 0.6 ----- 4--“
+ 10960.3 “x
20 A .
3 4 5 6

 

 

 

 

 

 

 

 

15' + 1596 0.6 ----- ’3 ““““““ <1
—e—-15%50§3

10 ‘ ‘ ‘

3 4 5 6

Number of frames per GOV, 3 stream ISC (Solid) vs. Traditional (Dashed)

Figure 32 Average PSNR over ten packet loss traces: three sub-stream ISC vs. Non-ISC for
Foreman Number of frames per GOV based performance evaluation

95

Mobile @250kbps Mobile @500kbps

 

 

 

 

 

 

M
N

 

 

 

N
O)

28*

M
014

““““ _- 26:

 

MNN
walla

uk

[0
.5

al-
v

 

8

 

 

 

 

 

 

 

19
5

Average PSNR over Ten Packet loss traces (dB)

  
    
 

 

  

—9— 15% 0.9
—A- 15% 0.6

—<}— 15% 0.3
10 ‘

5 is i e {9 5 6 7 e :3
Number of frames per GOV, 2 stream ISC (Solid) vs. Traditional (Dashed)

 

 

 

 

 

 

Figure 33 Average PSNR over ten packet loss traces: four sub-stream ISC vs. Non-ISC for
Foreman Number of frames per GOV based performance evaluation

96

Foreman @250kbps Foreman @500kbps

 

 

 

 

 

3% ____________ 5--“ 38-
________ B l __ .. 4..
35* 36 “““ f3- ———————————— +3
$1533: < 0 —+>
34 34<

 

 

 

 

 

 

 

 

33<?\\ 32* ------------ e ~~~~~~~~~
\\\\ ~‘0
32;“ \‘\v ~~~~~~~~~~~ 0 3°<L-_
““““““““ 0--_____ —a— 5% 0.9 _
31» ~~~~~ o 28 —e— 5% 0.6 0 ““““““““ 0
—e— 5% 0.3
30 26 ‘ - J
3 35 4 45 5 3 35 4 45 5

8
8
l

u

 

. 3.3

 

o)
. Q

0)
O

 

N
00

sh

 

 

 

 

8

 

 

 

Average PSNR over Ten Packet loss traces (dB)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

34 .. JV. 40
v- ------------ 3: _________
A ““9
32m 35 :7 _%
<
‘ —_ N
30* 30<I\4\<1
---------- A‘—~_“
28‘?" ~~~~~~~ is 25);. ____________ A ““““
26' 2% “A
5 15% 0.9
24‘?" ___________ <15 ~~~~~~ 15’ :15%0.6 E}- ____________ <1
\“Q + 15% 0.3
22 A 1 1 g 10 L A . j
3 3.5 4 4.5 5 3 3.5 4 4.5 5

Number of frames per GOV, 4 stream ISC (Solid) vs. Traditional (Dashed)

Figure 34 Average PSNR over ten packet loss traces: two sub-stream ISC vs. Non-ISC for Mo-
bfle: Number of flames per GOV based perforrnanoe evaluation

97

Mobile @250kbps Mobile @500kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Average PSNR over Ten Packet loss traces (dB)

to
O)

 

 

 

20‘ —v— 15% 0.9 \\
‘ \ 14* -A- 15% 0.6 “<1

\.\G ________ -4— 15% 0.3
3 4 5 6 3 4 5 6

Number of frames per GOV, 3 stream ISC (Solid) vs. Traditional (Dashed)

 

 

 

 

 

 

 

Figure 35 Average PSNR over ten packet loss traces: three sub-stream ISC vs. Non-ISC for Mo-
bfle: Number of frames per GOV based performance evaluation

98

26

25.

25

24.5

24

23.5

8

N
O)

N
01

M
Q

N
_L

Mobile @250kbps

 

——___

 

 

M
A 1

\Ir
-

 

 

Average PSNR over Ten Packet loss traces (dB)
to
o)

 

Figure 36 Average PSNR over ten packet loss traces: four sub-stream ISC vs. Non-ISC for Mo-

18
3

 

 

3.5 4 4.5

5

 

Mobile @500kbps

 

 

 

 

 

 

 

 

 

  
 
 

 

 

 

 

 
 
 
     
 
  
 
 

 

 

 

 

 

 

 

 

 

 

16’ —A— 15% 0.6 4,,
—4— 15% 0.3

14 A L A A

3 3.5 4 4.5 5

Number of frames per GOV, 4 stream ISC (Solid) vs. Traditional (Dashed)

bile: Number of frames per GOV based performance evaluation

99

Akiyo @250kbps Akiyo @500kbps

42.5
42
41.5

 

 

 

 

    

h
N

A
_x

.4

 

 

 

 

 

 

 

 

Average PSNR of MultiStream ISC (dB)
8

39; 36 +10%0.9
—+— 10% 0.6 ‘*
* + 10% 0.3
38 34 A
3 4 5 6 3 4 5 6
44
VW
42 "V

  

 
 

—v— 15% 0.9
< ~A— 15% 0.6
—<+— 15% 0.3

  

 

 

 

 

5
Number of frames per GOV, 2 stream (Dotted) vs. 3 stream (Solid) vs. 4 stream (Dashed) ISC

Figure 37 Average PSNR over ten packet loss traces: two, three and four sub-stream for Akiyo:
Number of frames per GOV based performance evaluation

100

Coastguard @250kbps Coastguard @500kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Average PSNR of Multi-Stream ISC (dB)

 

 

 

 

 

 

 

 

 

30’ —e-— 15% 0.6
—4— 15% 0.3 q
A A 28 4 A A
3 4 5 6 3 4 5 6

Number of frames per GOV, 2 stream (Dotted) vs. 3 stream (Solid) vs. 4 stream (Dashed) ISC

Figure 38 Average PSNR over ten packet loss traces: two, three and four sub-stream for Coast-
guamf Number of frames per GOV based performance evaluation

101

Mobile @250kbps Mobile @500kbps

 

 

 

 

8

N
U1

 

to
All
I

L

   

i3

  
     
  

+ 10% 0.9
—+— 10% 0.6
+ 10% 0.3

'3
8

 

 

 

Average PSNR of Multi-Stream ISC (dB)
N
9’

8
63

 

 

 

 

 

 

 

+15%oe \
16 +15%06 *
—4— 15% 0.3

15 3 ' A 14
3 4 5 6 3 4 5 6
Number of frames per GOV, 2 stream (Dotted) vs. 3 steam (Solid) vs. 4 stream (Dashed) ISC

Figure 39 Average PSNR over ten packet loss traces: two, three and four sub-stream for Fana-
man: Number of frames per GOV based performance evaluation

102

Foreman @250kbps Foreman @500kbps

35>

 

34.5 ~

34

 

 

 

 

 

 

 

 

 

 

 

 

Average PSNR of Multi-Stream ISC (dB)

 

 

 

-v— 15% 0.9 “
26 + 15% 0.6 ........... ._
—4— 15% 0.3 ‘1

 

 

 

 

 

 

A . A 24 A
3 4 5 6 3 4 5 6
Number of frames per GOV, 2 stream (Dotted) vs. 3 stream (Solid) vs. 4 stream (Dashed) ISC

 

Figure 40 Average PSNR over ten packet loss traces: two, three and four sub-stream for Mobile:
Number of frames per GOV based performance evaluation

103

5.2.3 Analysis

5.2.3.1 Channel condition and GOV size variation effects

When the sequences are coded into the same number of frames per GOV, ISC
cases have shown performance enhancements when compared to that of non-
ISC cases. Only when the sequences are coded with low bitrates and transmit-
ted over Iow-Ioss-rate channels with very high memory, 191 — p pair of 5% and
0.9, non-ISC has performed better than ISC. This is due to the fact that seldom,
but long burst losses can be confined into one GOV of non-ISC and the impact
from losses is quickly recovered with prediction refreshment. However, even
though ISC algorithm tries to confine the bursts into one of the sub-streams, due
to the higher temporal prediction distortion from interleaving, the prediction re-
freshment without interleaving can easily outperform in such special cases.
However, under the same channel condition, p1 — p pair , with the increment of
GOV size, the performance of ISC gets closer to that of non-ISC and in some
cases ISC exceeded in its performance. This is due to the fact that increasing
the GOV size increases the number of packet-loss impacted frames of non-ISC

cases until prediction refreshment; and the quality degradation from such impact

104

 

also increases enough to exceed temporal prediction distortion from interleaving.
For similar reason, under the same channel condition, when the sequence is
coded into same GOV size but with higher bitrate, ISO is expected to outperform
non-ISC, and yet, the simulation also show performance improvement as ex-
pected. ISC reduces packet loss impact and minimizes quality degradation,
hence outperforms non-ISC in most cases. The most noticeable improvement
of ISC is that the variation in quality from channel condition and bitrate variations.
is far less than of non-ISC. In fact, the most noticeable quality variation factor in
ISC is the strength of memory. As the memory gets weaker, in other words, as
the randomness of packet losses increases, it becomes harder for ISC to confine
losses into minimum number of sub-streams, hence number of loss impacted
frames increases which in tern degrades overall quality of the transmitted se-
quence. However, for non-ISC cases, since it does not have any packet loss
resilient mechanism other than prediction refreshment, chances of packet loss
impact can multiply with the decrement of memory strength, increment of packet

loss rate, and/or increment of coding bitrate.

105

 

5.2.3.2 Multi-stneam interleaving effects

When the sequences were coded into more than two sub-streams, regardless of
the search method or channel condition, it was difficult to distinguish the flexibility
or benefit of having multiple sub-streams when the output qualities were com-
pared based on the number of frames per sub-stream or ISO GOV size. There-
fore, considering the size of K , (Table 8 and Table 9 ) two stream ISC would be

the most sufficient choice.

5.2.3.3 Evaluation Summary

Overall results have shown that the proposed ISC method improves over the tra-
ditional approach even with the same prediction refresh rate on most of the
cases. Except for some extreme cases where the channel has low loss rate
with strong memory, lSC’s performance improvement was very noticeable, in
some cases, up to 10dB. The most important observation made from this part
of dissertation is that the most significant quality degrading factor is randomness
of packet losses, or strength of memory; however, for ISC, the quality variation
from memory strength variation is far less than that of traditional method, and yet,

this proves again that ISC is resilient to not only long loss bursts, but also to short

106

random bursts. In addition, as long as ISO algorithm is considered in the sys-
tem, the number of sub-stream increment has almost no effect in quality im-
provement, therefore, it is safe to say that two sub-stream ISC would be the most
efficient way to code a stream without risking long search time to find an optimal

interleaving set.

107

Chapter 6

Forward Error Correction for Interleaved Source Coding

6.1 Forward Error Correction

 

   

A FEc:Eri’°oog§r§

 

 

 

 

iii F cl}:
—’ :11: Parameter: u—n.

     

Input Stream

Video

Sequence

Interleaver Merger

 

 

 

 

 

 

:4 ng§ss0ljg

 

 

 

 

 

Network Channel
3 F§C§De§o&§r,,

Figure 41 ISC-FEC illustration

 

   
 

   

  
     

 

  

 

 

 

Stream Sequence Output

 

 

 

 

 

 

Interleaver Merger Video

 

 

 

One of the error resilient techniques that are commonly employed over packet

networks is the deployment of Forward Error Correction (FEC) [6, 49-51, 53-58,

60-63]. FEC is usually used in realtime applications where retransmissions of

the lost or error prone packets are not feasible. FEC recovers lost packets that

occurred during transmission; however, it increases the number of redundant

packets sent over the channel, which in turn could lead to an inefficient utilization

108

of bandwidth. In particular, the FEC redundant packets reduce available bitrate
for coding the original video stream, and hence this increases the overall distor-
tion of the transmitted stream. Therefore, the key issue in the FEC scheme for
the realtime media transport is measuring the acceptable number of redundancy
packets to recover the lost packets in a given channel condition, while minimizing
network overhead to guarantee on-time presentation of the transmitted media

and coding distortion.

6.2 Forward Error Correction for Interleaved Source Coding (ISC-FEC)

For ISC-FEC (Figure 41), the main focus of interleave

ng is to improve transmitted video quality over erasure channel without retrans-
mission. Meanwhile, the available source coding rate is decreased to a level
such that the overall transmission bitrate with FEC does not exceed the total bi-

trate of non-FEC protected ISC (Figure 42).

 

Total encoding bitrate per GOV (Non-interleaving GOV size)

 

 

 

 

 

 

 

FEC enabled encoding bitrate per GOV (Non-interleaving GOV size) FEC Bitrate

 

Figure 42 Comparison of bitrate between non-FEC coding vs. FEC protected coding

Therefore, the main focus in ISC-FEC is finding a suitable bitrate for FEC redun-

109

dancy packets while minimizing distortion while reducing the source encoding bi-
trate accordingly.

In addition, the design of ISC-FEC is intended to maximize the benefits of both
the packet erasure recovery nature of F EC, and erasure concealment and short
frame replacement distance of ISC. An example of ISC-FEC is illustrated in

Figure 43.

 

 

 

 

 

 

 

FEC Protected Frames

(b) ISC-FEC
X J

V
FEC Protected Frames
(c) ISC-FEC packet transmission order

 

Figure 43 ISC-FEC design scheme and packet transmission illustration

Different from the original ISC design, ISC-F EC sub-streams share a common

110

intra- coded frame] -frame and some inter- coded framesP - frame based on the
assumption that F EC will protect those frames from losses. Hence the length of
FEC redundancy packets and the number of F EC protected frames vary depend-
ing on the packet loss rate and maximum distortion variation allowed with FEC.
From the observations in Chapter 5, since multi-stream interleaving perforrn-
ances are very close to each other, the frames that are not protected with FEC
will be interleaved into two sub-streams. There is a risk involved with this de-
sign, if FEC fails, the quality degradation risk is almost the same as that of non-
ISC coding. Therefore, when computing the number of R-D optimized FEC pro-
tected frame, this number should be carefully selected so that it does not exceed
the FEC recovery rate. The design will consider maximum distance separable
(MDS) codes, i.e., Reed Solomon codes, for FEC[49-59]. ln MDS code, usually
defined as [12,16], where k is the number of message packets and h = n — k is
the number of FEC redundancy packets. If any 1: packets are received out of
the 1?. packet transmission block, the system will be able to recover any lost
packets in k. Hence h / n is the loss recovery rate of the given system. How-
ever, if the number of lost packets are greater than h, the risk factor of ISC-FEC,

the system won’t be able to recover the lost packets, hence, ISC-FEC will use

111

any received message packets to decode some frames that are not affected by
packet losses, and yet, partial decoding is still not an option as in the original ISC.
In addition, the rate assigned to the FEC-protected portion of the video stream
and the rate allocated to each interleaving sub-stream is determined based on
the number of frames in each corresponding part such that the overall (total) bi-

trate remains the same regardless the pattern of ISC-FEC.

6.2.1 Rate-Distortion Optimized ISC-FEC

Since ISC finds optimal interleaving pattern specific to channel condition and se-
quence, ISC-FEC assumes that R-D function can also be found empirically as a
preprocess information. A curve fitting method with the Minimum Mean Square

Estimator (MMSE) is used to obtain a R-D function for a given sequence.

arg min [MSE {D (R),a x exp (—R) + c, VR}] (42)
{a,b,c}

D (R) = a x exp (-Rb ) + c (43)
With a sequence specific R-D function, ISC-FEC computes the number of FEC
redundancy packets for MDS with. predefined variable, e.g., number of frames in
a GOV (un-interleaved size), frame rate, bitrate, packet loss rate, and maximum

coding distortion (%). The computation process to find number of FEC packets,

112

FEC protected frames, as well as FEC protected packets is shown in the follow-

ing pseudo code:

 

 

Compute total number of packets per second, totP/rt using current bitrate, bi-
trate/ packetsize ‘
Compute current distortion, iD/lsto/tion using current bitrate (43)
fDistortion = iDisto/Tion * (1 + AD/‘stortion /100)
using fDistortion new coding bitrate, compute nbitrate, by reversing (43)
if nbitrate < bitrate
k = Lnewbz'trate / packetsizeJ : Number of message packets per second
h = totht — k : Number of F EC packets per second
7; = l'k/ framemte‘l: Average number of packets per frame
tothtGOV = 17 x NGOV : Total number of packets in a GOV
hGOV = (h / framemte) x Nam/3 Number of FEC packets per GOV
If ham; 2 1 :break point1
"GOV = LhGOV / p1_| : Maximum number of packets protected over chan-
nel with packet loss rate p1
If "GOV < tothtGOV : break point 2
kGOV = "GOV — hGOV : Number of FEC protected packets
N FEC = [_(kGOV / n) _1 : Number of frames protected with FEC in a
GOV
If NGOV — NFEC 2 odd
N FEC = N FEC —1 : Make the remainder of GOV into even number
for two sub-streams without sacrificing recovery
rate
end
If NFEC > 0 : break point3

Return NFEC,nGOVa hGOV
end

 

113

 

There are break points set up in the code: break point 1 stops the process if the
number of F EC packets, hGOV , is less one since there is no full packet length
FEC redundancy packet. It is possible to pad the packet with zero to make a full
length F EC redundancy packet, however, if the bits used in zero padding add up
over time, it could result the coded video to exceed assigned bitrate. Therefore,
to prevent such ripple effect, FEC parameter computation process terminates in
such case. Break pointZis set to stop process if the total number of F EC
coded packets in a GOV, ”GOV: is greater than total number of packets as-
signed to a GOV. In such case, all the frames in a GOV can be protected with
FEC, therefore, interleaving is not necessary. This usually happens when the
maximum distortion variation allowance is set to high. Braakpoint3is set before
returning the computed values to the Sequence Separator. It is set to stop the
process if number of FEC protected frame, N FEC , is equal to zero. Since the
system encodes the remainder of non-FEC protected frames in a GOV into two
sub-streams, the number of frames associated with interleaving must be in even
number. To satisfy such constraint, N FEC is reduced by one frame, which in
some cases, become zero. It is possible to increase N FEC by one frame,

however, this can cause the system to exceed the FEC recovery rate.

114

6.2.2 ISC-FEC
When FEC parameter computing process returns values, N FEC , "GOV: and
hem; , ISC-MDP goes through the optimal interleaving set search process for the

remainder of the frames in a GOV that are not protected with FEC with following:

2 .

fl 3(k’j) = E, Vj, size (3(k’j)) = (NGOV — NFEC)/2

Since these interleaving patterns references to the last frame in FEC protected

portion of the GOV, the interleaving set is redefined as:

5*(k’1)(n) = {o 303%» — 3W) (0) + 1}
5*(k’2)(n) = {O 3(k’2)(n) + 1}

, fOYOSnS(NGOV-NFEC)/2(45)
Once the set is defined, ISC-FEC process finds an optimal interleaving set for the
frames in the remainder of the GOV with the ISC-MDP equations, (31)-(38), and
the decision criteria and equations, (19)-(30)

For encoding and decoding of the interleaving portion of the GOV, the system

uses interleaving set parameter from (44) instead of (45).

115

6.3 ISC-FEC Evaluation and Analysis

6.3.1 Simulation Setup

For evaluation, CIF sequences of Akiyo, Foreman, Coastguard, and Mobile were
coded into an IPPP... GOV structure using an MPEG-4 encoder. GOV sizes (un-
interleaved size) of 10, 12, 14, 16, 18, and 20 were used to protect and partition
the evaluation sequences. Frame rate of 15 frames per second, bitrate of
250kbps and 500kbps, and packet size of 512 8er were used for encoding the
sequences. Network conditions were set to 5%, 10%, and 15% packet loss
rates, p1, with varying packet correlation value, p, of 0.3, 0.6, and 0.9. For
each network condition p1 — p pair, ten packet loss traces were generated. For
FEC parameter computation, maximum distortion variation of 1~5% were used.
Since ISC-FEC R-D optimized protection and interleaving pattern search algo-
rithm targets to a specific sequence, sequence specific correlation model is used
to find an optimal interleaving set for ISO. In addition, the ISC-FEC results were
compared with sequence specific ISC, ISC-SC. Each evaluation case is fitted
into these packet loss traces and the PSNR values are averaged to provide sta-

tistically satisfying results for analysis.

116

6.3.2 Simulation Results

20 - 400
18 -
350 -
16 -
14 _ 300 -
NA - A
e 12 "9, 250 —
§ 10 - 5
g g 200 .
o 3 ' '5
6* 150 -
4 _
100 ~
2 -
0 * ' 4‘ 50
0 5 10 15 0
Rate (bps) x 105
Coastguard 15 frames per second
100
90 -
80 »
70 ~
100 -

”e “9. 5° '
§ .5 50 .
5 5
'5 5 4° -

50 —
30 —
20 -
10 -
o 1 I 4| 0
0 5 10 15 0
Rate (bps) x 105

 

Akiyo 15 frames per second

 

 

 

117

 

Mobile 15 frames per second

; l l

 

 

5 10 15
Rate (bps)

Foreman 15 frames per second

; L J

 

5 10 15
Rate (bps) x 10

Figure 44 Rate-Distortion of the evaluation sequences

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

43 Hldyo @250kbps
. I/B—-~——B”’da
42.5- /,/B"'
B/
42r////
41.5“ ’/,6\\
’,,4y’ \\
41 ’0’ ,0--~'9\\ \ ”id-‘0
<>~—/-e"" \\ 6~
40.5- A Y9“
‘5— 5% 0.9 \\0
40' + 5% 0.6
54 -€- 5% 0.3
39. - . . L .
1‘3
$43.
0
gm» xflflflxflflfl, _____ PH“),
- /
41- /
g / +~-~"+~~-
54OJK // ~4_____k\\

/ \\
E39159. +
(5 “an“

9 \\*
838' \\\
o: + 10960.9 \*‘~
§§37> -+-‘KN60£5 “wg\\
0- + 10% 0.3 a“?
”36 - A , _ .
3
42'
IV“"V“--v-—-~-v-~__V
40,7”
I’IAx““A_
3845” —---A\\
\
“*‘M-e \\ 1,1:
36. \\ A”
\x /4.\\
34» +15%0.9 /// \(K
"9*— 15% 0.6 x \\
-4— 15% 0.3 \ {I
32 A . , .

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

 

 

 

 

 

 

Akion500kbps
45-
ii r”?’3\“€
44’\ /
\ [I
\\ ,
439k \\ / W
42 \\ is, /Q\‘\
Q \\ // \m
\\ \ /
41' \ \\ / Q
\\ // \\\
40 ' \O// \e_‘__‘0“
‘re
39 A 4 A k A
10 12 14 16 18 20
443K )‘uz
\\ // “hm
42. \xx/
40'»‘\ \
\\+”’f,—+—~-_~+\\
‘~ _-——+
38-
36T\\alez
“*"T'HK
\
34» \
\\
._...--%lE
32 A A n f A
10 12 14 16 18 20
45
+‘- \ , f/v----V~~s_.v-__~,7
40- V
..-—A-~
” "‘"“A-————A\ xA
35$:_-v-—Q\ \\\\8///
\\ /’/4\\\\
30» ‘4’ ‘<l\
\
\
\
25- \
\
{I
20 A A A A A
10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 1%

Figure 45 ISC-FEC for Akiyo, Maximum coding distortion = 7%

118

43'

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

42.5-
42’
415EJr
41 *
40.5<'
40-
5d —e— 5% 0.3
39. A A
10 12 14 16 18 20
8
'U
4’43-
342' x..--—-—-*"’"""x ————— >e-—-"
_ /
941 - /
’T / +~~~""“-~-
640’K /// +--——k‘ \
Lu / W
339t‘“‘w\
92 “x
338' ‘\
I: —*—‘HE%(L9 \*g‘
537» —+— 10%0.6 “an“
o. + 10%0.3 \111
36 . . A J
g 10 12 14 16 18 20
>
<
42
,V'-——‘V“——-9'"”V""‘V
’1‘, WV
4
/”A\“~
3 ’ fir—_--§\\g’—A
\\
__~ \ [A
361 ‘K A, ’
\\ /4\\
34. +15%0.9 // \k
—A— 15% 0.6 , \\
—<1— 15% 0.3 \\
32 ‘ st]

 

 

101214 16 18 20

Nunber of frames per GOV, Maximun Coding Distortion = 2%

Figure 46 ISC-FEC for Akiyo, Maximum ooding distortion = 2%

119

 

2

 

 

 
 

 

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

 

20 A A
10 12 14

4o .
38- “‘*'"'"4'
at“
36» “11“
‘\*fl,-,*\
\
34- ‘\
\\
”-4
32 4 x:
10 12 14 16 1a 20

 

43 _
42.5 -

 

r

40

 

 

 

 

 

5‘3
493
<3W

 

 

 

 

 

 

E?
3
$43,
8
(542' ”PH“ _____ 4”“
.‘2 ,""’
41- /
g / +~~~~+~-- +._———-+~——+
5401/ // ~.+_____+‘\\
1i: / N
839"‘~~..*\\
“ ‘x
'538- \\
o: + 10% 0.9 \*‘~‘
02,37- + 10%0.6 4*“
0- + 10%0.3 \‘wr
3,36 . 1 A l _
3
<
42-
/V'""V“--v—~v-v-~__v
”I HR?
4017
Ad

 

 

 

 

 

 

<1-..~ \ (A
36» h4\\ A” ’
\\ /4\\
34. «#159603 // ‘4
—A— 15%0.6 / \\
—<1— 15% 0.3 x
32 . L + L *9
10 12 14 16 18 20

 

 

45-
EJ\ /|3-""G“‘*B-—’*‘G
44 -\ /
\ ,’
\ /
43<§ \\ I/
42 \\ \E{ f“
- \ \\
q \ // ‘0‘ ~~ .9
\ \\ / 91-,—
41 h \\ \\ //
\ 5’ £1
\\ // \\\
40’ \O/ \9“--‘O~..‘
‘0
39 A A
10 12 14 16 18 20

 

 

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

 

20 A A A
10 12 14 16

Number of frames per GOV, Maximun Coding Distortion = 3%

Figure 47 ISC-FEC for Akiyo, Maximum coding distortion = 3%

120

 

 

 

 

 

 

 

 

 

 

43' -a 45
.a’E}_—~—_G"f E] 6g ,pf’fi‘“~—E}_—-———-‘E}
425- ’3”, F
' x’ 44'\ /
ta” \ ,
42- / \ /
// G 43<k \\ ,I
/ \
\ - \ \\
I’d-i \ \ / \G‘“
41 f3 _,,ev'”’e\ \ ”#6 (K \ / “ewe-re
<>v—~’-9” \ ‘9 41» \ x /
40.5» / 62 X \ ’ Q
—B— 5% 0.9 ‘\\0 \‘V // \\\
40- -6— 5% 0.6 40' \ / ‘e-~—-e
-e— 5% 0.3 G “‘~o
39.55 + . a . . 39 , A - - -
1o 12 14 16 18 20 1o 12 14 16 1a 20

 

 

 

 

 

 

 

 

 

 

 

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)
Average PSNR of ISC-FEC (-) vs ISC-SC (—) (dB)

 

 

 

 

 

 

 

 

 

 

\ / ‘4‘
34, +15%o.9 // ‘4
"A— 15% 0.6 / \\
—<1— 15% 0.3 x
32 . . 4 . ‘1 20 L - - - .
1o 12 14 16 18 20 1o 12 14 16 13 20

Number of frames per GOV, Maximun Coding Distortion = 4%

Figure 48 ISC-FEC for Akiyo, Maximum coding distortion = 4%

121

Akiyo @250kbps Akiyo @500kbps

 

 

 

 

 

 

 

 

43, "a 45,
’,,El——-—-—B*" 5‘ ’#',.B.__‘__B_F'_._B
425. ’3' E )3
. /,I 44-\ /
/ra \ /
42F // 43 \\ l/
r/ ’9 <k \\ I
41.5E 4”/ \\\ 42 \\ Ei/ /G~.\\
a’f’ _'_ \ C \ / \fi‘N“
<>--'-"‘L9"’ \\ 41, \ \ /
40.5 , 15A \ x / Q
'5" 5% 0.9 \‘\0 \\6 // \\\
40 —6— 5% 0.6 40‘ \ / ‘e—~--~5
—e— 5% 0.3 0 ‘ Lo
39.54 L L 39 L L L
10 12 14 16 18 20 10 12 14 16 18 20

hub
NO)

    

 

 

 

 

 

 

 

 

36 “an“
2‘*~,,,*\
\\
7 —+—10%0.6 “~4\ 34* \\
+ 10%0.3 \4 \ ”4
36 A . A 32 A A f" A
10 12 14 16 18 20 10 12 14 16 18 20

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)
50 o) to A 5.
Average PSNR of ISC-FEC (-) vs ISO-SC (-) (dB)

 

 

 

 

 

 

 

 

 

 

 

 

< —_._ \ ,A
36?- -<1\ \A”’
\\ /Q\\
34. +15%0.9 // ‘<1\
-A— 15%0.6 / \
—<1— 15% 0.3 x
32 L L L L 81 20 L . L L L
10 12 14 16 18 20 10 12 14 16 18 20

Number of frames per GOV, Maximun Codng Distortion = 5%

Figure 49 ISC-FEC for Akiyo, Maximum coding distortion = 5%

122

Coastguard @250kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

37-
{Ia—mv-fi-----Ei
[}~——~B_~-'"'B'f’
36.5-
36- ,,4>~-~
¢>===zzg” ~9\
“‘~<l\ \
35.5 \\\ \\
®~\“:@____.0
35 —B'— 5% 0.9 \\\‘
—0— 5% 0.6 0
—e— 5% 0.3
34.5 ‘ ‘
10 12 14 16 18 20
8
3
£37
336* “““ *“w”
A ~‘+——"”
\L’35' +_~_‘+~~hb‘1‘“~~~+
- ‘*“
‘8 “K
m33 + 10960.9 \\
5 —1—— 10% 0.6 X‘Nx
o. + 10% 0.3 1‘
332 ‘ ‘ ‘ ' '
e 10 12 14 16 18 20
3%
3667. ........ ____ _-_
V 'V" V— fi"“*—-V
35'
34, \A‘~*--A\
\\A‘~\
“A
384_ \\‘\
~-- ‘A
~€~~~~fl
32. +15%0.9 \
+15%0.6 \
-<1— 15% 0.3 \qfl ,4
31 A . . “—59%: 4
1o 12 14 16 18 20

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

Coastguard @500kbps
39
38 ________ W
1:}- E ~13“..- ____B_'__’fl
371
<|>~- M‘s
a‘e~~h~
36 *}‘“~~9K\
cx‘x \\‘9~.
\‘e‘ ‘K‘
\~ '0
35 ' ~‘O-—~..e____
-Q\
\
\
\
L ‘0

 

 

 

 

*«a \r
32 r “*~~~...*~_“*
30 \\ /’*
. at,
28 4
10 12 14 16 18 20

 

26 n L n .
10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 1%

Figure 50 ISC-FEC for Coastguard, Maximum coding distortion = 1%

123

Coastguard @250kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

37.5-
37~ B”//£L\\\§E
,,EF-*-43—---
sescf‘”“‘a""'&’
36,
<>==2::8~::’d0.5‘5‘9\
35.5» QL\ \\
\9-~ \
—-e— 5% 0.9 “fix—‘9
35' —e— 5% 0.6 “0
—e— 5% 0.3
34.5 L L L L L
10 12 14 16 18 20
a
3
.L37_
0
a) :b-—-—x‘—--*@~———¥==::jt*——F*
836- ““‘x
g -55“ +\+———+
$35» +““‘*‘“-~+~~-_2+_~__
8 4‘ *\»\‘:
n- \2
034- ”“4“
<2 was
6 “R
0:33. —*— 10%0.9 ‘\\
5 + 10% 0.6 *\\
Q + 10%0.3 ‘6
832 . . l n J
210 12 14 16 18 20
3
36:
35.
4
34
334— \\
~~-‘ A
€“~~-A
32, +15%0.9 \\
—£~.— 15% 0.6 \
-<1— 15% 0.3 ‘2,“ #4
31 L L - “new" L
10 12 14 16 18 20

Coastguard @500kbps

 

 

 

 

Average PSNR of ISC-FEC (-) vs ISC-SC (—) (dB)

 

 

 

 

30- ‘A
<1.“~~
28 ’ ~€“\‘ _ ’4
%‘-_ Q‘Q-‘Ibflfi”
26 . 1 1 1 .
10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 2%

Figure 51 ISC-FEC for Coastguard, Maximum coding distortion = 2%

124

Coastguard @250kbps Coastguard @500kbps
37.5 -

37 C]

 

—a— 5% 0.9 “-@:::'*>
35' + 5% 0.6 ‘0
—e— 5% 0.3

 

 

 

 

 

 

 

 

 

 

 

K
I
I
l
J
T
f
t
I

w
M
I
1
ate
1
l

 

8
a|</

 

 

 

 

 

 

 

Average PSNR of ISC-F EC (-) vs ISC-SC (-) (dB)
I
I
Average PSNR of ISC-F EC H vs ISC-SC (—) (dB)
_.'63
1'
I
I
4.

 

 

 

 

 

 

 

 

 

 

“Ax
33¢~~_ A \\‘A
—v— 15% 0.9
32. +15%o.6 \\
—<1— 15% 0.3 \14 ”4
31 L - L “KL-4L" L 26 L L L L L
10 12 14 16 18 20 10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 3%

Figure 52 ISC-FEC for Coastguard Maximum coding distortion = 3%

125

Coastguard @250kbps Coastguard @500kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

37- 3881- ———————— -~..
,,EF-*-43--——E a {} ‘5“‘-~4¥::::g
E}~--2B_--—{r”
36.5
37 h M
36 02“ 0/0
“84
,4>~2 ~~~
<°====z L” “e\ 36» “(k-«49
“‘~<x \ \\\
355 \\\ \\ <>~\\\ \\@~.
\&\ \e\‘ \\\
“:@:'“*Q 35 ~ “0“.— ‘9
35. _B- 50160.9 \\\0 9—~~__Q
+ 5% 0.6 X
—e— 5% 0.3 \\
345 L L i i Q 34 . L . 4 g5)
10 12 14 16 18 20 10 12 14 16 18 20
6 6
3; El
3 :38
9 a 36M
3 334- ‘+~---+~-~-
8 a +“‘“*\
:3 :3 *LL 4
32L “ ~"~~.
<2 9 *“~~*\
\ As
2 33 . —x—— 10% 0.9 \‘\\ 2 30 . \\\ ///
oz) —1— 10% 0.6 *L\\ 5:, L"
Q —*— 10%0.3 1" D.
332 ‘ ‘ ‘ ‘ a 328 s L ‘ %
g 10 12 14 16 18 20 g 10 12 14 16 18 20
o
3E &

 

 

 

 

 

 

 

 

 

 

 

33b \\ 30’
<}~~‘__+ A ‘fl
—9— 15% 0.9 “wags
32‘ +15%0.6 \\ 28' “xvflflegn (”a
—e— 15% 0.3 \°G~ ,4 “6"
31 1 l . ‘~~,.£.--"¥ 26 L 4 . . .
10 12 14 16 18 20 10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 4%

Figure 53 ISC-FEC for Coastguard, Maximum coding distortion = 4%

126

Coastguard @250kbps

 

 

 

 

 

 

37,
,,a——--—B----+3
C}~-_~B___-{r”
365
36- 4>
<x=::: 4”’ “‘~sk
“*~<1
35.5» \\ \\
®“‘~~§@————o
35. -B— 5% 0.9 \‘x\
-+»-5%505 ‘0
-e»-59503
34.5 1 1 1 1 1
10 12 14 16 13 20

0)
\l

U
0)

0° 0°
00 -h

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)
o:
_Jo

37'

31
1

\l
v

l

03
0|

 

 

. —><— 10%o.9 \\
+ 10% 0.6 *\

 

 

 

+ 10% 0.3 ‘1‘

 

 

 

o 12 14 16 1a 20

H 1',

 

 

 

 

 

 

N
r ‘N‘A—q—"A'x
\‘\A_\
2~~A\

}‘*'~;—1 A \\‘A

—~?— 15960.9 \
‘ +15%05 \

‘6'— 1150/6013 \§‘ ,n'd

‘~§ 0”

 

01214 1618 20

Average PSNR of ISC-FEC (-) vs ISC-SC (—) (dB)

Coastguard @500kbps

38 ________ _

E}- B {3— ”“‘E’“~—~—g——--£1
37~

<x\\

‘\9“‘~~1}

36* ~“‘-9\

<)~\\ \\\G‘\
35’ ‘1}‘*-~e»-~_

_Q\
\
\
\

34 L L is

 

 

 

 

 

 

 

 

26 1 1 1 1 1
10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 5%

Figum 54 ISC-FEC for Coastguard, Maximum coding distortion = 5%

127

Foreman @250kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

35.5'
35 ' ’,’B--“-B-———qa.——--"B‘—~_.a
:3”
34.5
3464‘
2‘9_‘,,4>~‘~~9\
33‘5<>‘""*fle\\\ \\
\ \ _——a--O
33 _ E‘“‘“O\ G
-B- 5% 0.9 \\
32.5 —e— 5% 0.6 \ “,0
-e— 5% 0.3 ‘9’
32 . 1 - . .
1o 12 14 16 1a 20
E
B
9"”
JP-~——x-_~_fie___d*‘_~fi
34- ‘*----*
8
swunaufix
3 “+\‘
831K ‘\+-~-+
u. ‘\
31 _ are
8 “*4“
‘5 30r ‘9IE\\
m + 10960.9 \\\*‘
5291 —i—10%O.6 unfib
n. + 10% 0.3
828 1 1 1 1 1
g 10 12 14 16 18 20
'>’
'(
3 ________ ,-_, _~
4‘5— V V- v- ~—V._____-V
32A“___“A“
2‘13‘~“
30- 6\
<1...-_‘€“ \\ ”A
“‘4‘ \A’,
28- ~~1 ‘_
Q ~‘*&\
26. -s«— 15% 0.9 ‘\
+ 15% 0.6 ‘<l
-€— 15% 0.3
24 1 - - 4 4
10 12 14 16 18 20

 

 

 

 

 

 

ForemanQSOOkbps
38-
E}-—--a———--&---~W
-fl3__,-
36*
1m.“
6~-~‘«}\\
34- x \
<>~~—~e\ “\e~~
32- \°““9\
\\G—~___o
10 12 14 16 18 20
8
E
¢3Gik-~‘
o ‘*-~--~+<—~—~-
(0 mm
834-
9
4’32““ \\
o ‘~+2\
E ‘~+—~_..-
030’ \e—ak
9 “*‘+
‘52:an
1:: ~ \
\\ ’,*\
g Y” \\
”nu—die
326 1 1 1 ‘1" .
E 10 12 14 16 18 20
3%
356;-
““9““ “_—
1v 'V‘*‘~-v--~
“v
30-
Abr-—~—-A—--—'A~~\
\
\
\\
25¢qu \A
“‘<*~~~-q\\
20 . . 1 NM"?
10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 1%

Figure 55 ISC-FEC for Foreman, Maximum coding distortion = 1%

128

Foreman @250kbps

 

 

 

 

 

 

 

Foreman @500kbps

 

 

 

 

 

 

 

 

 

 

6 E?
E 3
$351 0‘!”
)E_-__x-_~_*__—~H1:*__:_x (D
§34L 1+ a< ‘5
g Q
£33 “‘M“+---—+“~ -.\1\‘- g
532*\ ‘~\ _.._—+ 5
1: ‘x E
31L ‘1‘»
301 “N
2 —x— 10% 0.9 \\\*. .3
02329. —i— 10% 0.6 “~M* 3’
a + 10% 0.3 0'
028 * ‘ ' I l v
8 10 12 14 16 16 20 3
g 3
< <

 

 

 

 

 

 

 

 

 

. —-a— 15% 0.9
—A— 15% 0.6 ”<1
—<1— 15% 0.3
24 1 1 1 1 1 20 1 1 1 1
1o 12 14 16 18 20 1o 12 14 16 18

Number of frames per GOV, Maximun Coding Distortion = 2%

Figure 56 ISC-FEC for Foreman, Maximum coding distortion = 2%

129

 

11%.!-
F.

 

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

 

 

 

 

 

 

ForemanQZSOkbps
36-
[3
35’ ,xB-----i3-~-—-Ei--—"3‘~~~£1
c3"
8
34<?*~-
ag9_,--4}-g~he\
C "fide\\ \\
\\“-~—‘ \G__———0
33 —a— 5%0.9 9\
—6— 59606 \ #0
—e— 59603 \0”
32 1 1 1 1
10 12 14 16 18 20

 

 

 

 

 

 

 

 

 

3 .

3 .

30- “*\\
—><—- 10% 0.9 ‘\*

29» —1—10%0.6 ‘ «1*
+ 10%0.3

28 1 1 1 1 1

10 12 14 16 18 20

 

 

 

 

 

 

 

 

26 . ‘9'— 15% 0.9 \\
—A— 15% 0.6 ‘1
-<1— 15% 0.3
24 g 1 1 1 1
1o 12 14 16 16 20

Average PSNR of ISC-FEC (-) vs ISC-SC (--) (dB)

Foreman @500kbps

 

 

 

_£3

0 12 14 16 18 20

:3

5‘

“%—‘ 3E
~h-

h~~~%

 

_~_+R\

30 ’ \\+“~
“‘1'

zakhfl“

' \
\\ ’,,*\
W!” \\\
26 4 EF”'**

 

—l
O
_L
N
1.;
15
_l
m b
_x
03
N
O

3

 

 

 

20 1 4 -
1o 12 14 16 16 20

Number of frames per GOV, Maximun Coding Distortion = 3%

Figure 57 ISC-FEC for Foreman, Maximum coding distortion = 3%

130

 

‘fu-Icu-u —-—

 

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

 

 

 

 

 

 

 

 

Foreman @250kbps Foreman @500kbps
36. 37E}-—‘——a—--—-B-___-Ek\
13 NW
36 1
351 ,,a-——~4}~___E,_-,_43-2-~B 0* M
if” 35' ““642 0‘" 5
0 “‘Ox
34 ’ \\\
0 ex
‘3‘." '0‘ ‘~‘e\ 33<’>- “—~e\\ \‘9“~--~.O
(rid—H. \ \ \\ \\
33 \“Mue ‘9““*> 32 B'—‘-€>\
-B— 56609 \ \\G [.I
—e— 5% 0.6 \ 31 “no ,
—e— 5% 0.3 Raw-"0
3 ‘ 30 J 1 1

_L
O
_L
N
_L
#
_L
O)
_l
m
N 1
0
_I1
0
_L
N
1.11
1h
.3
CD
_I.
(D
N
O

 

 

 

 

 

 

 

 

 

 

 

6
B
:38
6 i i
0361W l'
3 ‘‘‘‘
A34_W
E3
E32m‘“‘~+‘
3 ' C) “Ki—
\*‘\ Q30 ——-~_+\‘\‘
30' \fiK,‘ '8 \F“‘~1+
+10%0.9 xx m arm-«1k
29. +10%06 “~11 $28 ‘\ "flak
+ 10% 0.3 a. 2" \ ____}k
10 12 14 16 18 20 E 10 12 14 16 18 20
2

 

 

 

 

 

 

 

 

 

 

25. —v— 15% 0.9 \ 25<l~~u~€ ‘16
—A— 15% 0.6 ‘1 ““-<%—~--<1\

—<1— 15% 0.3 \\ .._-<1

24 I I I . 4 20 I . I ‘4’” I

10 12 14 16 16 20 10 12 14 16 16 20

Number of frames per GOV, Maximun Coding Distortion = 4%

Figure 58 ISC-FEC for Foreman, Maximum coding distortion = 4%

131

Foreman @250kbps Foreman @500mps

 

 

 

 

 

 

 

 

35.5r 37E}----—B—-—-'B'---~EL‘
35 ’ ’r’B———‘fi___‘a’—’-—th~-fi 36 '
El”
34.5» 35<}~‘~‘9‘~
a~<%\\
340- \\\\\\\ 4y~~~ 34. \\€K
9 ‘9\ (>5- \\\
33.5cyspr'@\ \ 33» ‘*®\ ‘6““0
\\ \\ ___...() \\
33» ‘0‘““62 ‘3" 32. 0---—e\
£— 5% 0.9 \\ \\
32.5- ‘9‘ 5% 0.6 \ ”,0 31 , \G~~-‘0
—e— 5% 0.3 ‘0’
32 1 1 1 1 1 3o 1 L 1 1 1
10 12 14 16 18 20 10 12 14 16 18 20

  

 

 

+ 10% 0.9 x
9- —+— 10%0.6 n.1,
+ 10% 0.3

0 12 14 16 18 20

 

 

 

 

 

 

 

Average PSNR of ISC-FEC (-) vs ISC-SC (—) (dB)
N co co co w 00
Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

 

 

 

 

 

 

 

 

 

-v— 15% 0.9 “fl 251““ ‘71
—e1— 15% 0.6 \\ *9“---<+—-_1€\

—<1— 15% 0.3 ‘<] ‘xfiflfla

25 I I . I I 20 . I I I

10 12 14 16 16 20 10 12 14 16 16 20

Number of frames per GOV, Maximun Coding Distortion = 5%

Figure 59 ISC-FEC for Foreman, Maximum coding distortion = 5%

132

Mobile @250kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

26»
255?HWM_B_---*}q~_‘fik‘““*3""€
<>~2~
25 ~~e\ M\
245 x>~—-—e\\
<>~-~~e “\9\
24, ‘\\ “4)
‘O~\
23.5- \\
—B— 5% 0.9 \9’”/G\\\
23» -9— 5% 0.6 ‘0
—e— 5% 0.3
225 I - ~ I I
10 12 14 16 18 20
ES
'3
é:26_
>$~-_~ ~_ x———~ash~“~x
825 )4 ~__.,(_____.,( ~~~~~ x—————-x
Eéz4+
3 ‘‘‘‘‘‘ ~1-n \+
23I ‘*~F-_~q
C322-
”—’ L
'621 “we — 1
m + 10%0.9 “~42
023201 -—1— 10%0.6 “wk“
a + 10% 0.3 “2*
319 1 1 1 1 1
E 10 12 14 16 18 20
E
25.
V‘s-Jan-
22}__~_ _‘ .
24. gfi~::§:::::g
22A~~~NA~ “fl
""~A—————A—-___A
20’ V
131—v— 15%o.9 \
+15%0.6 \\ ”242%
_<1— 15% 0.3 '3'”
16 1 1 L 1 1
10 12 14 16 18 20

Average PSNR of ISC-FEC (-) vs ISC-SC (—) (dB)

 

 

 

 

 

 

MobileQSOOkbps
28I
Efl’__,__B_——---‘B~-—-— ..-— ~-_..
261
<>‘—~~‘ NW
6 “‘<>~2‘
2461‘ W
. ‘\eag
ail‘
22 \\&
\\O‘_fl___.o
20 1 1 1 1 1
10 12 14 16 18 20
26x~~.__,,6____~_%_~_~
_§———.fi:::::: _____
24+
22 ““‘~+“\\ +~hfihhk\\‘\+
*\\
‘\+\\
20*\ \‘w-H'"
\~‘*\\
18- ‘»1__-_*\
\
\\ ”’*
16 . A 1 \1 .
10 12 14 16 18 20
251
VF----V~~~
24- ‘*?-—--vk~~2
V“‘“v
22.
zqfi___~A____{L\
. \\ ,2;
18 \ar,, \\\
16<i---~—q\ \‘A
14- ‘*$~2\\
€5‘~“G’"”4
12 1 1 1 1 1
10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 1%

Figure 60 ISC-FEC for Mobile, Maximum coding distortion = 1%

133

Mobile @250kbps

 

 

 

 

 

 

26A-
}—»--a----4}_~__ ~_~ ,__
25.5, a“ ‘9' ‘5
<>~..
25- w.
6\\\ o
24.5 “oh-441
<>~-‘~e “\@\\
24- ‘x\ “g
‘0‘\
23.5 ' \\ ,Q
—B— 5% 0.9 ‘9’” ‘\
23. —e— 5% 0.6 ‘0
—e— 5% 0.3
22.5 1 1 1 1 J
10 12 14 16 18 20

M
O)

N
0)

Average PSNR of ISC-FEC (-) vs ISC-SC (—) (dB)
10
N

   

 

 

 

 

 

21*~~— 4
-><— 10% 0.9 “Ink
20. —i- 10%06 ‘~\*“
+ 10% 0.3 “2*
19 1 1 1 1 1
10 12 14 16 18 20

A
2%“

W
“

 

 

 

 

 

 

20-
13. +15%0.9 \
—A— 15% 0.6 \\ #242236
-4— 15% 0.3 Q”
16 1 1 1 1 4
10 12 14 16 18 20

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

Mobile @500kbps

 

 

 

 

 

 

 

 

16<}~—-——-q\ 1‘-
14- ‘*$~2\\

€h‘~‘“€—""4

12 1 1 1 1 J

10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 2%

Figure 61 ISC-FEC for Mobile, Maximum coding distortion = 2%

134

Mobile @250kbps

 

 

 

 

 

 

26-
255:3"———a—----1}_~_-B___~_B_H_fl
25<'>~“‘\9\
24.5- 8~---6\
<>~2~~e ‘\3\‘
24» \\ “6
n\
23.5- x
—a— 5% 0.9 \V””@\\
23- —<>— 5% 0.6 ‘0
—e— 5% 0.3
22.5 1 1 1 1 1
10 12 14 16 18 20

Average PSNR of ISC-FEC (-) vs ISC-SC (—) (dB)

 

 

 

 

 

 

 

26-

2 1

2 .

2 .

22. \f”

21T:x:10%69 ““4“

20. —+— 10%0.6 \‘\*~_
+ 10% 0.3 f “wk

1910 12 14 16 18 20

 

 

 

 

 

 

 

 

18‘? —e— 15%0.9 \
—e.— 15% 0.6 \\ ”flahufl
—<1— 15% 0.3 '3”
16 I 4 I I -
10 12 14 16 18 20

Average PSNR of ISC-FEC (-) vs ISC-SC (-) (dB)

 

 

Mobile @500kbps
28 '
[}—-—-B-—"”{}“—“a—-"i;==::g
26-
<>-—--‘ ‘. :8
9 -“<>\N
“9~~~
24<>~\\ “G-~~-..0
“G~~‘
‘il\
22 ' \\&
\\\G_._._—-0
20 1 . 1 1 1
10 12 14 16 18 20

 

12
1

 

 

‘
\\
‘

 

‘~-—_Q._,.---—4
18 20

 

0 12 14 16

Number of frames per GOV, Maximun Coding Distortion = 3%

Figure 62 ISC-FEC for Mobile, Maximum coding distortion = 3%

Mobile @250kbps

 

 

 

 

 

 

     
 

 

 

 

 

26
25.5 "“B_-"fl““e‘““9"”€
25"“~9\
24.5 ‘~<>~—--e\
<>-—.__~G ‘x9‘~
24 \‘x “e
0‘\
23.5- \\
—a— 5% 0.9 \V””G\\
23. —e— 5%0.6 ‘0
-e—5%0.3
22.5 1 1 1 1 J

10 12 14 16 18 20
a
3
$26-

JF~“_~ *‘__F*“*‘*fisafififi*
825 x~_‘h*——__* “““““ x--—----x
'§2m_ +“~‘+_—__%~“-+
3 m“‘~+-_ m

23+ ‘“~F~_~_

8 +‘\\ ,..+
£21 ~~ 1

m +10%0.9 “11‘

5201 —i—10%O.6 \\\*_‘_~

0_ -iP-10%Hl3 ~‘*‘
319 I A - 4 I
g 10 12 14 16 18 20
<1:

 

 

 

 

 

 

 

13, +15%0.9 \
.1.— 15% 0.6 \\ “flauhfl
—e— 15% 0.3 ‘1”
16 ‘ 4 ' 4 '
10 12 14 16 18 20

Average PSNR of ISC-F EC (-) vs ISC-SC (—) (dB)

 

 

IWONKHQ§OG¢pS
28r
fiE.-———B--”"B"-*-Ei-----l3-~--~-El
26'
Wfi-h-h" s
6 ~“<}‘\
\‘$“\
24<>~\._ 5‘6“““0
“6\~‘
‘<1\
22* \\€K
\v-‘P’O
20 1 1 1 1 1
10 12 14 16 18 20

N
03

N
O)

_L
a)

r

F——~mfi:::::::::::::::j€“““x

~

N
1 h
Jj/f
i
i
I
+

 

 

.1
_LO)

 

\xk“
‘ ‘\-+-""'-F-'
~.~*\\
1 \*-————.*\
\
\\ ”’,*.
. . 1 3|?" ,

O 12 14 16 18 20

 

 

 

14’ n‘x‘x
Q““«&~**'4
12 1 A 1 1 1
10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 4%

Figure 63 ISC-FEC for Mobile, Maximum coding distortion = 4%

136

 

Mobile @250kbps

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

26
25.5 _fl-Bunfluna“““3“"fl
25' “‘«9\
24.5- ‘o~———e\
<>-e-~e \“6\‘
24» \\ “I0
n\
23.5 x
—B— 5% 0.9 \er”/G\\\
23 -e— 5%0.6 ‘0
—e— 5% 0.3
22.5 1 1 1 1 1
10 12 14 16 18 20
E?
3
$26-
8 X
825' ‘**‘*--x———--* ~~~~~ x——————x
E24-
3 "“*~+~~o
(223' “‘F"~~+\
E \“ «**'+
822’ ‘4'
£2
1521 “~44 - N
m +10%0.9 “‘*\\
$20. —+— 10%0.6 ‘51..“
a. + 10% 0.3 “it
819 1 1 1 1 1
e 10 12 14 16 18 20
3
26
24
224
20-
131 —v— 15% 0.9 \
-A— 15% 0.6 \x ”#4--“4
+ 15% 0.3 ‘1
16 1 1 1 1 1
10 12 14 16 18 20

Average PSNR of ISC-FEC (-) vs ISC-SC (—) (dB)

 

 

Mobile @500kbps
28-
n._———a—----t3‘--~—EJ—--——B--—~£l
26 .
<>_-~‘6~~‘
“$“‘
\6\~.\
24<>‘\\ \~G‘~‘~'O
“9~“
‘“O.\
22 ’ \\9\
\Ya-H-"O
10 12 14 16 18 20

 

 

 

 

 

28I
26
X-
24. ‘\‘\/Q
"*“~~+e
22 “~ka~
eg+K\
20*». “\+**'"4'
eg*\‘
18- ‘WF---—*\
\\\ I”*
16 1 1 1 \'*I 1
10 12 14 16 18 20

 

 

 

16<i~-—-—Q\ ‘13
14- \‘<¥~e\

eee-,~&,,,,4

12 1 1 1 1 1

10 12 14 16 18 20

Number of frames per GOV, Maximun Coding Distortion = 5%

Figure 64 ISC-FEC for Mobile, Maximum coding distortion = 5%

137

6.3.3 Analysis

6.3.3.1 Channel condition and GOV size variation effects

When FEC parameter search process successfully returned R-D optimized ISC-
FEC encoding parameters, following observations are made: when the channel
shows strong memory, p = 0.9, ISC-FEC shows quality improvement over ISC-
SC regardless of packet loss rate or GOV size in most of the cases. This is due
to the fact that the long packet loss burst in F EC protected portion can exceed
FEC recovery rate, hence results ISC-FEC to lose following non-FEC protected,
interleaved frames. However, as the memory strength gets weaker, in other
words, as the randomness of packet loss increases, ISC-FEC shows noticeable
improvement over ISC-SC. As the chance of FEC protected portion to be fully
decoded increases, the cascaded loss effect on interleaving portion decreases;
hence, the total number of frames that can be successfully decoded is higher

than that of ISC-SC.

6.3.3.2 Distortion Variation
With the increment of maximum coding distortion allowance, all the evaluation

cases have shown performance improvement compare to the ones with less cod-

138

ing distortion allowance. In addition, overall performance compared with ISC-
SC has also improved. Even though coding distortion increment decreases per
frame quality in lossless decoding, the increased number of successfully de-
coded frame in ISC-FEC and the overall lossless decoding quality from them can
easily overcome the effect from coding distortion increment. In fact, if a sequence
is coded under the same channel condition and/or with the same total GOV size,
increased coding distortion in ISC-FEC reduces the number of unprotected
frames in a GOV. This results smaller sum of temporal prediction distances in
the interleaving portion of ISC-FEC compared with ISC-SC; hence can also in-

crease overall quality of decoded stream.

6.3.3.3 Evaluation Summary

Overall observation shows that the proposed ISC-FEC method improves per-
formance of ISC-SC, even with the risk of total GOV loss or coding distortion, ex-
cept for some cases where the channel has very strong memory. Similar to the
observation made in previous parts, the most significant quality degrading factor
is randomness of packet losses, or strength of memory. However, with the ad-

aptation of FEC, the effect of memory is reduced, and yet provides more flexibil-

139

ity to ISC-MDP for different channel models.

140

Chapter 7

Conclusions and Future Work

7.1 Conclusion

In this dissertation, lnterleaving Source Coding framework for packet video over
erasure channel has introduced and investigated. The main purpose of ISC
framework is to provide packet loss resilient video-coding for predictive video se-
quences over single erasure channel by reducing the frequency and impact of
the cascaded effect of packet erasures and related propagation of decoding er-
rors. In design, ISC finds optimal interleaving pattern that separates single
video stream into multiple sub-streams based on the channel conditions, packet
loss rate and packet correlation ratio.

Simulations have shown that ISC advances traditional method for most of the
cases; however, the performance highly depends on the channel’s loss rate and
the packet correlation. Since most of the packet losses over the best-effort net-
work channel are caused by the buffer overflow of the routers on the path, hence

the high frequency, short burst (low packet correlation) losses, or isolated single

14]

losses (BEC channel) are more likely to be observed.

In addition to the canonical ISC approach, the proposed ISC-FEC method also

has shown performance improvement. Similar to the observation made in previ-

ous parts, the most significant quality degrading factor is randomness of packet

losses, or strength of memory. However, with the adaptation of F EC, the effect

of memory is reduced, and yet provides more flexibility to ISC-MDP for different

channel models.

In summary, adopting various models of Interleaved Source Coding to the real-

time streaming services over the best-effort IP network would be beneficial.

7.2 Future work

Future extension of ISO framework would be sole R—D optimized FEC protection

for Intra- coded frame. This will require embedding R-D optimized FEC process

into the reward computation process of ISC-MDP framework. With this exten-

sion, we expect to find the advantage of lntra- coded frame protection algorithm

compared with multi-frame protection algorithm of ISC-F EC.

142

Refe re nces

[1]

[2]

[3]

[4]

[5]

[6]

[7]

Borella, M.S., et al. Self-sirni/arihr of Internet packet delay. in IEEE Intema-
tiona/ Conference on Communications (ICC). 1997. Montreal, Canada.

Leland, W. E., et al., 0n the self-similar nature of Ethernet trafi‘ic (extended
version). lEEE/ACM Trans. Netw., 1994. 2(1): p. 1-15.

van der Schaar, M. and H. Rad ha. Packet-loss resilient lntemet video us-
ing MPEG4 fine granularsca/abiliOI. in IEEE lntemet/anal Conference on
Image Processing (lC/P). 2000. Vancouver, Canada.

Yajnik, M., J. Kurose, and D. Towsley. Packet loss correlation in the
MBone multicast netwodr. in IEEE Global lntemet Min/conference, part of
GLOBECOMM. 1996. London.

Yajnik, M., S. Moon, and J. Kurose. Measurement and modeling of the
temporal dependence in packet loss. in INF OCOM ’99. Eighteenth Annual
Joint Conference of the IEEE Computer and Communications Societies.
1999. New York, NY.

Karande, S., M. Wu, and H. Radha. Network embedded FEC (NEF) for
Video mull/cast in presence of packet loss correlation. in IEEE lnteme-

tional Conference on Image Processing (lC/P). 2005.

Van Der Schaar, M., H. Radha, and C. Dufour. Scalable MPEG4 Video
coding with gracefl/l packet-loss resilience over bandwidth- varying net-
works. in IEEE lntemationa/ Conference on Multimedia and Expo. 2000.
New York, NY.

143

[3]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

Lee, H. and H. Lee, A Packet-Loss Recovery Scheme Based on the Gap
Statistics. LNCS lnforrnation Networking, 2005. 3391: p. 627-34.

Chung, S.-H., et al. Detection and analysis of packet loss on underutil'zed
enterprise network links. in Workshop on End-to-End Monitoring Tech-
niques. 2005.

Bi, J., O. Wu, and Z. Li. Packet delay and packet loss in the lntemet. in
lntemational Symposium on Computers and Communications. 2002.

Clark, A.D. Modeling the Effects of Burst Packet Loss and Recency on
Subjective Voice Quality. in IP Telephony Workshop. 2001.

Xue, F ., V. Markovski, and L. Trajkovic. Packet loss in video transfers over
IP networks. in IEEE lntemational Symposium on Circuits and Systems
(lSCAS). 2001. Sydney, Australia.

Feamster, N. and H. Balakrishnan. Packet Loss Recovery for Streaming
Vrdeo. in lntemal Packet Video Workshop. 2002.

Pyun, J .-Y., et al. Packet loss resilience for MPEG-4 video stream over the
Internet. in lntemational Conference on Consumers Electronics (ICCE).
2002.

Su, Y.-C., C.-S. Yang, and C.-W. Lee, The analysis of packet loss predic-
tion for Gilbert-model with loss rate up/ink. Inf. Process. Lett., 2004. 90(3):
p. 1 55-9.

Claypool, M. and Y. Zhu, Using Inter/ea ving to Ameliorate the Effects of
Packet Loss in a l/rdeo Stream, in International Conference on Distributed
Computing Systems. 2003, IEEE Computer Society.

144

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

Liu, Y. and M. Claypool. Using Redundancy to Repair Video Damaged by
Network Data Loss. in IS& T/A CM/SPIE Multimedia Computing and Net-
working (MMCN). 2000. San Jose, CA, USA.

Radha, H., M.v.d. Scharr, and Y. Chen, The MPEG4 FGS video coding
method for multmedla streaming over IP. IEEE Transaction on Multimedia,
2001. 3(1): p. 53-68.

Rad ha, H., et al., Scalable lntemet video using MPEG-4, in Signal Proc-
essing: Image Communication. 1999. p. 95-126.

P. de Cuetos, M. Reisslein, and K.W. Ross, Evaluating the Streaming of
F GS-Encoded Vrdeo with Rate—Distortion Traces. 2003, lnstitut Eurécom
Technical Report.

Lin, S. and Y. Wang. Error resilience property of multihypothesis motion-
compensated prediction. in IEEE lntemafl'onal Conference on Image
Processing (ICIP). 2002.

Girod, B., Efficiency analysis of multihypothesis motion-compensated pre-
diction. IEEE Transaction on Image Processing, 2000. 9(2): p. 173-83.

Apostolopoulos, J.G. Error-Resilient Vrdeo Compression Through me Use
of Multiple States. in lntemational Conference on Image Processing (ICIP).
2000. Vancouver, BC, Canada.

Apostolopoulos, J .G. and SJ. Wee. Unbalanced Multiple Description
Vrdeo Communication Using Path Diversity. in IEEE lntemational Confer-
ence on Image Processing (ICIP). 2001 . Thessaloniki, Greece.

Reibman, A.R., et al. Multiple description coding for video using motion
compensated prediction in IEEE lntemational Conference on Image Proc-

145

[25]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

essing (ICIP). 1999. Kobe, Japan.

Barrenechea, G., et al. Multiple description source coding and diversity
routing: A joint source channel coding approach to real-time services over
dense networks. in lntemational Packet Vrdeo Workshop. 2003.

Began, A.C., Y. Altunbasak, and O. Ergun. Multi-path selection for multiple
description encoded video streaming. in IEEE lntemational Conference on
Communications (ICC). 2003.

Franchi, N., et al., Multiple description video coding for scalable and ro-
bust transmission over IP. IEEE Transaction on Circuits and Systems for
Video Technology, 2005. 15(3): p. 321-34.

Khayam, SA. and H. Radha. Markov-based Modeling of Wireless Local
Area Networks. in ACM Mobicom lntemational Workshop on Modeling,
Analysis and Simulation of Wireless and Mobile Systems (MSlMM). 2003.

Konrad, A., et al., A Marko v-based channel model algorithm for wireless
networks. WireleSs Networks, 2003. 9(3): p. 189-99.

Levenshtein, V.l. Efi‘icient Reconstruction at the Cutout of a Discrete
Memoryless Channel. in IEEE lntemational Symposium on lnfonnation
Theory (IS/77. 2000. Sorrento, Italy.

Brian A.- Banister, 33., Thomas R. Fischer. Robust Video Transmission
over Binary Symmetric Channels with Packet Erasures. in Data Compres-
sion Conference, DCC 2002. 2002.

Adler, M. and F. Fich, The complexity of end-to-end communicau'on in
memoryless networks, in Proceedings of the eighteenth annual ACM
symposium on Principles of distrrbuted computing. 1999, ACM: Atlanta,

146

[34]

[35]

[35]

[37]

[38]

[39]

[40]

[41]

[42]

Georgia, United States.

Gallager, R.G., Discrete stochastic processes. 1996, Boston: Kluwer Aca-
demic Publishers. xiii, 271 p.

Putennan, M ., Markov Decision Processes: Discrete Stochastic Dynamic
Programming. 1994, New York, NY: John Wiley & Sons, Inc.

We kamiya, N., et al., Dynamic Quality Adaptation Mechanisms for TCP-
fiiendly MPEG-4 l/rdeo Transfer, in lntemational Workshop on Quality of
Service in Multiservice IP Networks. 2003, Springer-Verlag.

Bertesekas, D.P., Dynamic Programming and Optimal Conb'ol. 1995:
Athena Scientific.

INFORM TION TECHNOLOGY — CODING OF AUD/O- VISUAL OB-
JEC TS - Part 2’: Visual, in ISO/IEC J TC I/SC 2.9/WG I 1 , ISO/IEC 14496-
2.2001. 2001: Sydney.

Beynier, A. and A.-I. Mouaddib, A polynomial algorithm for decentralized
Markov decision processes with temporal constraints, in lntemational joint
conference on Autonomous agents and multiagent systems. 2005: Nether-
lands.

Lane, T. and LP. Kaelbling. Approaches to macro decompositions of large
markov decision process planning problems. in SP/E Conference on Mo-
bile Robotics. 2001. Newton, MA, USA.

CODING 0F MOVING PICTURES AND AUDIO, in IS 0/IEC J TC I/SC
29/WG 11 M9706“. 2007: Marrakech, Morocco.

Ebert, JP. and A. Willig, A Gilbert-Elliot Bit Error Model and the Efl‘icient

147

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

Use in Packet Level Simulation, in TKN Technical Reports Series. 1999,

Technical University Berlin: Berlin.

Yee, JR. and E.J. Weldon, Jr., Evaluation of the performance of error-
correcting codes on 3 Gilbert Channel. IEEE Transaction on Multimedia,
1995. 43(8): p. 2316-23.

Chande, V. and N. F avardin. Progressive Image Transmission over Com-
pund Packet Erasure Channels. in IEEE lntemational Symposium on In-
formation 777eory (IS/77. 2000. Sorrento, Italy.

Kizhakkemadam, S.N., et al. Tradeofir between source and channel coding
for erasure channels. in IEEE lntemational Symposium on Information
Theory (IS/U. 2005. Adelaide SA, Australia.

Liang, Y.J., J.G. Apostolopoulos, and B. Girod. Model-based delay-
distortion optimization for video streaming using packet interiea ving. in

Asi/omar Conference on Signals, Systems and Computers. 2002.

Chande, V. and N. Farvardin. Progressive Image Transmission over
Compound Packet Erasure channels. in IEEE lntemational Symposium on
Information 777eory (IS/77. 2000. Sorrento, Italy.

Yousefi'zadeh, H., H. Jafarkhani, and F. Etemadi, Distortion-Optimal
Transmission of Progressive Images over Channels with Random Bit Er-
rors and Packet Erasures, in Proceedings of the Conference on Data
Compression. 2004, IEEE Computer Society.

Wolf, J.K., ECC Performance of Inter/ea ved RS Codes with Burst Enors.
IEEE Transaction on Magnetics, 1998. 34(1): p. 75-9.

Su, Y.-C., C.-S. Yang, and C.-W. Lee. Optimal FEC Assignment for Scal-

148

[51]

[52]

[53]

[54]

[55]

[56]

[57]

[53]

able Vrdeo Transmission over Burst Enor Channel with Loss Rate Uplink.
in lntemational Packet Vrdeo Workshop. 2003. Nantes, France.

Youshi, X. and Z. Tingting, Variable shortened-and-punctured Reed-
So/omon codes for packet loss protection. Broadcasting, IEEE Transac-
tions on, 2002. 48(3): p. 237-45.

Chin, SK. and R. Braun. lmpro ving video quality using packet inter/ea ving,
randomisationand redundancy. in IEEE Conference on Local Computer
Networks. 2001. Tampa, FL, USA.

Matsuzono, K., et al., Analysis of FEC Funcu'on for Real- Time DV Stream-
ing Sustainable lntemet, Lecture Notes in Computer Science, 2007.
4866/2007: p. 114-22.

Huitema, 0., 777a case for packet Ie vel FEC, in T06 W66. I/6‘.4 Fifth Inter-
national Workshop on Protocols for High-Speed Networks V. 1997.

Malik, R., A. Munjal, and S. Chakraverty. Adaptive Forward Enor Correc-
tion (AFEC) based Streaming using RTSP and RTP. in Advanced lntema-
tional Conference on Telecommunications - lntemational Conference on
lntemet and Web Applications and Services (AIC T-ICIM. 2006.

Stan kovic, V. and Z. Xiong, Packet Erasure Protection for Multicasting, in
Conference on Data Compression. 2004.

Mudugamuwa, U. and B.G. Evans, Impact of packet-level FEC and exist-
ing link Ie vel reliability mechanisms to deliver MBMS content over
WCDMA mobile satellite systems. International Journal of Satellite Com-
munications and Networking, 2006. 24(2): p. 169-84.

Vanhaverbeke, F., et al. On the Application of FEC to Counter Packet

149

[59]

[60]

[61]

[52]

[63]

Loss Caused by Buffer Overflow. in 28th Symposium on Information The-
cry in the Benelux. 2007. Enschede, Netherlands.

Luby, M.G., et al., Efi‘icient Erasure Correcting Codes. IEEE Transaction
on lnforrnation Theory, 2001. 47(2): p. 569-84.

Nafaa, A., T. Ahmed, and A. Mehaoua. Unequal and Interleaved FEC Pro-
tocol for Robust MPEG-4 Multicasting Over l/Vrreless LANs. in IEEE Inter-
national Conference on Communications. 2004.

Choi, T.-u., et al., An Adaptive Periodic FEC Scheme for lntemet I/rdeo
Applications, in Proceedings of the Th yrrhenian lntemational Workshop on
Digital Communications: Evolutionary Trends of the lntemet. 2001,
Springer-Venag.

Hayasaka, M., M. Gamage, and T. Miki. Cell Loss Recovery using Adap-
tive LDPC-based FEC Method for Real Time Services over Best Efi‘ort
Networks. in Joint Conference of the 10th Asia-Pacific Conference on
Communications and the 5th International Symposium on Multi-
Dimenslona/ Mobile Communications. 2004.

Hasimoto-Beltran, R. and AA. Khokhar, Dynamic Adaptive Forward Error
Control for Image Transmission over Loss y Networks, in lntemational
Conference on Information Technology: Coding and Computing (ITCC).
2000.

150

ill

III

II
7

I

Will
6 2

VIC

1 will"

llllllllllllll