I“MINIMUM“WWI”!”WINHUIHWHIW MESIS lltlllllllllill}UIHHHIllllllllllllllllllllllllllllllll 3 1293 009096 This is to certify that the dissertation entitled Autoregressive Expansion of Linear Predictor for Stationary Stochastic Processes presented by Jamshid Farshidi has been accepted towards fulfillment of the requirements for PhoDo degree in StatiStiCS 7AM 9M9! ajor professor Date September 27, 1991 M5 U is an Affirmative Action/Equal Opportunity Institution ( W LIBRARY Michigan State University \ 1' PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. DATE DUE DATE DUE DATE DUE l __J| fil ——_l . _ l MSU Is An Affirmative Action/Equal Opportunity Institution chimeras-9.1 AUTOREGRESSIVE EXPANSION OF LINEAR PREDICTOR FOR STATIONARY STOCHASTIC PROCESSES by Jamshid Farshidi A DISSERTATION submitted to Michi an State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Statistics and Probability 1991 A but" 4 >97 - 9.33 ABSTRACT AUTOREGRESSIVE EXPANSION OF LINEAR PREDICTOR FOR STATIONARY STOCHASTIC PROCESSES In this thesis autoregressiue expansion (AR—expansion) of l—step predictor for a univariate stationary stochastic process (SSP) X = {X,; tel} with spectral density f and the Optimal factor «p is studied. The main goal is to find necessary and sufficient conditions on f or p for the existence of a mean—squared convergent All—expansion. This problem, and it’s multivariate case, has been the subject of the study of several authors. N. Wiener and P. Masani [Acta Math; 1958], P. Masani [Acts Math; 1960, and Academic Press; 1966], A.G. Miamee and H. Salehi [Math Mexicana; 1983], M. Pourahmadi [Proc. Am. Math. Soc.; 1984] and others. A The important results in this thesis are: (i) The uniqueness of AR- expansion, (ii) necessary and sufficient conditions based on f and cp guaranteeing the existence and the uniqueness of the AR—expansion, (iii) equivalence of the existence of the AR—expansion with the inuertibility of X, and (iv) sufficiency of the set ofconditions "(l/f) e LP(A) and f EL‘KA) for some p, l 5 p < m and 1 < q 5 m with 1/p+ l/q = 1" for the existence and the uniqueness of AR—expansion. The important feature of this thesis is to give spectral characterization for an AIL-expansion of 5(1, without undue attention to the condition (1/f) in L‘(A) which emerges as a basic restriction in the work of earlier authors. This is accomplished by the consideration of the Optimal factor, it’s reciprocal, the examination of their Taylor coefficients, and the use of some facts from probability theory, harmonic and functional analysis. Dedicated to my family from Mahmoud to Ali iii ACKNOWLEDGEMENT I extend my most sincere gratitude to Professor Habib Salehi for introducing me to this historical problem, and for his encouragement and help during the preparation of this dissertation. I would like to express my appreciation to Professor James Stapleton for having served on my thesis committee and for his support in performing my teaching duties in the Department. My special thanks go to Professor V. Mandrekar for his valuable comments on the final draft of this dissertation, to Professor Vaclav Fabian for having served on my thesis committee and for mathematical discussions which enhanced my insight into some of the subjects used in this dissertation, and to Professor Shlomo Levental for having served on my thesis committee. Special appreciation to Professor James Hannan for offering valuable advice. Thanks are also due to Cathy Sparks for her excellent typing of this thesis. To my wife, I must extend a special appreciation because of her patience with me, and her support of my academic work during my stay at Michigan State University. iv Introduction TABLE OF CONTENTS Chapter 1 Preliminaries 1.1 1.2 Definitions and Theorem from Harmonic and Functional Analysis Definitions and Theorems from Probability Theory Chapter 2 Autorqressive Expansion 2.1 2.2 2.3 2.4 2.5 2.6 2.7 Definition of AR—expansion, and Historical Notes The Role of the Optimal Factor Uniqueness of Autoregressive Expansion Weak Convergence of Alt-expansion Necessary and Sufficient Conditions for the Existence and the Uniqueness of the Ali—expansion Invertible Processes and Connection with All—expansion Suficiency of the Condition (1 /f)eL‘(A) 10 21 22 24 26 27 29 37 38 39 INTRODUCTION Let X = {X,; tel} be a univariate Stationary Stochastic Process (SSP) with spectral density 1' and the optimal factor p. An important problem in the prediction theory of SSP’s and in time series analysis is to find conditions on f or «p, which are necessary and suficient for the existence of a mean—squared convergent autoregressive expansion (AR-expansion) for the linear least—square predictor it", = Proj (XMISE {X,; s g t}) of a future value X", based on the observations X'; 1, X,; i.e. an expansion of the form A (I) Xm = n20 an('r) XHI; tel, 1'2 1, for some sequence {an( r); nel}. The problem is fundamental to the entire discussion of SSP’s, time series analysis, the problem of rate of convergence of the predictor, and the estimation of the predictor. This problem, or its equivalent, but simpler form (*7 5‘1 = n50 an x-m and its multivariate extension was the subject of study by several authors. N. Wiener and P. Masani [Acta. Math.; 1958] showed the boundedness of f, namely "0 < c 5 is d < an" is sufficient for the existence and mean—squared convergence of the series in (*). Later P. Masani [Acta. Math.; 1960] weakened these conditions to "(l/f) eL‘(A) and feL‘”(A)", where A is the Lebestue measure on Borel subsets of [4,1]. Since then several attempts have been made to reduce the restrictive condition "feL°(A)" or to weaken this set of conditions. Recently A.G. Miamee and H. Salehi [Bo]. Soc. Math. Mexicans; 1983] gave the necessary and sufficient conditions "(l/fleL‘Q), and the convergence of the Fourier series of 1/«p to l/cp D in L’(f)" to guarantee an AR—expansion (*) with the condition n20 |an| 2 < m. Other authors, following the fundamental work of "Helson and SzegO" [Ann Math. Pura. Appl.; 1960], looked at the problem from a "geometrical" point of view. Among them M. Pourahmadi [Proc. Am. Math. Soc.; 1984] showed that "the positivity of the angle between the past — present and the future of the process X is sufficient to guarantee the existence of a representation as (*)". The common feature of all these works is the imposition of the condition (l/f)eL1(A), and as a result obtaining an expansion of the form (*) with the additional condition £0 |an| 2 < m as in Miamee—Salehi’s. With a fresh look at the problem, in this thesis, the following main results are obtained: (i) the uniqueness of the expansion (*), (ii) providing necessary and sufficient conditions based on f and p, guaranteeing the existence and the convergence and the uniqueness of the AR—expansion, (iii) establishing the equivalence between the existence of the Alt—expansion and the invertibility of X, and (iv) sufficiency of the set of conditions "(1/f)eLP(A) and feLq(A) for some 1 S p < m and 1 < q 5 up with l/p + l/q = 1" for the existence and the uniqueness AR—expansion. The basic tool employed to get the results is the study Of the Optimal factor p and it’s reciprocal l/tp, and to look at the Taylor expansions of 1 / p and it, the L’(f)—analog of 12,. The results are obtained with the aid of some standard fact in probability theory, harmonic and functional analysis. CHAPTER 1 PRELIMINARIES This chapter is devoted to essential definitions and theorems to be used in this thesis. Other definition and theorem which are considered as general mathematical knowledge will also be used. The chapter is divided into two sections, 1.1 and 1.2, covering materials from Harmonic and Functional analysis, and Probability theory. Some of these theorems are so modified to suit the work. 51.1. Definitions and Theorems from Harmonic and Functional Analysis This section contains definitions, theorems and concepts in harmonic and functional analysis which will be used in consequent sections. 1.1.1 Notations. Throughout this thesis: (i) ll, 1 and It stand for the sets of all natural numbers, integers and real numbers respectively. The set of all complex numbers is denoted by (3. (ii) C(0,1) denotes the unit circle; i.e. C(0,l) = {zetz |z| = 1}. (iii) D(0,r) denotes the Open disk with radius r in C; i.e. D(0,r) = {zetz |z| < r}, and D(0,r) denotes the closed disk with radius r in C; i.e. D(0,r) = {zetz |z| g r}. 1.1.2 Definition (Hilbert Spaces). A complex vector space H with an inner product < -,- >: HxH -o C (and norm ||x|| = < x,x >i for er) which is complete (i.e. every Cauchy sequence {xn} in H converges in norm to some x in H) is called a Hilbert space. Special types Of Hilbert spaces are La—spaces, which are used extensively in this thesis. The following definition extends this notion. 1.1.3 Definition (LP—spaces). Let 0 be a non-empty set, .7 a o—algebra of subsets of {l and u a positive measure on .9.’ (i) If 0 < p < m, and X is a complex 3—measurable function on 0, define nxu: == ;, lepdu. and let Lp(p) consist of all equivalent classes of X (in the sense that X, ~X if and only if X, = X, a.e. M) for which ||X||p < a). We call ||X||p the Lp—norm of X. (ii) If Y is a complex measurable function on 0, we define ||X||m to be the essential supremum Of |X| , and we let L"(u) consist of all equivalent classes of X for which ||X||m < on. Note that H = L201) with the inner product = ex? = i x? (111 is a Hilbert space. 1.1.4 Notation. Throughout this thesis the symbol A is used for the normalized Lebesgue measure on $[—r,1r], the Borel sets of [—1r,1r], i.e. dA = %? db and Ak stands for the Lebesgue measure on the Borel sets of It“, 3(8“). If a pofitive measure a on 3([—4nr]) or .201“) has a density f w.r.t. A or Ak, we will write Lp(f) or L:(f) for Lp(u). 1.1.5 Theorem (the projection theorem). If M is a closed subspace of the Hilbert space H, then (i) there is a unique element xeM such that ux—icn = int {llx-yll; yeM}. and (a) QM and ux—alu = inf{x—y||; yeM} if and only if xeM and (x—x) eM", where M‘ = {yeH: (y,x) = 0, VxeM}. Proof. cf. [16] Theorems 4.10 and 4.11; or [3], p 51. 1.1.6 Notation. The element x in Theorem 1.1.5 is called the orthogonal projection of x onto M, and is denoted by £= Proj(x|M). Proj(- |M) defines a mapping from H onto M which is called a projection mapping. 1.1.7 Theorem (Properties of Projection Mappings). Let H be a Hilbert space and let P(- |M) denote the projection mapping from H onto a closed subspace M of H. Then (i) P(am+fiYIM) = aP(X|M)+l91’(y|M) (ii) P(xn|M) -* P(X|M) if len - xll -' 0 (iii) Mt C M, if and only if P(P(x|M,)|M,) = P(x|M,), for all er. (iv) If a family {Hg tel} of subspaces of H has the prOperty that H8 C Ht for s < t and 9 H, = {0}, then for any er Proj(x|Ht) -b O as t -o -m. Proof. For proofs of (i) — (iii) see [3] p 52. (iv) Denote Proj(x| H,) by h,, tel. For any sequence {tn} in l with the property t1 > t2 > and tending to — m, the elements ht1— htz, ht2 - ht3,... are mutually orthogonal, and II u>uh-h II-uia —h )II-[inh h "’1‘” new hit ' ‘1 tan - i'1 ii tin — i'l ti- tin ’ . (I) it follows that the series i21(h‘i — h,i ) converges in H. Since the ith partial sum 3 +1 of this series is h, -- ht, lim hti exists, and since it is evidently contained in each 1 1 i-Hn subspace Ht , for each i, hence lim ht = 0. u. i i-Hn Weak convergence and weak topology play a role in some of the theorems in Chapter 2. The following definition and theorem are adapted forms of these notions in Hilbert spaces, which will be used. 1.1.8 Definition (Weak convergence in Hilbert Spaces). Let H be a Hilbert space over C. (i) A sequence {xn} in H is said to be weakly convergent, if there is an er with lim < me > = < x,y > for every yeH. The point x is called a n4!!! weak limit of {xn}, the sequence {xn} is said to converge weakly to x, and we write x 3—9 x. (ii) A set A g H is said to be weakly sequentially compact if every sequence {xn} in A contains a subsequence which converges weakly to a point in H. (iii) Every sequence {xn} in H such that {} is a Cauchy sequence of complex numbers for each yeH is called a weak Cauchy sequence. 1.1.9 Theorem. Let H be a Hilbert space over (I. Then (i) A weakly convergent sequence in H has a unique limit. (ii) A weakly convergent sequence {xn} in H is bounded It’s limit x is in the can; new} and lell 5 .112"an- n-r (I) Proof. cf. [5] p 68, Lemmas 26 and 27. 1.1.10 Definition (Fourier Coefficients and Partial Sums). Let p e L‘(A). The nth Fourier coefficient of «p is defined as . 1 ‘1 -int . tp(n) = 2?!” «p(t) 8 dt, nel, and the nth order Fourier partial sum of p is defined to be Sn(so)(0) = 53,. (5(1) 2.10. -«s o s «. new. Note that show) = .151: at) Duh-0) at = (v .D.) (a) where sin| 111+”? sin ’ 0* 0 _ *1 fie- Dn(0) ’13-)" e - 2n+l ; 0 = 0. Dn is called the Dirichlet kernel In the Hilbert space H = L’(A), consider the elements en, nel, defined by en(0) = sue for —rr$ 051. Then 1 rr i(m—n)t =fi£xe dt=6m.n i.e. (en; nel} is an orthonormal set in H, «p(n) = < «p, 61, >, nel and snw) = Pronouns-j; In sn}). new. 1.1.11 Theorem. (a) The orthonormal set {en; nell} is complete in H = L’(A), in the sense that the only vector in H orthogonal to it is the zero vector. (b) For every peL’(A) the sequence {Sn(«p)} converges in L’(A) to p. (c) (Parseval) For every cp and w in L’(A), < = n23“ <$,en>, and as a special case 2 P 2 ll¢||2= n?,,l<| . Proof. cf. [8] pp 28 and 29. It was conjectured in 1915 by Lusin that if peL’(A) then Sn(¢p) -r (p a.e. [A]. In 1966 Carleson proved this conjecture, and also showed that for p e LP(A) 1 < p < 2, Sn(go)(0) = 0 (log log log n) a.e. [A], (cf. [4]). One year later, Hunt modified Carleson’s technique and proved for p > 1, Sn(¢p) converges a.e. [A]. 1.1.12 Theorem (Carleson). Let peL3(A). Then Sn(¢p) -r (p a,e,[A]. Furthermore, if cpeLP(A), 1 < p < 2, then Sn(¢)(0) = 0 (log log log n), a.e.[A]. Proof. cf. [4]. To any function g: D(0,l) -r C, we may associate a family {gr: 0 g r < 1} of flmctions defined on T, by 9 9 g,(¢ia)=g(ré ); 0_<_r_ 0. 12 This property connects the autocovariance functions to the so—called distribution functions, via the following theorem: 1.2.4 Theorem (Herglota—Bochner). A function 7: l -o C is non—negative definite if and only if 7 'its 7(t) = I e dF(s) for all tel, —T where F is a right—continuous, non—decreasing, bounded fimction on [-rr,1r] and F(-rr) = 0. Proof. of. [3] pp 115—116, or [11]. Using Theorem 1.2.4, we have 7.0) = 1' 'é“dF.(x) where F1! is as above. 1.2.5 Definition. For a SSP X, the corresponding function Fx in Theorem 1.2.4 is called the spectral distribution function of X. If Fx(t) = It f(s)ds; —rr$ A 3 r, then f is called the spectral density of X. 1.2.6 Notation. For the SSP X = {X ; tel} we introduce the following subspaces: (i) Hx(t) = §5{st s g t}, tel (the past of the process). (ii) Hx = 351th tel} (the entire space of the process). (iii) Hx(— m) = tnl Hx(t) (the remote past of the process). e Here EA denotes the closed linear subspace of L’(fl,.9,’ P) generated by the elements in the subset A of this space. 13 The Linear Predictor of a SSP 1.2.7 Definition. Let x = {x,; tel} be an SSP. Let tel and relll. (i) The (best) linear r—step predictor of xt+r based on Hx(t) is an element X(t,r) of Hx(t) which minimizes the distance between Xt d Hx(t). (ii) The mean-squared error in prediction in (i): r 2 02W") = EIXUJ) _ xt-I-TI is called the (mean—squared) r—step error of the prediction +r'an By projection theorem (Theorem 1.1.5) X(t,r') := Proj(Xt+T|Hx(t)), tel, nu. An important consequence of stationarity of SP’s is the existence of a family of linear operators {Ut; tel} on HI, which shift the elements and subspaces of Hz. The following theorem proves the existence of such a. family, and establishes their basic properties. For a proof of [15], pp 14, 54 (Relations 1.4 and 1.5), or [11]. 1.2.8 Theorem (shift operators). There exists a unique family of unitary operators Ut, tel, on H! such that for s,t el: (a) Utxs = xt+sr (b) U.H.(s) = H.(s+t). (c) UtX(s,r) = X(t+s,r). {U,; tel} is called the family of shift operators associated with X. Property (c) easily implies that o‘(t,r) is, in fact, independent of the variable t. 14 1.2.9 Corollary. For every s,tel and rel, o’(t,'r) = o’(s,r). 1.2.10 Notation. For 33? x = {x,; tel} (i) Denote the 1—step predictor X(t—1,1) by X,; i.e. it, := pioxxd H,(t-1)); tel. (ii) Denote the one—step error of prediction o’(t—1,1) by a“, for every tel (c.f. Corollary 1.2.9); i.e. 03:: EIXt—thz; tel, The Wold Decomposition In this part, we summarize results that are related to the Wold’s decomposition which play an important role in the analysis of SSP’s. 1.2.11 Definition. A SSP X = {Xg tel} is called (a) deterministic, if X(to,ro) = xto‘l'To for some toel and roell - {0}. (b) regular, if lim X(to,r) = 0 for some toel T-H-m (c) white noise with mean 0 and variance (1’, denoted by WN(0,a’), if EX, = o, tel, and 7.(t)={3"§ it :3. Using the family of the associated shift Operators of X, it is easily proved that if X is deterministic, then X(t,r) = xt+r for all tel, rel, and if X is regular then lim X(t,r) = 0 for every tel (c.f. [11], or [15] p 52). T-H-tn 15 1.2.12 Theorem. A SSP X = {X,: tel} (i) is deterministic if and only if H, = H,(— in), (ii) is regular if and only if H,(— m) = {0}. Proof. of [11] or [15] p 52. 1.2.13 Theorem (Wold). Every SSP X = {X,; tel} can be represented as X, = W, + V,; tel where W = {W,: tel} and V = {V,; tel} are regular and deterministic SSP’s, respectively, such that (i) H,(t) c H,(t) and H,(t) c H,(t), tel and (ii) V and W are mutually orthogonal; i.e. EW,V, = 0, Vt, sel. (iii) Processes V and W with above properties are unique. Proof. cf. [15] Theorem 2.2; or [11]. 1.2.14 Theorem (Moving average representation). Let X = {X,: tel} be a regular SSP, then k H,(t) = H ((t), X, = 11:11:30 c, (Vim! H,); tel where Co = (Ila) IX. - Xe]; 8611 and c1 = EX, :0; Jeri. m Moreover, C = {(,} ~ WN(0,1), j}_l°|cj | 2 = E|Xo| 2 (C is sometimes called the innovation process of X). Proof. cf. [15] p 56; or [11]. 16 The following corollary gives the relation between the decomposition in Theorem 1.2.14 of a SSP X = {X,; tel}, and the Lebesque decomposition of its spectral distribution function F. 1.2.15 Corollary (Wold—Cramer concordance). Let X = {X,; tel} be a non—deterministic SSP with spectral distribution function F,. Let W and V be SSP’s as in Theorem 1.2.13 with spectral distribution functions F, , and F" , respectively. Then 0 (a) W,=j}30 cht-i for tel, where {c,; jell} and C areasin Theorem 1.2.12; i.e. (I) Xt = 1230 Cj Ct-j + Vt; t6”. '0 (b) F" has the spectral density f'(0) = %; |p(é )|2 a.e.[dl], where is 0 no e = 2 c e , ¢( ) 1,0 , F, = F, + F" is the Lebesque decomposition of F,. Proof. Use Theorems 1.2.13, 1.2.14, and [3] p 180 and 183, Theorems 5.7.1 and 5.7.2. Clearly this theorem implies that every regular SSP has a spectral density, determined completely by Wold decomposition. The next theorem explores the fact that for these processes the coefficients in Wold decompositon can be recovered from the density function. 1.2.16 Theorem. (a) The SSP X = {X,; tel} is regular if and only if it has an almost 17 it everywhere positive spectral density f such that I log f( 0)d0 > — m. .1 (b) If X = {X,; tel} is a regular SSiP, then tp(z)= exp{i—f-: Tiogiww} is in H1. Moreover,f(0)= ltp(ti39)|2a..e[d0],e W“) = 2 cjzji [Z] < 1 1'0 where {c,: jell} is the set of coefficients appeared in the moving average representation (Theorem 1.2.14), and (c) co: exp{1-—I_qr logf(0) d0} = o where a is the one step error. Proof. Proofs can be seen respectively in [15] Theorem 5.1 p 64, Theorem 5.2 p 65 and also pp 57 and 58, and relation (5.12) p 66; see also [11]. u The outer function tp appeared in part (b) of Theorem 1.2.16 plays an important role in the prediction theory of regular SSP’s. It has a maximal prOperty in the sense that among all their2 satisfying the boundary condition |¢(é°)| ’ = rm, «(0) is positive, «1(0) 2 |¢(0)|, and that such a to is unique. (c.f. [15] p 60 Theorem 4.2; see also [11]). 1.2.17 Definition. If X = {X,; tel} is a regular SSP with density function f, the outer function 1' '+o o(z)=exp {2157]. e _:+1ogi(l)do} Ii is called the optimal factor of X. 18 The second formulation of w stated in the paragraph preceding Definition 1.2.17 extends to the multivariate case; see [11]. Spectral Representation of SSP’s 1.2.18 Theorem. If X = {X,; tel} is a SSP with EX0 = 0 and spectral distribution function F,, there exists a right continuous orthogonal-increment process {Z,( 0); -r < 0 < r} such that (i) X,= If- todZ ,(0) a.e.[P], tel (n) EIZ(0) — Z(—w)| = Flo); —«s as «r (representation (1) is called the spectral representation of X). Proof. of [3] Theorem 4.8.2 p 140. 1.2.19 Theorem. Let X = {X,; tel} be a regular SSP with Optimal factor to. Then (a) for tel and tell—{0}, 120,7) = If i(é°,t,r) Z,(d0) where , :Zlcne inO «it ri- - W”) [1- ( ) Tl: ré‘(‘+")”to(e°)1./w(e°); ae [do] 6 Here for «é°)= hi3", s,t“ in L’(A), (a) (e0 )— .. ii at“, (cf [11]). (b) If C = {(,} ~ WN(0,1) is the white noise (innovation process) corresponsing to X, then T -iot g, = L,“ Z,(d0) a.e. [P]; tel, _, v(e ) where to is the Optimal factor of X. 19 Proof. of [11] or [15] Theorem 5.3 p 68, and use Theorem 1.2.14 and Theorem 1.2.16 part (c). 1.2.20 Theorem. Let X = {X,; tel} be a SSP with spectral distribution function F. Let a, be the positive measure induced by F on 3([—rr,rr]). Then the linear transformation L: L205.) --4 H, defined by Lthé’» = 1: ht'é") 2.010) is the unique isomorphism between L,(u,) and H, with the prOperty that L(E“') = x,; tel. Proof of. [3] Theorem 4.8.1 p 139. 1.2.21 Notation. If It, has a density f with respect to A, we denote L’(u,,) by L’(f). (see Notation 1.1.4). A geometric way of looking at the problem of autoregressive expansion is the approach "the angle between the past - present and the future", followed by some autors in this area. The following definition and theorem concern this concept to be used in this thesis. For detailed discussion see [7] and [9]. 1.2.22 Definition (Angle between "the past - present" and "the future"). For SSP X with spectral density f, let so = sup{l(x.y)l= er.(0) and yeHil) and uxu s 1. uyu s 1} where H,(l) = Fp'{X,: t 2 1}. We say the "the past and present" and "the future" of X are at positive angle if p(f) < 1. 20 The definition naturally can be extended for any two closed subspaces of a Hilbert space (c.f. [9], p 107, Definition 2.1). The important contribution of "angle approach" to the discussion of autoregressive expansion for linear predictor of SSP’s is the following theorem which is a consequence of a theorem of Helson and SzegO. 1.2.23 Theorem. Let X = {X,; tel} be a SSP with spectral density f, such that log fe L‘(A). (a) Then p(f) < 1, if and only if L’(f) c L‘(A) and the Fourier series of any tpeL’(f) converges to tp in the L’(f) norm; and (b) p(f) < 1 implies l/f e L'(A). Proof For part (a) cf. [9], p 131. Also see [13] p 318 Theorem 2.1. For part (b) of. [9]; p 110 Corollary 2.9. CEAPTER2 AUTOREGRESSIVE EXPANSION In this chapter, first, autoregressive (AR) expansion of the predictor for a stationary stochastic process (SSP) is introduced (c.f. Definition 2.2.1). The existence and uniqueness of this expansion for regular SSP’s, and the invertibility of processes admitting such expansions are the subject of study Of the rest of this chapter. The necessary and sufficient conditions to achieve such a representation are given in terms of the spectral density function f and the optimal factor tp of regular SSP’s; however, all suflicient condition(s) are based on the density function. Section 1 consists of the definition, and a summary of the earlier study of several authors on AR—expansion. Section 2 is devoted to a new approach to the problem, using the Optimal factor as the main tool. The role of the Optimal factor is completely clarified in Section 3. In this section, moreover, the uniqueness of AR—expansion of the linear predictor for a regular—SSP is proved. In Sections 4 and 5 two types of convergence of the AR—expansion are explored. Especially, in Section 5 the strong convergence of the Alt-expansion, based on a condition on the density function and the Optimal factor, is achieved. Consequently in Section 5, also an important extension of earlier results (1958 and 1960) of N. Wiener and P. Masani is derived. The invertibility of regular—SSP’s and the equivalence with the existence (and the uniqueness) of AR—expansion Of one—step predictor is discussed in Section 6. The Chapter is concluded with a brief discussion of sufficiency of the condition 1/feL'(A), in Section 7. 21 22 52.1. Definition of All—expansion, and Historical Notes Let X = {X,; tel} be a Stationary Stochastic Process (SSP). The r—step ahead linear predictor X(t,r), as defined earlier (Definition 1.2.7) and by projection theorem (Theorem 1.1.5), is X(t,r) := Proj(X,+T|H,(t)); tel, Tell — {0}. 2.1.1 Definition. Let X = {X,; tel} be an SSP. Let rel - {0}. Ifthere exists a sequence {b,( 1)} of complex numbers such that A 0 (2.1.1) X(t,T) = kilo bk(T) X,.k (in H1), we say that r—step predictors have autoregressive expansions with the coefficients {bk(7)}- Using the family of shift Operators of X (c.f. Theorem 1.2.8), it is necessary and sufficient to have (2.1.1) holds for t = 0; i.e. a 0 (2.1.2) X(0,r) = kEo b,(r) X_,,; Tell. This is because (2.1.1) implies (2.1.2), and if (2.1.2) holds, then 5cm) = Unique» = U. (,§,b1(r) xi) = ,5, bite) me.) = 1:230 bk”) xt-k' In this chapter we are mostly concerned with the special case of t = 0, r = 1; i.e. X, = X(0,1) (c.f. Notation 1.2.10), the 1—step ahead linear predictor of X, based on H,(O). With a minor change of notation, taking b,(1) = b, for kel, (2.1.1) in this case is restated as follows: (2.1.3) X1 = kzo bk X, 23 An important problem in time series analysis and prediction theory of SSP’s is to find condition(s), preferably on the spectral density frmction of the regular part of the process (c.f. Definitions 1.2.5 and 1.2.11 also Theorems 1.2.12 and 1.2.16) required to achieve the existence and the mean-squared convergence Of (2.1.1) (or its equivalent form (2.1.3)), and its generalization to the multivariate case. This problem has been the subject of study of several authors. Among all, the following works, demonstrating the diversity of approaches to the problem, are notable. N. Wiener and P. Masani, in their paper [17; 1958] proved that: " the boundedness ofthe density fitnction f; i.eO < c 5 f 5 d < an, is sufficient for the existence of an Alt—expansion for X,". Later P. Masani [10; 1960] weakened these conditions to: "(1 /f) eL‘(A) and feL°(A)". Since then, several attempts have been made to relax or weaken the very restrictive condition "feL"’(A)". A.G. Miamee and H. Salehi [12; 1983] gave the following necessary and sufficient condition to maintain an AR—expansion as in (2.1.3) with the condition rzolbkl 3 < on for X,: (i) (1/f)eL‘(A), (ii) The convergence of the Fourier series of (1/ p) the reciprocal of the optimal factor (c.f. Definition 1.2.17) to (1/ p) in the space L’( f). The second condition on their set of conditions, although weaker than Masani’s "feL'"(A)", is not easily verifiable. In addition their condition k$0]le 2 < an restrict the applicability of their work. M. Pourahmadi [13; 1984], with an eye on the famous theorem of Helson and Szegii on the positively of the angle between past — present and fitture of a stationary process (c.f. Theorem 1.2.23), examined the problem and proved: "the positivity of the angle between the past—present, and the future of the process is sufficient to guarantee the existence of an Alt—expansion." In the forementioned works the basic imposition (1 /f) e L‘(A) emerges as a common restriction on the spectral density f. This requirement translates into the squared summability of the coefficients in the resulting AR—expansion. In the 24 following sections, by a fresh look at the problem, this restrictive condition is removed, and especially the results of Wiener-Masani, Masani, Miamee—Salehi and Pourahmadi will be derived as a consequence of an essential theorem (see Corollaries 2.5.2 through 2.5.5). The stepping stone of the entire discussion is the analysis of the Optimal factor tp of f, discussed below. 52.2. The Role ofthe Optimal Factor Let X = {X,; tel} be a regular SSP with density function f and Optimal factor «p: , i9 F(Z)=exp{i];f inf—510mm} a; e — z =j§oejzi; |z| < 1, i(a)= Win”; -«s vita-em (c.f. Theorem 1.2.16 and Definition 1.2.17). For the sake of clarity, the assumptions on f that have been used so far are stated here: (1) feL‘(A) and (2) log feL‘(A). The l—step predictor X, has the spectral representation (2.2.1) 5:. =1: é‘°[1-c./e(é°)1 we). where C} = o2 = EIX,—X,|’ > 0 (c.f. Theorem 1.2.19). Since to is in H2 and has no zero in D(0,1), Up is analytic (holomorphic) in D(0,1). Let (2.2.2) 1M2) = ago a, an; |z| < 1 be the Taylor expansion of 1/tp in D(0,1). Define (2.2.3) {i(a) = a" [1 -e,/o(a)]; |z| < 1. Clearly y is analytic in D(0,l) since codo = l and consequently the constant term of the Taylor series of the function 1 - co/p equals 0. Let 25 A D W)=,3°8nzns IZI < 1 be its Taylor expansion. The relations between the coefficients an, (1,, and en (which are the coeficients in the moving average representation, of Theorem 1.2.14) sheds light on the entire discussion. In fact, since i(z) = n35 a. z" = [1 — come] in _ -1 _ n - z [1 co DIED dnz ] O (codo = 1) = "' n2,(°odn) ”I” m = ”20 (--c,,d,,,,)zn ; [z] < 1, we have (224) 8n = — c0 dnrl ; DE“, and by multiplying both sides of (2.2.3) by «1(2), we get rive) - col = (113,308.. x") (.2 can); Izl < 1. This is equivalent to D k U k k 3,9! '1: ,3, {,2}, (Ck-n'annz ; M < 1- 0 Notice that, since “20 ens“ is absolutely convergent in D(0,1), the so called Cauchy product theorem is applicable. The last equality implies k (2.2.5) ch, = nil, chm-an; ketl. Equations in (2.2.5) form a triangular system of equations in unknowns {a,; nell}, which has a unique solution. The unique solution, by a little efiort is found as follows: n-i 30 = c1/‘30 3 3n = 1/"o [on-t - ,3, an cit-kl; As a result, using (2.2.4) (2.2.6) 2(a) = “21(— c,d,,,,) an 26 where c,, = 00 > 0 and (1,, ml, are the coefficients of Taylor expansion of 1/ go. The relation (2.2.5) between a,’s and c,’s and expansion (2.2.6) will play important roles in the analysis of AR—expansion of X, in subsequent sections. It should be noted that the Taylor expansions of tp and 1/ tp, and the relations between the coeficients cn’s and dn’s in multivariate case are discussed by P. Masani in his 1960 and 1966 paper (c.f. [10] p 146, and [11] p 375). §2.3. Uniqueness of Autoregressive Expansion In this section the uniqueness of AR—expansion for regular SSP’s, which admit an AR—expansion, is proved. 2.3.1 Theorem (uniqueness). Let X = {X,; tel} be a regular SSP with the optimal factor (p. Let 1/tp(z)= “20 d, an for [z | < 1, and co: 0. Iffor some sequence {bn}. (2.3.1) 12,: )3 b, x," then bn = --c0 d,,, for nefll. Proof. By the the moving average representation (c.f. Theorem 1.2.14) for tel D X, = jgo C) (,1, {Cj} ~ WN(0,1), co = 00. Therefore, since H,(O) = H C(0) and (,1 H ((0), using Theorem 1.1.7 (1) we have a or X, = PIOKX,[H,(0)) = j§0 C,“ (2,, and m x —ni3°b,, x = 1: c,,,(_,,— “fob: 30c, c.“ 00 (2.3.2) = 112°C 0,", (m- 530 ngobn cs C-n-s mo =r2°cy+1(_rr-§[ikbncr-n]C-r rsOnao 27 (inthesecondsum,take n+s=r, then 05r 0 such that app HR," = M < 111. Let heH,. For every kell, define h, = Proj(h|H,(—k—1)). Since 2 H,(—k—1) = {0}, by 28 regularity of X, we have hk a 0 (in H!) as k -+ m. (c.f. Theorem l.l.7(iv)). On the other hand for each kell, Rkth(-k—1) (using Relations (2.3.2) and (2.2.5)). Therefore, (h—thk) = 0 for every hell. Hence Imam = law.» s uh." - Ilell 5 M uh." -+ o as 1.. o, k . which means R], L 0 (in Ex) (c.f. Definition 1.1.8). Conversely if “EOanX.1| L X,, k then {ngoanxfl} is bounded (c.f. Theorem 1.1.9 (ii)). a As it was stated earlier, the goal is to find necessary and suficient condition(s), preferably in term of density functions, to achieve the strong convergence of the series in All-expansion to X,. Theorem 2.4.1 points to the type of conditions to be considered. The following corollary, in part, is the frequency domain analogue of the theorem above. 2.4.2 Corollary. Using the same notation as in Theorem 2.4.1. let 0 rs 1' 2 s,(é)= E (1,3: for nel. Let a=sup§— j |sn| fd0. Then R80 11 I —f (a) Sn L 1/¢ (in L’(f)) if and only if a < co . (b) If 5}. w and a< o,then 5,... W (in mm). Proof (a) Using the isometry L between H! and L’(f) (c.f. Theorem i- . . 1.2.20), since Sn(e ) =éodk¢iak = L-l(k§od"x-k) for MN, and L"(X1) = -1. e [l-co/cp] (Theorem 1.2.19), by Theorem 2.4.1 )3 ill."'.i.1 ifdnlif )3 “'2 'boded hoake —-+e[-co/:p] an o y llhoake "1.19)“ un , However, for nel and 0e[-1r,1r], 1% O + ' (2.4.1) Snflé ) = :g: dk e = (10 _ l/Co é n 1% 0 1303* e (do = I/co) = we. {1 - é“ - 3‘3, a. 3°}. 29 so that . -t s,“ L W (in no) if and only if 3:50 nk 3“ L e [1 -— e0/e] in L’(f)). The result follows, by the fact that Ilsnll’,2(,, =%; I: Isnl’fdo. (b) Define MA) = (1/2rllflli)) { f d0. Add-ml)- Clearly it is a probability measure on 3([—1r,11), and Lp(u) = Lp(f) for o < p g n. Since a 5 Mo < n, by part (a) s, L l/tp (in L’(p)). This implies that lim I Sn dl‘ = I, lie) dl‘ E6 30-16m- n-Hn E Using this and the fact that IISnHLKn) S IsnllL2(p) S 05 Mo < “’1 by the moment inequality, we get Sn L 1/ lp (in L‘(p)) (c.f. [5], p 291 Theorem 7). On the other hand Sn IL. 1/tp because of Sn L Up and u << A. Hence Sn -o Up in L‘(p) = L‘(f). [c.f. [5], p 295, Theorem 12]. n 52.5. Necessary and Sufficient Conditions for the Existence and the Uniqueness of the All—expansion Theorem 2.5.1 below provides necessary and suficient conditions for the existence and uniqueness of All—expansion for X1. Of importance in this theorem is condition (c). This condition is slightly stronger than the condition of " boundedness of partial sums" of the series in this expansion which guarantees the weak convergence of this series. 30 2.5.1 Theorem. Let X = {X,; tel} be a regular SSP with spectral density f and Optimal factor tp. Let l/to(z) = 1:230 dnzn for |z| < 1. Set an = — c0 dim: for nell, where co = a. Let Sn(z) = llillodkz“ for set. Then the following statements are equivalent: (a) a 0 (2.5.1) x1 = “go aux-n (in Hz): k A X 11.nX_n -t X1(in HI) as k 4 m; i.e. and {an} is the unique sequence for which (2.5.1) holds. (b) The sequence {lp-Sn} converges in H’ to 1. (Equivalently {dé'finén converges in L’(A) to 1). (c) mtg: Is.(é°)l’r(o)dost lP‘m Proof Define it?) = 3% — OM35]; eel—m1. The following relations will be used in the course of proof: 0 ' - m - . (2.5.2) 130 cnfleno = ei0 a?! an?" = ei0 e(é°) [1 — eo/¢é°)] =¢(é°){l(é°), ”sage and 3 mo k inO 'iO k i(n+1)9 (2.5.3) n{Jeane = - co “23° dune = — co e 2 (in,1 e - k 1 - k 1 - =—co ei0 )5 dnt1=.na=—co ei9 [—1/c0 + id], 3"] us! n80 .. 0 '0 = ei [1 — cos,.,(é )1; 044,11, kell. (a H b) using the isomorphism L between L’(f) and 151K (Theorem 1.2.20). We have R - k _ a. A 2 tine“ = L ‘(ngo anxfl) —-» L 1(19) = 1(3); as k .. m, 31 if and only if T k a J )1: nné“°—¢(é°)|’r(o)do—.o as he... --1 n=0 if and only if T - O k (2.5.4) I (Jinx! cn é“) - 43530 the“) ’ d0 —. o as k e e. x . . Now, since (peL’(A) and tpSneL’Q) for nell, (2.5.4) holds if and only if (2.5.5) (I Ea cm, é” - ode) ”in anew", .. o; as k .. .1. Using relations (2.5.2) and (2.5.3), (2.5.5) is true if and only if v - é‘ll — 005m] . 6." «ll - Co/SO] (in 19(1)) or, equivalently, if and only if ell-cosine wll-co/sp] cum» and, since L’(A) is a Banach space, if and only if «asks-t 1 (inL’(A))- The result follow from the fact that for every nell, tp SneH2 and that the H2—norm of est. - 1 equals Ilw s. - lllr (b H c). We first prove that 1' (2.5.6) (0 sn .. 1 (in L’(A)) if and only if g j | sn| 2fdo -» 1 as n -+ n. 4 To this end, note that «p SneL’(A) for each nell. We have 2 T 10 i9 2 ||1“‘Psn||2=%;£x|1—%3)3n(e)| do = 1 — med; 1' we") s.(é°)do} 1 1r io 10 2 +5! |¢(e)3n(e)l <10. nelJEl-ml- 4 However, 1’ $7 I o(é°)sn(é°)do=c,o, =1, nelll. .1 Since tpSneL’(A), and 32 1 i0 2 1 i0 2 figldeo )Sn(e )| d0=fi£w| Sn(e)| f(0)d0, nelll, itfollowsthat lo 2 ||1-¢Sn||n=1-2+g,-,I ISn(e)| f(0)d0 116"» which proves (2.5.6). As a result of this 1 7 2 I |S| fd021 21? 4 n which implies 1—"1‘12—1 |s,’| fd0>1. Therefore, by (2.5.6) and the fact the li_m < Ii— for any sequence {a }, a11-311 n es -11(inL’(,\))ifandon1yif Bin 2'1 (s, (e°)|21(o) d0<1. 11" III The uniqueness of AR—expansion follows from Theorem 2.3. 1. n The following example is due to Topsoe [see [12], p 92], to demonstrate that a process with spectral density f(0) = |1+é°| 2, - 1r 3 05 1r, does not admit an AR—expansion for X1. Our Theorem 2.5.1 confirms his conclusion. Example Let X = {X,; tel} be any SSP with spectral density f(0) = II + 3’) 2, — x5 0 5 1r. Note that f is continuous, log f e L10), its Optimal factoris tp(z) = l + z for |z| 51, and 1M2) = :15, (-1)"z“; lzl < 1. Therefore (1n = (--1)n and Sn(z)¢(z) = 1 + (-1)nz“t‘, for every nell. Hence tel” Ism’ do= 2, _, which using 2.5.1(c), shows AR—expansion does not exist for X1. 33 Theorem 2.5.1 subsumes the results of Wiener—Masani, and that of Miamee—Salehi. Note that the latter authors assume the conditions (1/f)eL1(A) A O and S,I -1 1/ (p (in L’(f)) to achieve X, = “20 a,,X.n for some {an} with D n80 | anl 2 < o and conversely. As we’ll see later (see example following Corollary 2.6) the condition (1 /f) eLKA) by no means is necessary to have the AR—expansion of 12,. 2.5.2. Corollary (Masani). If (1 /f)eL‘(A) and feL°°(A), then for some sequence {an} A D x, = D230 2, x_n (in Hz). Proof. Let Ess. sup f = a. By assumption a < tn. Since (1/f)eL‘(A), we have (1/ tp)eL’(A) and "SI, — 1/(p||, -o 0. Therefore, as n -1 m 7 2 7 2 2 7 2 I IsoSn-ll dA=J |s,—1/o| -|tp|dA$a] |S,,—1/a] [f>a] forevery a> 0. 35 Proof. Let X, = $0 bn X.,, for some sequence {bn}. By Theorem 2.3.1, b,I = - co d,,, for nell, and by part (b) of Theorem 2.5.1, Sn -1 1/(p in L’(f) which implies |S,,|2 -o l/f in L‘(f). As a result {|Snla} is uniformly integrable w.r.t. the probability measure defined by #(A) = (1/21r)llf||1 { f d0; A6 3(l-1rml). So,forevery a>0,as C-HD wehave 3 2 Ot—supf |s,| tdazsupJ‘ |S,,| fx , do n [I 811' 2>C] n [I Snl >C]n[f> a] 2 Zaosup |S| x d0 .. f " us.l’>c1nlf>a1 2 w 33p, Is“xlf>a1' xlls.l’>c1d0 3 2”" 83"! 'S”"lr>al' "us.x,,>,]|’>cl As a result, for each a e (0,+tn), { | Snx f Ila} is uniformly integrable with >0 respect to A. Since Sn L) 1/ (p (because Sntp-A—t 1 by (b) in Theorem 2.5.1), we d0. get S""lr>a1—' w xM (in 11.11)). a In view of condition (2.5.7) of this corollary, it is tempting to conjecture that the condition "Sn -1 l/lp (in L’(A))" would hold under the assumption of existence of the Alt—expansion of X, Had this been true, one would get (1/f) eL‘(A). However, as it is shown (see also [14] p 317), the example f(0) = |1+tia9| 2). for any 1/2 5 A < 1 demonstrates that the condition "(1 /f)L‘e(A)" is not necessary for the existence of AR—expansion of X, Example. Consider SSP X = {X,; tel} with spectral density 19 .. f(0)=|1+e Ia,—1r$0$1r,and 1/2$A<1. X1 hasanAR—expansion(c.f. 36 x [14], p 317). However (l/f) t L‘(A). Clearly (p(z) = (1+z) for [z] _<_ 1, and using Taylor expansion, we have m (A) _ k _ k k _ ._ l/(p(z) .. h’EoTT'( 1) z ,where ()1)k ._ A(A+1)...(A+k 1). Therefore by Theorem 2.5.1 we have it. = kins-1)“ (no/(21)] x, (in H.) u The corollary below, whose proof is deduced from Theorem 2.5. 1, puts the conditions on the density fimction f. 2.5.7 Corollary. With the same notation as in Theorem 2.5.1, let l/f e LP(A) for some p,15p