\1 {ill -I* I [ll

ABSTRACT
ASYMPTOTIC DISTRIBUTIONS IN SOME NON-REGULAR
STATISTICAL PROBLEMS

by B. L. S. Prakasa Rao

As the title indicates, we consider here two different
prOblems. The first problem deals with estimation of
distributions with unimodal density and estimation of
distributions with monotone failure rate. The second
problem deals with the estimation of the location of the
cusp of a continuous density.

Recently Marshall and Proschan (Ann. Math. Statist.
lgé' 69-77) have derived the maximum likelihood estimates
for distributions with monotone failure rate and they have
shown that these estimators are consistent. In Chapter 2,
we obtain the asymptotic distribution of these estimators
using the results of Chernoff in his paper on the estimation
of mode. The estimation problem is reduced at first to that
of a stochastic process and the asymptotic distribution is
obtained by means of theorems on convergence of distributions
of stochastic processes. Similar results are obtained for
distributions with unimodal densities in Chapter 1.

Under the usual regularity conditions on the density,
it is well known that the maximum likelihood estimator is
consistent, asymptotically normal, and asymptotically ef-
ficient. Unfortunately, these conditions are not satisfied

for distributions like double-exponential with location

B. L. S. Prakasa Rao

parameter 9. Daniels, in his paper in the fourth Berkeley
Symposium, has shown that there exist modified maximum
likelihood estimators which are asymptotically efficient
for the family of densities f(x,9kCEXp[—|x - elk}, where

x and 9 range over (-oo,oo) and.%-< k < 1. In Chapter 3,
we show that hyper-efficient estimators exist for 6 when

0 < k <-% and 9 is restricted to a finite interval for a
wider class of densities. We relate its asymptotic
distribution to the distribution of the position of the
maximum for a non-stationary Gaussian process. The estima-
tion problem is reduced to that of a stochastic process and
the asymptotic distribution is obtained by using theorems
on convergence of distributions of stochastic processes in

C[O,1].

1'—

 

ll‘llrl! fit “!l.lxl..‘» . I... ’zf'l‘ll-l

ASYMPTOTIC DISTRIBUTIONS IN SOME NON-REGULAR

STATISTICAL PROBLEMS

BY

Bhagavatula Lakshmi Surya Prakasa Rao

A THESIS

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Statistics and Probability

1966

To my grand-parents and parents

ACKNOWLEDGMENTS

It is with pleasure that I express my sincere grati-
tude to my advisor Professor Herman Rubin for suggesting
the problems treated in the thesis and for his stimulating
advice and guidance throughout the entire work. The time
and effort spent by him in developing the thesis are
greatly appreciated.

I would like to thank the Department of Statistics
and Probability, Michigan State University, the Office of
Naval Research, and the National Science Foundation for
their financial support during my studies in the writing

of this thesis.

iii

TABLE OF CONTENTS

CHAPTER PAGE
0. INTRODUCTION . . . . . . . . . . . . . . . . . l
1. ESTIMATION OF A UNIMODAL DENSITY . . . . . . . 3

1.1 Introduction . . . . . . . . . . . . . .3
1.2. Maximum likelihood estimation of the
density . . . . . . . . . . . . . . . . 4
1.3 Consistency of the maximum likelihood
estimator . . . . . . . . . . . . 6
1.4- Some results related to the asymptotic
.. properties of the maximum likelihood
estimator . . . . . . . . . . . . . . . 8
1.5 Reduction to a problem in stochastic
processes .. . . . . . . . . . . . 18
1.6 Asymptotic distribution of the maximum
likelihood estimator . . . . . . . . . . 23

2. ESTIMATION FOR DISTRIBUTIONS WITH MONOTONE
FAILURE RATE 0 O C O O O O O O O 0 O O O O O 3 1

2.1 Introduction . . . . . . . . . . . . . . 31
2.2 Definition.and properties of distribu-
tions with monotone failure rate . . . . 32
2.3 Maximum likelihood estimation for
.. increasing failure rate distributions . 33
2.4 Some results related to the asymptotic
properties of the maximum likelihood
estimator . . . . . . . . . . . . . . . 34
2.5 Reduction to a problem in stochastic
processes .. . . . . . . . . . . . . . 44

2.6 Asymptotic distribution of the maximum
likelihood estimator for increasing
failure rate distributions . . . . . . . 56
2.7 Asymptotic distribution of the maximum
likelihood estimator for decreasing
failure rate distributions . . . . . . . 57

3. ESTIMATION OF THE LOCATION OF THE CUSP OF A
CONTINUOUS DENSITY . . . . . . . . . . . . . 60

3.1 Introduction . . . . . . . . . . . . . . 60
3.2 Some results related to the asymptotic
properties of the maximum likelihood

estimator . . . . . . . . . . . . . . . 61
3.3 Reduction to a problem in stochastic

processes . . . . . . . . . . . . . . 86
3.4 Asymptotic distribution of the maximum

likelihood estimator . . . . . . . . . . 96
3.5 Evaluation of integrals . . . . . . . . 97

BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . 110
iv

CHAPTER 0

INTRODUCTION

As the title of the thesis indicates appropriately, we
consider here two different problems. The first problem
deals with estimation of distributions with unimodal density
and estimation of distributions with monotone failure rate.
The second problem deals with the estimation of the location
of the cusp of a continuous density.

Grenander [10] derived the maximum likelihood estimators
for distributions with unimodal density and for distributions
with monotone failure rate. He did not derive the asymptotic
distributions for these estimators. It is interesting to note
that the maximum likelihood estimators can also be derived
by methods used in Brunk [2] or in vanlkflen [19]° Recently
Marshall and Proschan [14] showed that the maximum likeli-
hood estimator is consistent. In Chapters 1 and 2, we derive
the asymptotic distributions of the estimators in both cases.
Even though we do not obtain their distributions explicitly,
we show that they are related to a solution of the heat
equation as was done in Chernoff [4] in the case of the
estimation of the mode.

Under the usual regularity conditions on the density,
it is well known that maximum likelihood estimator is con-
sistent and asymptotically normal. See Cramer [6], Kulldorf
[12], Gurland [11], etc. Their estimators are also asymptotic—
ally efficient. In certain cases like double exponential

1

2
distribution with location parameter 6, these regularity
conditions are not satisfied. Daniels [7] has shown that
there exist modified maximum likelihood estimators (M.L.E.)

which are asymptotically efficient for the family of densities

f(x,e) = ——-1-—1— exp {-|x—e|k}, -%-< k < 1,
2N1?)
- a) < x < oo/ - a) < e < a).

Recently, Huber has generalized Daniels' results in
his paper presented at the fifth Berkeley Symposium and he
has shown. that M.L.E. is consistent and asymptotically
normal for cusps of order between é-and 1. We show in
Chapter 3 that hyper-efficient estimators exist,when the
exponent k lies between 0 and-% and e is in a finite inter-
val,for a wider class of densities. We relate its asymptotic
distribution to the distribution of the position of the
maximum for a nonwstationary Gaussian process. In fact, it
can be shown that Bayes estimators for smooth prior densities
for 9 are also hyper-efficient for the above class of den-
sities and asymptotically the estimation problem is equiva—
lent to estimation of location parameter for a non-stationary
Gaussian process. It should be mentioned here that some of
the Bayes estimators are asymptotically better than the

M.L.E.

CHAPTER 1

ESTIMATION OF A UNIMODAL DENSITY

1.1 Introduction:

 

Given a set of observations X1,..., Xn from a common
distribution F, it is natural to estimate F by the usual
empirical distribution function in the absence of additional
information. However, one would not use such an estimate
if there is some a priori information about the distribu-
tion F. In this chapter, we shall investigate the problem
of estimation when F is known to be unimodal. Grenander [10]
derived the maximum likelihood estimator for f, where f is
the density of F. Even though it is well known that the
maximum likelihood estimator (M.L.E.) of f is consistent,
we shall give a proof for completeness. We shall relate its
asymptotic distribution to a solution of a heat equation as
was done by Chernoff [4] in the case of the estimation of
the psuedo—mode.

Section 1.2 deals with the maximum likelihood estimation
of the density. The consistency of the M.L.E. is proved in
Section 1.3. Some results related to the asymptotic prop-
erties of the M.L.E. are Obtained in Section 1.4. In Sec-
tion 1.5, the estimation problem is reduced to that of a
stochastic process. We Obtain the asymptotic distribution

of the M.L.E. in Section 1.6.

4

1.2 Maximum likelihood estimation of the density:

We shall assume that the distribution F(x) is absolutely
continuous with density f which is unimodal with known mode

u. If u is the mode and F = aF

+ + (1 - a)F_ where F+ is

the conditional distribution on [u,oo) and F_ is the condi-
tional distribution on (~00, u), then it can be shown that
the M.L.E. of F is dF+ + (1 - a)F_ where a is the sample
proportion on [uqoo) and F+ and F_ are the M.L.E.'s of the
conditional distributions F+ and F_ reSpectively. Let f+
and f_ denote the densities of F+ and F_ respectively and
let f+ and f; denote their M.L.E.'s. We shall show later on
that for any 5 Z.u, [f+(§) - f+(§)] a Op(n-1/3) and for any
: < u, [E_(;) - f_<;>1 = op<n‘1/3>. Since a - . = op<n'1/2>
we get that for anyg , f(§) - f(§) = Op(n-1/3). Therefore
it is sufficient to obtain the M.L.E. of f+. flét us assume
that u = 0 without loss of generality. Therefore F(x) ='0
for x < 0. Since F is unimodal, f is non-increasing for x :.0.
Suppose X1 i-- o - E'Xn are n observations obtained by
ordering a random sample of size n from the pOpulation with
unknown distribution F. Let B-denote the class of unimodal

distributions F. Let X0 = 0. Let
n
L(F) = 2 log f(Xi) (1.2.1)

be the logarithm of the likelihood for J?€E}. For any F GE},

define F* to be the distribution with density

5

O for x.i O
* _ < - <
f (x) — Cf(Xi) for Xi_1 < x j_Xi, 1 _.1 _.n
0 for x > Xn (1.2.2).'

where C is a normalizing constant. It is easy to see that
C.: 1 and L(F*) = CnL(F). Therefore, L(F).i L(F*). In other
words, the M.L.E. fh(x) of f(x) will be a step function with
steps at the order statistics X1,X2,...,Xn.
Hence the problem of maximizing L(F) for F 6:? reduces
to the problem of determining numbers f1,f2,...,fn such that
(i)f1_>_f2:--o_>_fn,

(1].) lel + (X2 " X1)f2 + 000 + (Xn " Xn_1)fn = 1, and

n

(iii) Wfi is maximal. (1.2.3.)
1

This has been done in Grenander [10]. It can also be

done as an application of results obtained by Brunk [2] or

van Eeden [19].

This yields for the M.L.E. of f(x),

 

< -< - < _
f (x) = {fn(xi+1) for Xi < X -Xi+1' O — 1 ‘n 1
n 0 for x j.X0 or x > X (1.2.4)
n
where
1 _ Max Min v - u . 1.2.5
fn(xi) — n Z.v 2.1 O fi.u i.i-1 n(XV - Xu) ‘ )

The estimator fn(x) can also be written in the form

..

F (V) -F (11)
{~ SUP Inf n n for X0 < x :.x
c v > x u < x v - u n

0 otherwise.

 

(1.2.6)

6
In other words, the M.L.E. fh(x) is the slope of the

concave majorant of empirical distribution Fn at .

1.3 Consistency of the maximum likelihood estimator:

Theorem 1.3.1.

For every x

 

 

fn(x) -> f(x) inyprobability as n —> oo.

Egggfig If x < 0, then
fn(x) = O for all n and f(x) = 0.
Therefore fn(x) -> f(x) in probability.
Let x 2.0. Let Fn(x) denote the smallest concave

majorant of Fn(x). For any a > 0,

P[JE' Sup IFn(x) - F(x)| < g]-—> £(e) as n —> oo
‘00 ”“00 (1.3.1)
where
00 . .
2(a) = 2 (-1)3e‘28232
j=-oo

by the Kolmogorov-Smirnov theorem.
Let us choose 5 > 0. Then there exists an integer

N0(é) such that for every n > N0(é),
P[Fn(X) < F(x) + sn-l/2 and Fn(x) > F(x) _ en‘l/z

for all x] > 2(a) - g- by (1.3.1).

Since Fn(x) Z.Fn(x) for all x by the definition of Fn,
Fn(X) > F(X) - sn-l/2 for all x
=> §n(X) > F(X) - sn-l/2 for all x. (1.3.2)

Since Fn(x) is the smallest concave majorant of Fn(x) and

F(x) is concave,

Fn(x) < F(x) + en-l/2 for all x

:9 §h(x) < F(x) + gn-l/2 for all x. (1.3.3)

From (1.3.1) - (1.3.3) it follows that for n > No(é),
P[Fn(x) §.F(x) + en—l/2 for all x and
Fn(x) > F(x) - en-l/2 for all x] 5 2(8) - %-.
Therefore, for all n > NO(5),

Fn(x + n-1/4) - Fn(x) < F(x + n-1/4) - F(x)

 

 

 

 

‘1/4
P[ n‘171 _' n‘1/1 + Zen
and
E (x - n‘1/4) — E (x) F(x — n‘1/4) — F(x) _1/
n _1/ n > 17 - Zen 4 for all x]
—n 4 -n- 4
> 2(8) -3-. (1.3.4)

Let C > 0. Since f(x) exists for all x, ther exists an

integer N1(C) such that for every n > N1(C).

 

 

 

 

Fix + n-1/4) - F(x) <
11-1/4 - f(X) CI
and
Fix r n-1/1l - F(x) _ f(x) < C0 (1.3.5)
_n'174

 

 

Now (1.3.4), (1.3.5) together imply that for every
n > Max<No<a>.N.<:)z

. _1/ 1
F (x + n 4) - F (x) _

P[ n _17Z n < f(X) + C + Zen 1/4

 

and

8

F (x ‘ n-1/4) - Fn(x)

n -1/ 6
-n_1/4 > f(x) - C - Zen 4] > £(p)—§ .

Now, since fn(x) is the leftehand derivative of Fn(x) at x,

it follows that for every n > Max (NO(5). N1(5)).
Pan(x)‘< f(x) + C + 25n‘1/4 and fn(x) > f(x) - C ~2€n_1/4] >
£(e) -'g -

In other words

Ptfn<x> - f(x>|< C + zen‘1/41 > 1(a) - 5 (1.3.6)

'5
for every n > Max (N0(5),N1(C)). We choose 8 such that
2(5) > 1 --% and n such that

- 4
2sn 1/4 < C or equivalently n > (2%) = N2(§,o).

Hence, from (1.3.6), it follows that for every

n > M3X(NO(O),N1(C) IN2 (C05))I
p[|£“n(x) - f(x)] < 2c1> 1 - 5.

Therefore,

fn(x) -> f(x) in probability as n -> 00.

1.4 Some results related to the asymptotic_properties of

the-maximum likelihood estimator;

Before we proceed to obtain the asymptotic distribution
of M.L.E. fn(x), we shall prove some lemmas which simplify
the problem. We shall assume that f is differentiable at
the point x and that f‘(x) is different from zero.

Let f;’c(x) denote the lepe of the concave majorant
of Fn restricted to the interval [x - 2cn—1/3, x + 2cn_1/3],

evaluated at x. We shall now prove that

Lemma 1.4.1.

There is a function ¢.such that

 

(i) 1%? mgcm ¢ En(x)1 5. (MC)
Elli

(ii) ¢(c)‘—> O as c —> 00.

Proof: It is enough to prove that

 

lim p[¢n(y) 1.Fn(x + Cn-l/a) _ (x + cn‘1/3 _ y)(f(x)-An‘1/3)
n .

for all y,: x and all y 2.x + 2cn-1/3:
x - cn-1/3)+(y - x + cn 1/3)()-+-.?31n 1/3)

<

for all y 2.x and all y._ x - 2cn-1/3]f

3. 2p(c,A) (1.4.1)
where ¢(c,A) -> 1 as C‘_> 00 and A = -df'(x).
We shall show that for A = -cf'(x),

lim lim P[Fn(y):-Fn(x+cn-1/3)— XHCD-1/3-y)(f(x)-An-1/3)
C-‘>CID n

for all y i.x and all y 2.x + 2cn-1/3] = 1.(1.4.2)
In a similar way, it can be shown that

111“ ii!!! PIFn(Y) iFn(X-cn1/3 )+(y—x+cn-1/3)(f(x)+An-1/3)
c—>oo n .

for all y 2.x and all y 5-x - 2cn-1/3] = 1. (1.4.3)
(1.4.2) and (1.4.3) together imply (1.4.1) which in turn
proves the lemma.
We note that A > 0 since f'(x) < 0. Let us obtain a
lower bound for
p a PtFn<y>srn<x+cn'1/3>—<x+cn'1/3-y>(f(x)-An'l/a>

for all y i x] . (1.4.5)

10

LGt
In<x> = n[Fn(x + cn‘1/3>- .n-1/3(f<.> - An‘1/3> - Fn<x>1
= n[F(x + cn-1/3) - F(x) - cn-1/3(f(x) - An_1/3)]
+ n[{Fn(x + cn-1/3)—Fn(x)) - {F(x + cn‘1/3)-F(x)}]
= n[F(x) + cn-1/3f(x) + %c2n-2/3f'(x) + o(n—2/3)
— F(x) - cn-1/3f(x) + cAn-2/3]
+ n[{Fn(x + cn-l/S) - Fn<x>3 - {F(x-+ cn‘1/39-Féx)ll
= n1/3[%c2f'(x) + cA + 0(1)]
+ n[{Fn(x + cn'1/3)-Fn(x)} - {F(x + cn’1/3) - F(X)}]
= n1/3Bn + c1/2n1/3[f(x)]1/2Vn (1.4.6)
where
. _12. _ c2.
(1) Bn — 2C f (x) + cA + 0(1) - --§—f (x) + 0(1)
and
(ii) Vn = c-1/2[f(x)]-1/2n2/3[{Fn(x + cn-1/3) - Fn(x)}
- {F(x + cn-1/3) - F(x)}]. (1.4.7)
Obviously,
E(Vn) = 0.
and
Var(V ) = n[F(x + cn‘l/a) - F(x)111 + 6(n‘1/311

 

cn2/;f(x)

n1/3

EETET [cn-1/3f(x) + 0(n_1/3)][1 + o(n-1/3)]

1 + 0(1).

11

Therefore from (1.4.6) it follows that

In(x) = n1/3Bn + c1/2n1/3[f(x)]1/2Vn

where

E(Vn) = O and Var(Vn) = 1 + 0(1). (1.4.8)

Let 0 < A < 1. By Chebyshev's inequality

2 2
P[In(x) > -N%rf'(x)n1/3]=P[Bn + c1/2[f(x)]1/2Vn > - A%rf'(x)]

2 1

 

=p[vn > {(1-1)32—9 (x)+(o(1)}cl A2[f(x)]1/L2]

 

 

 

 

 

’ 1 [1 + 3(1)]Cf(x) . (1.4.9)
[(1 - MEZ—f'bc) + 0(1)]2
As n -> oo,
[1 + o(1)]cf(x) _5 cf(x)
CZ 2 c2 2
[(1 - wag—F(x) + 0(1)] [(1 — Mtg-F(x)]
= 4f(x)
(1 - x>2c31f-<x>12
Let
Q(C) _ 4f(X)

(1 - x)2c3[f-(x)12
From (1.4.9), it follows that there exists an integer N1 such

that for every n > N1,

c2 1/ 3
P[In(x) > - Airf'(x)n 3] 2.1 - §Q(c). (1.4.10)

From (1.4.5), we have

p = P[Fn(x + cn-1/3)-(x + cn-1/3- y)(f(x) - An-1/3)- Fn(y) 2.0

for all y 2.x]

12

= P[{Fn(x+cn-1/3)-F (y)} - (x + cn-1/3- y)(f(x)-An-1/3) 3.0

n
<

for all y _.x]
= P[n[Fn(x+cn-1/3)- Fn(X)} - cn2/3(f(X) - An_1/3)
: n(Fn(y)-Fn(x)] - n(y — x) (f(x)-An_1/3) for all y 1 x]
= P[In(x).: n[[Fn(y)-Fn(x)] - (y - X)(f(X)-An-1/3)]

<

for all y _.x]

2
: P[n{Fn (y) - an) - n(y - x) (f(x)-An-1/3) : 492.... (mg/3

<

for all y _.x]

- 321cm» (1.4.11)
for every n > N1 by (1.4.10).

Let F*(y) be the distribution defined by its density,

0 for y < -a
f* (y) = f(x) for -a _<. y I. x (1.4.12)
f(y) for y > x

where a is chosen so as to make F*(y) a distribution function.
Since F*(y).i F(y) for all y,
_1/ < c2 1/
PF[n{Fn(y)-Fn(X)} — n(y - X)(f(X)-An 3).. -erf'(x)n 3

for all y.i x]

2
-: PF*[n{Fn(Y)-Fn(x)) - n(y - x)(f(x)-An-1/3) E.- grf'(x)n1/3

<

for all y _.x] (1.4.13)
where PF denotes the probability when F is the underlying

distribution.

(1.4.11) and (1.4.13) imply that for every n > N1,

13
> ‘1/3 > szn 1/3
p _ PF*[n{Fn(X)-Fn(y)] - n(x - y) (f(X)-An ) _. Ng- (X)n

for all y E.x] - %Q(c)

' { ) )] f '1/3 sz- 1/3>
- PF*[n Fn(x -Fn(y - n(x - y)( (x)-An ) - 7\-2-- (x)n _ O

for all y f. x] - %Q(c). (1.4.14)

Let Z(B,t) for t 3.. 0 be distributed as
’ 2
N(tq) - nt(f(x) - An-1/3) - n1/3X%rf'(x) (1.4.15)

where N(q) is a Poisson process with parameter
q = [nf(x) - Bnl/z] and B is a constant > O.
From (1.4.13) we have for all n > N1,
p Z. PF*[n[Fn(x)-Fn(y)} - n(x -y) (f(x)-An-1/3) - n1/3A322—f' (x) Z O
“for all y f. x] - % Q(C)
- EF*{PF*[n{Fn(x)-Fn(y)] - n(x — y)(f(x)-An-1/3)
-n1/3)\-C72-f' (x) Z O for all y i x] )Fn(x) }- g- Q(c)
= EF* [P[Z(B,t) Z. 0 for all t such that O .1 t f. x + a] I11(x+a)q)

- .3».
— nFn(x)]] - 2 Q(c).
Let
_1/ 1/ c2
T = nFn(X) - n(x + a)(f(X) - An 3) - n 3N§rf'(x).
From our earlier remarks it follows that for every n 5 N1,
p 2'. EF*{P[Z(-B,t) Z. 0 for O f. t i x + a] |Z(B,x + a) = Tn]
3
- §'Q(C)

.i EF*{P[Z(B,t) : 0 for all t _>_ 0] |z(B,x + a) = Tn} - % Q(c).
(1.4.16)

Now

14
P[z(B,t).Z.O for all t 3.0].2

EF*[P{z(B,t):o for all t2L0]|z(B,x+a)-Tn]
+ PF*[Z(B,x + a) < Tn].

Therefore from (1.4.16), it follows that

pZP[Z(B,t)ZO for all t:O]-PF*[Z(B,x+a)< Tn]- %Q(c)
for all n > N1. (1.4.17)
Now

PF*[Z(B,x + a) < Tn] = P [N(q(x + a)) < nFn(x)].

F-X-
By Chebyshev's inequality
< F(x) (1 - F(x))

Let us now choose an e > 0. Then there exists a constant

PF.[N(q<x + a)) < aFn(x>1

BO > 0 such that

PF*[N(q(x + a)) < nFn(X)] < e

where q = nf(x) - Bonl/z.

Hence, from (1.4.17), we have for all n > N1
p z. P[Z(B0,t) : o for all t 1 0] - g- Q(c) - 5. (1.4.18)

It is obvious that for any real number u

E[exp{uZ(BO,t))]= exp[tq(eu-1)—utr-ukn1/3 ggf'(x)]
where
q = nf(x) - Bonl/2
and
r = nf(x) - Ana/3.
Therefore

2
E[exp[uZ(B0,t) - tq(eu-1)+utr+hun1/&% f'(x)}] = 1

for all u.

15
Let
T = inf {t : Z(B0,t) = 0].
By Wald's fundamental identity in the continuous parameter
case, (See A. Dvoretzky, J. Kiefer and J. Wolfwitz [9]) it

follows that

2
E[eXp{uz(Bo:T) - T{q(e”—1) - ur} + Anni/ag'f'(X)]] = 1-
LJ“o
Let us choose no such that q(e - 1) = nor.
Now we have

1/ (:2
E[exp{uOZ(B0,t) + kuon 35-f'(x)}] = 1.

This implies that
1/ c2
P[Z(BO,T) = 0] i.exp[-kuon 3\§ f'(x)]

In other words,

P[Z(B0,t)‘i.0 for all t.: 0]
1/ c2
.1 1 - exp[-Xu0n 3-§ f'(x)] (1.4.19)
From (1.4.18) and (1.4.19), we have
2
p.2 1 - expf—ku0n1/3 g- f'(x))-%-Q(c) - 8 (1.4.20)

for every n > N1.

 

u
Since q(e O - 1) ; nor where q = nf(x) - B0n1/2 and r = nf(x)-An2/3,
_1/
= - 2An 3 “'1/3
”0 f(x) -+ o(n. ). (1.4.21)

From (1.4.20) and(1.4.21), it follows that for every n > N1

2
p 2,1 - exp[-A[-2??X) 3 + o(n'-1/3)]n1/3 %- f'(x)]-e-%-Q(c)

 

2

= 1 - exp{-x[%%l + 0(1)]521 f'(x)} - e — % Q(c). (1.4.22)

16

Therefore,

. 2
11m , _ _ 3 f' x) _ _.3
T P— 1 8*“ AC “£731.74 .- 2 W’-

Since a is arbitrary, we have

. 2
11m > _ _ 3 f' (X) §_
Let us now take limit as c‘—> G).
Since Q(c) -2 0 as c —> oo,
11m 11m p = 1. (1.4.23)

C n

Let

s - P[Fn(y).i Fn(x + cal/3) — (x + CHI/3-y)(f(x) - Afil/3)

for all y 2.x + 2c 51/3].
It can be shown, by the same methods which were used
in proving (1.4.23), that

lim lim

s = 1. (1.4.24)
c n

(1.4.23) and(1.4.24) together prove (1.4.2).

Lemma 1.4.2.

Suppose that [ch}, [Xn] are collections of random

variables such that

 

(i) lim lim P[X C # x ] = 0
c->CO n->oo n n

(ii) lim P[xC ¢ X] = 0
C—‘>OO

(iii) ch converges to XC in law as n -2 00 for every c.

17
Then

Xn converges to X in law.

 

Proof:
Let L(X,Y) denote the Levy distance between the distribu-
tion functions of X and Y.

Since L(X,Y) i.P[X # Y], we have

(1) lim Ilm L(X ,x ) = 0
nc n
C—>CD n->oo
and (1.4.25)
(ii) lim L(XC,X) = 0 .

Since L is a metric,

< <
0 _.L(Xn,X) _.L(xn,xnc) + L(ch'xc) + L(XC,X).

Taking limit as n-vaa, we have for any fixed c

 

< - <‘_‘-'— _-—"'
0.. lim L(Xn,X)_l;m L(Xn,XnC)+l;m L(XnC,XC)+L(XC,X)

 

,xnc) + L(XC,X)

since lim L(ch.XC) = 0 by (iii) of the hypothesis. (Con-
n

vergence in Levy distance is equivalent to convergence in

law.) So we have for any c

o :. lim L(xn,x) 1 lim L(xn,x
n n

nC) + L(XC,X).

The expression in the right hand side of the above
inequality is equal to zero by (1.4.25). Therefore,
limL(Xn,X) = 0. In otherwords Xn converges to X in law.

As a consequence of lemmas 1.4.1, 1.4.2, it follows

that it is enough to find the asymptotic distribution of

18
f*n C(x) as n—>a> and then prove a result analogous to that
in lemma 1.4.1 for limiting random variables in order to

obtain the asymptotic distribution of fn(x).

1.5_fiReduction to ayproblem in stochastic processes:

In this section, we shall reduce the problem of calculat-
ing the asymptotic distribution of the slope of the concave
majorant of Fn(Y) over [§-2c 1.11/3, §+2c 51/3] at Y =§ to
the corresponding problem of a Wiener process over [~2c,2c]
after suitable normalization. We assume that f is differ-
entiable at g with f'(§) ¥ 0. Let us now consider

Fn(§+6) - Fn(§) for 6 in [-2cnl/3,2cnl/3].

Now
Fn(§+5)-Fn(§) =[F(§+fi)-f%§)] + [[Fn(§+5)-F(§+5)]-[Fn(§)-F(§)1l
62
= 5f(§) +'§ f'(§)[1 + 0(1)] +
{[Fn(§+é) - F(§+é)1-[Fn(§) - F(gm
= 5 f(g) — D62[1 + 0(1)] + 51/2 Yn(6) (1.5.1)
where

Y (6) = 111/2 [[Fn(§+6) - F(§+6)] - [Fn(g) - F(§)]] (1.5.2)

and

13:23:31.0. (1.5.3)

2
Let an(§) +6Bn(§) denote the tangent to the concave majorant

2

of fil/zYn(6) - D6 [1 + 0(1)] at 6 - 0. In other words,

Bn(g) is the lepe of the concave majorant of

2

56/2 Yn(6) - D6 [1 + 0(1)] at 6 = 0.

19
From (1.5.1), we note that
f;,C (g) = f(g) + 5n(§). (1.5.4)
We are now interested in determining the limiting

distribution of 6n(§) after suitable normalization.

 

LEt
5 ; rnC where rn = [fD-znnl]1/3 and f = f(i).(1.5.5)
LEt
( ) 51/2 Yn(6)
W =
n C r2 D
n

51/2[fD‘2n"1]”2/3 D_1 Yn(6)

= n1/6 fz/B nl/s yn(a). (1.5.6)
Let on = an(§) and an = Bn(g).
Let us now consider

51/2 Yn(6) - D62[1 + 0(1)] - an - and

[51/2 Yn(6) - riCZD - an - BnrnC - rfi C2 0(1)]

 

 

'1/2

2 n Yn(5) 2 an BnC 2
= r D[ ' - C --—-- --—-—- - 0(1)] (1.5.7)

n r2 D r2 D r D

n n n
2
B 2 a B

2 n n n 2
=rD[W(C)- +-—-) - (T‘- )-Co(1)]-'

n n (I; 2rnD rnD 4rr21D2

(1.5.8)

From (1.5.7), we observe that

B

rnD is the slope of the concave majorant at C = 0 of the
n

 

process

xn(C) = Wn(C) - cztl + 0(1)] (1.5.9)
2CD2

on [-q,q] where q = f

20

Let D[a,B] denote the space of all functions on the
interval [n.B] with discontinuities of.first kind and let
us introduce the convergence in D[a,8] by J1 - topology
(see Sethuraman [18]).

Let W(§) be the Wiener process over [-q,q]. It is
obvious that the trajectories of the process Wn(C) belong
to D[-q,q] with probability one. It is well known that
the process W(C) had trajectories in C[-q,q] with probability
one and C[-q,q] is a closed subset of D[-q,q].

Let un be the distribution induced by the process Wn
on D[-q,q]. Let u be the distribution induced by the process
W on D[-q,q]. Our aim is to prove that un converges to u

weakly. We shall prove some lemmas which lead to the result.

Lemma 1.5.1. For any C.£E [-q,q], Wn(§) is asymptotic-

ally normal with mean 0 and variance |C|.

Proof: By definition

Wn(§) = nl/sfz/alf/a Yn(6)
= n1/6f2/3D1/3{[Fn(§+6) - Fn(§)] - [F(g+5)-F(g)]]n1/2°
Obviously
Etwn(t:)] = o,
and
Var[Wn(C)] = n1/334/3 92/3 n%[F(g+6) - F(§)][1+o(1)]

= nl/a 24/3 D2/3 (5| f(§)[1 + 0(1)]

n1/3 E4/3 D2/3 rn (C) f(g) [1 + 0(1)]

21
= [CI [1 +0(1)1,
by (1.5.5).
. Therefore by the central limit theorem for independent

and identically distributed random variables, we get that

c

 

 

Wn(C) is asymptotically normal with mean 0 and variance

Remarks:

 

In a similar way, it can be shown that for any collection
C1,---,§k such that [Ci| j,q, the joint distribution of
[Wn(C1),Wn(C2),...,Wn(Ck)] converges to a multivariate nor-
mal distribution with mean 0 and variance-covariance matrix

(6(ci.cj) m1n<)g).lcjl>>
where 6(a,b) is defined by
1 if a,b are of the same sign
6(a,b) = ‘{
0 otherwise.

The next lemma consists of showing that the processes

Wn(C) satisfy an equicontinuity condition.

(Lemma 1.5.2.
E2£_anx C1<C2<C3.in {-q.q],
2
E[|Wn(C1) - wn(C2))2|Wn(C2) ‘ Wn(C3)|2]:C|C3 ‘ C1)

where C is a constant independent of n.

Proof:

From the definition

Ellwn<c.) - wn(c.)|2)wn<c.) - wn(c3)|21

 

 

= E
2
rnD rnD

EIAY (61)-r-11‘/2Y (6 ) 2 51/2)! (62)-51/2Y (6 ) 2-
g n 2 n 2 }. _{ n n 3 }.

ll
:3

 

where CO is a constant independent of n, by Chenstov [3],

 

 

-2
_ Con 2 2
‘ 8 4 rnlC3 ' C1)
r D
n
C n.-2
= O -2 -1 e/ 4 |C3 ‘ C112
[fD n ] 3D
by (1.5.5):
C
- 0 2
_ ;§]C3 _ C1) . (1.5.10)
Let C = Cof-Z. From (1.5.10), we have

E[an(C1)- wn<c.))2|wn(c.) - wn(c.))2|1:c|c. -c1I2

where C is independent of n.
We shall now state a theorem connected with convergence

of distributions of stochastic processes on D[a,B].

Theorem 1.5.3.
Suppose {Xn} is a sequence of stochasticpprocesses on

D[a,B] such that

 

(i) for any ti’ 1 i.i i.k ig,[a,B]. the joint distri-
bution of [Xn(t1),...,Xn(tk)] converges to the
joint distribution of [X(t1),...,X(tk)]

and

23
(ii). for any t11<t21<t3 ip_[a,6]

El IXn(t1)-Xn(t2) [71 lxn(t2)-xn(t3) [72 I] :5 ch:3 - t1|1+y3

where 71 > 0, 72 > 0, 73 > 0 and C > 0 are independent
2£n-
Let [vn] and v be the distributions induced by {Xn} and [X]

respectively on D[a,B]. Then Vn converges to v weakly.

 

Proof:

See Sethuraman [18].

As a consequence of lemmas 1.5.1 and 1.5.2 and the
remarks made after lemma 1.5.1, it follows as a particular
case of theorem 1.5.3 that the distribution ”n converges
Weakly to the distribution u.

Furthermore C2[1 + 0(1)] converges to C2 uniformly in
C since C is in [-q,q]. Hence, by a simple extension of

Slutsky's theorem for processes, it follows that
Theorem 1.5.4.

The sequence of processes Xn(C) = Wn(C) - C2[1 + 0(1)]

pp [-q,q] converges in distribution to the process

x(c) = w(c) 4:2

where W(C) is the Wiener_process on [-q,q].

1.6 Asymptotic distribution of the maximum likelihood

estimator..

For any x e D[-q,q], let g(x) denote the lepe of the

24

concave majorant of x(§) at C - 0. It is easy to see that
if xn -2 x in Jl-tOpology and x is continuous, then xn —¢ x
in the supremum norm tOpology. (See SethuramanlIS] pp. 129).
But, if xn -e x in the supremum norm topology and concave
majorant of x has unique slope at C = 0,then g(xn) —> g(x).
Further, it is well known that the process X is continuous
with probability one. Therefore g is a functional on

D[-q,q] whose set of discontinuities has probability zero
with respect to the distribution of X. FurtherXn converges
in distribution to X by Theorem 1.5.4. Therefore the distri—
bution of g(xn) converges weakly to the distribution of g(x).

Hence we have the following lemma.

Lemma 1.6.1.

.LEE f(x) be a unimodal density. ‘Lgp f;,c(§) denote
the slope of the concave majprant of Fn(y).gp [g - 2cn‘1/3,
g + 2cn-1/3] §£_y = 5. Further suppose that f'(g) exists
and is non-zero.

EBEE

nl/a 14;,C<g) - f(;)1

 

is distributed apymptotically as the lep§_of the concave

majorant of the pppcess

W(t)-t2, -q:t:q

at t = 0 where W(t) is the Wiener process with mean 0 and
_ I
variance 1 per unit t, W(0) = 0; D = .251397 f = f(g)

and q =2cf-1D2.

25

Proof: This theorem follows from the remarks made in

(1.5.4) and (1.5.9).

The next lemma shows that the lepe of the concave
majorant of the process X over [-q,q] at C = 0 and the slope
of the concave majorant of the process X over (-oo,oo) at

C = 0 are essentially the same.

Lemma 1.6.2:
TheyprobabilityL that the lepe of the concave majorant

of the process X(C) = W(C) - C2 over [-q,q] pp C = 0 is dif-

 

ferent from the slope of the concave majorant of the process
X(§) = W(C) - C2 pp_(-oo,oo) §p_c = 0, tends to zero as

q —> 00.

Proof:

For any a, let PC denote the probability that there
exist points u < a-c and v > a+c such that L(u,v,x) Z.X(a)
where L(u,v,x) denotes the line joining (u,X(u)) and
(v,X(v)). It is obvious that PC is independent of a.

Let us choose a to be zero.

Then PC = P(there exist points u < -c or V > c such that

L(u,v,x) 2.x(o) = 0). (1.6.1)
We notice that for any c,
x(g) :.w(§) +-c2 - 2c|c|. (1.6.2)
Therefore,
PC f. 2P[x(c) _>_ 0'__ , for some t > c]
.1 29[w(c) 2.2e: - c2 for some g > c]

2

2E[P[W(C).Z 2c§ - c for some C > c}|W(c)]

Q

26
1c2
= 2f:;o P{W(C).Z 2cC - c2 for some C > c|W(c)=x]d¢(x)

oo
+2f£222 P[W(C) Z. 2cC - c

2 for some C > ch(c)=x]d¢(x)

where 6 is normal with mean 0 and variance c,

112J" P[W(Q)-W(c) Z-2cC-c2-x for some C > c|W(c)=x]d¢(x)

: 2) P[W(t).Z 2c t + c2 - x for some t > 0|W(0)=x]d¢(x)

CD
d
+ 2f1c2 @(X)

I
. ' -

4

' 4
since W(C) is a stationary process with independent increments,
. l

2
c

= 2f4 p(w'(t) : 2ct + c2-2x for some t > O|W'(0)=0]d¢(x)
-CD

CD

+ 2f1 d¢(x)
/’ZC
where W'(t) is a Wiener process with W'(O) = 0.
But
P{w'(t) 2: 2ct + c2 - 2x for some t > 0 |w'(0) = 0]

= exp {8 cx - 4 c3] .

This can be proved by means of Wald's fundamental identity.

Therefore,
2

6° 1
< ' 4 A _ 3 _—
PC _-2] exp [85x 41c }. Jz—m? dx

CI)

27

 

+

CD
—x .
2f 2 exp {2c (Ev—c dx

1/4c
. C .

3 2 exp [- 55:} .
1 2 exp {-ZC] +J-2—7TE 5/4 C)

 

 

Taking limit at c -> 00, we get that

PC -> 0 as c ->'oo o (1.6.3)

Now let p be the probability that the slope of the concave
majorant of X on [-q,q] at C = 0 differs from the slope
of the concave majorant of X on (-oo,oo) evaluated at C = 0.
Then
p :.P[There exist points ul < 0, v1 > 2c such that
L(u1,v1,x) 2.x(c) or that there exists points

u2 < -2c,v2 > 0 such that

L(u2,v2,x) z-X(-C)]

by remarks made earlier.

Therefore,by (1.6.3), p —> 0 as q —> 00. (1.6.4)
(1.6.4) proves the lemma. .

In view of lemmas 1.4.1, 1.4.2 and 1.6.1, 1.6.2, we

obtain the following theorem.

Theorem 1.6.3.
Let f(x) be a unimodal density. Let fn(§) denote the
M.L.E. of f(g) based on n observations. Further suppose

that f is differentiable at g with non-zero derivative. Then

n1/3[fD]-1/3[fn(§) - f(§)] is asymptotically distributed

as the lepe of the concave majorant of the process

28

2

 

 

 

W(t)-t, -CD<t<oo
.3; t = 0, yhggg.w(t) is the Wiener process on (-oo,oo) ygggl
W(0) = 0 and mean 0, variance 1 per unit t, D s --£L§§l ,
f=f(§)-

The next step consists of deriving the distribution of
the lepe of the concave majorant of the process W(C) - C2
over (-oo,oo) at C = 0° It seems to be impossible to obtain
an eXplicit evaluation of the distribution. We shall show
that it is related to a solution of a heat equation as was
done by Chernoff [4] in the case of the distribution of the
location of the maximum for the process W(Q) —.§2 over (—oo,oo).

Let a + BC denote the tangent to the concave majorant
of the process W(C) - C2 at C = 0. We are now interested in
obtaining the distribution of B. Let h(B) denote the value
of C for which

Wm - (c + a)? (1.6.5)
is maximized over (—oo,oo) . .

Consider
)2 2

W(C) - C2 - a - at = W(C) - (C +'§- - (a - %—).(1.6.6)

By Theorem 1 of Section 4 in Chernoff [4], the prob-

ability density function of h(B) is ¢(C - B) where

MC) = $— Ux(t:2.c)ux(cz.—c) (1.6.7)

where Ux is the partial derivative of U(x,C) with respect

to x, U(x,C) being a solution of the heat equation

1
EUXX - UC (1.6.8)

29
subject to the boundary conditions
(i) U(x,§) = 1 for x Z.C2
and . . (1.6.9)
(ii) U(x,§) —> 0 as x-—> -oo. .

We note that

Prob[B - 53 :.3 :.B + 53]

= Prob[h(-B-%—§§-) .1 0, h(B $513): 0]
= Prob[h(§—g——§§) .<_ 0] - Prob[h(-B—;5—§§) £5 0]

= h(c - 13—5—36: - l w: - 9-3—9544
-d) -oo

_(3 5 53) _ (3 g 53)
fw(c)dt;. - f ((4)42:
—00

 

-00
_(B E OB)
=f + 6 Mama.
_(§_§_;§)

Therefore the density of B is
%z/1(-%). (1.6.10)

we note that ¢ is symmetric from (1.6.7).

Hence we have the following theorem.

Theorem 1.6.4.

The probability density function of B, viz. the slope
of the concave majorant of the process W(C) - C2 pp C - 0
where W(C) is a two-sided Wienep;pevy process with mean 0

and variance 1 per unit, is
1
5' (Mg)

where ¢ is defined in (1.6.7).

30

Combining the results in Theorems 1.6.3 and 1.6.4, we

have the following final result.

Theorem 1.6.5.
Let f(x) be a unimodal density. _Iigt fnfi) denote the

M.L.E. of f(g) based on n observations. Further suppose

 

that f is differentiable at g with non-zero derivative

f'(§). Then the asymptotic distribution of
n1/3[_f(§)§' (§)j'1/3[‘fn(§) _ “W
has the density
fi-wg)

where p is defined in (1.6.7).

CHAPTER 2
ESTIMATION FOR DISTRIBUTIONS WITH MONOTONE FAILURE RATE

2.1 Introduction :

In this chapter, we shall investigate a problem analogous
to the problem treated in Chapter 1. We shall now suppose
that the distribution F has the monotone failure rate r.
(definitions are given in 2.2). Suppose X1, ..., Xn are n
independent observations from F. Grenander [10] and Marshall
and Proschan [14] have obtained the maximum likelihood
estimator (M.L.E.) of r and the latter showed that these
estimators are consistent. We shall obtain the asymptotic
distribution of the M. L. E. as a function of a solution of
a heat equation as was done by Chernoff [4] in the case of
estimation of mode. Methods used in the chapter are similar
to those in Chapter 1 and therefore, proofs are given only
at places where they seem to be necessary.

We mention here that Murthy [15] has obtained some
estimators of failure rate which are consistent and asymp-
totically normal. He does not assume a priori that the
failure rate is monotone and his estimators are based on
the choice of "window". Watson and Leadbetter [20] have
also obtained similar estimators.~

We shall give proofs only for the case of distributions
with increasing railure rate (IFR). Results in the case of

distributions with decreasing failure rate (DFR) are

31

32

analogous to those in the case of IFR and we shall mention
them in section 2.7.

Sections 2.2 and 2.3 deal with definition and properties
of distributions with monotone failure rate. Some results
related to the asymptotic properties of the M.L.E. of r
are given in Section 2.4. The problem is reduced to that
of a stochastic process in Section 2.5. The asymptotic dis-

tribution of the M. L. E. is obtained in Section 2.6.

2.2 Definition andpprOperties of distributions with monotone

failure rate:

 

Let F be a distribution function with density Pf.
The failure rate r of F is defined for x such

that F(x) < 1 by

r(x) =-1——§L-)-;y . (2.2.1)

Let R(x) a - log (1 - F(x)

"EIX

. It is easily seen that R is

V

convex on the support of F if and only if r is non-de-
creasing and that R is concave on the support of F if
and only if r is non-increasing. We say that F is an
IFR (increasing failure rate) distribution or a DFR (de-
creasing failure rate) distribution according as r is non-
decreasing or non-increasing. PrOperties of distributions
with monotone failure rate are discussed in Barlow, Marshall

and Proschan [1].

33
2.3 Maximum likelihood estimation for increasing failure
rate distributions:
Suppose F is an IFR with failure rate r. Let
X1 §.X2 i.--- §.Xn be an ordered sample from F. Let c3Lbe
the class of IFR distributions. It is not possible to ob-
tain the maximum likelihood estimator for F e y—directly

by maximizing the likelihood
n
L(F) = W f(X

f(Xn) can be arbitrarily large for F e 2}. Therefore, we
consider a sub-familyng’ of El consisting of distributions

. . . n < n
F(x) With r :.M, obtaining Sup M F f(Xi) —-M . There
F e 3. 1

is a unique distribution FnM €2¥M for which the supremum

is attained. The failure rate an of En“ converges to

a failure rate in as M —> 00 for argument x < Xn. For

X Z.Xn, fn = M for all M and therefore an'-> 00 as

M -> 00. This estimator fn’ which is infinite for x Z'Xn’

is called the M. L. E. of r.
From the results of Grenander [10] or as an application
of Van Eeden [19], the estimator En can be derived and it

is given by

0 for x < X1

1 _ 1 < < ' < _
rn(x) rn(Xi) Xi _.x < Xi+1'1'_ 1._ n 1

00 x :.x

n

where
V-rl _1

rn(Xi) = min max [[V-u][ Z (n—j)(Xj+1-Xj)] }.

VZi+1 uii j-u

34
Marshall and Proschan [14] showed that this estimator

is consistent.

The estimator En can also be written in the form

 

~ =~ F(v)-F(u)

rn(x) 32: iii 3 n (2.3.1)
I [l-Fn(y)]dy
1.1

where Fn(x) is the empirical distribution function.

 

In fact
[rn(Xi)] = Sup Inf ¢n(v) - ¢n(u) (2.3.2)
>1 +1 <£
_-—- u— .
n n V-u
where

1 Xj
¢n(n) = ID [1 - Fn(x)1dx°

A

Let 9n be the concave majorant of ¢n' Then, from (2.3.2),

it follows that [531(k)].1 is the slope of the concave

 

majorant 5n 2;. Fn(x).

2.4 _Some results related to the asymptotic prOperties of

the maximum likelihood estimator:

x 1
Let rn(x) denote the M.L.E. of r at X. Let r; C(X)
I

denote the slope of the concave majorant of ¢n at Fn(x),
when the argument of ¢n is restricted to the interval

_1 _
[F(x) - cn /3 , F(x) + cn 1/3]. It can be shown, by
methods analogous to those used in Section 1.4 of Chapter 1,

that

35

Lemma 2.4.1.

Cligb TEE. P[r;'c(x) # fn(x)] = 0.
Let g be such that O < F(g) < 1.
We shall now obtain some asymptotic expansions of §n(y)
for y in the interval [F(g) - cal/3, F(g) + c51/3] I
We shall assume that _
(i) f is differentiable,
(ii) f' is continuous, and (2.4.1)
(iii) the failure rate r is differentiable at
the point g and r'(§) is non-zero.
(For any function h, h' denotes the derivative of h).
As a consequence of assumptions made above, it follows
that for x in a sufficiently small neighborhood of g ,
(1) f(x) is bounded away from zero,
(ii) r(x) is bounded away from zero, and
(iii) -£%%§% is bounded.
Suppose that
(i) f(x).: 7,
(ii) r(x) 2.0 and (2.4.2)
(iii) )fé’}: I: k

for x in that neighborhood of’g

Let Fn(§) = nn' Let F(g) n. It is well known that

Tln ... T] = op(fil/2).
LE’C

U. = F(Xj+1) - F(Xj) - E[F(X.

3 3+1) — F(xj)| xj] (2.4.3)

where Xi’ 1 2.1 §.n are the order statistics and E[Y)X]

36
denotes the conditional expectation of Y given X.

It is easy to see that

 

Uj = F(Xj+1) - F(Xj) - 1(xj) (2.4.5)
where 1 - F X.)
1(xj) = n _ j(+31 .

We shall obtain the necessary asymptotic expansions in
a series of lemmas which will be combined at the end to
give the final result.

We mention here that the approximations which are of
the order Op are all satisfied uniformly for 6 in [—c,c]
in the following lemmas. Let

a = [nn] and b = [nn + 6n2/3], (2.4.6)

where 0 < n < 1 and -c-1 6._ c.

Lemma 2.4.2.

 

b—1 X(X.)+U.
nu) (A) - «v E‘-)1 = 2 (n-')( 3 Jim (51/3).
n n n(n j-a J '_ftf;7__ p

Proof:
By definition of Uj in (2.4.5), we have
F X. = F X. + X. + U..
(3.1) (3) M3) 3
Therefore, Xj+1 = F-1[F(Xj) + 1(xj) + Uj].
Expanding F-1(Y) by Taylor's theorem up to second order

terms, we get that

 

-1
_ dF (y) =
xj+1 — x3. + [1(xj) + Uj] dY Y F(Xj)
2 d2F-1(Y) - «
+ 1/2[7((x.) + U.] 2 Y = 6. (2.4.7)
3 3 dY j .

where Gj lies between F(Xj) and F(Xj+1).

37

It is easy to see that

 

 

dF Y __ _ 1
dY I Y ‘ F(x)) ” f(xj)
and 2 1 . (2.4.8)
d F- (Y) Y = e, = “f (Cj)
de 3 f3(cj)
where
g -1
Cj F (9])
By definition,
b a b-l .
n[¢n(a) - ¢n§591 = _2 (n-J)(Xj+1 - Xj)
3:3
b-1 7\(Xj) + Uj] ( )
= Z n —j
j=a f(x).)
b'l f'(Cj ) 2
_1/2 8 —3————(n-j ){)(x. ) + v.) (2.4.9)
3' =a f (r. ) 3

j
by (2.4.7) and (2.4.8).

Now for n sufficiently large, we have by (2.4.2)

b-1 f (q.)

E |,2 "3—-l— (n-j) {6(Xj) + Uj}2|

J=a

 

k b-1
.-1 )2¢§ E[( n- -j)(5(X ) + Uj )]2
v 3- a
b-l -
_ . 2
- fg-jga (n-J) E[F(Xj+1) - F(Xj)1
b-1
5_. 2 _.
= y2 (n+1)(n+2) an (n 3)
i.h§' 25-(b-a) n = O (n-1/3) (2.4.10)
7 n

38
since b-a = 0(n2/3).

(2.44» and (2.4.10) together prove that

 

10-1 ij )+U.
2- _ é. : _- j_ -1
n[¢n(n) ¢n(n)] jE-a (n j){ f(Xj) }+op (n /3).

Lemma 2.4.3.

b-1 (n-j)U- b-1 _
n[¢n(9) — ¢ (3)] = z ?TE‘T1 + z ?7%;T' + 0p(n1/3).

n n n . . ._
3'3 J 3'3

Proof: By Lemma 2.4.2,

b- 1 MX )+Uj
n(enéf) - ¢n(§-)1= z [ £013.) 1(n-j )+0p (n /3)

j=a
b-l (n-j)Uj .b-l (n_j) 1-F(Xj)

—1
=an 7-5?)— +an (n-j+1)I f(x).) } + "10‘n A)

J

 

b-1 (n-j)Uj b-1
f(Xj )3 + jZ ar(Xj )

b-l 1

a(n-j+1)'(r(xj ) + 0p(fil/3). (2.4.11)

Now
b-1 b-l
1
< ._______
Elj— 2 am FDIC—:37 l _ajia (n-j+1) for n large by
(2.402)
n-b+2 1
.1 d -dx
n-a

£212.12. = 0(51/3)

n- a (2.4.12)

= alog

since b-a = 0(n2/3) and n < 1.
From (2.4.11) and (2.4.12), we have
b-1 (n-j)U. b—1 1

b a _ —1/
n[¢n(';1')- ¢n (H)]-j§a Wl 4-an W + 0p (n 3) o

39

Lemma 2.4.4.

b-l

1 _ 'b-a r'[Z(fl)] 1 Abra
Z — _
j=a r(XjS . r[Z(T])] r2[z(q)] f[z(n)] n

)2

 

+ o (nl/G)
P
where Z(t) = F- (t) for O fi.t 5.1.

Proof:

Let zj = z(%§.

-)

3 for some aj between

 

F(x.) - F(Z
Now x. — z. = 3
3 J f(ajf
X. and Z.
3 J
and f is bounded away from zero.

By the Kolmorov-Smirnov theorem,

-1
S§p|F(Xj) — F(Zj)| = Op(n /2).

Therefore

_1/
u X0 _ Z. — 0 II 2 0 2.4.13

Since. [, - ,

° ' ' '(-r?(§-)
riijj W .ri;.5 :.r2QCE)‘ (Xj 7 Zj)

J

for some Cj between Xj and Zj' we have

b-1 1 b-1 1 b-1 r'(§.)( )
Z - Z = - Z X. - Z.
j=a r(xj5 j=a rzzj) j=a r2(cj) 3 3

- Op(fil/2 ,nZ/a) - 0p(n1/6)

by (2.4.13) and the fact that 'E2 is bounded.
r

40

In other words,

b 1 b—l

- 1 : 1 1/6
i? inc—3.7 32%;) + °p<n >'

3

(2.4.14)

By Taylor's Theorem, we have

1 _ 1 + (1__ n {_ r'[z(n)J 1 }

r(sz - r[z(n)] n ) r2[Z(n)] f[z(n)]

+ (% - n) 0(1):

which implies that

 

b-l b-1 .
1 _ b-a r'[Z(n)] 1 .1
z _ - Z ( ‘ )
j=a rizjf r[Z(n5] r2[Z(n)] f[z?n)] j=a n n
b'11 )<)
+ Z — o 1
j=a (n n (2.4.15)

_ b-a _.ELLELDI] 1 (b—a)? n1/
firmw— rzmw mom n *°‘ 3’

since b-a - 0(n2/3).

From (2.4.14) and (2.4.15) we get that

b-l 2
1 = b-a _ r'|Z{DZ| 1 (b-32 1/
an r(xj) r[z(q)] r2[Z(n)] f[z(n)] n + Op(n 6).

Lemma 2.4.5.

b-1 (n-j)U. b-1

' x. = [z1 ] Z (n-j)Uj + op(n1/6) ‘
j=a j n j=a
Proof:

By the.Kolmogorov-SmiIDOV' Theorem, it can be shown

as before that

f(zlcj") = 27%;) + OP(J_%_ )- (2.4.16)

Therefore

41

b-1 (n-j)U. b-l (n-j)U. b-1

2' -T-—7—1'= f Z + Z’(n-j)Uj Opc-—). (2.4. 17)

. f . . .
J-a x3 3=a J j= a pJE'

But

2 2
{El(n-j)Uj0p(31-fi)l} : Bun-nu 12m 147:)

= o(%) uniformly in j. (2.4.18)

Therefore (2.4.17) and (2.4.18) imply that

 

 

 

bgl (n-j)Uja b-1 (n-j)Uj 0 1/6) 2 4 19)
+ . .
=a f(xj f(zj ) p(n (
since b-a = 0(n2/3).
‘1 _ 1 j f'UzCnxL
But ETET7'—'EEZIETT + (H - U) [— + 0(1)]-
3 f2 [2(2)]
So we have,
b-l (n-j)Uj_b-1 (n-j)Ujb-1 j f2 (ZQ ))
z —— >3 —— + z u.(—-n)[- ’1 +o<1>1<n-—j)
j-a f(zj ) j=a f[Z(”)] j=a 3 n f2 (Z(n))
1 b‘1 '.
" W jia ‘n'J’Uj + Zn (““2”
where
b-l . .
2n = 2 (%--q)[-f2(z(n)) + o(1)](n—j)U..
j=a f <z<n)) 3
Since E(Uj) = O for each j, E(Zn)= 0.
‘ 2-b-1 .
4, . k .
But Var-(Zn) 1 :2. 3.251% _ n)2(n_3 )2 var(Uj)
‘ 2 by (2.4.2)
2 b-l . - ~
15235-2- 18 (% - n)2.$2_:§.l)_ since Va'r(U.) :92—
v j=a n v 3 =n

for some constant d

= 0(1) since b-a = 0(n2/3).

42
Therefore Z = 0 (1).
n P
Now (2.4-19), (2-4.20) together imply that

b-l (n-j)Uj b-l

= _. 1 1/6
jia “f(x'j) [3.2a (’1 J)Uj]f[z(n)] + 0.1:>(n ) + 0pm

b-1
=[Z

+ O (n1/6).
j=a P

. 1
(n—3)Uj] f[Z (1.1)]
From lemmas (2.4.3) — (2.4.5), we have the following theorem.

Theorem 2.4.6.

Lg£.§ be such that 0 < F(é) < 1 and let F(g) = n.
Suppose that -c.i 6.1 c. ‘
Ls}; a= [nn] grid b= [nn+ 6n2/31-
21121

b a
nt¢n(;) - “31".?”

 

=fl.3__ W (5) -2311: 1 r'LZ(n)]
r[Z(n)] n 2 f[Z(n)] r2[Z(n)]
2/ /'
on 3 11
+ rtz‘mn + op‘n 6"
where
b-l .
w (5) a Z n2/3 i2:lill.u_°
n ._ (n-a) 3
3-3
Proof:

By lemmas (2.4.3) - (2.4.5), we have

n[¢n(%) - “GM?”

_ 130—12— BM) + Op(n1/6) (2.4.21)

43

where
_ 1 r' Z .
But [(b-a) - én2/3|.i 1.

Therefore, from (2.4.21), we have

-a b-1 (n-j)Uj

b a _ n
nwnw - wan 'W .2 T37—

J=a

+ én2/3 52n1/3
r[Z(n)] - 2

b‘1 - b-1 .
.2112._ £1.21). n-a (n- )
“2‘11” jfam'a) ”j + fIz—Tm) jig—£7 Uj

+ 5n2/3 _ 62n1/3
r[Z(Tfi] 2

B(n) + Op(n1/6).

= _E:BE__.b§1 (n-j) U + T'* 5n?/3 _ 5 n /3
f[Z(n)] . (n-a) j n f[z(‘—_Anl)] 2

(n)+0 (nl/G):
3=a p

(2.4.23)

where

b-1 .
_ gn-a Z (n-J
n f[Z(n)] j:

T U .

a (n-a) j

Obviously

< 1 b-l

BIT | _. 39:11-E|U.| = 0(51/3)
n J

f[Z (71)] j=a (n-a)

since b-a = 0(n3/3) and E|Uj| = 0(n-1).

Therefore, form (2.4.23), we have

b—1 -
Q __ a _ n|1—g| (n-J)
énz/a 52n1/3

________ ________ 1/6
+r[Z(n)] 2 ’3‘”) ’“Op‘n ’

44

= n b-l (n-j+1) U _ R
r[Z(n)] j=a (n-a) j n

én2/3 52n1/3

_ _____ 1/
+ W 2 B(T]) + 0p (n 6), (2.4.24)

where
R =__2__b§;1_l_u
n r[Z(T])] j=a n-a 3
Obviously
_ -1/ . -1
Eanl — 0(n 3) Since E )Ujl = 0(n ).

Therefore, from (2.4.24), it follows that

1/3 52n1/

2. a _ n” 3 ,6n2/3- 1
n[¢n (n) - ¢n(I—1')] - r—[Z—(TTTT Wn(5)- —2——B(T]) +W+Op(n /6)
where '
b-l -
wnw) = 2 n2/3 412—3? U.. (2.4.25)
j=a 3

Remark: Since r(x) is non—decreasing and since r'(§)# O,

we have B(n) > 0.
2.5 Reduction to a problem in stochastic_processes:

In this section, we shall reduce the problem of calculat-
ing the asymptotic distribution of the slope of the concave
majorant of ¢n(Y) over [F(g) - cfiI/3, F(g) + oil/3] at
Y = Fn(§) to the corresponding problem of a Wiener process
over [-c,c] after suitable normalization-WEIShall assume
that the conditions in (2.4.1) are satisfied.

.Let Fn(§) = nn and a = [nn], b = [nn + én2/3], and

F(g) = n where -c 3.5 :.c.

45

By theorem 2.4.6,

 

b a _ n n
n[<bn(;:) “MK” - MW] - 2 am)
2/
on 3
r[Z(n)] *0 (n 6’

where B(n) and Wn(d) are defined in (2.4.22) and (2.4.25).

Therefore,
W (6) 2
2/ b a = n 7 5
n 3[¢n(n-) " “31‘3” r[Z('q] ‘ 2 13m)
r[Z(T])] + 0p(n 6). (205.1)

Let an(n) + 6 fin(n) denote the tangent to the concave

majorant of

W (5) 52 _1
Fig—(37] - 71301) + 0p(n l6) over {-C.C]. at
5n = (nn _ n)n1/3.

We notice that

 

1/ .1 _ __.__;____ =

where [r; C[Z(T])]]-1 denotes the slope of the concave
I

majorant of ¢n(Y) over
0
- —1
[n-cn1/3, q+cn l3] at Y = nn.

We are now interested in determining the limiting dis-
tribution of Bn(n).

Let 5 = AC where

)\ = {W}. . r (2.5.3)

_ -1/
Let vn(c) - 7\ 2 wnw). (2.5.4)

46
Let us now consider

HMJT “m?“aJM-BH<Mé+om )

kl/zvn(é) B(n)x2§2

 

- an(n) - Bn<n)xc + op<51/6)

r[Z(n)] - 2'
1/ 2d (T1) 21.13 (Tl)
_ x 2 2 _1/
- r[z(n llvn(C) - c --;Z§7;;- 2-————;—— 1 + 0p (n 6)
, (2.5.5)

by (2.5.3),

2
1
x 2 Bn(n) 2 2an(n) an (n)
=———(—T— [V (C) - (C + )-( )1
r[Z n J n A B(n) h23(n) x232(n)

 

 

+ op(fil/6). (2.5.6)

23 (q)
From (2.5.5), we notice that R§%fiT—' is the s10pe of

the concave majorant at C = Cn = k-1(nn - 1])n1/3 of the

process
Xn(C) . vn(C) - c2 + op(fil/6) (2.5.7)
on [-q,q] where q =-%.

= 0 (51/6). (2.5.8)

Notice that Cn p

Let X(C) = W(C) - C2, where W(C) is the Wiener process
on {-q.q].

Let D[a,B] denote the space of all functions on the
interval [a,B] with discontinuities of first kind and let
us introduce the convergence in D[a,B] by Jl-t0pology.

(See Sethuraman [18]).
Let W(C) be the Wiener process over [-q,q]. It is

obvious that the trajectories of the process Vn(C) belong

47
to D[-q,q] with probability one. It is well known that the
process W(C). has trajectories in C[—q,q] with probability
one and C[-q,q] is a closed subset of D[-q,q].

Let un be the distribution induced by the process Vn
on D[-q,q].. Let u be the distribution induced by the
process W on D[-q,q]. .

Our aim is to prove that un converges to u weakly.

We shall prove some lemmas which lead to the result.

Lemma 2.5.}.
For any 5 in. [-c,c] , Wn(é) is asymptotically

 

normal with mean 0 and variance lé

Proof:

Let A1,...,A be independent random variables each

n+1

with the exponential distribution ne-nx, x.: 0. Let

 

Dn = A1 +. o o + An+10 (2.509)
A1 + + Ai
Then Zi - D , 1 §.i i.n form order statistics
n

of a sample of size n form the uniform distribution on
[0’1]. (205010)

From (2.4.25), we have

2/3 b_1 n-j+1

W (5) = n 2 U.
n . j-a n-a 3
b-1 - 1-F(X )
_ 2/3 n-J+1 .
- n jia --—n_a [F(x)-+1) - F(xj) - n_j+1 1.

Therefore, from (2.5.10), it follows that Wn(5) has the

same distribution as

 

 

 

 

 

 

 

 

 

1_ 'i
-1/3 b-1 A. - D ' ‘
. +1
W305)? ' a .3 (n-J+1)[-l— - n_j$1 ]
1-(-) j=a
n «y -
-1/3 b-1 A + +... +A
: n a z (n-j+1)[A.+1 J71: jfljnflfl]
Dn(1-E') j-a
_1/3 b-1
.... n a z (n-j+1)[(A.+1 711-) ~(Aj+1+°°'+An+1--I-];-)].
Dn(1-'r;) j=a n-j+1
(2.5.11)
Let /
—1 b-1
n 3 . 1
(1) Anus) - 1.2 (n-3+1>(Aj.1 - H)
n 3
and
.. __ n n-J+1
(11) Bn(c5) 1_ E j=a 3+1+'°°+An+1 n ).
n (2.5.12)
Then
A (<5) B (<5)
w*(5) a n — 3 (2.5.13)
n n
Since E(Aj) = .1]: for 1 S. 3 f. n+1,
E(Bn(<5)) - 0. (2.5.14)
Now
-2/3 b-l
Var(Bn(6)) - (1- 2)2 Var[j:a (Aj+1 + ... + An+1)]
n
n.2/3 b-1 n+1
='———5—§- Var[ Z (j-a)A. + (b-a) Z Aj].
(1- 3) 3'3 3":
—2/3 b-l

49

 

 

since A]. are independent and Var (Aj) = $2- .
n

Therefore

-2/ b-a-l
3 . .2
Var(Bn(é)) = n2 a 2 [ Z 3 + (b-a) (n-b)]
(1- g) 3‘1
= 1 [(b-a-1)(b-a)[2(b-a-1) + 1)

6

+ (b-a)2(n-b)]

3
1 b-
n8/3(1_ 292 [$ 33)+ (b

-a)2n].

IA

Since b-a = 0(n2/3), the term on the right hand side
tends to zero. Therefore, from (2.5.14), it follows that
Bn(é) -¢-O in probability.

Since Dn converges to 1 in probability, by Slutsky's

theorem (See Cramer [6]),

Bn(6)

D
n

7> O in probability. (2-5-15)

 

Imn: fn(t) be the characteristic function of An(é). Then,

 

 

 

 

-1/ b-1
_ tn 3 - .l
fn(t) - EIeXPfl 1_ _ 33a ((n-3+1)(Aj.1 - nn )1 .‘
n
-4 b—1 Q )b-l n ‘_
= exp{-i 2 n-j+1 }w _ - /
._ — j=a j—a n 1[(ni+1):n 3]
"5

since Aj's are independent and exponentially distributed.

Therefore

50

 

-4/ b-l b-1
log f (t) = _.££E___§ z (n-j+1) + 2 log n
n a .
(1-';) j=a 3:?
b'l n-'+1 -1/3
- Z logEn - {imL-lz—l tn )]
j=a 1' '-
n
-4/ b-l b-1 . -4/3
=‘1tn a3 z (n-j+1)- z log[1 - 1t(“’j+1)n J
1" E j=a j=a 1- 3
11
2-8/ b-l
= _ £;2__2.2 2 (n-j+l)2 + 0(1)
2(1- 3) j=a
n
2 -8/ 5-1
_ -t n a g z [(n-a)2+ (a-j+1)2+ 2(n-a)(a-j+1)] + 0(1)
2(1- -) J=a
n
2 -8/
= - 1:__n___3_:§_ (b-a)(n-a)2 + o(1)
2(1--9)
n
2 -8/
= - 5—3—3 nzlzslnm + 0(1)

2
- g- lél + o(l).

Therefore, fn(t) -' exp { - g— (5|) as n -+ °°.
Hence by the continuity theorem for characteristic
functions, it follows that
An(6) is asymptotically normal with mean 0 and
variance '6) . (2.5.16)
Then, by Slutsky's theorem (See Cramer [6]),
An(6)

D
n

 

is asymptotically normal with mean 0 and

variance (6| (2.5.17)

51

since Dn converges to 1 in probability.

From (2.5.13), (2.5-15) and (2.5.17), we get that Wh(é)

isamymptotically normal with mean 0 and variance (5|.

Lemma 2.5.2-

For any C in [-q,q], Vn(C) is asymptotically normal

 

widinman O and variance [C

Proof: This lemma follows immediately from lemma

2.5.1, since by definition
= '1/2
vn(C) x wn(6).

Remark: In a similar manner, it can be shown that for any

collection C, ... , Ck such that Ci 6 [-q,q], the joint

distribution of [Vn(C1), ... , Vn(Ck)] converges to the

multivariate normal distribution with mean 0 and variance
- covariance matrix (6(Ci’cj)" min (|§i|,|Cj())

where'
if a,B are of the same sign

_ 1
é(o,fi) - {0 otherwise.
The next lemma proves that the processes {Dnvn(C)) on

[-q,q] (Dn is defined in (2.5.9)) satisfy an equi-continuity

condition.

Iggmma 2.5.3.

Lesser C1IC2 _i_r.1_ l-q:q1:

4
EIDnVn(C1) ‘ DnVn(C2)|

2
:.C|C1 - Czl + ICI - C2) 0(1)

Where C is a constant independent of n.

52

Proof: We have

E(nnvn(s.) - nnvn(é.)(4

_ 4
= x 2 EanWn(c1) - ann(c2)l

‘ K-z EilAn(51) ‘ Bn(51)} - {An(52) ‘ Bn(52)}|4

where An(é), Bn(é) are defined in (2.5.12),
<srfiEm(a)-A(6H4+Em(a)-B(aHfi
—- n 1 n 2 n 1 n) 2
(2.5.18)
by the elementary inequality
ElX + Y|4 1.8[Elxl4 + EIYI4].
Let

bl = [nn + 61n2/3] 7 b2 = [nn + 52n2/3].(2.5.19)

Let us first compute

ElAn(51) ‘ An(52)|4

_.2 . 1+1
(1 n) 3 b1
-4/3 bz-l
. 4 9
' n a 4 [ 2 (“‘3+1) ‘4'
[1" '_'] j=b1 n
n
b2-1 b -1 ‘
+ 6 Z Z (n-i+1) (n-j+1)2'lz ]:
1-b1 j-b1 n
1 j (2.5.20)
since Aj are independent and
1 _ _1" _ 1.4 -._g .
E(Aj) a 37 Var(Aj) — n2, E(Aj n) — n4 (2.5.21)

for all j such that 1.: j.: n+1.

From (2.5.20), we have

 

 

 

-4 b -1
9 3 2 . 2 2
IA (5)‘An(52)|4 in ,-=(4[,z (fl-#1)]
4( “‘3) J=b1
_‘/
9 3 2 2
:2 ‘a4 [(152 -b1)n]
n 1- -)
n
"4/3
.5. 9n 4 4 (e32 — 51)2 n4/3n4
(1-n) n . ,
2
= c1|52 - 511 . (2 5.22)
Let us now compute
4
E|Bn(61) - Bn(éz)|
"4/3 b -1
-n E1 2 (A. -l)+... (A --1-)}l4
a 4 ._ 3+1 n n+1 n
(1' 39 J‘bi
-4 b —b n+1
3 2 1' 1 l- 1 4
- n a 4 I Z 3( 1+3- 39+(b2'b1)_~2 (AD-"))
(1 a) = J=b2 1
-4/ b -b n+1
< 3n 3 2 1 1 1 4—
...———[E j -—) +b-b)4E (A--)]
(1-.',a;)4lj-1(Ab1+3n|4(21 lj 41:3anl
-4/3 bZ-bl bZ—bl bz-bl .2-2
= 8n [{ 2 9-4 + 6 Z Z 1 J 3
a 4 . -l— . ._ 4
(1- —) 3=1 n4 3:1 1-1 n
i755:
n+1
+ (bz-b1)4{ z 2+62 z 14.)]
j=b2+1n i n
i#jj
-4/ b -b
7 3 2 1 . 2
_<_ 2“ z: 3212 + (152-151)4 (n-bz) 1

4 a 4 .=
n_ (1- n) j 1

 

4:/

54

 

”4/3
-= 73?. a)4 [Is-61:2 n 4’3 n4 + 152-6114 rug/3 n21
n __
n by (2.5.19),
_ 72 2 —2/ 3
- ?I_—333'[l52‘51| + I52‘51I n 3 3C 1
n since [52-51l 1 2c,
:c2|52-51|2 + [52-51| 0(1). (2.5.23)
where C2 = -Zgji
(l-n)

Combine (2.5.8), (2.5.22) and 2.5.23) we get that
2< .‘2 2 2
Eanvn(§1)-Dnvn(q2)|._ 8k [C1|52-61] + C2|52-51| +|62-61|o(1)]

2
= ClCz'C1l + ICz—C1l 0(1)
where C is a constant independent of n.

This proves lemma 2.5.3.

Remarks:

Since Dn converges to 1 in probability, DnVn(C)
is asymptotically normal with mean 0 and variance [Cl by
lemma 2.5.2. Further from the remarks made at the end of
lemma 2.5.2, it follows that the joint distribution of
[DnVn(C1), ... , DnVn(Ck)] lS asymptotically multivariate
normal with mean 0 and variance - covariance matrix.

<6<ci.cj) min <|c1|.|cj|))
where 6 is defined by

_ 1 if a,b are of the same sign
6(a,b) - {0 otherwise.

When Dth(C) is represented in terms of exponentials

as in lemma 2.5.1, we note that DnVn(C) is a process with

55
independent increments.
We shall now state a theorem connected with convergence
of distributions of stochastic processes with independent

increments in D[Q,B].

Theorem 2.5.4.

Let_ Xn be a sequence of processes with independent
increments on D[a,B] and_ X be a_process on D[a,B] quh
2152

(i) for any ti, 1 i.i i.k “in [a,B] the joint distri-
bution of [Xn(t1), ... , Xn(tk)] converges to thegjoint
distribution of [X(t1), ... , X(tk)], and,

(ii) there exist constants 7 > 0, C > 0 independent

.9: n such that for every t1,t2 e [c.51,

2
E|Xn(t1) - xn(t2)|7 :.c|t1-t2| -+ |t2-t1| 0(1).

 

Let Vn and v be the distributions induced by_ Xn and X

respectively on D[d.B]- Then Vn converges to v weakly.

Proof: From the condition (ii) of the hypothesis it
follows that

E|Xn(t1) - xn(t2)|7 : Alt1‘tzl

for all n and for t1,t2 e [a,B] such that lt1‘t2I < 1
where A is a constant independent of n.

Therefore, for any 1 > O,

A t -t
'2 lliéé.
17 x7

 

P{|xn(t1) - xn(t2)| > M i

for all ItZ-tll :- 5 < 1.

56

Let w(5,x) = A%-. We note that Y(5,k) -9 0 as é-—> 0.

A

Now from the remarks made on page 140 of Sethuraman [18],
it follows that the distribution Vn converges weakly to v.
As a consequence of lemma 2.5.3 and the remarks made at
the end of the lemma, we get that the sequence of distributions
induced by the processes DnVn(C) on D[-q,q] converges
weakly to the distribution u induced by the process W(C)
on D[-q,q]. .
Since Dn converges to 1 in probability, the follow-
ing theorem can be obtained by Slutsky's theorem generalized

to processes. (Rubin [16]).

Theorem 2.5.5.
The sequence ofgprocesses Vn(C) pg. [-q,q] converges
in distribution to the process W(C) 9n. [-q,q].

Furthermore

_1
C2 + Op(n /6) converges to C2 uniformly in C , since C

belongs to a finite interval and 0p is uniform for

C 6 l-q.q].
2 '1/6
Hence the process Xn(C) = Vn(C) - C + Op(n ) con-
verges in distribution to the process

X(C) = W(C) - C2 on [-q.q].

where W(C) is the Wiener process on [-q,q].

2:6 Asymptotic distribution of the maximum likelihood

estimator for increasing failure rate distributions:

In view of the result obtained at the end of section 2.5

57
and the lemmas 2.4.1, 1.4.2, 1.6.2, the following final
result can be obtained by methods analogous to those used

in Section 1.6 of Chapter 10

Theorem 2.6.1.
Let F(x) be a distribution with non-decreasinggfailure
rate r(x). Suppose that fn(§) is the M.L.E. of r(§‘)

based on n observations. Further assume that the condi-

 

tions in (2.4.1) are satisfied. Then the asymptotic dis—

 

tribution of

n1/ _SleyfiiJ—r' r ‘4-1/3 __1.__-_1_
3[ 2f<§ 1 [Enm rm]

has densipy

 

l Q.
2“.)
where Y is defined in (1.6.7)o

2.7 Asymptotic distribution of the maximum likelihood

estimator for decreasing failure rate distributions;

In this section, we shall give results for distributions
with decreasing failure rate. Let F(x) be a DFR distri-
bution with failure rate rkfl. Let in1x) denote the M.L.E. of
r(x). It was shown by Marshall and Proschan [14] that the
estimate fn(x) is conshmnnt and [i:n(x)]"1 is the slope of
the convex minorant of ¢n(Y) at Y = Fn(x),

where

j 3
¢> (—> ‘I [1 - Fn(y)]dy.

Xj being the order statistics of a random sample of size n

58
and Fn is the empirical distribution.
The following theorem can be proved by methods analogous

to those used in the case of IFR.

Theorem 2.7.1.

 

Let F(x) be a distribution with non-increasing failure

rate r(x). Let fn(g) denote the M.L.E. of r(x) §£_

 

x = g. Further suppose that conditions in (2.4.1) are

 

satisfied. Then

 

 

1/3 -2 C(g) ‘1/3 1 — 1
n {My} 2 1 [—--——. (g) rm]

n
is asymptotically distributed as the slppe of the convex
minorant of thegprocess W(t) + t2, -a)< t < oo §E_t = 0

where W(t) is the two-sided Wiener process with mean 0

and variance 1 per unit t and

 

2.2::

From Chernoff [4], we have the following theorem.

Theorem 2.7.2

 

The probability density function of E, the value of C
which minimizes W(C) + C2 where W(C) is the two-sided
Wiener_process with mean 0 and variance 1 per unit C i§_

W(C)
where Y is defined in (1.6.7).

From theorems 2.7.1 and 2.7.2, we have the following

result for DFR distributions.

59
Theorem 2.7.3.
L§£_ F(x) be a distribution with npn-increasinq failure
page r(x). Suppose that fn(§) is the M.L.E. of r(g)

based on n observations. Further assume that the condi-

 

 

tions in (2.4.1) are satisfied. Then the asymptotic distri-

bution of

 

-r' r ‘4 "If3 1 1
“1A” 12955)} 1 [sum “E731

 

has the density'
1/2 1(g)

where Y is defined in (1.6.7).

Finally we conjecture that similar results can be
obtained for the asymptotic distribution for estimates of
T(X) = ¢[F(X)]f(X)

when T is monotone and ¢ has a special known form.

CHAPTER 3

ESTIMATION OF THE LOCATION OF THE CUSP
OF A CONTINUOUS DENSITY

3.1 Introduction:

 

Chernoff and Rubin [5] and Rubin [17] investigated the
problem of estimation of the location of a discontinuity in
density in their papers in the third and fourth Berkeley
symposiums respectively. They have shown that the maximum
likelihood estimator is hyper-efficient under some regularity
conditions on the density and that asymptotically the esti-
mation problem is equivalent to that for a non—stationary
process with unknown center of non-stationarity. Daniels
[7] has obtained an asymptotically efficient estimator of
6 (modified maximum likelihood estimator) for the family

of densities

1

f(x,9) = exp {-lx-G|x},'§ < h < 1.

2r(1+1)

I
In this chapter, we shall obtain a hyper-efficient
estimator for 9 where 9 is a parameter determining the

family of densities f(x,9) given by

 

 

A
_ 8 x,9 x-G + g x,9) for x :.A
log f(x,9) -‘{g(x,9)| I ( 'for x > A
(3.1.1)

where
. = B 6 if x < 6
(1) g(x,9) {7(9) if x > 9,

6O

61
(ii) 0 < 7. < 1/2, a_nd (3.1.2)

(iii) 9 e (0:5) where -A < a < B < A. .'

We shall prove that hyper-efficient estimators, among
them the maximum likelihood estimator (M.L.E.), exist for 6
under some regularity conditions and that asymptotically the
estimation problem is equivalent to the estimation of the
location parameter for a non-stationary Gaussian process.

We obtain some results related to the asymptotic pro-
perties of the M.L.E. in Section 3.2. The estimation problem
is reduced to that of a stochastic process in Section 3.3.
The asymptotic distribution of the M.L.E. is Obtained in
Section 3.4. Section 3.5 contains the evaluation of

integrals encountered in Section 3.2.

3.2 Some results related to the asymptotic properties of

the maximum likelihood estimator:

Since our interest centers around obtaining the
asymptotic distribution of the M.L.E. of 9, we can assume.
without loss of generality, that the true value of the
parameter is zero.

We shall assume that the following regularity conditions
are satisfied by f(x,9).

(i) For each 9 # 0 e [a,B], there corresponds a 6(9) > 0
such that

E0[|:Eg|:'é(e)[log f(x,¢) — log f(x,0)]] < 0. (3.2.1)

(ii) For every ee:[a,B],

2
W)— , W exist and

592

62
2
Eo[[[§9é%Lgl]e_ol] < a: and EO[|§-§é%*gl[].i.k < 00

for all 9. (3.2.2)

0

... a 9
(111) Eol logégixi)le=o] = 0' (3.2.3)

(iv) 3(9) and 7(9) are twice differentiable at all 9 with

bounded derivatives. (3.2.4)

(v) [f(x,0) - f(0,0)|.i K |x|x for all x e [-A,A]
(3.2.5)
for some constant K.
We would like to mention here that even if the density
is given by
g(x,9)lx—eIM g(x,9) for -A z. x r. 3
log f(x,9) =
g(x,9) for x < -A and x > B
where conditions (3.1.2) and (3.2.1)—(3.2.5) are satisfied,
it can be reduced to the form (3.1.1) by suitably modifying
the function g(x,9) and the conditions (3.1.2) and
(3.2.1)-(3.2.5) can be shown to be satisfied by the new
density very easily.
Let Xi’ 1 fi.i fi.n be n independent and identically

distributed observations from f(x,9).

Let 9n denote the M.L.E. of 9.

Lemma 3.2.1.

The M.L.E. 9n is stronqiy consistent under condition
(3.2.1).

Proof:

Let S(9,5e) denote the interval (9—6 9+ée) where

e!
69 is given for each 9 by condition (3.2.1). Choose any

63
number n > 0.
Let
k
Lk(9) = _2 log f(Xi,9). . (3.2.6)
1-1
Let Q = [9: Q.: 9.: 5 ] fl [9: [9[.i q].
We notice that U S(9,5 )DSZ and S2 is
9€Q 9
compact. Therefore, there exists a finite set 91, ..., q“
in 0 such that
M .
U s(ei,5e )39- (3.2.7)

i=1 1

Now

Pol U {lékl > 3}] iPol U {Sup L149) > Lk(0)}1
kin kin 9 6 G

M
.1 2 Po[ U { Sup Lk(9) > Lk(o)}]
': > .
1 k_n e e s(ei.5ei) (3.2.8)
by (3.2.7).
Choose an e > 0.
Since E{ Sup [log f(X,9) — log f(X,O)]} < 0
9 e S(9.,6
1 9i

by condition (3.2.1), it follows by the Strong law of large
numbers, that there exists an integer N(9i,g) such that
k
P[ U Sup 2 [log f(X.,9) — log f(X.,O)]>< 0]
kin 9 e S(9.,é ) j=1 3 J --
1 9.
1
> 1 -

.E
M
for every n > N(9i,e), i = 1, ... , M.

Since

64

Sup n
e es(e ,56 ) 2 [log f(X.,9) - log f(X.,O)]
_ i j:1 3 J
n Sup
< 2 [lo f X.,9 - lo f X.,O ,
._j=1 e 6 5(91'59.) g ( J ) g ( 3 )1
1
we get that

U Sup . L (9)> (O)}] < £-

P[k:n 9 € S(9i,5e ) ki . *Lk.‘ .7 M
i

for every n > Max[N(9i,g), i a 1, ..., M].
Therefore, from (3.2.7) and (3.2.8) we get that
NU IW|> H<M£=
kin ’k U M E
for every n > Max(N(9i,g), 1.: 1.: M).
In other words,

9n is strongly consistent.

Let us now consider the log-likelihood ratio

L (9) - Ln(0) where Ln is defined in (3.2.5).

n
We have
n x 7‘
Ln(e) - Ln(0) = 21 [g(Xi,9)|Xi-9| -s(xi.0)|Xil ]
i=1
n
+ Z [g(Xi,9) ‘ g(Xi,0)]
i=1

where- 21 denotes that the sum is extended over those Xi

such that |X.| :.A.
1

2
Let 9'(X,9) = £35.91 and g"(x,e) = a 2162(3) '
6

65

Then by Taylor's theorem (in view of (3.2-2)),

L(9) L(0)'§[(X9 97‘ o A
n - n '- 1 E ii )lxi- ‘ -' g(xil )lxil ]
i=1
n
+ 9 Z g'(X.,O)
i=1 1
92 1 n
+-§ f (1-t) .2 §f(xi,9t)dt. (3.2.9)
0 1-1
n 92
= 2 w(xi,e) + n9 E[g'(X,0)] +-Jfi'e w + n —- v
1-1 n 2 n
(3.2.10)

{8(X69)|X-9|7\- g(x,o)[x[7‘ for [x[ :1

otherwise,
1 n
(...) wn - [fi (.21 g'<xi.o> - n E(g'(X:0))],
l:
and n 1
(111) vn =3; 2 fg"(Xi,9t)(1-t)dt. (3.2.11)
i=1.0

Since g'(X.,O) are i.i.d. random variables, condi-
tion (3.2.2) implies that Wn is asymptotically normal with
mean 0 and finite variance by the central limit theorem
for i.i.d. random variables.

Since g"(Xi,9t) are i.i.d. random variables, by
Condition (3.2.2), we get that

Vn = 0p(1).
Therefore, from (3.2.10), we have

Ln(9) - Ln(0)

II
II M3

W(xi,e) + n e E[g'(X,O)]

i 1
920 1

90(1)+n§ p().

:fi

66
The lemmas which we will prove next lead to the calcu-
lation of. E0[Ln(9) - Ln(0)], Varo[Ln(9) - Ln(0)] and

Varo[Ln(9) - Ln(¢)].

Lemma.3.2-2.'

Forany 9.<1> € [01.6],

2 < 2K+1

Eo[‘1’(X,9) - w(x,¢)] _B [e - ¢[

where .B is a constant independent of 9 and ®,

 

and
EO[Y(X,9) — 1(x,¢)12 = [e—¢|2“+1 [2c f(0,0) + 0(1)]
is; 9 —*> O and (D —> 0
where
P A+ F 2 2
c = 42,..qu '1“ [a 2(o > + v <o)—2a<o>y<o)cos m
2 In (2M1)
(3.2.12)
Proof:
Let us assume without loss of generality that 9 > ¢.
By the definition of Y (X,9),
A
=f [e(x 9)]X-el- (x, ¢ )-|x ¢|7‘] 2f(x,0)dx
-A
= T1 '1' T2 (3.2.13)
where
. A 1 1 2
(1) T1‘= f[e(x,e)|x-e| — g(X,¢)IX-¢| ] f(0,0)dx
and

A 1] 2[
(ii) 12 = f[e(x,e)|x-e|7‘ - g(x, ¢ )|x- ¢|] [f(X,O)—f(0,0)]dx.
-A (3.2.14)

67
Let n = 9 - ¢.

By condition (3.2.5),

[Talin[e(,)X9 lX-9I- (,x¢> [x <1>| 121x931
-A
.1 2K (T3 + T4) (3.2.15)
where
(1) T3 = f[e(x-q,¢)[x-e|- (x, ¢ )lx- -¢|x] 2 lxlx'dx

and

A
(ii) T4 = f lx-e|2”|x|A[e(x-n,¢) - e(x,e)]2dx(3.2.16)

'For x < 9, g(x,9) _ €(X-fl:¢) 3(9) ' B(¢)
7(9) - 7(2)-

Therefore, integrand of T4 is of the order

and for X > 9, 8(X,9) — e(X-q,¢)

[9-¢|20(]X-9|2%|x|9) Since 9(9) and y(9) have bounded
derivatives.

Since 9 belongs to a finite interval, it follows that

[e-¢|2 0(1). (3.2.17)
Let y
A
(1) T5 = f [s (x-n, ¢) M|x-e| (x, ¢ )[x- ¢|] f(O,Q)dX
and
(ii) T. = ? [(e(x,e)|x—e|k — e(x,¢)|x-¢|A

-A. -(€(Xrn,¢)lX-9| - e(x,¢)[x-¢]A]]2f(o,0)dx.
(3.2.18)

From the inequality
Itnum + v(x)]2.x]1/2-[).2(x).x11/2l 2 [(v2<x)dx1 1/2.

it follows that

68
IT11/2 _ Tsl/zl-i T61/2. (3-2-19)

From (3.2-18), we have
A 2 21
T6 = f [g(x,9) - g(x-n,¢)] [x-el f(0,0)dX .
-A

[e-¢|2 0(1), (3.2.20)
since 9 -belongs to a finite interval and 6(9) and
7(9) have bounded derivatives.

Let us now consider T5. We have

A/n
T5 = n2k+1f(0,0)A; [a (nY-n, ¢)[Y - -]k- 5(nY ¢)[Y - glx]2
- n

by the substitution X = nY,

A-CD

n2A+1f(0'0) ? [s(nZ+¢-n,¢)[Z-llx-e(nz+¢,¢)lzlx]2dz
-A-¢
n

by the substitution Y = Z +-%,

A- n¢
= 32k+1f(0 0)fn [h(z-1)|z-1|A- h(z)|z|7‘]2 dZ
-A-¢
n
B(¢) if Z < 0
‘Where h(Z) -{7(¢) if Z > 0.
Therefore,
00
T5 = n2A+1f(0,0) f [h(ze1)|z—1|A- h(z)|z|7‘]2 dz
‘(IJ
- q2A+1f(0.0) f [h(z-1)|z-1|x— h(z)|z|k]2 dz
2 4 (3539-. 5:3)

n

69

2A+1

= n f<0.0) 2c<¢)-n2x+l

f(O,O)f[h(Z-1)[Zellx-h(Z)[le12dz

-A-¢ A-¢
z ¢ ( n n ) )3 2 2 )

P(A+1)r(%-A).

 

[Bz(¢)+V2(¢)-ZB(¢)V(¢)cos w A]
(3.2.22)

where C(¢)=
222+lf%(2k+1)

(The integral will be evaluated in Section 3.5, lemma 3.5.2).
Since 9 and ¢ belong to a finite interval,.it
follows from (3.2.21), that for any 9 and ¢' in [a,B]

T5 = n2x+1 0(1) (3.2.23)

and for 6 and ¢ in [a,B] such that ¢'-> 0 and n —# 0

2X+1 2k+1
0(

T5 = 2 c(o) n f(0,0) + n 1). (3.2.24)

Let us now consider T3.
We notice that
A A A A 2
|T3|.: |A| f [e(X-n,¢)|X-9l - €(X,¢)|X-¢l ] dx

< AKT
—'f'©,0) 5 '

Therefore, from (3.2.23), it follows that for any. 9 and

¢ in the interval [a,B],

Ta = n2x+l 0(1). (3.2.25)

On the other hand, for 9 and ¢ such that
n-> 0' and ¢ —> 0, let us evaluate T3.

A-¢

Now T3 - n2k+1 P [€(nZ+¢—n,¢)|Z-1|x-e(nz+¢,¢)lZIx]2|¢+nZ|de
-A-¢
n

Let Q¢ T1(Z) denote the integrand in the right hand

 

side. We observe that as ¢ and n —> 0, the range of

70

,integration tends to (-oo,a>). We note that
62(¢)[IZ-llk-Izlk]2|¢+n2)k for z < o,
Q¢’n(Z) = y2(¢)[[z-1|k-|zlx]2|¢+nz|x for z > 1,
[B(¢)[Z-llx-7(¢)(Z|k]2|¢+nzf\for o < z < 1.
By condition (3.2.4), B(¢) and y(¢) are bounded.

Let c1 2 Sup [IB(¢)|. lv(¢)|}.
¢€[d,B]

Therefore the integrand Q¢ T1(Z) is bounded by

I

c12[|z-1|7‘-|z|%]2|A|A for z < o,
Q(Z) = clzuz—lfl—IZIMZIA)x for z > 1,
2
c12[|z-1| K+C12|Z[2k+2C12|Z-1|%|Z|x]IAIA

forO <Z,< 1,.
for all o and n.

Further Q(Z) is integrable over (-oo,oo) since
0 < A < 1/2, and Q¢ n(z) —» o as ¢ —> o and n —» 0.
Therefore by the bounded convergence theorem,

A-¢
n
f Q¢ HZ) dZ —> 0 as ¢ -¢ 0 and n-—> 0.

-A-¢
W

In other words, for 9 and m such that n —> O and
¢ —> 0,
2
T3 = 3 3+1 0(1). (3.2.26)

From (3.2.19) and (3.2.20), it follows that

Ti/z = Tg/Z + |9-¢| 0(1).

Therefore, from (3.2.23), we have

71
T1 - T5 + [9— ¢[ Tg/z 0(1) + {9- -¢[2 0(1)

= T5 + [e—¢|x+3/2 0(1) + le-¢|2 0(1). (3.2.27)
Let us first.consider the general case when 6 and ¢ are
any numbers in [a,B].
Now

E0[Y(X,9)—Y(X,¢)]2 = T1 + T2 by (3.2.13),

.1 Tg+[e-¢|x+3/2 0(1)+|e-¢|2 O(1)+2K(T3+T4)

by (3.2.15) and (3.2.27),

IA

n2k+1 0(1)+n7‘+3/2 0(1)+n2 0(1)

23+l 0(1)+n7‘+3/2 0(1)+n2 0(1)]

+ 2K[n
by (3.2.23), (3.2.25) and (3.2.17),

T127x+1

= [0(1)+n /2‘* 0(1)+n1-2k

0(1)]

= n2k+1 0(1) (3.2.28)

since 0 < A < 1/2 and n is in a
‘ finite interval.
Suppose in addition that G .and ¢ approach 0.
Then

Eo[w(x,¢)-w(x,¢)]2 = T1 + T2

T5fi9-¢|x+3/2 0(1)+|e-¢|2 0(1)

+ n2” 0(1) + n2 0(1)

by (3.2.26),(3.2.27),(3.2.15) and (3.2.17)

qzx+1[2c f(O, 0)+o(1)1+n“‘°‘/2 0(1)

2h+1

+ n2 0(1) + n 0(1) by (3.2.24),

72
n2A+1

[2c f(O, 0)+o(1)+r]"-7\1/2 0(1)

n2x+1[2C f(O 0)+ 0(1)], (3.2.29)

since 0 < x < 1/2.

(3.2.18) and (3.2.29) together prove the lemma.

Let
¢(x e) = {e 5(X,0) x Sgn x |x|7"1 for |x[ :.A
’ 0 Otherwise
(3.2.30)

Lemma 3.2.3-

1+2k

E0[Y(X,9)+¢(X, =|9| [-C+o(1)]f(o,o)

+ f[e(X,9)-€(X-9,0)]IX-Gle(X,0)dX
-A

where o(l) is in 9 and C is given by (3.2.12).

Proof:
Let us suppose that 9 > 0.
By the definition of Y and w,

E0[Y(X,9) + ¢(x,e)]

A
f[e(x,e M|x-e| (x, 0) )|x|
-A

+ e e(x,0) x Sgn x|x|x_1]f(x,0)dx

T1 + T2 (3.2.31)

where

(1) T1 - ‘f[e(x-e, 0) M|x-e| (x, 0) )|x|

+ e e(x,0) A Sgn x|x|k‘1]f(x,0)dx

and

73
A
(ii) T2 = f[e(x,9) - g(x—e,0)][x-e|‘f(x,0)dX(3.2.32)
-A .

We now have

9
T1 [911+X7; [g(x-1,0)|x-1|x- g(x,o)|x[‘

-A 9
. + x"1
.. k 5(X,0) Sgn X|X| . ]f(x6,0)dX

1+X(T3 + T4)

[9|

where A/e x x
(i) T. = f te<x-1,0)Ix-1I - e(x,0)|x|
A/e

.+ A.e(X,O) Sgn x‘xlfi‘11f(o,o)dx

A/e x i
(ii) T = f [8(X-1,0)|X-1| - 5(X,0)|X[
-A/e
+ h 8(X.0) Sgn X [XIX-1][f(X9,O)-f(0,0)]dx.
(3.2.33)
we have .
_ 0° _ x_ A x-1
T3 — f(0,0) f [e(X-1,0)|X 1| e(x,o)|x[ +xg(X,O)Sgn x|x| 1dx
_m ,
k A
- f(0,0f f [a(X-1,0)|x-1| -e(x,0)|x|
x|:A/e
+ x e(X,O) Sgn x |x|%'1]dx
= —f(0,0f f [€(X-1,0)|X-1|x-e(x,0)[X|x
x|:A/e

+ A e(X,O) Sgn x |x|x"l]dx

by lemma 3.5.1 of Section 3.5.
Since the integrand on the right hand side of the above
equality is of the order 0(|X|x-2),

T3 = 0(1) |e|1‘* . (3.2.34)

74

Let us now evaluate T4. Let f = f(0,0)
From (3.2.33), we have
A/e
T4 - f [e(X—1,0)|X-1|xeX,O)IXI
- /6
k-l
+ x e(X.0) Sgn x | | ] x 0 [xel de
A/e
+ f [g(X- 1, 0) )[x- 1| €(X,O)|X|
-A/
+ k 8(X,0) Sgn X I Ix-l]
[f(X9,0)-f(0,0)-fe(x,0)[Xelx] dX.(3.2.35)
Let t
(1) T5 =f|9|k f [5(X- 1) olx— 1| - s(X,0)lX[x
+ k e(X,O) Sgn x [XIX-1]5(X,O)[X[A
.. A k
(11) T6 =-f|9|x [e (x-1,o)|x-1| - e(X,O)|X|
IXIIA/e -
+ 1 g(x.0) Sgn x |XI"'11 g(x.0) [x|"
and
A/e
(iii) T7 = f [g(X- -1, 0)|x-1| - g(x, 0)|x|
_A/e
+ x s(X,0) Sgn x |x|x‘1]
A
[f(xe,0) - f(0,0) -fe(x,0) [Xe] ] dx.
We have
T4 = T5 + T6 + T7. (302.36)
By lemma 3.5.4 of Section 3.5,
T5 = -Cf|6|x (3. 2. 37)
2A- 2)

Since the integrand of T6

is of the order O(|X|

75

T6 - [9'1 (9(1’2*o(1). (3.2.33)
Let us now evaluate T7.
We notice that the range of integration tends to (-oo,a))

as G -> 0.

Furthermore for each X,

fixem) — no.0) - fetx.0)Jx9Jl_> 0

w—

as G —9 O

191‘
. . R A
and the integrand of T7 15 bounded by C1 [Xl [9|
for each X, and is bounded by C2 lex‘2 for X large
for some constants C1 and C2.
Therefore, by the bounded convergence theorem,
T7 = |9|ko(1). (3.2.39)

From (3.2.34) - (3.2.39), we have

T3 + T4 = |e|1‘x 0(1) - Cf|9|k + |e|k 0(1) (3.2.40)
Therefore, 2

T1 = |9|1+A[-Cf|9|k + |e|“ 0(1) +-;ell‘“ 0(1)]

_ |e|1+2‘[-CE+ 0(1)].

Now, from (3.2.31) and (3.2.32), we have

Eo[‘1’(X:9) + ¢(x,e)]

1+2x .. A x
|e| [—cf+ o(1)]+ f[e(X,9)—e(X-9,0)]|X-9| f(X,O) dx.
-A (3.2.41)
Now, if 6 —a 0, then
1+2N-Cf+ 0(1)]

E0[Y(X,G) + ¢(x,e)] = |e|
. A x
+ f[e(x.9)- 5(X—9,0)]|x-e| f(x,0) dx (3.2.42)

-A

since 0 < A < 1/2, (3.2.42) proves lemma 3.2.3.

76
Using the results obtained in lemmas 3.22 and 3.2.3, we
shall compute E0(Ln(e) - Ln(0)], Varo(Ln(9) - Ln(0) and

Varo (Ln(9) - Ln(¢)) in the following lemmas 3.2.4 and 3.2.5.

Lemma 3.2.4.

 

1+2}

E0[Ln(9) - Ln(0)] = - n Cf|e| [1 + 0(1)]

where o(l) is in 9 and C is given in (3.2.12), and in

 

general, for any 9 e [a451,

EO[Ln(6) - Ln(0)].i -n H |e|1+zx

where H is a constant independent of n and 9.

Proof:
Let us assume that 9 > 0.

We have

EOILn(9) - Ln(o>1

n E[log f(X,9) - log f(X,O)]

n E[Y(X.9)] + n E[g(x,e) - g(X.0)1

n E[Y(X,9)] + n E[9 g'(X,0)

e2 1
+7-g(1-t)g"(x,9t) dt]

n E[Y(X.9) + 9 g'(X,O)] + n 92 0(1)

by condition (3.2.2),

n E[w(x,e) + 4(x,e)] + n 92 0(1)

+ n E[e g'(X,O) - ¢(x,e)]. (3.2.43)
Where ¢ is defined in (3.2.30). 2
Let

(1) T1 = E[W(X,9)+¢(X,9)]-?[e(x,9)-8(X-9,0)]|X-6|kf(X,O)dX
-A

and

77

(ii) T2 = E[6 g'(X,O) "' ¢(X,9)]

A
+f [g(x,9) - 6(X-9.0)] lx-el7‘ f(x,0) dx.
‘A ' ' (3.2.44)
From (3.2.43), we have .

EO[Ln(G) - Ln(0)] = n T1 + n T2 + n 92 0(1). (3.2.45)

From (3.2.3), we have

a log f(X,9) =
EO{ Be 9:0} 0°

 

Therefore,
A A 1-1
I [s'(x,0)|x| - x g(x,0) Sgn x |x| ] f(X,O) dX
00
+ f g'(X,O) f(X,O) dX - 0.
-oo
In other words,
A
E[e g'(X,0) - ¢(x,e)] = —e fg'(x,o)|x|A f(X,0) dx.
-A
From (3.2.44), it follows that
A 1 1
T2 = f([e(x,e)-g(x-e,0)]|x—e| -ee'(x,0)|x1 )f(x,0) dx.
-A
We note that T2 = 92 0(1) since 9 is in a finite interval

and B(6),7(9) have bounded second derivatives.
Therefore from (3.2.45), it follows that
2
E0[Ln(6) - Ln(0)] = n T1 + n 9 0(1).
By lemma 3.2.3, as 9 approaches zero,

I1+2)\

T1 = |9 {-c + o(1)]f.

Therefore, if 9-—> 0, then

78

l1+27\

[e [-Cf+ 0(1)] (3.2.46)

0
I

since 0 < A < 1/2 .
In other words there exists a number n > 0 such that

Cf

E0[Ln(9) — Ln(0)] : _ n 5- lel1+21 (4)

for all 9 such that 9 e [a,B] and [GI < n.
Let us now consider the set

Q = [9: a.: 9.: s]n[e: 9|.: n]-

 

Since this is a compact set, there exists a finite set

61, ... , 6 in 0 such that

m
m < >
use,6 an
1=1 l 91
where S(6i,5e.) denotes the interval (Bi-59 ,6i+5e )
1 1 1
and be is given by condition (3.2.1).
i
Therefore

Sup Eo[log f(X,6) - log f(X,O)]
GER

i. Sup E Sup {lo f(X,9)-log f(X,O)]
' m e e S(ei,ée )
i

by condition (3.2.1).
Since (9|.Z n > 0 , we have now

Sup Eo[ log f(X,9) - log f(X,O)] < 0 .
9 s Q |e|1+2h

Let D - - Sup E0 {log f(x,e) - log_fgx,0)

9 € 0 |e|1+27\ ‘

Notice that D r O.

79

Then for every 6 s Q,

E0[Ln(9) - Ln(0)] = n E[log f(X,9) - log f(X,O)]
1 - nD|e|1+2)‘.

This, together with (*), implies that

1+2}

E0[Ln(9) - Ln(0)].: - n H [9| (3.2.47)

for every 9 in [d.B],

where H is greater than zero°

Lemma 3.2.5.

For any 9 and ¢ in the interval [a,E]:

Varo[Ln(G) - Ln(¢)] :.n Q [e-¢|2“+1

where Q is a constant independent of 9,¢ and n,

and

Varo[Ln(9) - Ln(0)] = 2 nC f(0,0) [e12k+1(1 + 0(1))

where o(l) is in 6 and C is given by (3.2.12).

Proof:

Since Xi’ 1.: i.i n are i.i.d. random variables
Varoan(9) - Ln(0)1

= n Varo[log f(X,G) — log f(X,O)].
(3.2.48)

Let us now compute for 9 -> O,

E0[log f(X,6) — log f(X,O)]Z

2

Eo[Y(X:9) + g(X.9) - g(X:O)J

Eorw(x.e>12 + Emma) - g(x.o>12

+ 2 Eo[Y(X:9){g(X:9) - g(X:0)}]-

T1 + T2 + 2T3 (3.2.49)

80

where
(1) T1 = E0[Y(X,9)]2,
(ii) T. = notg<x.e) - g(x.o)12,
and ' '
(iii) T3 = Eo[1(x.9){g(x.9) - 9(X:0)}]o

Note that 12 = 92 0(1) by condition (3.2.2) and

 

T1 = lelgx+1 0(1) by lemma 3.2.2.
Further
3+2k
[T3] : JT1T2. Therefore T3 = [6| 2 0(1)°

Therefore, from (3.2.49), we have for 9-—> 0

Eo[log f(X,G) - log f(X,O)]Z

3+2}

27‘+1+|e|_§m0(1)+[e|20(1)

= [2 C f(0,0) + 0(1)] (9|

|9|2x+lt2 C f(0:0) + 0(1)] (3.2.50)

since 0 < A < 1/2.
Now

Varo[log f(X,6) - log f(X,O)]
a Eo[log f(X,9)-log f(X,O)]Z-[Eo[log f(X.9)-109 f(X,O)]}Z

I2M1 ' 2

= [9 [2c f(0,0) + o(1)]-{—c[e|1+zx[1 + o(1)])2 f

by (3.2.50) and lemma 3.2.4,

2h+1

)9) [2c f(0,0) + 0(1)].

Therefore, from (3.2.48), we have
varoth(9) - Ln(0)1

_ n Iel2M1

2c £(0,o)[1 + o(1)] (3.2.51)
Let us now consider Varo[Ln(9) - Ln(¢)] for any 9,¢ in

[a.B]-

81
Obviously
Varo[Ln(9) - Ln(¢)]

:_n E0[log f(X.9) - log f(X, )12

= n Eo[{‘i’(X,9) - 2(x.¢)} + [g(x,9) - g(X:¢)}]2

2h+1

2}

.1 2n [B|e-¢| + le—¢|2 0(1)}

by condition (3.2.2) and lemma 3.2.2,
+
1 n Q (9-4%“ 1 (3.2.52)
where Q is some constant since 9 and ¢ belong to a

finite interval. (3.2.51) and (3.2.52) prove lemma 3.2.5.

We shall now prove a theorem Which enables us to con-

clude that the probability, that the maximum of Ln(9) - Ln(0)

1 1

18 attained out31de the interval [—K n -'szi' K n ‘ 1+2h],

approacheszero for K sufficiently large. More precisely,

Theorem 3.2.6.

There exists n > 0 such that

1 Mn(9) > _ o
“1427. 21+1 - '71] ‘
n|e|

 

 

lim lim Po[ Sup
T-700 n |9|>Tn

where Mn(9) = Ln(9) - Ln(0).

Proof:

Since Mn(9) is continuous in 9, it is enough to prove

that

Mn(9)

 

lim lim P0 [ Sup -f£2i
T—>oo n leijkl>Tn nleijkl

82

 

 

 

1
for some set {eijk) dense in {9 : [9] > Tn 1+21 ].
1 . k.
+‘J
_ 1+2x 1 2 _
Let eijk 2 for 1 — O,1,2,...
j = O,1,2,...
k = O,1,2,...,23-1.
_ 1 (3.2.54)
Obviously eijk is dense in {6: 9 > n 1+2) ].
We shall prove (3.2.53) when 9 ranges over
_.;l__
{9: 9 > Tn 1+2K }.
The proof is analogous when 9 ranges over
_.l;_.
{9: 9 < -Tn 1+2K ].
Let y = 2% + 1.
Let us now define
= _ V
Tn(eijk) Mn(eijk) E[Mn(9ijk) + nH eioo], (3.2.55)
where H is defined in lemma 3.2.4.
- 7 < 7
Since 6100 _.eijk and
< _ Y < _ V
E[Mn(eijk)] -' “H eijk -' nH eioo’
it follows that
T 9.. M 9.. ~
.;E£_£IE1.:._E£_$JEE.. (3.2.56)
7 97
ioo ijk
Therefore
M (9.. )
1 k
Po[ Sup—37 n7 1 Z. 'Tl]
9 jk>Tn neijk
T (6.. )
:.Po[ Sup-l- n 13k :.-n] (3.2.57)
9 . >~cny 97

83

From (3.2.55),

z _ v = _ 7 iv
)] nH eio H T 2

130[anio o

o
and

E0[Tn(9iij’2k+1) - Tn(Gilj_1’k)] = 0. (3.2.58)
Now

Var[Tn(9. )] = Var[Mn(6

)1

100 ioo
:.n Q @100 = Q Tyzly' (3.2.59)
Let us now compute
var°[Tn(ei,j,2k+1) ‘ Tn(ei,j-1,k)]
= var°[Mn(ei,j,2k+1) " Mn(ei,j-1,k)]
< _ 7
_.n Q|ei,j,2k+1 ei,j-1,kl by lemma 3.2.5,
. 2k+1 ..EI
. "——'I'— '_ 'Y
:.Q TV 217 I2 23 _ 223 |
2k+1 y
. j _.
= Q 3V 2W 2 2 )1 - 2 3|7

IA

° 7
Q T7 2(1+1) [$993217
2

)7.

= Q TV (2 log 2)y 2(1'j (3.2.60)

We observe that

< _ 7
Tn(eioo) _' nC eioo
and
_ < 7
Tn(ei,j,2k+1) Tn(ei,3—1,k) “npjeioo

for all i,j and k = o,1,...,23‘1-1

imply that

84

< _ Y -
Tn(9ijk) _. nneioo for all 1
provided
a)
— g + z p..: -n- (3.2.61)
1'1 -

We shall choose C > 0 and sequence Pj suitably at the

end so as to satisfy the condition (3.2.61).

Tn(eijk) .: _ 9]

Therefore, Po[ Sup _._l__ -
>Tn 1+2x “9:00
ijk
00 v
>
i .2 P[Tn(eioo) _ -nC 9.1.00]
1-0
j-l
(1) C0 2-1 V
_ <
+ 2 2 z PlTn(9i,j,2k+1) Tn(9i,j-1.k)-“Pjeioo]

i=0 j=1 k-o

 

 

< 3; 4g 1721V . +C§ Cg 23‘119‘fl217’(21og2W2"3y
i-o [H-c12t27221y i=0 j=1 p? 127 22”

by (3.2.58), (3.2.59), (3.2.60) and Chebyshev's inequality,

CD

2
(H-C) Ty i=0

oo _. yoo . _ _
+,£%{ z 2 17] (2 139 2) 2 23(1 7) 9.2

T i=9 j=1 j

gL 1 [ 1 + (2 log 2)7 0; 2j(1-y) 2'2]

 

 

Ty 1-2-Y (H-Q)2 2 j=1 3
(3.2.62)
_lfi_
Let us choose 0 < C < H and Pj = 2 2 o where 6 > 0
and .
oo 5.5k/2

= —ZP,— — . .
n C 1 3 C 1:3:775

85

Then, from (3.2.72).

 

 

 

 

 

 

 

 

 

 

Sup Tn(eiik) >
Po[ 1 —"T]]
1 n9?
9.. >Tn +2h 100
13k
<_Q_ 1 [1 +L21032)7 1 1.
TV (142—7) (H—zg)2 23+162 1-2‘3
(3.2.63)
Therefore, by (3.2.57)
M (9.. )
11m Po[ Sup 1 n yljk > ‘2]
n -- ..
7 13k
eijk>Tn
< 11. 1 [ 1 + (2 log 2)7 1 l
17 1-2"7 (Hr 02 2M1 62 1-2““
5.2-1/2
where C < H, 6 > 0 and n = C - and y = 2h+1.
1_2-h72
Taking limits as T-—> a), we get that
M (9.. )
lim lim po[ sup 1 n 11k 3-‘91 = o
7
T n —-— n9..k
9.. >Tn y 13
13k

since y > 0.
This proves theorem 3.2.6 in view Of the remarks

made at the beginning of the proof.

3.3 Reduction to a problem in stochastic processes:

We shall reduce now the problem of determining the
asymptotic distribution of 9n or equivalently the asymptotic

distribution of the maximum of Mn(9) to that of a limiting

86

process. In view of theorem 3.2.6, we can restrict our
1 1

attention to intervals of the type [-Tny, Tny] where T > 0
and 7-: 2% + 1 in order to locate the maximum of Mn(9).
For T > 0.

1
let Xn(2;) = Mn(n_ —1+27\C) for C 6 {-3.3}, and X(C)

be the continuous normal non-stationary process on {-1.1} with
E[x(c)1= -c|c|7f(0.0).
Var[x(q)]= 2(2f(0,0) |§|V,

and Cov[x(ci).x(:2)1 = c f<o.o>[Icily+lczlyelci-czlyl7

 

(3.3.1)
1 / - 1 2 2
Where = PfiziiiJ;(?2:+l)llB (0) + v (0) - 26(0)7(0)cos wk]-
Let
An(C) = Xn(C) - E(Xn€))
and A(C) = X(C) - E(X(C))- (3-3-2)

Theorem 3.3.1.

For any C,e [-T,T], An(C) is asymptotically_normal

with mean 0 and variance 2C f(0,0) |Q|7.

 

Proof:
By definition

An<c> = Mn(9) — E[M(9)1

n
where 1

e = n 1+21 C-

Let

An(c) = Bn(c) + cn(c)

where
B (c) = ; [5(X e)|x —9|)‘-s(X 0)|x [A]
n i=1 i’ i i’ i
- n E[¢(X,9)]
and
n.
Cn(C) = '2‘ [9(X1:9) - 9(X1.0)]-n E[9(X.9)-9(X.0)1
1"1

where 21 denotes that the sum is extended over only those

X.'s for which IXiI §.A.

 

1
Obviously
Eo(Cn(C)) = 0
and
2
varo(cn(c))-i n Eo[g(x.9) - g(x.0)1
.: k n 92 by condition 3.2.2
__2__
= k 1+21 C2
21-1
= k n2M1 C2 .

Since 0 < x < 1/2, it follows that
Varo (Cn(C)) —> 0 as n -> 00.
Therefore,
Cn(§) converges to 0 in probability as n -> 00.
Hence An(C) and Bn(C) have,the same asymptotic distri-
bution by Slutsky's Theorem.

Let F; be the distribution function of Bn(C) and

 

 

§(x) be the normal distribution with mean 0 and variance 1.

Let

Y = w(x,9)

88

where

c and w is as defined in (3.2.11).

n
Bn(c) = E [Yi - E(Yi)]
where Yi are i.i.d as Y.
By the normal approximation theorem, (See Loeve [13], pp. 288)

* x - x i. .90 . n - 3.
[Fn ( ) §( )I [Var ’Bn(C)]"7‘§ E[Y E(Y)I

where C0 is a numerical constant.
It can be easily shown by methods analogous to those used

in lemma 3.2.5, that

EIY - E(Y)|3 :. c1[e|37‘+1

where C1 is a constant independent of 9.

3k+1

- 1+2} 37‘ 1
Therefore, IF;(x) - §(x)|.i CO n C1 n [Cl '+
{2c f(0.0)ch7[1+o(1)113/2
A

- 1+2?\
= COC1|C| 3A+1 n

{2c f(0,0)|§|y[1+o(1)]):7;

This term tends to zero as n -> 00 since A > 0.

Therefore, _En££2____. is asymptotically normal with mean 0
JVar(Bn(C)

and variance 1. But we note that
Var (Bn(c)) = 2c f(o,0)|c|7[1 + 0(1)],
from the proof of lemma 3.2.5.

This establishes that Bn(C) is distributed asymptotically

89

as normal with mean 0 and variance 2C f(0,0)|C|y. Therefore,
An(C) is distributed asymptotically as normal with mean 0
and variance 2C f(0,0)[§[y.
Remark: By the normal approximation theorem again, it can
be shown that for any real numbers a1, ..., ak and
C1: C2: ..., Ck in [‘TIT]:

k .
.2 anAn(Ci) is asymptotically normal with mean 0
i=1

and variance

 

k2 kk
2c f(0,0)[ 2 ai |§i|7+ 2 z ai
i=1 i=1 j=
i<j

aj[lCi 17+|Cj ly‘lci-Cj [7}]

The next theorem shows that the process An(C) on

[-T,T] satisfy an equi-continuity condition.

Theorem 3.3.2.

 

For any C1,C2 if; [’TIT]!
EIAn(c1) - An(C2)|2 291:1 - czly

where Q is a constant independent of n,§1,§2 and

7 = 21 + 1.

Proof:

By the definition of Xn(C)

313nm.) - An<c.)12 - varomnw.) - we.»
1

where 91 = n Y C1 and 92 = n V C2.

H

But from lemma 3.2.5,

90
Varo [Mn(91) - Mn(92)] _<_ n Q [91 - 92))”.

Therefore

E[An(C1) ‘ An(C2)]2 j-nQ n-llcl'C2ly

= Q |C1 ‘ Czly-
From Kolmogonmfs theorem, (see Doob [8]) the process An(§)
has trajectories in C[-T,T] by theorem 3.3.2.
We shall now state a theorem connected with convergence
of distributions of stochastic processes on C[a,b], where
C[a,b] denotes the space of continuous functions on [a,b]

with supremum norm topology.

Theorem 3.3.3.

 

Let Xn be a sequence of stochastic processes on
C[a,b] and X be another process on C[a,b] suCh that
(i) for any ti 6 [a,b], 1 i.itj.k, the joint distri-

bution of [Xn(t1, ..., X tk)] converges to the joint

n(
distribution of [X(t1), ..., X(tk)], and
(ii) there exist. constants A,B,C, > 0 independent of

n such that for every n

A 1+B
E|Xn(t1) - Xn(t2)| < c t1 - t2| .

Then the sequence of processes Xn converge in distri-
bution to the process X.

For a proof of the above theorem, see theorem 2.4 of
Sethuraman [18].

Theorem 3.3.1 and the remarks made at the end of its
proof together with theorem 3.3.2 imply the following result

in View of theorem 3.3.3.

91
Theorem 3.3.4
Thejprocesses- An(§) gg_ [-T,T] converge in distribu-
tion to the process A(§) on [-T,T].

Therefore we have

Theorem 3.3.5
The process Xn(C) 92_ [-T,T] converge in distribu-

tion to the process X(C) ‘QQ [~T.T].

2.422;:

Since E[Xn(§)] = -c |c|7[1 + 0(1)]f
where 0(1) is uniform for C E [-T,T] as n'-> a) by
lemma 3.2.4, and since

E[X(C)] = -CE|C|7, it follows that

E[xn(c)1 -2 E[x(c)1 as n —2 co
uniformly for C in the interval {-1.1}.

Therefore, by an extension of Slutsky's theorem to
stochastic processes (See Rubin [16]), it follows from
theorem 3.3.4, that the process An(§) + E[Xn(C)] converges
in distribution to the process A(§) + E[X(§)].

In other words,

The process Xn(C) converges in distribution to

the process x(C) on [-T,T].

For any x e C[-T,T], let g(x) be the value of t
that maximizes x(t) over [-T,T]. Obviously g(x) is a
continuous functional in the supremum norm topology on

C[-T,T], provided x has a unique maximum.

92
Therefore by theorem 3.3.5, the distribution of
g(Xn(C)) converges to the distribution of g(X(C)) for

C E [-T,T]. Hence we have the following theorem.

Theorem 3.3.6.

The distribution of the position of the maximum of
' 1 1

M 9) over [-Tn 7, Tn v] converges to the distribution

n(
of the position of the maximum of non-stationary Gaussian

process X(C) defined in (3.3.1) over [-T,T].

The next theorem proves that the process X(C) over
(-oo,oo) has its maximum in a finite interval with probability

one 0

.Theorem 3.3.7

Prob [lim SUP -§Jll-i.-1] = 1
[Tl-eoo Clrlyf

where C is given in (3.2.12).

Proof:

We shall first prove that

Prob[llm sup EJII.:__1] = 1.

T—'>+CD CTyf
Define 'A(T) = X(T) - E[X(T)]
=X(T) +c|T|7£. (3.3.3)
Let
Sup A(T)
ZO = and

1 :.T §.2

93

Sup
2 = A f - 0,1,2, ...
n 2n.: 4.: 2n+1 (T) or n
_ Sup IA(T) - A(1) (3.3.4)
and U - 1 :.T fi.2 I

Since A(T) is normally distributed with mean 0 and
variance 2C f(0,0)[T|y and covariance of A(T1) and A(T2) is

c f(0,0)[|T1|7 +|72|7 - [Tl-Tzlv], it follows that zn
21

and Z0 22 have identical distributions. Therefore, for

any 5 > O,

21
p[zn > c 2n7] = p[z0 > e 22 1. (3.3.5)

Let k = C f(0,0).
We note that k > 0 and 1 < y < 2.
Since A(T) is continuous on any finite interval with

probability one and since dyadic rationals are dense in [1,2],

-1 s
U = Sup A(§-9 - A(-—9
| 2, 2, l

with probability one.

 

Therefore
00
U i. 2 T2 with probability one (3.3.6)
£=1
s-1 5
where TE = 2 Sup 3+1 |A( 22) - A(2£)|.
2 + 1.1 5.1 2
Now for any a > 0 and 1 > r > 0
P[T£ > arg]
_ Sup 44“) - Min > an: 1
‘ P[ z z 1 23 22
2 + 1 2.5 i.2 +

 

 

 

 

 

2+1
< 2 5-1 s E
_ z P[|A(£)-A(—Z)I>ar.]
+1 2 s-1 5 E
<22 2 > P(IA(——) - 3(7)) > a.)
2 2
. 5-1 s . .
for some 5 Since A( 2) - A(—Z) are i.i.d as normal
2
with mean 0 and variance k 2 -37.
Therefore, by Chebyshev's inequality
1-27
B 2
P[TE > ar ]_<_ 2'2 m k. (3.3.7)
a r
Now, from (3.3.6), we have
Prob[U > lfrl i.Prob [TE > arg for some 1 1.3 < 00]
0° 2
i. Z P[Tg > ar ]
i=1
5. a; 217’ 21-“ k
£=0 a2r2£
by (3.3.7) I
_ 2 1
— 2 k l-y °
a :2
(1‘ 2 )
r
-Ll
Let r = 2 4 .
Therefore,
P[U > a ]:2—2k 1 .
_.1 a 1-3L
1-2 4 1-2 2
Equivalently
P[U > a] 1—2 (3.3.3)
a
where D is a constant.

95

Therefore
00
E(U) = f P(U > a)da
0
D> a)
- f P[U > a]da + f P[U > a]da
0 D
00
:D +f Eida
D a
by (3.3.9)!
fi.D + 1 < a) . (3.3.9)
Since

IZOI j |A(1)| + Sup [A(T) - A(1)|
< :2

= |A(1)| + U,

Elzol : E(|A(1)])+ E(U) < 00

by (3.3.9).
Let EIZOI = J.
Now
00 oo ‘
z P[zn > 5 zny] = z P[zO > g 2n7/2]
n= n=
by (3.3.5):
00
J 1 '
.1 Z - by (3.3.9)
n=0 8 2n7 2
and Chebyshev's inequality,
= g--z‘-----l‘--7'-- since 1 < y < 2.
8 1_2-Y 2
Therefore
00 n
z p[zn > a 2 y1 < so for all e > 0.
n-O

Then, Borel-Cantelli lemma implies that

96

P[Zn > s Zny infinitely often] = 0 for every. 5 > 0.

In other words

 

Z
Prob [lim Sup 2 1.0] = 1. (3.3.10)
n 2 7
Z
Since J—A T) 1 __n if 2n 1 T 1 2n+1
Ty _.2n7 -_ I

it follows that,

Prob[ lim Sup AllL10] = 1.
T -> 00 TV

Since A(T) = X(T) + CTyf for T > O,

we have

Prob [ lim Sup 5131'1.-1] = 1. (3.3.11)
T'—9+OO CTyf

Similarly we can prove that

Prob [ lim Sup élll— .1 -1] = 1. (3.3.12)
T—> -co C[lef

(3.3.11) and (3.3.12) together prove the theorem 3.3.7.

3.4 Asymptotic distribution of the maximum likelihood
estimator :
From theorems 3.2.6, 3.3.6, and 3.3.7, we get the follow-

ing final theorem.

Theorem 3.4.1 .

Consider the family of densities f(x,9) given by

<
log f(x,9) = [E(X'e :'§

)[x-9|x + g(x,9) for x
g(x,9) for X

 

 

where (i) A‘;§ a finiteynumber
(ii), 0 < A < 1/2
... _ B 9 if x < 9
(111) E(XIG) - {ygeg if X > e

97
agg_ (iv) 9 belongs to a finite interval- (a,B)
satisfying the regularity conditions (3.2.1) - (3.2.5).
Let_ 9n denote the M.L.E. of 9 based on) n independ-

ent observations of f(x,9). Let 90 denoteythe true value

 

f 9. Then

 

- 90] has a limitingdistribution
and it is the distribution of the position of the maximum

of the non-stationary Gaussianyprocess X(T) 2n_(-oo,oo)
£1111

E[X(T)] = -c I |29+1f

f(9o,90 ) EYE.

21+1+

2R+1 2k+1
I 'lTl‘Tzl 1

COV[X(T1) .X(T2)]'C f(OOIQI) [ [71!

[72

 

where C = P(giii;;1<:;:i3[6 6(90 )+)’2 (9 0)-26(90)y(90)cos wk].

In other words, the M.L.E. 9n is a hyper-efficient

estimator since 1

1/2<T$27{‘1 for 0<)\<1/2.

In fact, by analogous methods, it can be shown that
Bayes estimators for 9 , for smooth prior densities, are
also hyper-efficient and asymptotically the Bayes estimation
of 9 is equivalent to the estimation of the location

parameter for a non-stationary Gaussian process.

3.5 Evaluation of integrals:

We shall now evaluate the integrals encountered in

Section 3.2.

98

Lama 3.5.15
0° h 1

Let H(1)E f [s(X-1,0)[X+1[ —5(X,0)|X| +K5(X,0)Sgn X
—oo

Then H(h) = 0 for 0 < 1 < 1/2.

3192:
Since the integrand is of the order 0([Xlk-2) the
integral H(k) < 00 for 0 < 1 < 1/2. Let us now compute
A
f[e(x-1,0)|x-1|9-s(x,0)|x[§%g(x,0) Sgn xlxlx'11dx

O
= {lama—x)x - mom-x)x - 46(0)(-X>x"11dx

+ g[B(0)(1_x)% - 7(0)x9 + h7(0)xx-1]dx

A

+ {mom-1)x - 7(0)x“ + ly(0)x*'11dx
A+1 h A yx_1
=B(0)f ydy-6(0) 'ZYXdY-ha(0)gy dy
1 x 1 x h- 1dx
+ 6(0) f y dy - 7(0) f x dx + 17(0) f x

A-l A A
+ 7(0) f xkdx - 7(0) f xxdx + xy(0) f xk-ldx
0 1 1

A+1

B(O)fy )‘ldx

-1
kdy - kB<°>fy dy-v(0)f dey + 47(0)f X

A-l

h+1_ h+1 h+1 h+1
[(A 1 _ _

 

 

Since 0 < h < 1/2.

)7\+1_ )\+1

(A+1 1+1 A - Ak-—> 0 as A -> oo

[XIX-1

]dX.

'99.

and

(A-1)_f1

A + 1 -> O as A -> a).

Therefore H(x) = O for all k such that O < A < 1/2.

Lemma 3.5.2.

 

 

Let h(y) = )|y|k for all y
where
_ B for y < O
E(Y) - { y for y > 0.
Then for any T1.T2 and O < k < 1/2,
00
R(T1.T2) E f [h(y-T1)-h(y)][h(y-T2)-h(Y)]dY
-oo
; 2k+1 2A+1 2x+1
- C[IT1I -4T2[ -[T1-T2l ] (3.5.1)
where
C E 2k+1 [B +7 -267 cos wk]. (3.5.2)
2 J%(21+1)
Proof:

Since the integrand of R(T1,T2) is of the order
|Y|2?\-2 for Y sufficiently large and since 0 < A < 1/2,
the integral R(T1,T2) is finite.

Define

tha(Y) = E(Y) |Y|xe-a|yl for a > 0 . (3.5.3)
Let us now consider

lha (y-11)-ha(y)|

“aiy'T1|_

emu/Fe all".

Xe -a|y||

|€(y-11)ly-i1lxe
GIY‘T1|_

|8(y)|y-I1|Ae- -e(y)lY|e

100

for |y| > [T1[,

IA

Max <lBl:lvl>le'a‘Y’Tllily-Tllx - lylle‘alyi+a|Y-Tll}l

= Max (lfilrl7l) e-aly-Tlll{ly_11|%_ lylm e-aIT1l}i,
(3.5.4)
Let Co = Max (|fi[,[y|).

For [yl > [11|, we have from (3.5.4),

[ha(y-Tl) -ha(y)[

:.Co e—QIY'T1||([y-Tl[X-[y[x)-ly|x(e'alT1l-1)|

.1 Co RITIIIny-1+Co[y|x(ealT1|-1)e-Qlyl‘ (3.5.5)
For lyl > Max ([11},172I).

[ha(y-11)-ha(y)Ilha(Y'T2)'ha(Y)l

j-{Co 7\I'Tlllylkml +ColY|x(ealT1r‘1)e-alyl}
{Co AIT2FIth—1+ ColY|x(ea[T2+_1)e-QIYI}.

2 2 - Zla 2 -2.
Co A [Tszille-A 2 alYl

IA

+ C02|y[ [Tsz [e

+ 2C02 k aIrszllylzl-l e-alyl. (3.5.6)
Let us observe that for O < A < 1/2,

OD
<1) I I IZ*' < a» ,
Max(|11|, TTzl)

a)
(ii) I |y|zxaz e-Zalyl dy = Pg1+212 1-2x

a < oo ,
-oo 21+2?\

and

00
(iii) f a|y|2x-1 e-alyldy = P(ZK) a _ < oo.
-oo

101
Therefore, from (3.5.6), we get that
00
flhOz (y- T1) ‘ha (WIlha (y- 12) -h 0;”de < oo.
-oo

In particular, we get that

ha(y-T) - ha(y) e L2(R)

for every T , where L2(R) denote the set of square
integrable functions on the real line.
From (3.5.6) and (3.5.7), we observe that for any

A > max ((Tll’lTZI)

flh (y—Tl) - ha <y>|lh (Y‘Tz) - ha (y)| dY
IYI>A

:.C1 A27”1 + C2 al-Zx, (3.5.8)

where C1 and C2 are constants.

Letfkfiy)= [ha(y-T1) - ha(y)][ha(y-T2)-ha(y)].
Obviously 9a(y) —> 90(y) as a —9 0 for each y.

Let us consider

 

 

(I) (I)
f 9a(y)dy - f 90(y)dy
"CD -CD
{< |9a(y)-90(y)ldy,+ f (9a(y)|dy + { [90(y)[dy
:(Y —A |y|>A~ Iy >A
—| (m l9a(y)-eo(y)ldy + 2C1AZ7"1 + ca a1-2A
Y—A
by (3.5.8).

Choose an e > 0.

Since 0 < A < 1/2, we can choose a number A0 such

that

Therefore

00 00 162A
f 9a(y)dy - f 90(y)dY-1 I f l9a(y)-eo(y)ldy+8+caa

-oo -oo y(1Ao
By the bounded convergence theorem
|9a(y) - 60(y)|dy —> o as a-—> o .
(Y 30
Therefore there exists an no > 0 such that

for a > (10:

00 CD
I 9a<y>dy - f 90(y)dy .1 3s
-a> -oo

In other words

lim oo oo
a->0 f 9a(y)dy = f 90(y)dy - (3.5.9)
-(X) -CD

Let ha(t) denote the Fourier transform of ha(y).

we have
1 a) ity
ha(t) - -6; ha(y) e dy

oo . oo .
A - - t A - + t
37 Y e (a 1 )ydy+é-B Y e (a 1 )y dy

= y P(1+A)

1+x + B P(1+x) (3.5.10)
(a-it)

(a+it)1+x

Let ga(t,T) be the Fourier transform of ha(y-T)-ha(y).

Now
(t )= )91h < - >-h < )1 eityd
9a ,T - -00 a Y T a Y Y
CD CD

't 't
= f ha(y-T)el ydy - f ha(y)el Y dy
-CD -(I)

103

 

 

 

 

 

 

 

 

 

 

 

.. 1 P(1+A)_ LP (1+A) it'r
_ [:(a-it)1+“ + (a+1t)1+*:](e _1) (3.5.11)
by (3.5.1.0) .
By Parseval's theorem,
00
f [ha(y-T1) - ha(y)][ha(y-Tz) - ha(y)] dy
‘00
1 0° -———————
= 5; _é ga(t.11) ga(t.'rz) dt
= 31‘ 9‘1“)? f 7 1+1 + B 1+1} (€11:th
7T -00 (a-it) (a+it)
7 + B (e‘itT2-1)dt
{ (a+it)1+)‘ (a-it)1+7‘}
1 2 0°. 72+62 1 1
.-.- -— 1" (1+A) A + YB + "—"‘—_"
2v _in (a2+t2)1+x { (a_it)2+2x (a+it)2+21):]
[eit(Tl-T2)_eitT1_e-itT2+ 1] dt
(3.5.12)
Therefore,
00
R(T1,T2) E f [h(y-Tl) -h(y)][h(y-Tz) - h(y)]dy
-oo
_ C0lim
- f a—50{[ha(Y'T1)‘ha(y)][ha(Y‘T2)‘ha(Y)]dY
-oo
_ 1im 0° h ) d
-a,,0_£gha(y-T1)-ha(y)][ha(y-Tz)- O,(y 1 y
lim. 1 2 0° yZ +52
= — 1‘ (1+A) [
a->O 2w _i; (a2+t2)1+A
1 1
+ 76{ + }]
(a-it)2+2k (a+it)2+2x

[elt(T1-T2)_ e 1tT1_ e-lth +1]dt

104

by (3 .5 .12.) ,

_ 1 2 0° lim 72 + 2
27T -00 (1 >0 [(a2+t2) A

 

 

(a-it) (a+it)

+ 75-) 1 2+2A + 1 2+2AE|

[elt(T1-T2) _ e-1t11 _ e-lth + 1]dti

by the bounded convergence theorem,

_ 1 2 0° 1 2 2
”-57? P (1+A)-(j;o mm {7 + B -276 COS WA}

{e1t(T1*T2) _ e ltT1 _ e-lth + 1}dt

='§% P2(1+A)(72+B2 - 26y cos WA)
0° 1 it(T -r ) it? -itT
f I |§I§X {e 1 2 - e 1 - e 2+ lldt .
-oo t

(3.5.13)
Let T3 = Tl-Tzo

For any a > 0, define

OD . . . .
G(a.s) E f |t|a-1(e1t73 _ eltT1_e-ltT24—1)e’5ltl dt
‘00

 

 

 

 

=I‘(a)[ 1 3+ 1 a_ 1 a_ 1 a
(e-ira) (€+iT3) (8-iT1) (e+irl)
____1___3._ —-1—:+-351
(8+iT2) (8-iT2) E
. . . . a-1+2
Since the integrand 1n G(a,e) 13 of the order It] ,

the integral can be defined for a > —1.

In other words, for a > -1

105

 

 

 

 

G(a.e) = P(a)[ 1 a + 1‘J§"‘"_l"‘35 ‘ 1 a
(e-iT3) (8+iT3) (8-iT1) (5+i11)
_ ,1.1 a -.___—l——3 +-§5 1 (3.5.14)
1€+iT2) (8-iT2) E
In particular, the above equality is true for a'= - (1+2A).
Let n = -(1+2x).
Since

Itin-l (eitT3_ eitT1_e-itT2 + 1) e-Eltl .
.1 [tin-1 leith-eitTl-e-ith + ll
and f|t[n-1 (eitT3-eitT1-e-itfr2 + ll dt < CD.

it follows by the bounded convergence theorem,

oo . . .
lim G(n,e) = f [tln-1 (eltT3-eltT1-e lth + 1)dt.
e—>O -00
Therefore, from (3.5.14), we have

00 _ . . _.
f It)“ 1 (eltT3_eltT1_e 1t12 + 1)dt
-OO

-2P(n) sin WA [[T3I-n-IT1I-n-IT2I-n]

1+2).+| 1+2x_lT1_T2l1+2x

2P(n) sin WA [(Tll ]

(3.5.15)

12]

Therefore, from (3.5.13) and (3.5.15), we have
_ 1 2 2 2 . .
R(T1,T2) -'§F P (1+A)(y +3 -267 coswA)2P(-1-2A) Sln WA

1+2h+IT2[1+2A_[T1_T2|1+2*1, (3.5.16)

[ lTll
Let

c --%; P2(1+A)2P(-1-2A)sinwAA(72+BZ-ZBY cos VA) (3.5.17)

2
.-£—L%ibl [ g%:%bl] sinWAF(y2+Bz-26y cos VA)

106

by the formula P(x + 1) = x F(x),

 

 

 

1
2 IT.(e-A)I‘(->.+—)
- - F (;+A) i42x* * “ 2 ‘lsinwAI(72+BZ-28y cos WA)
2 WW(2.A-F1) .2
1
P(x)P(x-+-0
by the duplication formula P(2x) 2 22X = th.
1
P(7\+1)P(§"7\) 2 2
- [6 +7 -25) cos WA] (3.5.18)

220+¥Fh(2x+1)

by the formula P(x)P(1-x) = W/sin FX .

Combining (3.5.16) and (3.5.17), we have

 

 

R(T1IT2) = C[(T1(1+2x + (T2(1+2x ‘ (Tl-T2(1+2k]

(3.5.19)
where
_was-M 2 2

c - 223*¥}(2131(B +y — 26y cos WA]. (3.5.20)

Lemma 3.5.3.

For any T

00 ,

_é)[e(y-T)ly-Tlx - s(y) (ylxlzdy = 2c|T[7 (3.5.21)

where 7': 2A+1 and C is defined in (3.5.20).

 

Proof:
Note that the integral is R(T.T) where R is defined

in the previous lemma.

Lemma 3 .5.4 .
1‘ 0° A A A-l x
Q(A) E f[e(x-1)|x-1| -e(x)|x| +A sgn xlxl e(x)]g(x)|x| dx
-a)

= -c

107'

where C is defined in (3.5.20).

Proof:

Since the integrand of Q(A) is of the order

2A-2)’ and since 0 < )\ < 1/2 , Q(A) iS finite.
0

= f [fi(1-x)x- B(-X)x- A(-X)x-1 B] 5('X)x dx

-oo

A A-

1
+ f [3(1-x>* - vx + hx 1)] v xA dx
0

(X)
+ f [fix-1)k - vxx + AKA-17] 7x“ dx
1
(I)
= 62 f [(Y+1)“ - Y) - 1YA'11 yA oy
0

CD
+ 72 f [(x-1)x - Xx + xxx'l] x“ dx
1
1 1 1
+ BY f (1-X)xxxdx + 72 f A x2)“.1 dx - 72 f x2x dx
0 o 0
(I)
= 62 f [(Y+1)x - Yx - hyx‘l] Yx dY
0
(I)
+ 72 f [<x-1)* - xx + AXx-l] x) dx
1
2 ‘A 12
+ By B(A+1,A+1) + y 'EA - 2A+1
CD .
= 62 f [(Y+1)x - Yx - hyx'l] Y0 dY
0
(I)
+ v2 f [(x—1)k - xx + Axx_1] x“ dx
1
2
+ BY B(A+1,A+1) + 2A-1) (3.5.22)

212A 1)

108
Let us now compute

CD
I [(Y+1)x - Yx - AYK'IJ Yx-dY- (3.5.23)
0

Let G(€) (Y+1)7‘ - (Y+s)7‘ - A(Y+s)7‘-1] Y7‘ dY. (3.5.24)

Ill
0's. 8

For any Q's -1 and a’< -1/2,

(I) I CD I
f Ya’(e+Y)B’dY = sl+a+5 f YO"(1+y)B dY
0 V 0

, 1 ,
O

I
= el+a+5'3(o#1, —23L1).

Therefore, for -1 < A < -1/2,

1+2A
- e

G(€) = B(A+1, —2A-1) B(A+1, -2A-1)

\
- A 82% B(A+1, -2A+1).

Since G(s) is analytic for A < 1/2, it follows that

1+2A
- s

G(€) = B(A+1, -2A—1) B(A+1, -2A-1)

- A 82% B(A+1, -2x+1) (3.5.25)

for all A < 1/2 and in particular for 0 < A < 1/2.

Furthermore the integrand of G(e) is bounded uniformly in
8 by an integrable function since 0 < A < 1/2.

We shall now take limit as e -> 0.

By bounded convergence theorem, it follows that

CD

f [(Y+1)k - YA
0

A

- xyx‘l] Y dy = s(1+1, -21—1).

(3.5.26)

Similarly, we can show that

109

2
2A-1

A dx = 3(1+1, -27\-1)-2 21+1 .

CD
I [(x_1)x _ Xx + 7935-1] x
1

Therefore, from (3.5-22), we have

[B52 B(A+1, -2A~1) + 7’2 B(A+1, ~2A-1)

Q(A)
+ B‘Y‘ E(A+1.A+1)] (3.5.27)

="’C v

(3.5.27) proves the lemma.

[1]

[2]

[3]

[4]

[5]

[5]

[7]

[3]

[9]

[10]

[111

BIBLIOGRAPHY

Barlow, R. E., Marshall, A. W., and Proschan, F.
1(1963). Properties of probability distributions
with monotone hazard rate. Ann.Math. Statist.
34,375-389 .

Brunk, H. D. (1958). On the estimation of parameters
restricted by inequalities. ,Ann.Math. Statist.
23, 437-454.

Chenstov, N. N. (1956). Weak Convergence of Stochastic
Processes.whose trajectories have no discontinu-
ities of the second kind and the {heuristic ap-
.proach to the Kolmogorov-Smiranov tests'.

Theor. of Prob. and its Appl. 11, 140-144.

Chernoff, H. (1964). Estimation of the mode. Ann.
Inst._§tatist. Math. Tokyo ‘16,.31-41.

Chernoff, H. and Rubin, H. (1955). The estimation
,of the location of a discontinuity in density.
Proc. Third Berkeley Symp. Math. Statist. Prob.
11, 19-37, Univ. of California, Berkeley.

Cramer,.H. (1946). Mathematical Methods of Statistics,
Princeton Univ. Press, Princeton.

Daniels, H. (1960). The Asymptotic efficiency of a
maximum likelihood estimator. Proc. Fourth
Berkeley Symp. Math. Statist. Prdb. ’1, 151-163,
Univ. of California, Berkeley.

Doob, J. L. (1953). Stochastic Processes, Wiley,
New York.

Dvoretzky, A.,.Kiefer, J., and Wolfowitz, J. (1953).
Sequential-decision problems for processes with

continuous time parameter. AnnLyMath. Statist.
2%,-254-264.

Grenander, U- (1956). On the theory of mortality
measurements, Part II. Skan. Aktuarietidskr
39, 125-153.

Gurland, J. (1954). On regularity conditions for

maximum likelihood estimators. Skan. Aktuarie-
tidskr. ‘31, 71-76.

110

111

[12] Kulldorf, G. (1957). On the conditions for consist-
ency and asymptotic efficiency of maximum likeli-

hood estimators. Skan. Aktuarietidskr. 40,
130-144.

 

[13] Loeve, M. (1963). Probability Theory, (3rd. ed.).
Van Nostrand, Princeton.

[14] Marshall, A. W..and.Proschan, F. (1965)- Maximum
likelihood estimation for distributions with
monotone failure rate. Ann. Math. Statist. ‘36,
69-77.

[15] Murthy, V. K. (1965). Estimation of jumps, reliability
and hazard rate. Ann. Math. Statist. ‘36, 1032-1040.

[16] Rubin, H. (1963). Unpublished lecture notes,
Michigan State University.

[17] Rubin, H. (1960). The estimation of discontinuities
in multivariatemdensities, and related problems
in stochastic processes. Proc. Fourth Berkeley
gym . Math. Statist. Prob. ‘1, 563-574. Univ. of
California, Berkeley.

[18] Sethuraman, J. (1965). Limit theorems for stochastic
processes. Technical report no. 10, Department
of Statisticsr.Stanford Univ.

[19] Van Eeden, C. (1958). Testing and estimating ordered
parameters of probability distributions. Ph.D.
Thesis, Univ. of Amsterdam.

[20] Watson, G. S. and Leadbetter, M. R. (1964). Hazard
Analysis II. Sankhya. 26, 101-116.

TE UNWERSITY LIBRARIES

III]! II)” III)

3 3175 7986

MICH

l )I
9

(71)”)
3 1

2