‘l

        

WWW

\
I

EQME DECESEQN PRQCEDURES IN
EXPONENTEAL FAMELEES

$5; ilHWHllHHWUWNWWWII

THS

Thesis 50v Hm Degree of DR. D.
MiCHlGiE‘a’ SHE‘E UNIVERSE“
David Wallace Macky

1966

{Ephesus

llUHlHllllHlllUllHlllllfl‘lllllllllllHIHUIIIUHIIHHI

3 1293 01591 4058

This is to certify that the

thesis entitled
SOME DECISION PROCEDURES

IN EXPONENTIAL FAMILIES

presented by
David Wallace Macky
has been accepted towards fulfillment

of the requirements for

__.Eh..D..___ deqree in __S.ta.1;.i_s_t.ic s
and Probability

Wt LLMM
0

Major professor

Date November 21, 1956

0-169

' LIBRARY "

Michigan State
University

 

 

 

ABSMT
80E DECISION PROCDURBS IN
WOW FAIILIBS

by

David Wallace lanky

Consider a statistical decision problen where the pair (as...)
have Joint density p.(x) with respect to p x G where p is a known
neasure over R1 and G is an unknown probability neasure over a
subset a of 11. L is a loss function defined on 9(0):: 11 where
0 is a function on (1. 2(6) is defined to be the Bayes risk of
using the Bayes procedure asan estimate era. The first part
of the thesis shows estinatcrs of e‘ and. based on alsecpence
of observations with density h I: G(~)whose Bayes risk converges
to 2(G) when the loss is squred deviation. , A theorem is also
proved givinéiufficient conditions for an estimator to have its
Bayes risk to converge to R(G).

Let s =- (s1....,s) denote the vector of observations with
Q“, for g in a, a probability measure on the la'm” his the
action space and L ) O is a; loss defined on D n a; After observ-

ins each s a decision is nade on the basis of the‘ifirstj cheer-

3
vaticus.‘ ne'secend part of the thesis considers the proble-
conposed of the n component problems described above, with risk
equal to the sun of theoalponent risks's-v l'or n a {1,...,-},
define v a {:(a) . (Ld(1),,..,1.d(s))}. menif v is compact, the
mm... is seen to be an S-gane in which 3 is compact and convex
and the Bayes procedures are complete and the ednisdible procedures

are then characterised.

David vensoe Iacky

low let D have two elements and rencve restrictions on a.
Let the s's be independent and identically distributed fru an
exponential fuily. A class of nonotone procedures is defined
and shown to be essentially cupl'ete. fires them are proved
shoving tint under certain conditions several classes of none-
tene procedures are adnissible . 'nie problem considered here was
the basis for a paper by Girshick, Berlin and Roydcn (1957).
Iultistage statistical decision procedures. inn, lath, Statist.
£8_, _111-125. he thesis contains two counterexamples to theorems
3 and 5 of the above paper. Also it is'shown that under certain
conditions there exists a procedure s whoserisk pans) =
n film-3(a)} + 0(1).

son DECISION PROCEDURES IN
manner. mums
by

David vmsoe new

A 1313813

Submitted to
lichigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Statistics and Probability

1 966

/ l, p " ’ 1‘ V"
'5’, ‘\..'~ 5/ /
/_/ f . 4
I ’v’ "
'1 It I ‘

immcnmrs

I an greatly indebted to Professor Janos 2. Man for his
patieme, encouragement and above all, the guidance he gave to
my thesis work. His suggestions were of great value in enabling
no to obtain the results d’ this thesis.

I would also like to acknowledge with gratitude the patieme
and encouragement of Iv wife during the long course of aw graduate
studies.

PAR!

IABLB OI CON'JEITTS

I. man. HAYES BSTIIATIOI IN AK
WOWPMI--.........

1.1
1.2

mdmtion Elli Naution . o o o o

Anini-isercfanestinateasan
estinateofaaininiser .. .. ..

Squared error loss with unbounded o

p equivalent to counting measure on
the non-negative integers . . . . .

p equivalent to Lebesgue measure on R

1

II. M DEISIONB WITH CONSTANT murmur!
BUT'ITHIICREASING IMA'IIOI . . -. . . .

2.1
, 2.2
2.3
2.1.

2.5

2.6

2.7

MOOOOOOOOOOOOOO
notatioosndnsyes procedures '. . .

A nininal couplete class when I is
cupect anin is finite .. . . . .

Prelininarylennasconcerningan
Womofaflly.........

Bayes procedures and an essentially
caplOtCOl‘lfleeeeee'eeeee

”-1.31me e__eeeeeeeeee
Asynptctic cptinality . . . .1. . .

WWOOOOOOOOOOOOOOOOO

PA.

1 2
21
26

29

Pm I
1, Introduction and notation.

Io consider several empirical Bayes estimation problems, i.e.
the pair (2,.) have Joint density P'(x) with respect to u x G where
p is a neasure over 11 and G is an unknown probability measure over
some subset o of 21' and we wish to estimate 9(a) for some function
0. In all of the cases that we consider the density Pa(x) is an
exponential family. In the second section we prove a theorem
which gives sufficient conditions for an estimator of e" to have
its Bayes risk converge to the Bayes risk of the Bayes procedure
provided the loss function is of the type L(x,e) s x(:,.) + m(et')
where m is a signed measure. This loss structure is a generalisa-
tion of that used by Samuel (1963), see (23) p. 1375 of the latter.
We then give an example in the case of squared deviation loss in
which the estimator meets the conditions of Theorem 1. We have to
assume, however, that n is bounded in order to apply Theorem 1.

In section 3 we introduce the estimation problem with squared
error loss in several cases where the parameter space is unbounded.
Now in this situation the loss functions are unbounded and there is
no natural domination of our estimator as was the case in Robbins
(1961.) where he made the restriction that 6(s3p L(s,A)) (so or in
Samuel (1963) or Robbins (1963) where they consider hypothesis
testing problems. In sections 1. and 5 we consider two different
types of neasures p, one discrete and one equivalent to Lebesgue
measure. In both cases there is a natural estimator of the Bayes

-2-

procedure 1: however we have to truncate it in order to get the

at
desired convergence of risks. We truncate the estimator back to
an(x) and get a bound on the conditional squared deviation of

the estimator from tG for each 1. But in order to get the inte-
gral of this bound converging to zero we have to choose a sequence
{In} converging to infinity which is such that {an > o} Cilxl e In}.
The principal difficulty is that without knowing G we must let an
and In increase slowly enough to force the difference in risks to
approach zero. In both cases we consider, we show conditions on an
and In which are sufficient for the Bayes risk of our ecumator to

converge to the Bayes risk of t for simplicity in the case where

G‘
p is equivalent to Lebesgue measure, we let an be constant with re-
spect to x and truncate the estimator back to zero for [2' > In.

In this paper we will frequently abbreviate f f dm to m(f),
sometimes omitting the parenthesis, for any function f and measure

1 andlet

m. Let p be a measure on the Borel sets in R
or {ex -oc < 0 («band As“) <oo}. Let Gbe a probability

measure over a. For a in 0, let 0(0) 3 1/p(0“) and define
{1) h(:) s f.“ c dG.

Let Inn, n s 1,2,...} be a sequence of independent and identically
distributed random variables with common density h with respect

to p and lot (1,0) be a pair of random variables independent of
the sequence Inn} and laving Joint density 90(1) = c(s)e"‘ with
respect to p x G. Let P be the measure on the probability space
of the 1,05,11,12,... By the symbol Pxf we shall mean the condi-
tional expectation of f given I. Now applying Theorem 2.9 of
Lehman (1959) to h we get that the derivatives of all orders of

h can be taken under the integral sign for x in the interior of

-3-

the convex hull of the spectrum of :4. By applying the definition of
a conditional probability one can inediately verify that 0(a)e“'/h(x)
is a conditional density (with respect to a) of a given x. Then con-
ditional eqectations can be computed as integrals with respect to

this density, and doing so we obtain:

(2) P19“) = h(x + K)/h(x)

(3) P45) = swayed).

where I is any non-negative integer.

2, A mininizer of an estimate as an estimate of a minimiser.

In this section we prove a theorem which gives sufficient con-
ditions for the Bayes risk of a particular type of estimator to
converge to the Bayes risk of the Bayes function. We only consider
losses of the type: L(x,e) :- K(x,u) + m(et‘) where m is a signed
measure over translations t. Therefore choosing an estimate s to
minimize PIG. (x,o)) is equivalent to choosing m to minimize
Px('(°“)) a I(h(x + t)/h(x)). The result of the theorem is that,
under certain restrictions, we can obtain a satisfactory (in the
sense of Bayes risk convergence) estimator of the minimizer over
I of m(h(x + t)) by choosing an eluent of I which minimizes
m(hn(x + t)) where 1111 is an estimator of h. We note that in. any
application of the theorem it would probably be necessary for the
elements of I to concentrate on translations t which map the
support of is into itself since the values of h(x + t) for x + t
not in the convex hull of the spectrum of u may not be determined
by the values of h(x) for x in the spectrum of p. Let 8(m) be

the spectrum of m for any m in I.

'10-

!heorem 1, Let I be a set of signed measures for which

(a) P(lnl(e"")) < e.

If hn is independent of (x,u), if mn is (x,hn)-measurable and. mini-
nizes m(hn(x + t)) for m in I and if either (5) and (6) hold or
(7), (8), (9) and (10) hold where

(5) u is finite
(6) m(hnCi'+ t)) 3 m(h(x + t)) for all m in I
(7) there exists finite N such that 3(m) C.

NforallminI

(8) [nitll slfor alltandm inI
(9) hn(z+t)§h(x+t)roralltinn
(10) He“) < esror all t in N,

then

(11) P(nn(e"")) .. Phi: Px(m e“)).

Propr, Now (1.) implies that
rxcs I“) = nu; a“) = s<n<s + when»

and replacing m by [ml in the above we see that

(12) P(ln|(h(x + t)/h(x))) «a.

low 1"(mn e“) = P P(x h )(mn et') a: P(mn t')) = P(man(eu)) =
’ n

P(x.hn)(°
P(mn(h(x + t)/h(x))), the second equality holding since mn is (x,hn)-
measurable and the third equality holding since hn is independent of
(x,u)e

l'or the remainder of this proof we will abbreviate evaluations

at x + t by omission when it is convenient. Now

-5-

(13) mn(h(x + t)) = mn(h - hn) 4» (mn - m)hn + m(hn - h) + m h,

where m is in I. But

(14) . mn(h - hn) e ii Im(h - h 1n)I,
and
(1.5) (an - n)hn a 0 by hypothesis.
Therefore mn h e 2 li:[|1s(h - hn)| + m h and since an is an arbitrary
element of I,
(16) mn(h) s 2 l;:|1u(h - 5H + 1:: (m h).
Now clearly
(17) mn(h) ( zlmlh + inf (m h).
I I

Now if (5) holds then by (12) and (1.7), mn(h(x + t)/h(x)) -
inf(m(h(x + t)/h(x))) is dominated by an integrable function and.
tgerefore (6) and (16) together with the dominated convergence
theorem imply that (11) holds.

Now suppose that (7) and. (8) hold. Then

(18) mn(h - hn) s x :2 (h - hn)“,

and

(19) m(hn - h) e x z (h1m - h)+,
N

and by combining (13), (1.5), (18) and (19) we have

(20) nh-infnhexxlh-hl.
n n
. n N
Clearly
(21) nh-inf(nh)s21:xh.
n n N

lbw (20) and (9) imply that mnh - inf (m h) 3 on and since (21) to-
n

-6-

gether with (7) and (10) shows that mn(h(x + t)/h(m)) - i:f m(h(x + t)/h(x))
is dominated by an integrable function, by the dominated convergence
thcorem (11) holds.

An eagle, Let )1, equivalent to counting measure on the non:-
negative integers, be such that o = (- «,1! where (I < a. Let a loss
of (s - e”)2 result if we estimate e' by s. Then corresponding to
this loss function is the discrete signed measure m defined by miO} = r2,
.11} = -2r, um = 1 and sit} = o for all other t. Corresponding to
the set of all possible estimators s is I = {m x 0 s r e ed}. Then
(7) is satisfied with N = {0,1,2} and (8) is satisfied with

I a harmed, e21}.

Let hn( x + t) a #(11 a: x + t)/n p(1 + t).
Then hn -e h a.e.[P] and (9) is satisfied. we now assune that
6&2“) < so which then implies that (h) and. (10) hold.
Now for any m,
m(hn) = r2 hn(x) - 2rhn(x + 1) + hn(x + 2),
and this is minimized over I by mn(x, ha) a retraction of hn(x + 1)/h.(x)
to [0, ed]. Then by Theorem 1, the Bayes risk of using mn as an

estimator of e“ converges to the Bayes risk of the Bayes procedure.

:3 Sguared error loss with unbounded 0.

Now for any function 0 over 0, the Bayes procedure tG for
squared deviation estimation is defined by: tG(x) s 12(9) and
denoting by B. the Bayes risk of 1‘6 we get

(22) z = m - and)? = Page") - (cha)>2) s ”a . sea.

Let tn be any function of 11...":n and x and let Rn be the Bayes

risk of using tn to estimate 9. Then since P3 2"

(X 9x11“) ><szf1 and since Px(Px(9) - a) a o, Px(tn - d)2 a
bi

=7...

qutn - 13(9)) + (191(9) s 9)]2 = gen - £350»2 + P1010) - 02.
Therefore
(23) 13- n = P(Px([tm .. 912 - [21(9) - 912)) = haze, - Px(e))‘°')_.

In the next two sections we will consider two different cases falling
into the above general framework: one in which )1 is equivalent to
counting measure on the nonanegative integers and the other where

u is equivalent (in the sense of mutual absolute continuity) to
Lebesgue measure. In both cases we will obtain procedures whose

risks converge to R as n -e as.

A; p is a measure on the nonmnegative integers which is equivalent

to count measure in the sense of mutual absolute oontinui
In this section.we will let a = c“ and restrict G-by requiring
that:

(2t) mg) < use

For the remainder of this section we will abbreviate (for fixed x)
evaluations at x by omission, at x + 1 by ' and at x + 2 by "e
Let q(x) = h(x)u(x). Then q is the unconditional probability
function of x and we define qn to be the empirical probability
function of 1:, ice. qn(x) = (#(Xigooogxn) = x)/n. Let hn :- qn/u.
We now define a sequence of estimators tn of 151(0) whose risk

will later be shown to converge to R. For each x, let {an(x)} be
a sequence of nonanegative reels and let

= 0 if h' 3 h = O

n n

(25) t

= minian, (hé/hn) otherwise.

We now prove a lemma which is used in Theorem 2.

-8s-

Lens 1, If tn is as defined above then
Px(tn - h'/h)2 e [an > o] s + c
a... as q’ n =1I(an+ A/u')2 +' (w, + canal e
sat (a, + M11112 + waif . (s/u')2} =- e<n> i: a, . sub.
and, with e s- (h'/h - .5)“, 5
2 , n 2 2 , 2
6= c +(1-q-q) [(h'/h) -¢]‘ (h/h) .
Proof. If an a 0 then tn = 0 and c a (h'/h)2, and consequently
Px(tn - h'/h)2 s c. It therefore only remains to show that
P:(t‘ - h'lh)2 c B + 6 when 131:1 > 0.

Ne shall first show the bound on B given in the hypothesis
obtains. Now

a[(an + u/u')2 + (h'/h + 11/1102] <

(1 - 11")“: + 2 tun/M) + 2q(n/s')2 + q"n/n"" +
2q'(u/n')2 < I

(1 - onus, + 11/1102 + (11/1102) + awn/s") a
mi(&,, + 11/1102 + (MHZ e WW}.

nefine f1 e 1Px[t1| < h'/h - v] and f2 = Px[tn > h'/h + v].

Then
as

(26) Px(tn - h'/h)2 s {(131 + f2)2v dv.

Since tn s o,-f1= 0 on [v s h'/h] and. since t1n s an, f1 = 1 on

[v s 1]. Therefore

as h'/h
(27) {f1 2v dv = e2 + {:1 27 dv.

Tor c < v < h'/h, define

= -1 if 11 = x + 1
(28) :1 e (wow/n - v) if x1 .. x
8 O . otherwise.

Then since c < v < h'/h=$h'/h - v < an, and since I/p’ 2:
(h./h - ')hn " h;,

(29) [tn < h'/h - v] C [Y > 0] U [hIn :- h; . a].

Let

’6 '- qui) - q(h'/h - V)(u'/u) - q' = - h V n' < 0-

Then by applying Theorem 2 of Boeffding (1963) to the 11 we get:

-2nt2 ‘-2n1292

105(le1 > 0]) a ' ' 2 ' 2
t<§-v>g-+ 1.1 (3—17) _

which, together with (27) and (29), inplies that

as ea
(30) ff. 2v dv a c + {1. exp(-2n vzqz/[th + p/p']2)2v dv
0

= c + (th + u/u')2/2n <12.

We next obtain a bound on

(31) {f2 2v dv = f n , f2 2v dv,

the equality holding since an < h'/h + v=>f2 a O. for
O < v < (at - h'./h)+, define

=1 ifx1=x+1

'1 e -(h‘/h + v)(u'/u) 1‘ J‘i ’ 3‘
3 0 otherwise.

-10-

Ne then have:

'/p' e h; - hn(h'/h + v) and [tIll > h'/h + v] c [i > 0].
Therefore since 91611) s q' - q(h'/h + w)(u'/u) a -h v 11'; by
applying Thecrem 2 of hoeffding (1963) to t we get:

(32) £2 a osz-ZVZqzn/(p/u' + 11%| + v)2] ‘
oxp[-2v2q2n/(a/u' + of].
Therefore by (31) and (32) we have:
(33) Zr, 2v as s (a, + A/u')2/(2n «12).
which together with (so) eonpletes the proof.

Ne shall next prove a theorem which together with (23) implies
tut Rn Q 1.

Theorem 2, If

(31+) an = 001%) and an 1‘ on.
(35) s21) q} B s B’

where B is as in Isa-a 1 and where B" is constant with respect to

Gand
v.0 ash-sec,
and
. df°°
(36) 0‘ e-eaandhence d(m) a [6 dG-eG(G) >0,
“m
then

(37) ”(42(3) 2 q, B) a : (0:3)..2 B‘ y en(n) -e O as n was.
G ‘ 1:30 1:0 ‘

If in addition,

-1 11-

08) I"? es and en(In) -e o

and

(39) [an > o] C. [x e In]

M -

(w) P(tn - h'/h)2 s i'z(ln)0.(ln) + EKG) + 0-

.I__._a_r_§, Ife‘(m) -e0asn-eebandifo < ¢(m) 90thenthere
eonsts an I. < so such that n s N. ieplies that e3(m) e e(s), (the
smallest possible choice is I. a 1 + maxinle'nh) > e(m)}. l'er am
such sequence (NJ, if for n ) ll,I we define In a mainly: N,‘ n}
then I”). and since n > a“, en(In) s can). Therefore given
the {e‘} of (37) we can construct {I‘I which satisfies (33).

Also that e v satisfying (35) exists follows from (31.) and the
bousdonngiveninthehypothesisefLe-e1.

If the aIll of (31.) does not satisfy (39) we can then construct
a sequence {11;} satisfying (311-) and (39) by defining a; a [x a lull“.
That a; eeefollews from the feet that ‘n .eeend I. sac, and that
en(I.) still -e 0 follows from the fact that it only depends on
those x e I“.

mefofThgcremZ, Let0.=e a. lbw
q=ph=pfen6dG>pd:d(l)e

Therefore

I; a B e 2 (u 0: “t))-2113 B < 4’20!) '8' (n 0:)‘23‘
no no no

end the inequality of (37) holds. That en(n) -e 0 follows directly
’1'. B. -O 0.

-12.
By Le-a 1 and (39). he, - h'lh)2 s P(c)+ Pa: a 1,111).
New goods/sf < P(a)2 < a by (as) and therefore, since 1: e (h'lh)2,

by the dominated convergence theorem and (311.) we get: P(c) -e 0. Now
I

Puts a uni) - 3:211 n a {Zap-nun) by (37) a: a<n,> a o as

trivially if d(In) a 0. But by (36) end (38), d(In) .. c(c) > o and
by (38): .n(l‘) '* 00

2, u s uivalent to Lebes e measure on B1.
Let 0 = e in this case. Let {8“} and {an} be sequences of,

positive reals (for the remainder of this section we will abbreviate
6“ and an by eliminating their subscripts). Tor any distribution
function 1 define

:- 8-11.8(’]x+8/r]x ) if ””8 > O

x x-6‘ x-S

(t1) t(r.8)

a c OthOMSOe
Ie are permitting t to have extended real values and we will

abbreviate t(r,8) to t(r) when convenient. Define the probability
1

measure I by fi(-en,x] = f h(y)dp; we will abbreviate the latter to
h(x). Define
(1.2) and) =- [#(x1,.e.,xn) s x]/n

Suppose that d is an end-point of 0. Then, by lens 3 of
nrlin, Girshick and Royden (1957), c(.).“' e o as e .. d if d 5! o
and x is any real number. But if 0. G 0, C(11)." .. 0(a).“ < as as
s -e d. Therefore since C(u)en, defined on o, is continuous we get:

as) 3(a) 3‘ sup (cc-1.”) < a.
s Q

-13..

Now if le s m then en a made-nae”) and consequently

B(x) s sexis(-e),s(n)} and thus 3. 3‘ [nip (B(x)) < so. I. note
x <-

that the last term is independent of G and is a known quantity
sine p is known. for the remainder of this section we will ab-
breviate evaluations at x by omission.

Let 1"} be a sequence of positive reals which converges to
e

+cs and let d(m) :3 f. G dG. Then d(m) 9 G(C) > 0. We now restrict

.0.

G by requiring that
(UP) 9(02) < co.

Next we prove several lea-as which are later used to show that
the risk of our estimator (defined by (62)) converges to the risk
of the Bayes procedure.

Leena 2, If (1.1.) holdsandifhaudharees definedin
this section then:

(i) h'/h is square integrable with respect to E.

(ii) 0 s h"/h - (h'/h)2 s s e(.2)/a.

m Since h'/h :3 Px(e) we have: H(li'/h)2 s H(Px(ez)) 8
9(02) < oh: (“l-)0 Al”. by (#3).

(as) h" - fez-”c as s n 6(3).

Then since 0 e 'arx(s) s h"/h - (h'/h)2, (ii) follows by applying
(11-5) to the last expression.
m Ifhanddareas definedinthis seetionandif,

for any real v, [x] e v than for any positive integer m,
. t _1 vs. .
(‘15) 1/h e 11 (sh e

m h :- fe“c dG e [5.2% dG s eq.‘d(s).

-1»-

Lug, Ift,h,h,dands.aredefinedinthisseetien
and if, for any integer k, IxI ( k and if m is any positive integer
then

k+8 e
(#7) lt(s)| < 2 o(|e|)d'1(a) Bk+8.( ) "

Proof, By the mean value theorem applied to H and then to
log (h) we get, for some :1, x2 and :3 satisfying
x- 8<x2 <x3 <x1<x+8,

t(H) = 84(1og h(x1) - log h(x2)) a 8-1 (x1 - x2)h' (5)/h(x)).
But since,
(‘13) [W =U'ux‘c dd a s a(|~|)a~

we get by applying Lens 3 for M e h + 6‘, Ih'/hl e s(|.l)af1(a)

(kt-8).. _1
Bkvb‘. and (5.7) follows since 6‘ (x,I - x2) < 2.

Abbreviate (n65)-k to e and consider the following ccnditiens

holding as n -eg~
(#9) t-wo
(50) Sa-eO.
L-afi, Ifh, I, Inandtareasdefinedinthis section

and if (#9) and (50)hold then there exists an no such that if
n>n°,ifO<I<2a/¢andif (fl skthen

(51) gm.) - to!) a 1.] < when/4+)
and

<52) squat) - can e dc] < mafia/n)

-15.

forsemex<x‘<x+8andwherebyN1-wemeananynumbergreater
thanI.
Proof, LetA: x(n8)'1’andh = 8 t(n), and

s 1 for x, in (x,x + 8]
define I, a a“; for x1 in (x - 6‘,x]
m 0 otherwise.
Now
t(lin ) - t(n) a x e implies that Is an]:*5- a 1:5,?“3 )0
since I < 0 implies that Eng-6‘ > O and therefore that
8t(In) < A+B a 8(1 e + t(n)).
Now e3 a H]:+6/H]:_8 and thus
in Px(!,=) 3-le 31;, aIII]:"8(1--e‘)co

since I and therefore A is >0. Also
rxui) a 11st (l + on”)
and we define e2 :- Varx(11). Then defining v1 = Ii - n and applying
Theorem 3 of Hoeffding (1963) to the '1 we get
qu a a] a a," a .7.) a up)... pzc'za-1[(1 + e")i.g(1 + a) - 1])

wherea=(1-m)(-u)o-2>O. Ienowshowthata-eOasm-eme' Ie
firstnote thatO<A<2a(n 6‘3)i(n 8)'%=2a8-e0by(50): also

by Lee-a 11-. (t9) and (50) '0 set
M < 2 s cm e(|.|)d"(a) exp(o.(k + 5)) . o

where m is large enough to have d.1(m) < co. Therefore as A and
s . o, p 8/6..) a (1 + .“+3)/(.‘ - 1) > eB/(e‘ - 1) ..

Therefore we finally get:

-16-

2 212/ {-3) + s

-1
0
<“ WT m w

since p-eOasA-eOe

 

Returning new to the bound on P13 s o], as e co,
c.1'[(1 + a.1)log(1 + a) - 1] s i - 9/6 + 112/12 - 9/20 «I» ..e oi.
Also 7
n s n]:*5(-a - 112/2 - ...) < 431x;8

02 < le'z :- I]:+8(1 + .2A+B) -e 2B]:+8

as A,B «e 0. Consequently the bound on qu n 0] is bounded above
for sufficiently large n by exp(-n 123]:+8/k+) = enp(-lzh(x')/le+)
and therefore (51) holds.

Now define
a -1 for x1 in (x,x + 8]
' B-L
21:. fcrxiin(x-8.3]
0 otherwise.

If t(Hn) - t(s) e 4 c then Z a an]; on" - gt“ a 0 since

2 < 0 would imply that at“. > 0 and this together with

t(fin) < esimplies that Edi, > 0. It would then follow that

Men) > s - a a 8(t(h) - x e).
Let y a Px(z1) :- h]:*5(ef" - 1) < 0 since a > 0. Now

2 8 B- 2
Px(z1) a it]? (1 + e 2A) and we let a1 a Varx(z1) and v1 = z, - y.

Then applying Theorem 3 of soeffding (1963) to the v1 we get:

PIE 3 0] - P" s -y] e expi-n yzo-zfi-1[(1 + p'1)log(1 + p) -1]

-17..

B-ZA
where p = - 7(eH - 7)::2 > 0. Now Px(zi)/(-y) a: 1 + . _, one since

 

A and B «e 0. Therefore [3.1 a (Px(zz)/ (~11) + y)/(en" - 7) a. since
7 90 as A -e 0. Then, as was shown for a,p-1[(1 + p")log(1 + p) - 1] cit
and together with the facts that 012 < 31:”(1 + .3) -e 2 mi“ and i

y s n]:*8(-i + 112/2 - 9/3] +...) < arses/1+)

as l. -e o, the above bound on Px[2 a o]- is bounded for sufficiently
use a by arc-n mf‘ AZ/M) =- -ap<-x2 scrim»).

£233, A simpler but weaker version of Lane 5 is possible if
one uses the looser bound given by Theorem 1 of Hoeffding (1.963). In
this case (51) and (52) would be replaced by:

(i) Px[t(3n) - t(H) a re] < emp(-12&lz(x’)/2+) and

(ii) hymn) - cm) s 4.1 < eap(-s2n2(s-..)/2+).
Ifthisversionwereusedto show thatRn-ehthenwewouldhave to
replace (60) by en..(n8‘")-1 -e O.

m If h,H,t,d,B‘ and a. are as defined in this section,

if(t9)and(5o)holdandifais suchthatd(m) >0them

(53) |t(n) h'/h| a o( 2 20M)"
'3‘?“ Bk+8 '

m By the mean value theorems for first and second
differences, v 3’ {1(hjf" - n];a)/n];_8 . h'(x1)/(x2) with

x-c$‘<x1<x-o-15‘alldx-¢S‘<x2 (x. Ageinapplyingthemeam

value theorem we get

(51.) em) . r‘iegu + av) .. v/(1 + a“) with o < 51 < a.

-18-

e ”(12),. (h'(1 ) - 11412)).
(55) V -%' 3 -G'2T’ _ ;(12) _

 

andfor22<23<x,

(56) It?) #3711 = 1:-le %- #121,

(k 8)
s quash/m3) < a catfish“. * "'5
the equality following from the mean value theorem, the first inequality
following from Lemma 2 and the final inequality following from Lena 3.
Again by the mean value theorem, Ih' (x1) - h'(x2)l :- Ix1 - x2! h"(x,)

with s, between x1 and :2. But by (1.5), h" e 3h5°('2) and since
[x1-le <28, wegetbyLeI-a3:

. (kt-8)

: :(12) #1.... ‘2 28 %M' “2)‘1 ((1)3 ..e

which together with (55) and (56) gives us

 

8) s.

2 -1 (1“
(57) Iv - h'/hl < 36‘ Bk+8G(s )d (m)e .

Iorecver, by (1.8), (MI < k'wGfleI) and by Lama 3 we get:
(k+ 6‘)s

M < 2,, “(lens ‘(ah .

Thus by (1.9) and (50) and for sufficiently large n, [1 + dvl > 1} and
consequently:

Iv/(1+81v) - vl s: [81vf/(11-81v) I e 28 B12“ fl.2)d.2(.).2(k*8)“3.

rinelly combining the last result with (57) w. get

(k+8)u
lt(I) - h'/hl < 8 Bk+8G(102)d"1 (m)e .‘(3 + 2d 1(m)e and)“

-19..

Ne new state. some conditions, holding as n -e co, that are
sufficient to yield our final result. Let {In} be a sequence of
positive integers and abbreviate In to I.

(58) I 9 oo

(59) shamans-.1 » o
-1

(60) (n 65) dunno-.1 . o

(61) 5 cinnamon“) .. 0

Let gn be t(nn) truncated back to the [-a,a] interval. Ne
finally define our estigtor tn as follows:

as gn(x) if [xl ( I
(62) tutu)
a 0 if le > I

and lot In be the Bayes risk of using tn against Ge

Then by (23) and (£12).

I
(53) 1. " 1 " [,[wan - h'/h)2]an + [I lfallow/102ml.
x

Define a a [Px(‘n - t(n))2]‘1r and r a |t(n) - h'/h|. Then by the

flnkowski inequality, Px(5n - h'/h)2 ( (A + !)2 and therefore by

a a
the Schvarts inequality, iffAde-eOandflzdfi-eOasn-eo
-s -m

s
then flux“, - h'/h)2]dfi .. 0. Is now show that this situation

holds.

-20-
Since [gal s a, [t(H)I s a implies that {(gn - t(B))2 ) u}

{(tai.) - t(a))2 s us}. Since d(I) -o d(c) > o, Le-a a and (59)
inply: supdlt(fi)l < a for sufficiently large n.
I g

2 °° 2 . (2'92 2
Then a a (3’4“: - t(H)) s- aka: a g Px[(t(Hn) - t(n)) s u]du e

-1
of“ P [(t(hIn ) - t(h))2 s efflzx (II a 2”1f“"z‘("')/‘*2x d! <

(2(8+)/h(x"), the second inequality following fre- Leena 5. Then by
Leena 3 and (60),

(61.) 1:121:11 e e(84-)dn1(I)exp[(I-I»d)ul . 0]
Also by Lens. 6 and (61), for large I,

I
(65) [.er e 6‘2 bag“ exp[l.(I+6)e.]) . 0

Then since part (i) of Lemma 2 and (58) imply that { (h'/h)2dfl
[Ix >I]
we get by (63), (61.) and (65) that In «.1 as n eons

Remark, In the case we have Just considered we could, as in the
case where p was equivalent. to counting measure, have used ads which
were functions of x. If we did so the estimators tn would be defined
by tn :- t(a,) truncated back to the [firs] interval. Then since in
theproef ofLe-a5wewculdneedAandBeonvergingtoOunich-mly
fer |xl e 1:. (50) neuldbe rcplaccdby 8n-e0 uniformly for H oh
and we would replace (1.9) by lltip‘k(8) -e o. In th. statement of

x

Lee-a 6, (53) would be replaced by sup |t(n) - h‘/hl =
13161:

-21..

2(

2 k+8)m‘
lsTpde) O(Bk+8e )3. Similarly (59), (6c) and (61) would
x

be changed to:

-1
([21:71;th '31:, (Bmaqlflw‘hll) 0.

-1
:18} I 8 -e 0
[:11de ) ap[( + )5]
sup (8 Bi+8emp[2(I-1-6‘)m.]) -e 0.
lxld
respectively. Analogous to (39) of Theorem 1 is the condition that
[n1| > o] C [le a I]. If this holds then (63) is replaced by:
a - a a flat (t - h'/h)2)dB + (h'/h)2dI
‘ -I ‘ " [h(m]
andtheremainderof theproofisthe same.
Anenmple oftheabove case iswheng-gisthe standardnormal
2
density. Then B(x) a c" /2 and if we let 8 :- n'V", a, a (I+6‘)/2
and a a 81/5 than (59) becomes n-1/5.(I+8)2 -e 0, (60) becomes
n-1/h.i-(I+d)2 +0 and (61) becomes n.1/ 1'32““? .0. All of the
latter three conditions are satisfied for any I such that (I+8)2 <
lcg(n1/1o). If (60) holds then 8 must be greater than m-1/ 3 and if

in addition (61) holds then I can at most be o(log n1/ 6)e

PART II
him

The general problem considered in this part is as follows.
Is successively observe n random variables 81...”: n where the
distribution of the s's depends on the state of nature and after
observing each one we'make a decision based on the information
available up to that point. l'or the problem consisting of the
n component problems with total risk equal to the sum of the
risks of the component procedures we consider the question of
obtaining admissible procedures under several different sets of
conditions on the 13's, on the set D of possible decisions and on
the set D of the states of nature.

The first case we examine is a generalization of a case
studied by Samuel (1963s). I. assume that both a and n are
finite and show that the class of Bayes procedures is complete
and that if a procedure s is Bayes against an a priori distribu-
tion 5 over 0 then each of its component procedures «pi, i a 1,...,n

must also be Bayes against 5 in the 1th

component problem. to

finally state an extension of a theorem of Wald and Wolfowits

(1951) which characterise: the admissible procedures in this case.
The second case that we consider was the basis for a paper

by Girshick, Karlin and loyden (1957) (referred to hereafter as

[6311]). D is assumed to have only 2 elements although 0 is m

longer assumed to be finite. The s's are assumed to be independent

and identically distributed and to have an exponential distribution

.2-

whose parameter is an element cf 0. We define a class 3 of monotone
procedures which in theorems 1 and 2 we show to be essentially com-
plete. In section 6 we give counter-examples to theorems 3 and 5

of [on] and thus show that the authors of [GE] have not obtained
a minimal complete class of procedures for the problem. In our
theorems 3, A. and 5 we prove that under certain conditions several

classes of procedures are admissible.

2, Notation and Bayes procedures,
Let 5n = (‘1’“;"n) denote the vector of observations and. lot

51 = (s1....,si) for i = 1,...,n. Let Q“, 0 en, be a probability
measure on the gin-space and let Q ed. a 0.051)“. For each m in 0
let I.(.) be a function from n into [0,.) and let L(.,d) be the less
that occurs when decision 6. is taken and a E Q is the state of
nature. We will consider strategies 3 = (¢1,...,¢n) where eaeh

‘1 is a. 51-measurable probability measure over a m-field of subsets

of B. Let p(¢1)(u) be the risk at the 1th stage of using s, i.e.

mix.) = QU(¢1(I-(e))).

.0...) 3‘ 3 s(¢1)(0)
i=1

is the risk for the n-stage problem of using s when 0 holds.
Let p(§,s) denote the Bayes risk of using s against 6. Then
n n
(1) e(p(-.-)) . x 50011)) > x wte(p(¢1))].
i=1 1:1 (:11
Therefore s is Bayes against 5 iff

map) = in: [g(.(¢1))1.
4’i

--3-

2. A minimal cmlete class when 0 is finite and I is eggpact.
In this section we consider a generalization of a result of

Samuel (1963). Let c e {1,...,.} and define I = {L(1,d),...,h(n,d))z
d E D}. Ie now show that the class of Bayes procedures for the n-
stage problem is conplete and therefore to obtain an admissible
strategy for the n-stage problem we need only consider procedures
which are such that each of their components is Bayes with respect
to the same prior distribution.

Since there are only a finite number of pure strategies for
nature, our n-stage game can be represented as an S-game (for a

definition, see pg. 1.7 of Blackwell and Girshick (1951.)) with

(2) S 3 {0(3) 3 (P(1s3)s°"sP(‘sa))lfl-1 3}.

we now show that s is closed and convex. Lot 31 = {p(¢1)lall o1}.
'I'hen by Theorem 6.2.1 of Blackwell and Girshick (1951.). 2.3L is
compact because I is compact. Consequently by the rychanoff com-
pactness theorem 5311 x ... x 8n is compact relative to the product
tepology. But then S is compact since it is the image under a
continuous map of a compact set.

Let s and s' be arbitrary procedures and Let 0 < a < 1. Them
0: p(s) + (1 - a) p(s') = p(cs + (1 - a)s') and 8 is convex...

l'hen by Theorems 5.2.2 - 5.2.1. of Blackwell and Girshick (195k).
‘5 C 1103*:an is complete where his the class of
admissible strategiesz is the class of Bayes strategies, and ”5+
is the class of strategies which are Bayes against prior distributions
5 which have 5(a) > 0 for all a in a. be admissible strategies are
characterised by an extension of Theorem 2 of Wald and Iolfowits
(1951) which we now state.

Theorem.

In an S-game in which 8 is compact and convex, an element of
8 is admissible if and only if it is Bayes with respect to a
sequence of h(s m) a priori distributions (€1’°“’5h) such that
the matrix [£1319 i = 1,...,h, J = 1,...,m, satisfies:

(a) for any 3 there exists an i such that 51: > O,
(b) the matrix [513], i = 1,...,h-1, J = 1,...,m does not
possess property a.

The proof of Wald and Wolfowits remains valid for the above
extension because the only property of the space of risk vectors
used in their proof is the fact that it is compact and convex.
Then since our n-stage problem can be represented as an S-game
in which 8 is cmpact and convex, the procedures with risk vectors
of the type of the above theorem constitute the minimal complete

class for our problem.

Prel Lemmas conce an onential f

We will now consider a less general case where s1 is the sum
of k1 independent and identically distributed random variables
from an exponential family detenined by a non-degenerate measure
poof. Leta-19+ ... “:1. Let r: [..blwithaeb be the
convex hull of the spectrum of p. Lat

(3) Q:{us-o<c<mandp(e“)<oc}={c,d}

with e < d, and let o(e“) .-. 1/p(.). Then 21 = :1 + ... + :1 is

sufficient for Qui over the gi-space and therefore we only consider

the distribution function P .1 (which is the [it]: self-convolution

of the distribution function P. which is determined by the density

h(x) = Mode" with respect to p) and the density

-5-

mZ
(1») past) = 135000 1

the ladon-Nikodym derivative of P d with respect to p1 the a? self-
convolution of u. We restrict D to have only two elements, and re-
strict cur attention to losses Ma,” and 140,2) (written as L1(e)

and 1.2M in this section) which satisfy

(5) 11(0) - L200) 2 0 if so § ”o

for some “o in the interior of o. Ihen Z1 is the variable of inte-
gration we will abbreviate it to s. In this section ¢1(21) 8 £61.11).
to now show that if a class of decision procedures is an essentially
complete class for some decision problem where the losses satisfy (5)
then the class is essentially complete for any other set of loss
functions which also satisfy (5). Let s and s' be any two strategies.

‘0" if (5) hon-d3 th‘n P('s') ) P<Us3.') 11': (L1,- 3’2) ‘21P“(‘1 "' i) ) 0

on 0 since "’1’ s 1.10.) P d(si) e 1.20.) (1 - P ‘91)). hut then by

the continuity of 11““: '.. 95;),
(a) use) a poor) in: (.0 - .) 13‘ we, - .1) s: 0

an 0. Therefore the ordering of the risks of any pair of procedures

is not affected by the particular I.1 and L2

that satisfy (5). To prove the essential coIpletenese of a class

chosen from among those

of prooedm~es in a problem in which the losses satisfy (5), it is
therefore sufficient to prove the essential completeness for a pair
of losses which are such that L1 - L2 is continuous on n.

It was shown by xsrlin and Rubin (1956) that mninimal com-
plete class of procedures for the fixed sample size game is the

class of monotone procedtn'es of the form:

-6-

311. flziati
¢1=O flzi<t1
=1 ifzi>t1.

Ie therefore will only consider procedures s which are such that
their components $1, i = 1,...,n are of the above type. Hereafter
when we refer to monotone procedures we shall mean only those of
the above form.

Consider the following conditions on montone procedures:

(7) t1 and 15h1 are equal and infinite

(8) o(x1 - I“) < t1 - t1M < h(li - ‘i—1)

(9) "i"“i--1“‘(Ki‘l‘i-d)““n‘M‘i-A(13%?"o
(10) t1 - t5,1 = h(x1 - xi_1)end a“ > o = 211 a 1.

Is now define S to be the class of all monotone procedures which
for i = 2,...,n satisfy (7) or (8) or (9), or (10), or are equi-
valent a.e. [hi] to a monotone procedure which satisfies (7) or (8)
or (9) or (10). Let s1 be the set of all elements of s which are
equivalent to procedures with only finite critical points.

The problem considered in this section and in section 5 and
6 was discussed in Girshick, Karlin and Royden (1957) and our dis-
cussion is the result of an attempt to deal with some of the errors
of the above paper. The format of our discussion is designed to make
comparison with the above paper relatively straightforward. In
section 5 we give counteraamples to theorems 3 and 5 of [on].
Our theorem 1 does not use the assumption, made implicitly by, the

above authors, that finfii(m)(L1(u) - L2(a))dl(e) exists finitely for

.7.

every s and consequently we do not use Lena 1 in the proof of
Theorem 1. Is do however state a corrected version of Lemma 1
fer the sake of comparison. Ie now prove some lends which are
used. later in this paper. Le-sas 1,2 and 5 correspond to Lamas
1, 2 and 5 of [on] and Lemma 6 corresponds to Theorem 1 of [on].

£93.13. Let z s: {313(0“ Ihl) < on} for a measure E. If (s - a.)h(u) a
0 see. [B] then

(i) H(e”h) has at most one zero if

file! 0,10 [M >01) >001!

1

(ii)ifZ=l andifH((-«m,ao)n[h>0])>01dif

“he'd n [h < 0]) > 0 then 3(euh) has exactly one
8030.
M (i). Let g(s) - H(e“h). Then by Theorem 2.9 of Lehan (1959).

in the interior of z the derivatives of g can be taken under the inte-
gral sign. Therefore

-su 3(0-0)
%; (as). o, e f. .. (Wyn-Mao) p o,

and consequently g can have at most one zero.

(0-. )z _
(ii). is a increases across («g-tee), H(e ° h ) increases across
(h-(u.)fiiu.}, .) and H(e 0-0. 35") decreases across (h+(u°)H{e°}, + co).

Therefore by continuity g has emactly one zero.

M Bb‘b' (cothen fi(.)a’b. 91/141”) “......
m- (fi(e1)e"h.)"1 a ”(.(n-b'h) " firm} as 0 9.1)] the montone
convergence them...

Lug-3‘ If we define P'(y) = I’M-v) for v in 0' = In h(em) < co},

-8-

we have a new exponential family which is the dual of the one we have
been considering. Now v belongs to 0 iff -v belongs to 0' and conse-
quently 0' = {-d,-c}. It then follows that Lamas 2 - 5 each have a
dual which results from applying the lenas to the dual exponential
family. ‘

hen-s 1, If d is not inc then p(e)e“V ..o as e ..di'er-eeeyeb.
Proof. Io first consider the case where d s on. Then

[ma-”1“ = u(0'(w)) e a“! > , + «Jo"“”).

Now suppose d < do. Then by Patou's lens, 1/p(s) = Me“) -e.
as: a -e 11 since o(ed") = on.

£13 Ifdisnotinflandb' <bthenthere exists 0' suchthat

if s t u' then for all y a b', dude” is decreasing and converges to
0 83 U 9 Q.

m. %; (fi(u)ew) s p(u)e”(y - P.(s)). luv, as pointed out in
Hannah (1965), P‘(s) ob as u -e d and therefore there exists a“ such
that if u > 0' then P'(s) > b’. Then p(u)ew -.O for y ( Why the

aboveandLmaBasu-ed.

Lena 5, Suppose that d is not in a, s and s'are elements of S,
.1 > 1:, (I) 1,110.1(4’3 - 4.3) > c and (n) P.°k(¢k- We) a 0. Then

(i)ift (scand~(ABC)holdsthen‘-e0asa-edwhere

. = “(¢3 - ¢J)/P&(¢k " we

1 : tJ-tk=(J-k)be

J

B : pk“: > thu‘fik- fl” = Os

0 : nib} = 03

(ii) If sac holds then

Ea“ . 02-41(9)“.
Hoof. '5 first observe that

(I) implies -.. 1;J s t3 (a

E

an M11“ ’eoC ti>tk<o.

new by (I) and t . we get that t is finite.

a ‘ .1

Suppose A does not hold. men since neither (7) nor (1 O) .
hold, t J - tk . h(J - k) and consequently tk is finite. men
s e DP/Q with

D(.) .pkkeuaftk)

r1.) - ”as, - .93 1.
( .

9(a) "stuck ' 4Q: Ptk) 1b

We note that P is non-increasing and that Q is non—decreasing. -low

(H's)

by Lena sepplied to ”34,11... 0 as . .. d and therefore . .. o as
e o d. ~

ll‘cw suppose that A holds but that B doesn't. Then extending
the definition of 0/1) to this case by

k": 0(kt1}

Q/D‘lk[(fi'¢pp. ’ ls

J-kma(t-s)
Q/bamas...dbyl'atou'sLemmasinoep e 3 ..Oonfsstk]

and B doesn't.hold. hen-since P is non—increasing we again have

’-e0.

low suppose that A and B hold but C doesn't. But A and B

~10-

implythattkp-goandthatb<¢,. hereforedam. MD
increases to pbjib} < es by Lemma 2 applied to pk: and P de-
creases to (¢3(t3) - X1)p1{t1} a 0 since 1: 8 1 by (10) since
A,BandIIimp1ythatxk> o. ‘merefore t. 0.

(ii) We may assume without loss of generality that t J < co and

that t3 ) up since this decreases ‘. Then

(11) cc.) - (a, - yanking}.
x: '1
and

-k t"
(12) Dom) =5 :1.ng p’o‘f ,-

i-k... (VJ-tit) ; e (:43)

fl J-(‘I ’Wfi' dfla‘

how 1: =1 implies that #1 < t.1 inlightof (II) andtherefore

. J‘kla’d"
tg-tk((J-k)b. menbyLe-a3,p e ..Oasu-eoo,

which, since the integral in the negative tens of (12) is non-

increasing, implies that the negative term in (12) converges to

sero as g ...... Now since nib} a O,

k t.
p get: ”kitk} ‘ ? pkags ‘14 pinned”,

which together with (11) and (12) completes the proaf.
Remark. If p is atmless, then * ... O as e .e d since (II) then
ilpliel that B does not hold.

When writing integrals over subsets of a, the appearance of
o(d) as one of the limits of integration is to be interpreted as

.11;

Lemma 6. For purposes of this lemma abbreviate (L1 - L2)* to

L and (1.2-1.1)*te1. Iftandt' finite such that

1 2'
”.o +0 d t.

(13) f e p111” cf e pihzdl' (a,
c .0

and '
.° 1: ,1 d t:

(11;) f 0 “fl L131 3] O “pal-251(w
0 .0

for somea distribution function 1' which does not concentrate on “o

andintegersi) J) Othen
(want-1r. (i-J)b.

ProofI Suppose t - t! , (i - J)b. then by Lens 2, (15) and (11.),

foe 17"p:"1.:’:d1'=-=./'02e(t"1")"pepj'd'w'41sd1‘

(15.9 )e 1‘1 d ,
> e op (at) f at “1331.261!
'0

('3'? )e 1'3

= e °p (so) {0"VupJL1”

'o
’ .(t-t' )'p1'3svp151d!

. $3,315“,

But since 1b.} < 1, one of the inequalities must be strict which
then contradicts (1 3) e

. Similarly by applying the dual of Laura 2 in the case when
t-t’ r (i-J)awegetthatt-t' , (i-J)a.

5. B_ezes Eo_cedures and an esgentigz chete class.

In this section we prove two theorems, one showing that if
s is Bayes and depends only on the sufficient statistic then s
belongs to S, and the other showing that S is essentially cu-

plete. In this section we will abbreviate 21 to s when it is

clear what value of i is appropriate.

neorem 1,. . If (5) holds, if 1" does not concentrate on .0, if
. s: (4,1,...41.) is Bayes against 1 and if ,(r,s) ... then s

belongs to 3.

Proof, Define forawreal s
.0 8(e-m ) 1

11(8) 8/ e op i'(Is“--L,‘,)'.'d.l'
c
d s(.-. )

111(3) 8 f e o 35(L2-L1)+d1'.
"o

llowbyLemma2withb' abwegetthatifh<gpthem
. I .0 ab k1) +'('-b) x14

. e11“) 9/. p (111-132) e p d!

o .

b . .(Pb) ' ...
e .'° skit.) f- 3"" (31-5,) 111'
O
k s .
m p 1&0)... 11-1 (Pb) e
herefore

(15) b < . and 11;, (s-b) < .=>Ii(s) < ..

By similar arguments we get

.13-

(16) b < m and 1110:) “an“ (..e) s...
(17) e > -. and 11h) < .211... (H) < es

and

(18) a > -ee and H1_1(x-a) < «$111.51 (8) < so.

Now since s is Bayes against 1', its components must satisfy a.e. [#1]

K _ ’ K
(19) c1 = {1, if fienp 1LI) : Neup ‘12).

s I d K
Define e310.) , {0,0 p 11.3“ and b J1(a) a {fig ‘1.de for
o

.1 =1,2. New by (5),

(20) a21 ‘ a” and 1’11 ‘ ha.
merefore if either ‘21 or but is infinite, then
s I
(21) P(e. p 1113):.» for i =1 andz.
.3 “av-ea) K11
Now I e p 2dr is a decreasing function and therefore
e
(22) a21(s) 3m$3a(l') 8.. if s' g s.
d “0110) I ‘
Similarly f e p 11.1” is an increasing function and thus
'3
(23) huh) -e.=>bu(-') =1» 11" r s s.

. I x .
Now (19) implies that ,(r,.) =1}; ,1 [nnzr(e"p in), no", 1a,)”.

.11.»

men for any s such that
(211-) u1(’as3] > O ”3 flint”) > as

,(r,s) finite implies by (21), (22) and (23) that a21(s) and b11(s)
are finite. hen since the #1 measure of the set of s's which do

not satisfy (21+) is sero, we have a.e. [A1]

(25) r(e”p“x.1) E r(e”px’1.2) iff I1 5 111.
But (19) and (25) yield a.e. D11]

I1 1111<II

i
(26) $1
30 “111(110
Wewill dlow now that a.e. [p1]
=1 ifl)t1
(27) ¢1=O u8<t1
axiifs-ti

a supislIi >111}
where t
as -.o if above set is empty.

supislIi I 111}
Mill. ‘8

ti if above set is empty.

suppose that t1 < ti'. new if r(.°,d} , o and if II1 is finite

on (t1, t? (then 111 is increasing across (t1,t:) and since 11

is a decreasing function this contradicts t1 < t°. herefcre

”-15-.-

t

1 < t; and 10,04}, cisplies that 111 =- Ii e... on (t1,t;). But

1 is finite on “1"“? then Ii is decreasing

1 < t:. filers-

o .
fore t1 ( t1 and Home} , 0 implies that Ii - 111 e... on (t1,t:).

over (t1,t;) aml we again get a contradiction to t

low by hypothesis rise} < 1 and therefore either flog.) or r(.°,d}
is positive, and we have:

t t° implies that I

i‘i ién "°

1

But since ,(r,s) < a we then get p1(t1,t:) - o. It is possible

0 0
that ”it“ , o if t1 < t1 but in that case 111“?) a. and

therefor. 11(t'1’) e g, and gliti’} = 1. v. have thus shown that

(27) holds.
We now define
= sup {8'11 I'11:)!
(28) 1:;
rs -.., if above set is empty.
and
B inf {8"111 “a!
(29) t;

= tan if above set is empty.

new it follows directly from (15) and (16) that

(30) behaving. +1111» end tzet;.1+k1b.
Also free (17) and (18) we get

(31) 3>'D:‘v1_1+.k1 ‘ti mtg-1+.k1 ‘tze

-16-

It follows directly from their respective definitions that t t“.

i‘ i
We first consider the case where a aul b are finite and show

that in this case s belongs to S. We do this by considering the

five different possible relationships betwoen t1, t; and ti for
two successive values of i:
. .
(32) t1_1 a t1_1 g t3.”1 and v1<t1 < t;,
t. m
(33) "i-1 < t1.“1 < t1_1 and t1 =- t1. 1111,
. ’.
(311-) tip-1 e ti-1 < 1314 and *1 < *1 < 1319
t t m
(35) ti--1 ' t i-1 ‘ 1"i--1 “‘1 ti ‘ 1‘i ‘ ‘3'
(56) either ti_ 1 < t1_ 1 a t1_1 or t1 < t; - t1.
case I: (52) holds.
hen t1_1 is finite since ita1=°°¢°° a + “i ‘ 914 +
. _ _ a
ski ‘t' ,1 by (51), and since t1_1- a: .. e t;_1+bh1 sti
by (50). But by (50) and (51),t 1_1 finite=$t + .31 g
t3_1+ e31 #1 < t1 e t; 2 t1 1 +bk1 = t1_1 +bk; and therefore
(8) helds.

Case II: (33) holds. ‘

Now t A must be finite and by (50) and (31),

i-1

t +ek t‘ +ek (t’

1‘ 1.1 1 It‘t'gt' +bk <1: +bk

i—1 i i 1 1-1 1 i-‘I i

and therefore (8) also holds in this case.

Case 111: (35-) holds.

hen since I and 111 are continuous on intervals where they

i
are finite and since (5a) implies that they are finite on (ti , ti),

t1 is a sero of Ii - 111. Similarly t1_1 is a sero of 1191 - 111.1.

.1 7.

hen Lemma 6 yields ak

1<t1- t_1<bk and(8)helds.

i 1

Case IV: (55) holds.

hen either (7) holds or both t and t are finite since the

i i-‘I

assumption that (7) doesn't hold and that t1 and/«- t1_ 1 is in-

finite leads via (50) and (51) to a contradiction. However if

t and t areboth finit. then by (30) and (51),.1s1 ( t1-

i i- 1
t1_1 ( bki. We first consider the case of (55) and
(37) t1 II 1:14 '0' bki and 11.1 > 0.

Since s is Bayes, either Ii-1( t1_1) g 111,1 (t1.1) or [11.1it1-1) I 0.
If ‘1-1it1-1} a 0 then s is eqml a.e. [”1] to a procedure in 8,
But if Ii- 1(t1_ 1) g I111( t1_1), then by the analysis proceeding

(15) - (18) end by (37).

(38) 1&1.st 11111)epic>n11111)en<1).

hen we get that 11(t1) < 111(t1) because in light of Lew-a 6 and
(58), 11(t1) = 1:1(t1)=.>t‘1 < t1_1 + bki which contradicts (57).
herefcre if (55) and (57) hold then either (1 0) holds or
“1.1!t1-1} a O and there exists an s' in S’whioh is equivalent to
a. Now consider the other boundary situation: (35) and

(59) t1 = t1_1 + an:1 and 11.1 < 1.

If 11 8 0 then (9) holds. Suppose then that 11 ) O. .hen since a
is Bayes, either "1&1; =- 0 or 11(t1) g 111(t1). In the former case
there exists an s' in S which is equivalent to e. If the latter
holds then by the analysis which preceded (15) - (18),

k . k
(110) p it.) 11.1 (ti-1) c Ii (1:1) e II1 (t1) s p 1(”11) 1111.1 (1:111) .

-18-'

But then I1_1(t1_1) (IIi_1(t1f_1) because equality would-by (no)
and Lens 6 contradict t1 8 ti-1 + aki. herefore, since 114 ( 1,
we must have ”1;1{t1-13 = O and consequently there exists an s' in

S which is equivalent to s.

Case V: (36) holds.

O
New 91¢ t1 . ti inplies that t1

on the interval (avg), :11 increases to... and I

ago since t1 ‘< on would imply that

1 decreases fra

., which by the continuity of I1 and I11 would imply that I1 :- 111

at some interior point and consequently thatt 1 < t' . Furthermore

I
'0' 1
as 8 goes from ti to”, I1 decreases to (L1 (...) - L205» p Garbo}.
But since I1 > 111 for all s and since 1'0.de > 0 implies that
111(s) ..gp as s .eqr we must have

an:1 = (1.20.;- -n 1. >) p 11. m. u (r. 1.) r. 1. ,“n {warm}.

nerercre IIi ; 0, which implies 111.1 a o, and ti-‘I a. and thus
(7) holds. Similarly if tit-1 e t1_1 = t;_1 then (7) holds.

Since (32) - (36) cover all the possible cases, we have proved
the theorem for a ani b finite. 'nle theorem is immediate in the
case that a = ’m and b age. In the other two cases, a = “'00:,

b ¢ co and a > q,” 'b 89, we again consider the sane cases: (32) -
(35). file proof is identical in case (3A.), in part of case (35)
and in case (35)e In the other cases the proof is similar except
=.= “6%) and b =a(e = -ec).

that in the cases where t - t

i i— 1
the proof is .inediate since p11 .} = “ii'a’; = 0.

More: 2. S is essentially‘oonfiete.

Proof. Let (an, 11.8 1,2,...) be dense in a but not including so.
Now considering only 01..."... we know that the class of Bayes

.19-

procadurss for ).1,...,.n} is complete (see 1). 111.6 of Blackwell
and Girshick (1 9511.)). men by {mood-en 1, for any procedure s,

there exists an s' in s which is such that

(1+1) p013») r stays.) for J =1....,m.

Consider {s.,- = 1,2,...}. Let is”, r-=1,...} be a subsequence
of {s‘} which is such that

(1.2) 1:11. .. t: for sale t:

and “Jr; is either strictly monotone or is constant. In the case

1r 0

that t, = t1 for all r we can assume wifllout loss of generality

forsomeO‘a

that 44’“? .. a g 1. In the other cases

1 1

$11.“? a 1(0) 88 13: >(<) 13?. merefcre; 4.111;) -e ”0(2) where.

=1 if:>t?

4.1.(x) 1- o ifx< t:
alin 44%;) if: = tf.
r-em

Dy repeating this procedure n times we (can obtain a procedure
so a (t:,eee.t:) and a subsequence ~{at} 61‘ is“! with

¢:(x) .. ¢1(x) for all r and i a1,...,n. {Blerefore by the dcuie

' 10 Ia
nated convergence theorgn “(a 3&1 1- 4,?) -e [11(0 J(¢1 " d?)
as r .. go and consequently

(11-3) ij's) - pbj’ar) '0 'bj'a) - ijfiafio
But thelhsis. ctorr. be (1.1) andtherefore

(1.1.) ,(.,rs) - pays) . o for all a.

-20-

Let a be an arbitrary ale-ent of (2 ad let {.‘3 be a subsequence of

{an} which converges to 3. flow
out .s)- .1., .c°) = s 112111.011. 1.) r. 1. n)) .11. ”1.11m -

41:6»).

New
30" 10
“11“.!“ -0 1.")(931 o5]! c u1[°.-e |.. 0

since ”1(ex") is a continuous function of a in theOinterior at Q,
is right continuous at c in a and left continuous at d in n and -

since an. -e .. merefore
115) .11-“'1e, - .p) .. “13%., - .p) .. . ....

'nlerefore since p6,.) .. p0) and since we can without loss of

generality assme that L1 - L2 is continuous (see note in section

1.) , the above iuplies that

11' [#9. ,s) " P01. 930)] “the” rpm

I-ea
his by (1111-) 111311» that .01») - .61.?) e 0.

we finally will show that s° belongs to 3. If t1 a 1:111 31c

then (7) holds. If either t° or t° are infinite then we are dons

1 1- 1
since fl: 5.} I 0. We therefore assulse without loss of generality

o e‘
that both rt; and t:_1 are finite. hen at; ( t1 - t1,“1 ‘ bi:1

:-
because er belongs to S and ski ‘ t1 - ti“ ‘ bki.

Supposethatti-t1_1=bk1ad¢;_1i.1(1)>O.'.fllen

lil 1):. 1(t° .1) > c, which implies that t1- 1 . t1_ 1 ad consequently
r am

-21 r-

1‘ 1' 0
that t: a ti. If t: > t1 then ¢1(t°) - 1 ed (10) holds. 1:

+b‘to +batoltr.

c r o'
tat thent Itr becausetr‘tr 1,1 1 1

1 1 1-1 i-1 1 1-1

But then since (10) holds for t” and t: _1, 41:03? - 1 for large r

i
which ilplies that ‘31:?) .. 1 an! (10) holds.

0 o
Suppose that t1 - t1_1

8 .31 w .
(as) .111 11411) .1 a.

his inplies that li- d; 1(to. 1) < 1 ad consequently that
r

o

tad(t1_1whichinpliesthatt§¢t1sinoet13t14 +a;t;_1 +
o O

5311. nti’sti'thentw -t1_1 and, since (9) thenhcldsfcr

r .
t: and ti1s c1(1=°) .- 1:- 4.31:? - o. If t1 < t1 then clearly

4,;(ti’) - 0 ad (9) holds.

6. Addesibilin
In this section we provehthree theorems which show the

admissibility of different classes of procedures. He then give
counterexamples to theorems 3 ad 5 of Girshick, Karlin ad

Roydon (1957).

neuron :. If a is open and p is atculess then 81 is eddssible.
Proof. Let s be an elenent of 31 and suppose s is inadnissible.
then since 8 is essentially caplete there ”exists an s' in 8 such

that s‘ is better than c. hen

(1.7) A0.) - .(m) - .(m‘) - (1:10.) - 1121.)]

.
13:1;1Q1-‘1)) Ofor 8.11.1110.

-22-

. e
Suppose that the smallest i such that 1:1“: ¢1) ’1 0 occurs for
- e
i = i1 and that P.11 Q11 ¢11) < 0. Let 12,...pil he the ether

values of 1 tor which p.131 -.q) ‘ 0. Let
33 a P0393 - ¢3)/P011 “i1 - ¢i“)

for .1 > 11 and .1 K 11,, r s 2,..~.,R. hen since p is etc-less,

3:..Oas...cbythedualctl.elna5. Mom-east...”

(118) 1iPUi-(‘1 - 1.? has sane sign as P'11 ($11 - ¢;1) < 0.

But in light of (5), (1.8) implies that so.) ¢ 0 for . near c.
Since this contradicts (#7) we cannot have 15.11 “11, - ei') < 0.

By a symetric argment it follows that the assumption that
P (Q -¢"))OinpliesthatAC.)<Ofwgmardrhichagein
“11 1 11 .

- e
contradicts (1.7), .merefcre 13.1“: .51) I: Oi’cr e in a and.
i a 1,.."11. .But this contradicts the assupticn that s‘I in-
proses s. aurora-e there is no s‘ which improves s all con-

sequently S1 is canesible.

liteoreng,a Ifnis cpenandifsinSis suchthetak1<
t1 - t1_1 < bki for i a 1,2,...,n, then s is addssible.
m Suppose s is inadmissible. {men there exists an s" in
S which improves on s. Since neither condition A of Lens 5
nor the symmetric condition of the dual of Lena 5 holds for
any i, a choice of 3.1 and an argument identical to that of

theorem 3 yields the desired result.

hooren 5, I: c is Open then:
(a) s in S1 is admissible except when, for sue 1, either
nib} =CEufl-h.(rl>t_1($_-R_)) =0andt. -t. . an- a-

45-

.{s} . o M.pk([l < 113% - 4.9) = o and. t1 - 1:14 a ski.
(b) if s in S is such that g1 c O a.e. 511‘]!!- a 1,,..v,n or
such that ‘1 a 1 a.e. [p1], 1,...,n then s is adnissible.

m

(a) the proof here is again analogous. to that of theorel
3 and the exeeptimel cases are those of lens 5 and its dual,
(b) this result follows immediately fro- the assupticn that
L1“) '32“) : Oas. : .0 since for. < e. an strateg that
takes action one on any set with positive measure [p1].

1 - 1,...,n is iaproved (with respect to this a).by a procedure
with,1 a 0 a.e. ["11' J =1....,ns Similarly the'other case
covers those a ) ...

We next give counterexanples to theorems 3 and 5 of [M].
maple 1 is a counteremple to both theoreas but we give
Example 2 because it is a procedure with on]; finite critical
points ani shows that 31 is not in general an sanssible class.
he procednre 31103 in Example 2 is also s counterexalple to
thssrss 5 o: {an

A c e to he of an

ye show here the existence of 2 aonetene strategies
s a (t1,t2) an). e' I (tht'z) which are such that s' is a
better strategy than s. Let [be equivalent to Iuegesgae ’
measure and be such that n = (-.. , +.), if for exalple ft:
is the standard ncrlel density, then p satisfies these cen-
ditionss Lstt2 u... t < 12' and t, t; sad #2 finite. New
sous) -s(s.v) =- (1. (s) - 1.29))(1’ (.51- sq) - P 2(#2)) -
(I. (...) - 1.29))(Au) - 30.)) tumors A and 3 are defined by

positional correspondences . ‘

By Lou-n 3.2 (1) of Lem (1959)). Ms animus-ins
muss our 0. sisstcw"; t)) = «5 a.) o: - no» at
since (hnnan (1955)) P.(s) .. t. as e .. 3., there exists an

suehthatP (s)=t'andoonsequentlyfcr.;. ,lis
...1 1 ’1
a decreasing function but is always positive. nerefore for

"i

e

.’.%

, B/A is an increasing function. How writing

s(t'2 - t3) .. ...(s - v2)

 

3 Mn)- . «261-2
18' " t; ..(s - 91F '

f‘ (‘1 " ‘41)“1

*1

weseethatforusat,2_tz,where1’ (s)=t'2-—t3',

. -
t'2 *5

B/A is an increasing function. Since the only assunption en

t' and tr in the above analysis is that they are finite, we

1 2
choose then to satisfy t'2 I 2133' and thus we have that 3/1 is
an increasing function ever a. low as g .. -.., B/A. O by
thedual orbs-a 5. Onthe otherhand, «...... i(.).. o
by Lens 3 and the aonotone convergence theorem. But this
together with the fact that B is an increasing function ilplies
that B/A .... as e do.-

Row choose a. to be. such that. “DJ/AG.) = 1 ad choose
1.1and1sz such thsti.1 4.2: cross:.°. hen since
(1.1 - 1.2) (i - n) . o with strict inequality for seas ., we

have shown that s is inadmissible.

-25..
1 counter e to Dacron of on
Let a and b be finite, nib} a 0 sin s,"'s"eleuents of

31 which are such that 19.91 - .1) V, c, P.(¢'2 - 4.2) , o,

t'2-1t1 > c.11'zs1andt1ct1-t2-b. nissr.(¢1-¢1)t

.t1 .
(11 - v).{t1}s so.) and consequently 12 a 1 which together
with 34‘, a 1 ilplies that .2(t2) . .g.:2(.t2 ) . 0. low

(#9) p (139141) +P22(¢ -qn«'2))/M»)«"%1 ‘f.

n-(
(211 -11)sit1l -Ep(s)-M1)ds2 #011 -i')s!t1 3- AG).
. ‘ 2
But since ”{b} a O,
at
(so) P2122<.t21sP!z1tt13sp(s)o 1&1}.

Since P .(yz- a) s Oead¢2(t2) ayzflz ) we met have

t'é-1<t2-t1§bandthereforebyLeI-a3,pb)e .sO
0(8 - 92)“ '
as e «e d-Bsss hen since f2. e611.2 is non—increasim,

thereexistsang‘fcr0<(;<p{t}suohthat

(51) e o e‘ illiliospfi)0.(v 4:1)?” :48“?de < c.

moi-stars by (so) and (51), if s s ..t then

_1 t . a _ '. ' ‘
5(0) ' P (0):. 1 P fizfihflduz " 3(0)“ (#2 *1.) P :(kvhz

finial-(>0.

-26-
_ low by the dual of Lena It, there exists .1 such that if
e g .1 than p(.)e'y decreases to O as y. '60 for all

y 3 t5 - 1 > a. herefcre by the monotone convergence

theorem,

(52) at.) t 4.1) m- . < .1

andA(.) .s Olflgq-Qe

Since A is continuous and positive on [51“], it has a
positive mini-u: :- over this interval. Shows 1’, 11 and 131
such that 0‘ n‘ < n and n" I pit1lfi1 -).f1). Let 0., We such
that Ah“) 4- a“. then

(53) I‘ - A‘s? : 0 as e i 'o’
@0080 L1 and L2 such that
(51.) L10.) - L20.) ; o .. s ; ...

. .1 . _ .
as... A...) -.e,» --fi(s)0 -.1 (1.10.) - 1.2mm at») > o

for e )4... ,fiierefore s is inadmissible.

z. .motic muslin,
to show in this section that under certain conditions there

exists a procedure s whose risk Mus!) satisfies:
pans) = ntmin1tm2mn + om.

he idea is that after making 1 observations we choose the
action which would if taken at each of the first 1 stages have
given suller risk than taking the other action at each stage.

Assn-e:

-27..

there exists a randon variable v(s) which satisfies

(55)

., - >
P.(v)‘0asln1 L2< o.

(56) P.(et') < .. in some neighborhood of t s 0.1

Is assuae now that s1 ..ssss‘ are independent and identically

distributed but are no longer restricting P. to be an exponential
£83113.

1
1 <
Define V1 'é'uk)’ Define 91%) :- oas Vi’ Oandlet

s a (¢1,...,¢n). Den

.0...) ‘= 1’: [I-1(s)Q.IY1t o! + name. {V13 on -

n 1.10.) + (1.20.) - 11¢.» 11-; 9. m o 0;,

Lot . be such that 1.10.) - 1.20,) < 0. then by theorel1 of
Chennai? (1952),

Q {Y1)O}‘r1wherer-intl’(etv)and
D t O
r < 1 since, at t c O, %;[P.(et')] :- P.(v) 7‘ 0

tv ‘ i ‘
by (55) and ,P.(e ) c 1.. hen since 1:11!” ¢ 1/(1 -.r) for

1'<'1s

.t..-) t n 1.1 t.) + (1.20.) - n1 (mo/<1 - 2)).
Similarly if s is such that 1.1 (1..) 41.5..) , c then

'Qe‘) ‘ n 1'2“) + (L1G) " L2“))-(1/(1 "' r)e

-28-

to note flat if the losses satisfy Lab) II P. (1.1) for snap

functions 11, .1 - 1,2, then (55) is satisfied with v - 11 - 12.

-29-
BIBLIOGRAPHY

Elasbrell, D. and Girshick, a. (1 951.), nag of Genes and
Statistical Decisions. Wiley, New York.

Ghornoff, n. (1952) , A neasure of ssysptotio efficiency for
tests of a hypothesis based on the st- of observations.
Ann, Iath, Statist. 23,, 1133-507.

Girshiok, 1.. Karlin, "s. an! Hayden, n. (1957). Iultistage .
statistical decision procedures. Ann, Iath, Statist. 2_8.,

Hannah, J. (1965), Iathgtical Reviews. 29, 665.

Boeffding, I. (1963), Probability inequalities for suns of
bounded random variables. J Amer Statist Assoc. 5g,
13-30- .

Isrlin, s. ssa’nnbis. a. (1956), no theory cf assisisn
procedures for. distributions with nonotcne likelihood

bob-sh. 3.1.. (1959), lasting Statistical mothsgsg. Wiley,

Iew York.

Robbins, n. (1963). no empirical Bayes approach to testing
statistical hypotheses. Rev, Inst, Internet, Statist.

Robbins, H. (1961.). his expirichl Bayes approach to' statistical
decision problem. Ann law Statist. 21.1-20.

Rosenblatt, I. (1956), Remarks on sale nonparanetric estimates
of a density function. Ann, lath, Statist. 31, 832-837.

Sasual, E. (1963), An eapirical Bayes approach. to the testing
of certsgin paraaetrio hypotheses. Ann lath Statist. 25,
1370-1 5.

Smell, 1., (1 963a), late on a seguential classifical problen.
Ann, lath, Statist. 1&‘1095-1097e .

Wald, A. and voirosits. ‘J; (1' 951 )‘, Characterisation of the
tidal complete class of decision functions when the
nunber of distributionsand decisionsisfinite. Pros,

Second Barrels: m, lath, Statist, Prob. University
crnia Press.