MINIMUMDISTANCEESTIMATIONONREGRESSIONMODELSWITHDEPENDENTERRORSANDGOODNESSOFFITTESTOFERRORSByJiwoongKimADISSERTATIONSubmittedtoMichiganStateUniversityinpartialentoftherequirementsforthedegreeofStatistics-DoctorofPhilosophy2016ABSTRACTMINIMUMDISTANCEESTIMATIONONREGRESSIONMODELSWITHDEPENDENTERRORSANDGOODNESSOFFITTESTOFERRORSByJiwoongKimThisdissertationconsistsofthreechapters.Thechapterintroducesthemodelsunderconsiderationandmotivatesproblemsofinterest.Abriefliteraturereviewisalsoprovidedinthischapter.Thesecondchapterinvestigatesminimumdistanceestimatorsoftheparametersinthelinearregressionmodelwithdependenterrors:autoregressiveerrorsandpaneldataerrors.Asymptoticdistributionalpropertiesoftheseestimatorsarediscussed.AsimulationstudythatcomparestheperformanceofsomeoftheseminimumdistanceestimatorswithGaussianmaximumlikelihood,thegeneralizedleastsquares,andtheordinaryleastsquaresestimatorsisalsoincluded.Thissimulationshowsthesuperiorityoftheminimumdistanceestimatorovertheotherestimators.Thethirdchaptercomparestwoasymptoticdistributionfreemethodsforanerrordistributionintheonesamplelocation-scalemodel:Khmaladzetransformationandempiri-callikelihoodmethods.Thecomparisonismadefromtheperspectiveofempiricallevelandpowerobtainedfromsimulations.Whentestingfornormalandlogisticnulldistributions,wetryvariousalternativedistributionsandthatKhmaladzetransformationmethodhasbetterpowerinmostcases.ACKNOWLEDGMENTSIwishtoexpressmydeepappreciationandthankstoProfessorHiraL.Koulforsuggestingthesisproblemsandguidingmydoctoralstudy.IwouldalsoliketothankProfessorVidyadharS.Mandrekarforteachingcourseswhichwidenedmyunderstandingovertheprobability.Also,IthankProfessorsDanielHayes,ChaeYoungLim,andMarianneHuebnerforservingonmydissertationcommittee.Ithankmywonderfulparentsandbrotherfortheirconstantloveandsupportwhichenabledmetocompletedoctoralstudy.FinallyIwouldliketothankmyfriendsAkshitaChawla,AbishekKaul,andDaewooPakfortheirsupportduringmydoctoralstudy.iiiTABLEOFCONTENTSLISTOFTABLES....................................vLISTOFFIGURES...................................viKEYTOSYMBOLS..................................viiChapter1Introduction...............................1Chapter2M.d.EstimationMethodonRegressionModelwithDepen-dentErrors................................72.1RegressionmodelwithARerrors........................72.1.1Asymptoticdistributionofb.......................92.1.2Asymptoticdistributionofbˆ.......................162.1.3Simulation&applicationtorealdata..................242.1.3.1Simulationstudy........................242.1.3.2Globaltemperature.......................292.2Panelregressionmodel..............................352.2.1Randommodel...........................352.2.2Asymptoticdistributionofb.......................382.2.3Otherpaneldataestimators.......................432.2.4Simulation.................................452.3ProofsforChapter2...............................49Chapter3GooTesting:KhmaladzeTransformationvsEm-piricalLikelihood............................703.1KMT&ELmethods...............................703.2Simulationstudy.................................743.2.1Testingfornormaldistribution.....................763.2.2Testingforlogisticdistribution.....................79BIBLIOGRAPHY.................................83ivLISTOFTABLESTable2.1Bias,SE,andMSEwhenn=50....................27Table2.2Bias,SE,andMSEwhenn=100...................27Table2.3Bias,SE,andMSEwhenn=200...................28Table2.4AveragesofSE'sofbj'swithnormalandLaplaceinnovations....28Table2.5AveragesofSE'sofbj'swithlogisticandMTNinnovations.....29Table2.6M.d.andGMLestimatesofandˆ..................32Table2.7SEofm.d.estimatorof1and95%CI................32Table2.8Bias,SE,andMSEofestimators:isnormalwithn=10andT=5.46Table2.9Bias,SE,andMSEofestimators:islogisticwithn=10andT=5.46Table2.10Bias,SE,andMSEofestimators:isLaplacewithn=10andT=5.46Table2.11Bias,SE,andMSEofestimators:isMTNwithn=10andT=5.47Table2.12Bias,SE,andMSEofestimators:isnormalwithn=20andT=10.47Table2.13Bias,SE,andMSEofestimators:islogisticwithn=20andT=10.47Table2.14Bias,SE,andMSEofestimators:isLaplacewithn=20andT=10.48Table2.15Bias,SE,andMSEofestimators:isMTNwithn=20andT=10.48Table3.1CriticalvalueforKMTandEL.....................75Table3.2EmpiricallevelandpowerobtainedfromtestingH0:F=FN...78Table3.3EmpiricallevelandpowerobtainedfromtestingH0:F=FL....81vLISTOFFIGURESFigure2.1TheIPCCglobaltemperature(left)anderrors(right)from1916to2016...............................29Figure2.2Lagvs.sampleACFandPACFofOLSerrors.........31Figure2.3Lagvs.sampleACFwithARmodelsoforder1,2,3,and4.....33Figure2.4Lagvs.samplePACFwithARmodelsoforder1,2,3,and4.....34viKEYTOSYMBOLSInwhatfollows,foranyfunctionsf:R!Randg:R!R,jfj2Handjfgj2HdenotethatRf2(y)dH(y)andRff(y)g(y)g2dH(y),respectively.Forarealvectoru2Rq,kukdenotesEuclideannorm.ForanyrandomvariableY,kYkpdenotes(EjYjp)1=pwheneveritexists.viiChapter1IntroductionInthepast,extensiveresearchhasbeendoneonm.d.estimationmethod,wheretheesti-matorsareobtainedbyminimizingsomedispersionsorpseudodistancesbetweenthedataandtheunderlyingmodel.Manyauthorshavedemonstratedrobustnesspropertiesofava-rietyofm.d.estimators.Inthecontextoftheonesampleproblem,Beran(1977)showedthattheminimumHellingerdistanceestimatorthatminimizesHellingerdistancebetweenthemodeledparametricdensityanditsnonparametricestimatorisrobustagainstsmallperturbationintheunderlyingmodel.ThroughtheMonteCarlomethods,ParrandSchu-cany(1979)demonstratedrobustnessofseveralm.d.estimatorsofthelocationparameterbasedonvariousL2-typedistances.Intheirsimulation,theygeneratedavarietyofmix-turesofdistributionsandshowedseveralL2-typem.d.estimatorsarecompetitiveintermsofasymptoticvariancesandrobustness.Millar(1981)provedlocalasymptoticminimaxityofafairlylargeclassofCramer-vonMises(CvM)typem.d.estimators.DonohoandLiu(1988a)showedthatalargeclassofthesem.d.estimatorsis\automatically"robustinthesensethattheyhavethesmallestpossiblevarianceamongestimatorswhentherearedepar-turesfromtheassumedmodel.DonohoandLiu(1988b),however,demonstratedthattheminimumKolmogorov-Smirnovdistanceestimatorofnormallocationmodelcanbebyoutlierslocatedbeyondthe3˙pointsandhencehaveanarbitrarilylargeasymptoticvariance.Allofthesepapersdealwiththeoneortwosamplemodelsonly.1ConsiderthelinearregressionmodelYi=x0i+"i;(1.1)whereE"i0,xi=(1;xi1;:::;xip)02Rp+1withxij;j=1;;p;i=1;;nbeingnonrandomdesignvariables,andwhere=(0;1;:::;p)02Rp+1istheparametervectorofinterest.When"i'sareindependentidenticallydistributed(i.i.d.)randomvariables(r.v.'s)withaknowndistributionfunction(d.f.)F,KoulanddeWet(1983)proposedaclassofm.d.estimatorsbyminimizingaclassofL2-distancesbetweenaweightedresidualempiricalprocessesandtheerrord.f.F.Koul(1985,1986)extendedthismethodologytothecasewheretheerrordistributionisunknownbutsymmetricaroundzero,andtoautoregressivemodelswithsymmetricinnovation,respectively.Furthermore,itwasshowninthesepapersthatintermsoftheasymptoticvariancesomeofthesem.d.estimatorsremaincompetitivewithotherestimatorsincludingWilcoxonrank,theleastabsolutedeviation,theleastsquareandnormalscoresestimatorsof.Alsorobustnessofthesem.d.estimatorsagainsttheextremesintheerrorssuchascertaingrosserrorswasestablished.Formoredetail,seechapter5inthemonographbyKoul(2002).Giventheabovementionedpropertiesofthem.d.estimators,itisexpectedthatthem.d.estimationmethodwillstillremaincompetitivewhenweextendthedomainofitsapplicationstothelinearregressionmodelwithdependenterrors,whichistheaimofthisdissertation.Chapter2demonstratesempirical-intermsofbiasandasymptoticvariance-ofthem.d.estimationmethodthroughsimulationstudieswhenitisappliedtothelinearregressionmodelwithautoregressive(AR)errorsandonewithpaneldata.Section2.1considersthelinearregressionmodelwithARerrors.Inbothsocialsciences2(e.g.economics)andnaturalsciences(e.g.climatology,hydrology)linearregressionmodelswithARerrorsarefoundtobehighlyuseful.Forexample,timeseriesoftotalGrossNationalProduct(GNP)intheUnitedStatesduringthepostwarerahasshownthelineartimetrend-oftenreferredtoaslongtermtrend.Intheseries,thecyclicalaroundthelineartimetrend-oftenreferredtoasbusinesscyclesorshorttermshock-iscommonlyobserved.Thetrueunderlyingstructureofthesebusinesscycleshasbeenofinteresttothemacroeconomists.Aclassofmacroeconomists,knownasrealbusiness-cycleschool,believesthesebusinesscyclesrespondtothechangesinthelongtermtrend;see,e.g.,Prescott(1987).Itiswell-knownthatthecyclicalofmanymacroeconomicvariables-includingGNP,totalconsumption,etc.-isgenerallyautocorrelated,andARmodelcanexplainitwell.TheglobaltemperatureanditslongtermtrendintheclimatologyisanotherexamplewhichshowsthatthelinearmodelwithARerrorsisofgooduse.Blo(1992)decomposedthetimeseriesofthepast100yearglobaltemperatureintothelinearlongtermtrendandtheshorttermerrorsandvariousmodelsincludingARmodeltotheerrors.HeshowedthatARmodelisedbettertotheerrors.Section2.2considersthelinearregressionmodelwithpaneldata.Therepeatedobser-vationsonthesamecrosssectionunit,e.g.,household,individual,arecalledpaneldata;someauthorsprefertocallthemlongitudinaldata.ThemostfamousexampleofpaneldataisthePanelStudyofIncomeDynamics(PSID),collectedbytheInstituteofSocialResearchattheUniversityofMichigan.PSIDhascollectedinformationabouttheannualeconomicstatussuchasemploymentstatus,incomechanges,andmaritalstatusfrommorethan5000householdssince1968.Paneldataanalysishasbeenpopularintheliteratureofeconometricsduetoseveralmerits.Thesemeritsbroughtattentionofeconometricianstothepaneldataanalysis,andextensiveresearchhasbeendoneonthepaneldata;seeBalt-3agi(2001,p.5)andreferencestherein.Amongallmerits,theexemplarymeritofthepaneldataanalysisisthatitconsidersheterogeneityofcrosssectionunitsthroughtime-invariantindividualMoulton(1986)showedthatinappropriateusageofordinaryleastsquaresestimationwithoutconsideringtheindividualresultsinextremelyhugedownwardbiasofstandarderrorsoftheestimators.Anothermeritisattributedtothefundamentalstruc-tureofthepaneldata;repeatedobservationsonthesamecrosssectionunitisappropriatetostudythedynamicsofsomesocialphenomena,andhence,thepaneldataanalysispro-videspolicymakerswithgoodunderstandingofthem.Forexample,highunemploymenthasbeenplaguingmostcountriesduringthepostwarera,andpolicymakersstruggleagainstunemploymentsinceithasthedevastatingonoverallsocietysuchaspovertyandsurgesofcrimerate.Increasingtheunemploymentbduringthetimeoframpantunemploymentinthesocietyhasalwaysbeencontroversial.Onecriticismofunemploymentbisitcausesmoralhazard,thatis,itinduceslessofjob-search.Cross-sectionaldatacanprovideunemploymentrateatatimewhilepaneldataareabletoshowtheproportionofindividualswhoseemploymentstatuschangedoverthetimeperiods.Uponthetimeofdeterminingwhetherunemploymentischronicortransitory,paneldataareveryusefulandprovideguidanceforpolicymakersonhowtomakeagovernmentinitiativetoincreaseunemploymentbForexample,Ashenfelter(1978)investigatedhowfederaltrainingprogramfutureincomeofprogramparticipantsthroughcomparinggroupsofparticipantsandnonparticipants.Foreachgroup,heanalyzedtheincomerecordsbeforeandaftertheprogramanddrewaconclusionthatparticipatingintheprogrampositivelyanincreaseoffutureincome.Motivatedbytheabovementionedmeritsofpaneldata,Section2.2investigatesapplica-tionsofthem.d.estimationmethodtotheregressionmodelwithpaneldata.Inthesequel4werefertotheregressionmodelwithpaneldataaspanelregressionmodel.ConsiderpanelregressionmodelYit=x0it+"it;(1.2)"it=i+it;i=1;2;:::;n;t=1;2;:::;T;whereiandtdenotecrosssectionandtime,respectively,xit=(x1it;:::;xpit)02Rparenonrandomdesignvariables,and=(1;:::;p)02Rpistheparametervectorofinterest.Wedistinguishthevectorxitabovefromthescalarxitinthemodel(1.1)bywritingitinaboldfont.Thenotablebetweenthepanelregressionmodelandthemodel(1.1)isthefactthatthevariablesofthevarywithcrosssectionunitandtime.Anotheristhefactthattheerror"itisdecomposedintotwounobservablecomponents:time-invariantindividual(i)andtheremainderdisturbance(it)whichvarieswithcrosssectionandtime.WhenweimplementtheCvMtypem.d.estimationmethod,onecommonproblemweencounteristhechoiceoftheintegratingmeasure.Koul(2002)showedthatchoosingop-timalmeasureinthecaseofthei.i.d.errorsinthemodel(1.1)yieldsthemosttestimatorsofregressionparameters.Forexample,theLebesgueanddegeneratemeasuresarerecommendedforthetestimatorsoftheregressionparameterswhentheerrorfollowsthelogisticandLaplacedistributions,respectively;seeKoul(2002,p.207)forthedetail.Therefore,specifyingtheexactdistributionoftheerrorplaysanimportantroleinobtainingtheetestimatorsofregressionparameters.Chapter3discussesagoodtestingproblemwhichshedssomelightonspecifyingtheerrordistribution.Aclassicalgootestingproblem,i.e.,theproblemoftesting5whetherarandomsamplecomesfromaspdistributionorfromagivenparametricfamilyofdistributions,hasbeenofinteresttomanysforalongtime.Forexample,thenormalityofsamplehasbeencommonlyassumedinthevastliteratureofsocialandphysicalsciences.Sincethealresult,then,heavilydependsonnormalityassumption,gootestfornormalityhasbeenacriticalissue.Forgootestforgeneraldistributions,thevariousparametricandnonparamet-rictestshavebeenproposedintheliterature.Thebestknownexemplaryparametricandnonparametrictestsare˜2testandKolmogorov-Smirnov(K-S)test.ThemostattractiveadvantageoftheK-Stestisthatasymptoticdistributionofitsteststatisticunderthenullhypothesisdoesnotdependonthenulldistribution,whenaknowncontinuousdis-tribution.However,itlosesthispropertywhenaparametricfamilyofdistributions;see,e.g.,Durbin(1973).Seekingtestswhichpreservethedesirablefeatureofbeingdistributionfree,wecomeupwiththetwocelebratedmethods:Khmaladzemartingaletransformation(KMT)andempir-icallikelihood(EL).TheKMTandELmethodsareimplementedtotestforaparametriclocation-scalefamilyofdistributions;thenacomparisonofthesetwomethodsismadeinordertoshowwhichmethodissuperior.Tothatend,wereportempiricallevelsandpowersofKMTandELmethodsinChapter3.Inordertocomparethetwomethodsintermsoftheempiricallevels,wegeneratearandomsampleofobservationsfromthechosennulldistri-butionsandcomputethoseofthetwomethods.Similarly,wegeneratearandomsampleofobservationsfromvariousalternativedistributionsandcomputethecorrespondingempiricalpowersofthetwomethods.6Chapter2M.d.EstimationMethodonRegressionModelwithDependentErrors2.1RegressionmodelwithARerrorsRecallthelinearregressionmodel(1.1).Theerrors"i'sareassumedtoobeythefollowingautoregressivemodelofaknownorderq.Forsomeˆ=(ˆ1;:::;ˆq)02Rq,"i=Z0iˆ+˘i;(2.1)wheretheinnovationsf˘i;i=0;1;2;:::garei.i.d.r.v.'s,"i1isindependentof˘i,i2f0;1;g,andZi=("i1;:::;"iq)02Rq.Toensurethestationarityoftheerrorprocessweassumethatallrootsoftheequationzqˆ1zq1ˆq1zˆq=0arelessthanunityinmodulus.Inaddition,thedistributionof˘0isassumedtobesymmetricaroundzero,unknownotherwise.Next,weshallintroducethebasicprocessesandestimatorsofinteresthere.Tobegin7withweassumethefollowingaboutn(p+1)designmatrixX.(X0X)1exists;8np+1;maxix0i(X0X)1xi=o(1):(2.2)NowletA=(X0X)1=2andajdenotejthcolumnofA.LetD=((dik)),1in,1kp+1,beann(p+1)matrixofrealnumbersanddjdenotejthcolumnofD.AsstatedinKoul(2002,p.60),ifD=XA(i.e.,dik=x0iak),thenunder(2.2),nXid2ik=1;max1ind2ik=o(1);forall1kp+1:(2.3)Notethatthesymmetryof˘0aroundzeroimpliesthatoftheregressionerrors"i's.Thismotivatesonetointroduce,asinKoul(2002;5.3.1),Uk(y;t):=nXi=1a0kxinIYix0ityIYi+x0it<yo;U(y;t):=(U1(y;t);:::;Up+1(y;t))0;y2R;T(t):=ZkU(y;t)k2dH(y);t2Rp+1;whereHisa˙measureonRandsymmetricaround0,i.e.,dH(x)=dH(x);x2R.Subsequently,nebasT(b)=inftT(t):(2.4)Next,weanalogousestimatorsofˆ.Accordingly,letS(y;r):=1pnnXi=1ZinI"iZ0iryI"i+Z0ir<yo;y2R;(2.5)M(r):=ZkS(y;r)k2dH(y);r2Rq:8SinceZi'sareunobservable,wereplacethemin(2.5)bybZi=(b"i1;:::;b"iq),whereb"i=YiX0ib.Finally,wearemotivatedtointroducebS(y;r):=n1=2nXi=1bZinIb"ibZ0iryIb"i+bZ0ir<yo;(2.6)cM(r):=ZkbS(y;r)k2dH(y);andbˆascM(bˆ)=infrcM(r):Thissectionisorganizedasfollows.Sections2.1.1and2.1.2investigatetheasymptoticdistributionsofm.d.estimatorsofbothregressionandARparameters.FindingsofasamplesimulationsaredescribedinSubsection2.1.3.Thissectionalsoincludesadirectapplicationofthem.d.estimationmethodtosomerealtimeseriesdata-onehundredyearlydataofglobaltemperature.ProofsinthissectionarepostponedtoSection2.3.2.1.1AsymptoticdistributionofbInthissubsectionwederivetheasymptoticdistributionofbunderthecurrentsetup.Thebasicmethodoftheproofissimilartothatofsections5.4,5.5ofKoul(2002).ThismethodamountstoshowingthatT(+Au)isasymptoticallyuniformlyquadraticinubelongingtoaboundedsetandkA1(b)k=Op(1):Toachievethesegoalsweneedthefollowingassumptions,wheredik=x0iakandwhichinturnhaverootsinsection5.5ofKoul(2002).(A.1)ThematrixX0Xisnonsingularand,withA=(X0X)1=2,limsupn!1nmax1inkAxik2<1:9(A.2)TheintegratingmeasureHis˙andsymmetricaround0,andZ10(1F)1=2dH<1:(A.3)Foranyrealsequencesfang,fbng,bnan!0,limsupn!1ZbnanZf(y+x)dH(y)dx=0:(A.4)Fora2R,a+:=max(a;0),a:=a+a.Leti:=kAxik.Forallu2Rp+1,kukb,forall>0,andforall1kp+1,limsupn!1ZhnXi=1dikF(y+u0Axi+i)F(y+u0Axii)i2dH(y)2;wherecisaconstantnotdependingonuand.(A.5)Letek2Rp+1beanelementaryvectorwhosekthcoordinateis1.Foreachu2Rp+1andall1kp+1,ZhnXi=1dikF(y+u0Axi)F(y)u0ekf(y)i2dH(y)=o(1):(A.6)FhasacontinuousdensityfwithrespecttotheLebesguemeasureon(R;B).(A.7)0<R10frdH<1,r=1=2;1;2.Remark2.1.1.Notethat(A.1)implies(2.2)and(A.2)impliesR10(1F)dH<1.FromCorollary5.6.3ofKoul(2002),wenotethatinthecaseofi.i.d.errors,theasymptoticnormalityofbwasestablishedundertheweakerconditions(2.2)andR10(1F)dH<101.Theautoregressivedependenceoftheerrorsnowforcesustoassumethetwostrongerconditions(A.1)and(A.2).Remark2.1.2.HerewediscussexamplesofHandFthatsatisfy(A.2).ClearlyitisbyanymeasureH.Nextconsiderthe˙measureHgivenbydHfF(1F)g1dF,Facontinuousd.f.symmetricaroundzero.ThenF(0)=1=2andZ10(1F)1=2dH=Z10(1F)1=2F(1F)dF2Z11=2(1u)1=2du<1:Anotherusefulexampleofa˙measureHisgivenbyH(y)y.Forthismeasure,(A.2)isbymanysymmetricerrord.f.sincludingnormal,logistic,andLaplace.Forexample,fornormald.f.,wedonothaveaclosedformoftheintegral,butbyusingthewellcelebratedtailboundfornormaldistribution,seee.g.,Theorem1.4ofDurrett(2005),weobtainZ10f1F(y)g1=2dy(2ˇ)1=2Z10y1=2exp(y2=4)dy=(2=ˇ)1=2=4):RecallfromKoul(2002)thatthebandbˆcorrespondingtoH(y)yaretheextensionsoftheonesampleHodges-Lehmannestimatorofthelocationparametertotheaboveregressionandautoregressivemodels.Remark2.1.3.Considercondition(A.7).IffisboundedthenRf1=2dH<1impliestheothertwoconditionsin(A.7)forany˙measureH.ForH(y)y,Rf1=2(y)dy<1whenfisnormal,logisticorLaplacedensity.Inparticular,whendH=fF(1F)g1dFandFislogisticd.f.,sothatdH(y)dy,thisconditionisalso11Toproceedfurther,weneedanassumptiononthetimeseries"i;i2Z=f0;1;2g.LetFlmbethe˙generatedby"m;"m+1;:::;"l;ml.Thesequencef"j;j2Zgissaidtosatisfythestronglymixingconditionifforeveryl1,l(k):=supnjP(A\B)P(A)P(B)j:A2Fl;B2F1l+ko!0;ask!1.WerecallthefollowingtheoremfromAthreyaandPantula(1986),whichwillbeusedlater.Theorem2.1.1.Considertheqthorderautoregressiveprocessf"nggivenby"n=ˆ1"n1+ˆ2"n2++ˆq"nq+˘n.If(A.8.1)Eflogj˘1jg+<1,(A.8.2)thedistributionof˘1hasnon-trivialabsolutelycontinuouscomponents,(A.8.3)e0=("0;"1;:::;"1q)isindependentoff˘j;j=1;2;:::g,(A.8.4)f˘jgarei.i.d.r.v.'s,and(A.8.5)therootsofthecharacteristicequationzqˆ1zq1ˆq1zˆq=0arelessthanoneinmodulus,thenf"ngisstronglymixingandstationary.InadditiontotheassumptionsofTheorem2.1.1,weassumethefollowing.Thesequencef"ngisstronglymixingwithmixingnumberlsatisfying1Xk=1k2l(k)<1;8l1(A.8.6)12Theaboveassumptions(A.8.1)-(A.8.6)willbereferredtoasassumption(A.8).Next,fort2Rp+1,Q(t)=ZU(y;)+2A1(t)f(y)2dH(y):Wearereadytostatetheneededresults.Thetheoremestablishestheneededasymptoticuniformquadraticitywhilethecorollaryshowstheboundednessofasuitablystandardizedb.Theorem2.1.2.LetfYi;1ingbeasinthemodel(1.1).Assumethat(A.1)-(A.8)hold.Then,forany0<b<1,EsupkA1(t)bkT(t)Q(t)k=o(1):(2.7)Proof.SeeSection2.3.Corollary2.1.1.SupposethattheassumptionsofTheorem2.1.2hold.Thenforany>0,0<M<1thereexistsanN",and0<b<1suchthatP infkA1(t)bT(t)M!18nN:(2.8)Proof.SeeSection2.3.Theorem2.1.3.UndertheassumptionsofTheorem2.1.2,A1(b)=2jfj2Hg1Zf(y)U(y;)dH(y)+op(1):(2.9)Proof.TheprooffollowsfromTheorem2.1.2andCorollary2.1.1,asinthei.i.d.caseillustratedinKoul(2002).13Next, (x):=Zxf(y)dH(y)Zxf(y)dH(y);(2.10)snik:=dik ("i);znjk:=jXi=1snik;znj:=(znj1;:::;znj;p+1)0;nk:=Var(znnk)=Var( ("0))+2n1Xi=1nXj=i+1dikdjkE ("i) ("j);K(F;H):=Z10Z10(1F(x_y))f(x)f(y)dH(x)dH(y):SymmetryoftheFaround0yieldsE ("0)=0.NotethatVar( ("0))=8K(F;H)<1by(A.7).Moreover,n1Xi=1nXj=i+1jdikdjkE ("i) ("j)jnmaxid2ikn1n1Xi=1niXm=11=2l(m)k ("0)k2<1;by(2.38),Lemma2.3.1below,and(A.1).Letndenotecovariancematrixofznn.Writen=((nkh));1kp+1;1hp+1.Observethatnkh=Cov(znnk;znnh)=nXi=1nXj=1dikdjhE ("i) ("j);andhence,for1kp+1;1hp+1,jnkhj<1.WerecallatheoremfromMehraandRao(1975).Theorem2.1.4.Foreachn1,letf˘nj;1jngbestrongmixingwithmixingnumberl.Assumetheser.v.'stobeuniformlyboundedandflgsatisfying(A.8.6).Furtherassumemaxic2ni=˝2c!0;(2.11)14andliminfn!1˙2n=˝2c>0;(2.12)where˙2n=Var(Pni=1cni˘ni)and˝2c=Pni=1c2ni.Then˙1nnXi=1cni˘ni!DN(0;1):(2.13)AsaconsequenceofTheorem2.1.4,weobtainthefollowingresults.Corollary2.1.2.Assumenispositiveforallnp+1.Inaddition,assumethatsupu2Rp+1;kuk=1u01nu=O(1):(2.14)Then1=2nznn!DN(0;I(p+1)(p+1));(2.15)whereI(p+1)(p+1)isthe(p+1)(p+1)identitymatrix.Proof.Toprove(2.15),ittoshowthatforany2Rp+1,01=2nznnisasymptot-icallynormallydistributed.But01=2nznn=Pni=101=2nAxi ("i),whichisthesumasinTheorem2.1.4withcni=01=2nAxiand˘ni= ("i).Notethat˝2c=nXi=1(01=2nAxi)2=01n˙2n=EnnXi=1(01=2nAxi) ("i)o2=kk2:Also,wehavemax1inc2ni=˝2cmax1ink01=2nk2kAxik201n=max1inkAxik2!0;15byassumption(A.1).Finally,weobtainliminfn!1˙2n=˝2ckk2=(limsup01n)>0;by(2.14).Hence,thedesiredresultfollowsfromTheorem2.1.4.Corollary2.1.3.InadditiontotheassumptionsofTheorem2.1.2,assume(2.14).Then1=2nA1(b))Df2jfj2Hg1N(0;I(p+1)(p+1)):(2.16)Proof.Claim(2.16)followsfromCorollary2.1.2uponnotingthatZf(y)U(y;)dH(y)=znn:2.1.2AsymptoticdistributionofbˆInthissectionweinvestigateasymptoticdistributionofbˆ.RecallM(r)andcM(r)from(2.5)and(2.6).AsymptoticuniformquadraticityofM(ˆ+v=pn)invisillustratedintheSection7.4ofKoul(2002).SeealsoTheorem2.1.5below.SinceerrorsinMare,however,unobservable,Misnolongerforthepurposeofestimatingˆ.Tothatend,weusecMinstead.WeprovethatthesupremumovervincompactsetsoftheabsolutecebetweenM(ˆ+v=pn)andcM(ˆ+v=pn)isop(1).Asaconsequence,weaccomplishthedesiredasymptoticuniformquadraticityofcM(ˆ+v=pn)inv.Throughthissection,similarmethodoftheprooftothatusedintheSection7.4ofKoul(2002)willbeemployed.However,theproofswillbealittlebitmorecomplicatedsincetheycontainadditionaltermsincludingnianddni-see(2.18)below-asaconsequenceofusingresidualsinsteadofunobservableerrors.16Tobeginwithwestatethefollowinglemma.Lemma2.1.1.LetM1andM2bepositive(orsemi-positivennmatricessuchthatM1+M2=Inn:Let1and2beeigenvaluesofM1andM2.Then01;21:Proof.SinceM1andM2arepositive(semi-positive),1and2arenon-negative.Notethat0=det(M22I)=(1)ndet(M1(12)I);andhence(12)iseigenvalueofM1.Therefore,(12)0sinceM1ispositivewhichinturnimpliesthat21.Similarfactholdsfor1,therebycompletingtheproofofthelemma.Next,weneedtorecallthatforanym1m2realmatrixM,itsspectralnormiskMk2:=supu2Rm2;kuk=1kMuk2.RecallthatkMk2=maxf:isaneigenvalueofM0Mg1=2:(2.17)PartitionD=XAintothreeblocksD0=[D0iD0iD0+i]whereDi,Di,andD+iare(iq1)(p+1),q(p+1),(ni+1)(p+1)matricesrespectively.NotethatD0iDi+D0iDi+D0+iD+i=I(p+1)(p+1).Lemma2.1.117impliesD0iDihasnon-negativeeigenvalues,whicharelessthanorequaltoone.By(2.17),kDik21,andhence,foranyu2Rp+1kD0iukkDik2kukkuk;8i1:Moreover,foru2Rp+1,wealsohave,withci=Axi,max1inkDiuk2qkuk2max1inkcik2:Now,letu=A1(b)2Rp+1andv=n1=2(rˆ)2Rqwherer=(r1;:::;rq)02Rq.ni(u):="ib"i=(Yix0i)(Yix0ib)=u0ci;#n:=max1inkcik;(2.18)ni(u;v):=ni(u)qXk=1rkn;ik=u0ciˆ0Diuv0Diu=pn;dni(;u;v):=(1+qkˆk)#n+n1=2kZik+n1=2kuk+n1=2kvk+n1=2:Tobeginwithwestateasetofassumptions.ThefollowingassumptionsarefoundsimilartoonesintheSection7.4ofKoul(2002).Inordertorandomlyweightedresidualempiricalprocess,heusedameasurablefunctiongofr.v.'sasarandomweight.Bychoosingvariousg's,hedemonstratedm.d.estimatorsturnouttobewellcelebratedestimatorssuchasHodges-Lehmannestimator,theleastabsolutedeviationestimator,etc.Hereweonlyconsidereitherg(x)xorg1.Letgikdenoteg("ik).Hence,gikassumesonlythevaluesof"ikand1inthesubsequentassumptions.(B.1)(a)E"40<1andE˘2<1.18(b)Thed.f.F˘oftheinnovation˘hasacontinuousdensityf˘withrespecttotheLebesguemeasureon(R;R).(c)0<R10fr˘dH<1,r=0:5;1;2.(d)0<R10(1F˘)dH<1.(B.2)Forallkukb,kvkb,2R,andfork=1;2;:::;qEZn1nXi=1g2ikjF˘(y+n1=2v0Zi+ni(u;v)+dni(;u;v))F˘(y)jdH(y)=o(1):(B.3)Thereexistsaconstant0<c<1suchthatforall>0,kukb,kvkb,andforallk=1;2;:::;qliminfnP Zn1hnXi=1gikF˘(y+n1=2v0Zi+ni(u;v)+dni(;u;v))F˘(y+n1=2v0Zi+ni(u;v)dni(;u;v))i2dH(y)2!=1;wheregisasintheassumption(A.4).(B.4)Foreverykukb,kvkb,andfork=1;2;:::;qZn1hnXi=1gikF˘(y+n1=2v0Zi+ni(u;v))F˘(y)f˘(y)fn1=2v0Zi+ni(u;v)gi2dH(y)=op(1):(B.5)lims!0ZEhkZ1kf˘(y+skZ1k)i`dH(y)=EkZ1k`Zf`˘dH;`=1;2:19RecallS(y;r)in(2.5)andbS(y;r)in(2.6).LetSj(y;r)andbSj(y;r)denotetheirjthcoordinate,respectively.NotethatSj(y;r)=n1=2nXi=1"ijnI"iZ0iryIi+Z0ir<yo:Next,letQ(r):=qXj=1ZhSj(y;ˆ)+2pn(rˆ)0n1nXi=1"ijZif˘(y)i2dH(y):ThetheoremstatestheasymptoticuniformquadraticityofM,whichisprovedinKoul(2002,p.328).ThesecondtheoremstatesthatthebetweenMandcMissmallenoughtoestablishtheasymptoticuniformquadraticityofcMwhilethecorollaryshowstheboundednessofbˆ.Theorem2.1.5.Supposethattheautoregressivemodel(2.1)holds.Assumethat(B.1)-(B.5)hold.Then,forany0<b<1,suppnk(rˆ)bjM(r)Q(r)j=op(1):Theorem2.1.6.SupposetheassumptionsofTheorem2.1.5hold.Assumethatbusedforobtainingb"iin(2.1.6)A1kbk=Op(1).Then,forany0<b<1,suppnkrˆbjcM(r)M(r)j=op(1):Proof.SeeSection2.3.20Corollary2.1.4.SupposethattheassumptionsofTheorem2.1.5.Forany>0,0<B<1thereexistsanN,and0<b<1suchthatP infpnkrˆbcM(r)B!18nN:Proof.TheproofissimilartothatofLemma5.5.4ofKoul(2002).TheroleofninLemma5.5.4isplayedbyn1PiZiZ0i.Thentherestoftheproofisexactlythesamewiththeprovisothatweusenewn=n1PiZiZ0i.NotethatbythestationarityofZi;i2ZandtheErgodicTheorem,n1PiZiZ0i!E(Z1Z01),almostsurely.Moreover,thecovariancematrixE(Z1Z01)ispositiveThusforalltlylargen,n1PiZiZ0iisalmostsurelypositiveThisfactisusedthroughouttheproofsherewithoutfurthermention.Theorem2.1.7.UndertheassumptionsofTheorem2.1.5,pn(bˆˆ)=2jf˘j2Hn1nXi=1ZiZ0ig1ZS(y;ˆ)f˘(y)dH(y)+op(1):Consequently,pn(bˆˆ)!DN(0;(EZ1Z01)1~˝2)where~˝2:=Var( (˘1))=4(Rf2˘dH)2.Proof.TheprooffollowsfromTheorem2.1.6,Corollary2.1.4,andTheorem7.4.5ofKoul(2002).Remark2.1.4.Consideranestimatorof(EZ1Z01)1~˝2.NotethatVar( (˘1))=8K(F˘;H)whereK(F˘;H)isasin(2.10).WhenH(y)y,K(F˘;H)=1=3.Inordertoestimate21asymptoticvarianceofbˆ,weneedtoestimateEZ1Z01andRf2˘(x)dx.AvalidestimatorofEZ1Z01isn1PibZibZ0i,andhence,itremainstoestimateB(f˘):=Zf2˘(x)dx=Zf˘(x)dF˘(x):(2.19)Wereplacef˘andF˘withakerneltypedensityestimatorandtheempiricald.f.basedontheestimatedresiduals.LetGbeaprobabilitydensityonR,hnbeasequenceofpositivenumbers,hn!0.forx2R,b˘i:=b"ibZ0ibˆ;bFn(x):=n1nXi=1I(b˘ix);Fn(x):=n1nXi=1I(˘ix)bfn(x):=(nhn)1nXi=1G(xb˘i)=hn;fn(x):=(nhn)1nXi=1G(x˘i)=hnbBn:=Zbfn(x)dbFn(x)Lemma2.1.2.AssumethatF˘hasuniformlycontinuousdensityf˘w.r.t.theLebesguemeasureandf˘>0a.e..Then,foranydu;v,supx2RpnfbFn(x)Fn(x)gn1=2nXi=1Z0iv=pn+ni(u;v)f˘(x)=op(1):Proof.Notethatb˘i=˘iZ0iv=pnni(u;v):Therefore,pnfbFn(x)Fn(x)gn1=2nXi=1Z0iv=pn+ni(u;v)f˘(x)Wn1(x)+Wn2(x);22whereWn1(x):=n1=2nXi=1I(˘ix+Z0iv=pn+ni(u;v))F˘(x+Z0iv=pn+ni(u;v))n1=2nXi=1I(˘ix)F˘(x);Wn2(x):=n1=2nXi=1F˘(x+Z0iv=pn+ni(u;v))F˘(x)Z0iv=pn+ni(u;v)f˘(x):Thus,toprovethelemma,ittoshowthatsupx2RjWni(x)j=op(1);i=1;2:Togetherwiththeassumptionoftheuniformcontinuityoff˘and(u;v))=O(n1=2),weobtainsupxjWn1(x)j=op(1)afterapplyingtheTheorem2.2.3ofKoul(2002).NowconsiderWn2.Letzni:=Z0iv=pn+ni(u;v)andzn:=maxizni.Notethatzn=Op(n1=2).Therefore,supxjWn2(x)jn1=2nXi=1Zx+znixsupjyxznijf˘(y)f˘(x)jdypnznsupjyxznjf˘(y)f˘(x)j!p0;wheretheconvergencetozeroinprobabilityfollowsfromtheuniformcontinuityoff˘.Theorem2.1.8.InadditiontotheassumptionsofLemma2.1.2,assumethatthefollowingconditionshold.(i)hn>0;hn!0;nh1=2n!1.(ii)Gisabsolutelycontinuouswithitsa.e.derivative_GsatisfyingRj_G(z)jdz<1.23ThenjbBnB(f˘)j=op(1):Proof.TheproofisverysimilartothatofTheorem4.5.3ofKoul(2002).Onlyarisesinthepartofshowingkbfnfnk1=op(1)whichrequirestheaboveLemma2.1.2.Therestoftheproofisthesame.2.1.3Simulation&applicationtorealdata2.1.3.1SimulationstudyInthissection,wepresentasimulationstudycorrespondingtofoursymmetricinnovations:normal,Laplace,logistic,andmixtureofthetwonormals(MTN).Ineachcase,weestimateandˆbym.d.,generalizedleastsquares(GLS),ordinaryleastsquares(OLS),andGaussianmaximumlikelihood(GML)methods.Wereportempiricalbias,standarderror(SE),andmeansquarederror(MSE)oftheseestimators.ToobtainGLSestimators,weuseRpackageorcuttwhichimplementsCochrane-Orcutt(CO)iterativeestimationprocedure.Toobtainm.d.estimators,weemploy\two-stage"m.d.estimationmethodwhichisproposedbyKim(2016)todealwiththelinearregressionmodelwithautocorrelatederrors.Toimplementthetwo-stagem.d.estimationmethod,weuseRpackageKoulMde.ThepackageisavailablefromComprehensiveRArchiveNetwork(CRAN)athttps://cran.r-project.org/web/packages/KoulMde/index.html.Inthissimulationstudies,p=3andq=1,andH(y)y.Wesetthetrue=(2;3;1:5;4:3)0andtrueˆ=0:4.Foreachk=1;2;3,weobtainfxikgni=1in(1.1)asarandomsamplefromtheuniformdistributionon[0;50].TheinnovationhasaLaplace24distributionifitsdensityfunctionisfLa(x):=(2s1)1exp(x1j=s1)whilethedensityfunctionofLogisticinnovationisgivenbyfLo(x):=s12exp(x2j=s2)=(1+exp(x2j=s2))2:Whenwegeneratef˘gni=1,wesetmeanofnormal,Laplace,andlogisticinnovationsat0(i.e.,1=2=0)sinceweassumedtheinnovationissymmetric.Wesetthestandarddeviationofnormalinnovationat2whileboths1ands2aresetat5forLaplaceandlogisticinnovation,respectively.ForMTN,weconsider(1)N(0;22)+N(0;102)where=0:1.Thenwegeneratef"igni=1andfYigni=1subsequentlyusingmodels(1.1)and(2.1).Table1,2,and3reportbiases,SE'sandMSE'sofestimatorsforthesamplesizes50,100,and200,eachrepeated1,000times.AuthorusedHighPerformanceComputingCenter(HPCC)toacceleratethesimulations.AllofthesimulationsweredoneintheR-3.2.2.First,weconsiderthenormalinnovation.Wheninnovationisnormal,andnis50,theGMLestimatorsshowthesmallestbiasesamongallestimatorsofj's,j=0;1;2;3.However,asnincreases,thisfactdoesnotholdanymore;whennis200,theGMLestimatorsof0,1,and2displaythelargestbiases.WhenweconsidertheSE,theGLSestimatorsdisplaythesmallestSEregardlessofn;followingGLS,boththeGMLandthem.d.estimatorsshowsimilarSE's.WhenweevaluatetheperformanceoftheestimatorsintermsoftheMSE,weconclude,notsurprisingly,thattheGLSisthebest,theGMLandthem.d.aresimilarandthesecondbest,andtheOLSistheworst.25Fornon-Gaussianinnovations,wecomeupwithatconclusion:them.d.estimatorsoutperformallotherestimators.NotethattheOLSmethodexhibitsverypoorperformancewhichisnotsurprising.Also,notethattheGMLestimationmethodshowsthelossofcom-petitivenesscomparedtoGLSandm.d.estimatorssinceitassumesthenormalityoftheinnovation.Therefore,weleavetheOLSandGMLoutofdiscussionforthenon-Gaussianinnovations.NotethatweighingthemeritsoftheGLSandthem.d.estimatorsintermsofbiasishard.Forexample,fortheLaplaceinnovationwhenn=200,them.d.estimatorsof1andˆdisplaysmallerbiasesthantheGLSestimatorswhiletheoppositeistrueforj,j=0;2;3.Whenweconsiderlogisticinnovation,theresultissimilar.Them.d.estimatorsof1andˆagainshowsmallerbiasesthantheGLSestimatorswhiletheoppositeistrueforj,j=0;3;boththem.d.andtheGLSestimatorsof1showthesamebias.How-ever,them.d.estimatorsdisplaythesmallestSE'sregardlessofnandinnovations.Asaresult,them.d.estimatorsdisplaythesmallestMSE'sforallnon-Gaussianinnovations.OneremarkablefactistheMSE'scorrespondingtothem.d.estimatorsareapproximately60%ofthosecorrespondingtotheGLSestimatorswhentheinnovationisMTNwithn=100or200.Thereforeweconcludethatthem.d.estimatorsoutperformallotherestimatorsatthesechosennon-Gaussianinnovationdistributions,especiallywheninnovationdistributionisMTN.Tables2.4and2.5summarizetheaveragesoftheestimatedSE'sofm.d.estimatorsofj,j=0;1;2;3correspondingtofourinnovationswhennis100and200.Whenthesimulationisiterated1000times,weestimateSEofm.d.estimatorsofj,j=0;1;2;3eachtimeasdoneinRemark2.1.4.Next,wecalculatetheaverageof1000SE'sforeachestimatorofjandcompareitwithSEwhichisreportedinTables2.2and2.3.InordertodistinguishbetweentwoSE's,werefertotheaverageofSE'sandSEfromTables2.2and2.3as\estimated26OLSGLSGMLm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN0-0.18732.63336.9694-0.10082.42355.8837-0.01652.64627.0027-0.04042.51596.331210.00220.0560.00310.00110.04980.00256e-040.05050.00257e-040.05210.002720.00290.05490.0030.0010.04940.00243e-040.04970.00255e-040.05210.002730.00220.05330.00280.00150.04750.00238e-040.04830.00239e-040.04970.0025ˆ-0.07070.14120.0249-0.03340.15210.0242-0.04050.15280.025-0.0390.15680.0261La00.07071.96053.84840.0531.78173.17730.33571.99094.07640.06381.48432.20711-9e-040.0380.0014-8e-040.03220.001-0.00360.03320.0011-7e-040.02687e-042-0.0010.03960.00162e-040.03430.0012-0.00260.03530.0013-3e-040.02898e-0431e-040.03870.0015-4e-040.03330.0011-0.00320.03430.0012-6e-040.02737e-04ˆ-0.01640.06590.0046-0.00650.06620.0044-0.00850.06660.0045-0.00450.05650.0032Lo00.17164.662921.77190.14224.253418.11190.57014.724222.64270.13584.182717.51311-0.00360.10410.0108-0.00280.08960.008-0.00740.09220.0085-0.00320.08950.0082-0.00310.09360.0088-0.00240.08470.0072-0.00690.08550.0074-0.0020.08380.00734e-040.09950.00995e-040.08660.0075-0.00380.08930.0080.00110.08650.0075ˆ-0.06790.13180.022-0.02990.14160.0209-0.03780.14220.0217-0.03410.13820.0202M00.12044.100216.8260.0243.698613.68030.45414.107517.07770.06333.624213.13871-0.00340.08340.007-0.00220.07380.0054-0.00640.07550.0057-0.00220.07260.00532-0.00410.08590.0074-0.00210.07590.0058-0.00620.07750.0061-0.00330.07390.005530.00160.08710.00760.00120.07670.0059-0.00240.07830.00610.00190.07490.0056ˆ-0.07550.13320.0234-0.03980.14170.0217-0.04840.14290.0228-0.04430.1390.0213yN,La,Lo,andMdenotenormal,Laplace,logistic,andMTNinnovations,respectively.Table2.1:Bias,SE,andMSEwhenn=50OLSGLSGMLm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN0-0.04951.80323.2539-0.02221.59352.53970.1121.82193.3318-0.00971.66832.7834100.03730.0014-0.0010.03240.0011-0.00210.03330.0011-0.0010.03310.001120.00140.03810.00150.0010.0340.0012-4e-040.0350.00129e-040.03520.001232e-040.03950.00162e-040.03360.0011-0.00110.03480.0012-4e-040.03460.0012ˆ-0.03330.09430.01-0.01360.09620.0094-0.0180.09670.0097-0.01620.09840.0099La0-0.03992.5846.6788-0.11232.39275.73780.22552.58426.7289-0.10342.00574.033514e-040.05470.0030.00220.04740.0023-0.00120.04870.00240.00160.0410.00172-2e-040.05330.00289e-040.04640.0022-0.00290.04710.00220.0010.03990.00163-1e-040.05290.0028-3e-040.04680.0022-0.00310.04750.00232e-040.03990.0016ˆ-0.03050.08880.0088-0.01060.0910.0084-0.01570.09140.0086-0.00970.08010.0065Lo00.08913.227610.42510.06992.96278.78230.48333.25710.84140.1222.89698.40691-0.00490.06840.0047-0.00350.05940.0035-0.00780.06160.0039-0.00380.05810.00342-0.00110.07030.0049-8e-040.060.0036-0.0050.06190.0039-0.0010.05890.003530.00210.06740.00460.00130.06080.0037-0.00270.06130.00380.00110.0590.0035ˆ-0.03850.09280.0101-0.0190.09550.0095-0.02270.09620.0098-0.01910.09370.0091M00.02432.91328.48710.02812.64727.00840.35832.92768.69940.02922.02474.100312e-040.06110.0037-5e-040.0540.0029-0.00390.05530.0031-5e-040.04160.00172-8e-040.05950.003500.05230.0027-0.00350.05330.0029-3e-040.03940.001630.0010.05950.00357e-040.05260.0028-0.00250.05410.002900.03940.0016ˆ-0.02640.08490.0079-0.00710.08790.0078-0.01120.08890.008-0.00620.07230.0053Table2.2:Bias,SE,andMSEwhenn=100SE"and\simulatedSE,"respectively.AsshownintheTables2.4and2.5,therightchoiceofbandwidthhnresultsinthesmallrencebetweenestimatedSEandsimulatedSEofm.d.estimatorsofj'sexcept0.Forexample,theerencebetweentwoSE'sofeachm.d.estimatorofj-except0-rangesfrom0to0.0021.Furthermore,theoptimalbandwidth27OLSGLSGMLm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN0-0.03041.33331.7785-0.01471.16421.35560.11741.34771.83-0.02291.19441.42711-4e-040.02818e-04-1e-040.02356e-04-0.00150.02456e-04-3e-040.0246e-042-3e-040.02717e-04-6e-040.02335e-04-0.00180.02426e-04-6e-040.02416e-0430.00140.02657e-047e-040.02275e-04-6e-040.02356e-040.00110.02356e-04ˆ-0.01880.06560.0047-0.0080.06650.0045-0.01030.06680.0046-0.00840.06780.0047La00.07071.96053.84840.0531.78173.17730.33571.99094.07640.06381.48432.20711-9e-040.0380.0014-8e-040.03220.001-0.00360.03320.0011-7e-040.02687e-042-0.0010.03960.00162e-040.03430.0012-0.00260.03530.0013-3e-040.02898e-0431e-040.03870.0015-4e-040.03330.0011-0.00320.03430.0012-6e-040.02737e-04ˆ-0.01640.06590.0046-0.00650.06620.0044-0.00850.06660.0045-0.00450.05650.0032Lo00.09042.36215.58760.06122.0744.30510.48282.40966.03940.06982.01124.049710.00140.04710.00220.00140.04060.0017-0.00250.0420.00180.00120.03980.00162-0.00240.04950.0025-0.00220.04110.0017-0.00630.04360.0019-0.00220.040.00163-0.00110.04870.0024-3e-040.04210.0018-0.00450.04330.0019-4e-040.04080.0017ˆ-0.01540.0670.0047-0.00540.06780.0046-0.00760.06780.0047-0.00510.06550.0043M0-0.01952.11314.4655-0.03651.88343.54850.25632.14214.6544-0.03771.38691.924810.00130.04340.00190.00140.03680.0014-0.00310.03760.00141e-040.02667e-042-0.00110.04290.0018-7e-040.03570.0013-0.00290.03860.00151e-040.02788e-0430.00120.04130.00170.00130.03550.0013-0.00320.03850.0015-2e-040.02778e-04ˆ-0.01510.06330.0042-0.00530.06390.0041-0.00960.06380.0042-0.00570.04970.0025Table2.3:Bias,SE,andMSEwhenn=200hn-whichyieldstheestimatedSEclosesttothesimulatedSE-isshowntobeproportionaltonawherea>0forallinnovations.FornormalandLaplaceinnovations,hn/n1:32;forothertwoinnovations,hn/n1.NLaTable2.2hn=0.80.91Table2.2hn=0.50.60.7n=10001.66831.55621.57521.59002.00571.75431.81071.854810.03310.03440.03490.03520.04100.03870.03990.040920.03520.03450.03490.03520.03990.03870.04000.040830.03460.03470.03510.03550.03990.03860.03990.0409NLaTable2.3hn=0.340.360.38Table2.3hn=0.220.240.26n=20001.19441.07061.07881.08601.48431.24731.26431.279410.02400.02380.02400.02420.02680.02770.02800.028420.02410.02390.02410.02420.02890.02770.02810.028430.02350.02380.02400.02410.02730.02770.02810.0285Table2.4:AveragesofSE'sofbj'swithnormalandLaplaceinnovations28LoMTable2.2hn=1.31.41.5Table2.2hn=0.50.60.7n=10002.89692.61292.63372.65222.02471.73961.79551.838010.05810.05810.05850.05890.04160.03860.03990.040820.05890.05790.05840.05880.03940.03870.03990.040930.05900.05800.05840.05880.03940.03870.03990.0409LoMTable2.3hn=0.660.680.7Table2.3hn=0.280.30.32n=20002.01121.79551.80811.81981.38691.24701.26471.280310.03980.03970.04000.04030.02660.02750.02790.028220.04000.03970.03990.04020.02780.02740.02780.028230.04080.03990.04020.04040.02770.02750.02790.0282Table2.5:AveragesofSE'sofbj'swithlogisticandMTNinnovations2.1.3.2GlobaltemperatureInthissection,weapplym.d.estimationmethodtorealdata:onehundredyearlydataofglobaltemperature.ThedatacanbeobtainedatInternationalPanelonClimateChange(IPCC)website:http://www.ipcc.ch/.TheIPCCwebsiteprovideswithfreelyaccessibledatabase.TheleftpanelofFigure2.1showstheseriesofglobaltemperatureoverthepastFigure2.1:TheIPCCglobaltemperature(left)anderrors(right)from1916to2016100years.Asshowninthetheupwardlineartimetrendisobserved,whichmaymeanthereisanevidenceofglobalwarming.Therefore,weconsideralinearregressionmodelwith29t0=1916yt=0+1(tt0)+"t;t=1917;1918;:::;2016;(2.20)whereytisglobaltemperatureinyeart.TheOLSestimatesofthe0and1are-0.3432and0.0079,respectively.Next,weerrorb"ols;t=yt(0:03432+0:0079(tt0)):TherightpanelofFigure2.1showstheseriesofderrors;itisnottoseethereexistacorrelationbetweenthelaggederrors.Tocarryoutthecorrelationanalysisinthesequel,wechecksampleautocorrelationfunction(ACF)andpartialautocorrelationfunction(PACF)oftheseriesoftheinterest.Figure2.2showsthesampleACFandPACFoftheb"ols;t'softhe15lags.First,theACFexhibitsamixtureoftheslowexponentialdecayanddampedsinusoidalcomponent.Second,theseriesofb"ols;t'sdisplayssigntPACFcotsuptothefourthlag.ThesetwofactssuggestthattrueunderlyingerrormodelislikelytobeARmodeloforder4.Bearingthesefactsinmind,wewillARmodelofvariousorderstotheerrorsforthepurposeofspecifyingitsunknownorder.Undertheassumptionthat"tobeysARmodel(2.1)oftheunknownorder,weestimateboththeregressionandARparametersofmodel(2.20);fortheordersofARmodel,1,2,3,and4aretried.Table2.6summarizesm.d.andGMLestimationresults;COestimationisnotpresentedsinceorcuttworksforARerrorsoforder1only.Boththem.d.andtheGMLestimationmethodsgivesimilarestimatesofandˆexceptwhenARmodeloforder2isAssummarizedinthetable,thelineartrendofthem.d.estimationmethodbecomessteeperasARmodelofhigherorderistheslopescorrespondingtotheARorder1,2,3,and4are0.081,0.084,0.087,and0.096,respectively.TheGMLestimationmethod30Figure2.2:Lagvs.sampleACFandPACFofOLSerrorsexhibitsthesimilarresult.Itwasobservedabovethatwheninnovations(n=100)arenormal,Laplace,logistic,andMTN,theestimatedSEwhichistheclosesttothesimulatedSEcanbeobtainedwithbandwidthsof0.9,0.6,1.4,and0.6,respectively;see,e.g.,Tables2.4and2.5.WiththeassumptionthattheerrorobeysARmodelsoforders1,2,3,and4,theestimatedSEofthem.d.estimatorof1and95%interval(CI)of1correspondingtothesebandwidthsarereportedinTable2.7.Thevaluesinthetableareexpressedinthousandths:e.g.,SEcorrespondingtohn=0:6andARorderof1is1:4559103.WhenARmodelofthesameorderisconsidered,theSE'scorrespondingtothetbandwidthsonlybymillionths.Asaresult,95%CI'sfor1arealmostthesame.ButasARmodelofthetorderisconsidered,quitetSE'sarereported;soaretCI's.AllCI'sarewell-separatedfromzero,andhencewecanconclude16=0,whichimpliesthatthereisastrongevidenceoftheglobalwarming.Next,weproceedtocheckwhethertheassumptionofindependenceoftheinnovationsismetinordertoconcludeARmodelofwhichorderprovidesthebestTothatend,we31Orderofm.d.GMLARerrorˆˆ1(-0.3207,0.0081)'0.6842(-0.3540,0.0085)'0.71462(-0.3390,0.0084)'(0.6248,0.0946)'(-0.4430,0.0099)'(0.6500,0.1378)'3(-0.3540,0.0087)'(0.5992,0.0252,0.1682)'(-0.3574,0.0089)'(0.6152,0.0216,0.1897)'4(-0.4025,0.0096)'(0.5298,-0.0389,0.0420,0.3103)'(-0.3268,0.0091)'(0.5584,0.0076,0.0571,0.2757)'Table2.6:M.d.andGMLestimatesofandˆOrderofhn=0:6hn=0:9hn=1:4ARerrorSECISECISECI11.4559(5.2728,10.9800)1.4556(5.2734,10.9794)1.4554(5.2738,10.9790)21.5202(5.4452,11.4045)1.5188(5.4479,11.4018)1.5188(5.4478,11.4019)31.6190(5.6124,11.9589)1.6176(5.6150,11.9562)1.6176(5.6150,11.9563)41.9522(5.8703,13.5232)1.9485(5.8776,13.5159)1.9485(5.8776,13.5159)Table2.7:SEofm.d.estimatorof1and95%CIobtaininnovationsofARmodelbysolvingb˘md;t=b"md;tbZmd;tbˆmd:whereb"md;t'saretheerrorsafterthetwo-stagem.d.estimation,bZmd;tisavectorofthelaggedb"md;t's,andbˆmdisthetwo-stagem.d.estimatoroftheARparameter.Figure2.3showsthesampleACF'softheinnovationscorrespondingtotheARmodelsoforder1(topleft),2(topright),3(bottomleft),and4(bottomright).Itcanbeseenthattheindependenceassumptionbecomesviableastheorderincreases;thedependenceseemstodisappearwhenARorderis4asexpectedfromtheACFandthePACFoferrors.WhenARmodeloforder1istherearetwospikesoutoftheapproximate95%band[-0.2,0.2].However,thereisnospikeoutofthebandifARmodeloforder4istotheerrors.Therefore,weconcludethatARmodeloforder4providesbetterthanARmodelsofotherorders:Figure2.4-thesamplePACF'softheinnovations-theconclusion.32Figure2.3:Lagvs.sampleACFwithARmodelsoforder1,2,3,and4.33Figure2.4:Lagvs.samplePACFwithARmodelsoforder1,2,3,and4.342.2PanelregressionmodelInthissection,weapplythem.d.estimationmethodtopanelregressionmodelandinvesti-gatetheasymptoticpropertiesofm.d.estimatorsoftheregressionparameters.Wediscussonlythecasewherethenumberofcrosssectionunitsisrelativelylargerthanthelengthoftimeperiods,whichiscalledshortpaneldata.Section2.2.2willdiscusstheasymptoticpropertieswhenthenumberofcrosssectionunitsgoestoywhiletimeperiodsarekept.2.2.1RandommodelRecallthepanelregressionmodel(1.2).Yi=26666666664Yi1Yi2...YiT37777777775T1;Xi=26666666664x1i1x2i1xpi1x1i2x2i2xpi2............x1iTx2iTxpiT37777777775Tp;"i=26666666664"i1"i2..."iT37777777775T1;Y=26666666664Y1Y2...Yn37777777775nT1;X=26666666664X1X2...Xn37777777775nTp;"=26666666664"1"2..."n37777777775nT1:ThenstandardpanelregressionmodelcanbeexpressedasY=X+"35where=(1;2;:::;p)02Rp.Asseenin(1.2),theerrorcanberewrittenassumoftime-invariantindividualandremainderdisturbance:"it=i+it.Thepanelregressionmodelwhichassumesthattime-invariantindividualiisuncorrelatedwithxitiscalledrandomctmodel.Anotherclassicalassumptionontheerrorintherandommodelisthatcorrelationovertimedoesnotchangeacrossthepanel,i.e.,E("it"is)=˙2+˙2.However,errorsovercross-sectionsareindependent,i.e.,E("it"js)=0foralli6=j.AssumethatE()=0;E(0)=˙2InTnT;E(ij)=0;E(ii)=˙2;E(ijt)=0;E(i)=0;1i;jn;1tTwhere=(11;:::;nT)02RnT.LetdenoteE("i"0i).Then,wehave=˙2ITT+˙2ii0=26666666664˙2+˙2˙2˙2˙2˙2+˙2˙2............˙2˙2˙2+˙237777777775:whereiisaT1vectorofones.Letbethecovarianceoftheerrorforalltheobservations,i.e.,E(""0).Then,=Inn=266666666640000............003777777777536Beforeweinvestigatetheasymptoticbehaviorofthem.d.estimators,weneedassump-tionsonthestructureofX,similartothoseintheSection2.1.Hence,wereplaceXandxiwithXandxitin(2.2).Similarly,letA:=(X0X)1=2andD:=XA.NotethatDcanbepartitionedinton1blocksasfollows:D=266666666666664X1A...XiA...XnA377777777777775=266666666666664D1...Di...Dn377777777777775whereDi:=XiA=266666666666664di11di1kdi1p...............dit1...ditk...ditp...............diT1diTkdiTp377777777777775for1in,1tT,and1kp.Notethatforall1kpnXi=1TXt=1d2itk=1:(2.21)SimilartowhatwasdoneintheSection2.1.1forasymptoticuniformquadraticity,we37thedistancefunctionUk(y;b):=nXi=1TXt=1ditknIYitx0itbyIYit+x0itb<yo;U(y;b):=(U1(y;b);:::;Up(y;b))0;y2R;L(b):=ZkU(y;b)k2dH(y);b2Rp;whereHisa˙measureonRandsymmetricaround0,i.e.,dH(x)=dH(x);x2R.Subsequently,debasL(b):=infbL(b):(2.22)FortheanalogueofQintheTheorem2.1.2,forb2RpL(b):=ZkU(y;)+2A1(b)f(y)k2dH(y):2.2.2AsymptoticdistributionofbInthissectionwederivetheasymptoticdistributionofbunderthecurrentsetup.ThebasicmethodoftheproofissimilartothatoftheSection2.1.1.Westatethenecessaryassumptions.LetFdenotethed.f.of".(C.1)ThematrixX0Xisnonsingularand,withA=(X0X)1=2,limsupn!1nmax1inmax1tTkAxitk2<1:38(C.2)TheintegratingmeasureHis˙andsymmetricaround0,andZ10(1F)dH<1:(C.3)Foranyrealsequencesfang,fbng,bnan!0,limsupn!1ZbnanZf(y+x)dH(y)dx=0:(C.4)Fora2R,a+:=max(a;0),a:=a+a.Letit:=kAxitk.Forallu2Rp,kukb,forall>0,andforall1kp,limsupn!1ZhnXi=1TXt=1ditkF(y+u0Axit+it)F(y+u0Axitit)i2dH(y)2;wherecdoesnotdependonuand.(C.5)Letek2Rpbeelementaryvectorwhosekthcoordinateis1.Foreachu2Rpandall1kp+1,Z24nXi=1TXt=1ditkF(y+u0Axit)F(y)u0ekf(y)352dH(y)=o(1):(C.6)FhasacontinuousdensityfwithrespecttotheLebesguemeasureon(R;B).(C.7)0<R10frdH<1,r=1;2.Now,wearereadytostatetheresult.Theorem2.2.1.AssumefYit;1in;1tTgbeinthemodel(1.2).Assumethat39(C.1)-(C.7)hold.Thenforany0<c<1EsupkA1(b)ckL(b)L(b)k=o(1):(2.23)Corollary2.2.1.SupposethattheassumptionsofTheorem2.2.1hold.Thenforany">0,0<z<1thereexistsanN",and0<c<1(dependingon"andM)suchthatP infkA1(b)cL(b)z!1";8nN":(2.24)ProofsofTheorem2.2.1andCorollary2.2.1areverysimilartothoseofTheorem2.1.2andCorollary2.1.1;however,theproofsaremuchsimplersincethecorrelationoferrorsbetweentcrosssectionunitsiszero.Therefore,theyarenotpresentedhere.Theorem2.2.2.UndertheassumptionsofTheorem2.2.1,A1(b)=2jfj2Hg1Zf(y)U(y;)dH(y)+op(1):(2.25)Proof.ByTheorem2.2.1andCorollary2.2.1,L(b)isuniformlylocallyasymptoticallyquadraticandkA1(b)k=Op(1):Therefore,theproofisstraightforward,asini.i.d.caseillustratedinKoul(2002).Recall inthe(2.10)andzn:=(zn1;:::;znp)02Rpwhereznk:=nXi=1TXt=1ditk ("it);k=1;2;:::p:40Letndenotecovariancematrixofznandnkhdenoteits(k;h)entry.Notethatnkh=E24nXi=1TXt=1ditk ("it)nXj=1TXs=1djsh ("js)35=nXi=1TXt=1TXs=1ditkdiskE ("it) ("is)nmax1inmax1tTd2itkT2k ("11)k22<1Thesecondequalityfollowsfromtheindependenceof"itand"jsfori6=j;theinequalityfollowsfrom(A.1)and(A.7).LetakdenotekthcolumnvectorofA.Rewritenkh=a0k24nXi=18<:TXt=1TXs=1xitx0itE ("it) ("is)9=;35ah:(2.26)Consider2Rp.Leti:=PTt=101=2nAxit ("it).Bysymmetryoferrors,Ei=0.AlsonotethatnXi=1E2i=nXi=124ETXt=1TXs=101=2nAxit ("it)01=2nAxis ("is)35=01=2nA24nXi=18<:TXt=1TXs=1xitx0isE ("it) ("is)9=;351=2n=kk2wherethelastequalityfollowsfrom(2.26).RecallLyapunov'sTheorem.41Theorem2.2.3.(Lyapunov'sTheorem)SupposefX1;X2;:::gisasequenceofindepen-dentrandomvariables,eachwithexpectedvalueiandvariance˙2i.s2n=nXi=1˙2i:Ifthereissome>0suchthatlimn!11s2+nnXi=1EjXiij2+=0;(2.27)then1snnXi=1(Xii)!DN(0;1):AsaconsequenceofTheorem2.2.3,weobtainthefollowingcorollary.Corollary2.2.2.Assumenispositiveforallnp.Inaddition,assumethatsupu2Rp;kuk=1u01nu=O(1):(2.28)Then1=2nzn!DN(0;Ipp);(2.29)whereIppistheppidentitymatrix.Proof.Toprove(2.29),itcestoshowthatforany2Rp,01=2nznisasymptoticallynormallydistributed.Notethat01=2nzn=Pni=1iwheref1;2;:::;ngisasequenceof42independentrandomvariables.Observethat(C.1),(C.7),and(2.28)implyjij01=2nTXt=1Axit ("it)(0n)1=2Tmax1inmax1tTkAxitkj ("it)j=O(n1=2):Consequently,forany>0,wehavenXi=1Ejij2+=O(n=2)nXi=1E2i!0:Therefore(2.29)followsafterthedirectapplicationofTheorem2.2.3.Corollary2.2.3.Assumethat(C.1)-(C.7)hold.Then1=2nA1(b)!Df2jfj2Hg1N(0;Ipp)(2.30)Proof.Notethatzn=Zf(y)U(y;)dH(y);Hence(2.30)directlyfollowsfromCorollary2.2.2.2.2.3OtherpaneldataestimatorsInthissection,weintroduceotherestimatorscommonlyusedintheliteratureofeconomet-rics.AllarevariationsofOLSestimators.Recallthemodel(1.2).Whenweaverageresponsevariables,designvariables,anderrorsoveralltimeperiodsandsubtracttheaveragesfrom43theoriginalvariables,wethenobtainbetweenmodelYi=x0i+"i;i=1;2;:::;n;whereYi=T1PTt=1Yit,xi=T1PTt=1xit,and"i=T1PTt=1"it.Thebetweenestima-torisnothingbutOLSestimatorofregressionparameterofthebetweenmodel.Thepaneldatahasaspecialfeature-thatthevariablesvarywithtime-whichthemeritsofpaneldataoriginatesin.Asseenabove,thebetweenmodellosesthisfeaturebyaveragingtheoriginalvariablesoveralltimeperiods.Toredressthisissue,anotherpanelregressionmodelwhichiscalledwithinmodelisproposed:YitYi=(xitxi)0+("it"i);i=1;2;:::;n;t=1;2;:::;T:ThewithinestimatoristheOLSestimatorobtainedfromthewithinmodel.Notethatthetime-invariantindividualidoesnotexistinthewithinmodelaftertheaverageoftheerrorissubtractedfromtheoriginalerror.Anotherwell-celebratedpaneldataestimatorisrandomctestimator.TherandomestimatorisfeasibleGLSestimatorofthemodel(1.2);itcanbeobtainedbyapplyingOLSestimationtothefollowingmodelYitbYi=(xitbxi)0+("itb"i);wherebisconsistentfor:=1˙2=q˙2+T˙2.NotethatOLSandwithinestimatorarespecialcaseoftherandomestimatorscorrespondingtob=0andb=1,respectively.Inordertoobtainthem.d.estimatorsinthenextsection,weapplythem.d.estimationmethodtothewithinmodelinsteadofthemodel(1.2).Forthecaseoftheregression44modelwithi.i.d.non-Gaussianerrors,Kim(2016)empiricallyshowedthem.d.estimatorsoutperformOLSestimators.Notethattheextentofthedependenceoferrorsinthepanelregressionmodelisbetweenthoseoftheregressionwithi.i.d.andARerrors.Sincethesuperiorityofthem.d.estimatorstotheOLSfamilyisdemonstrated,theyareexpectedtostillremainsuperiortothepaneldataestimators.2.2.4SimulationInthissectionwepresentasimulationstudycorrespondingtosixteenpairsofsymmetricin-dividualandremainderdisturbance.Bothindividualandremainderdisturbancearegeneratedfromnormal,Laplace,logistic,andMTN.Forexample,Table2.8reportsthecorrespondingtothenormalindividualwithnormal,Laplace,logistic,andMTNremainderdisturbanceswhenn=10andT=5.ThesimulationsetupisverysimilartooneintheSection2.1.3.Thebias,SE,andMSEoftheestimatorsofregressionparam-etersarereportedinthetables;foreasycomparisonpurpose,weanalyzetheandevaluatetheperformanceofestimatorsintermsofMSEsincethosewhichdisplaytheleastMSEalsoapproximatelydisplaytheleastbiasandSE.Incaseofthenormalindividualthewithinestimatorsshowthebestperformanceregardlessofremainderdisturbance;therandomestimatorsdisplayalmostthesameperformanceasthewithinestimators.Them.d.estimatorsfollowsthewithinandtheran-domestimators,and,notsurprisingly,OLSestimatorsaretheworst.Forthenon-Gaussianindividualthem.d.estimatorsoutperformotherpaneldataestimatorsasexpected.ThesimulationstudyinthissectionexhibitsthesimilarpatterntooneintheSection2.1.3;thesuperiorityofthem.d.estimatorstootherestimatorsissalientespeciallywhenthemodelcontainstheMTNindividual45OLSWithinREm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN10.00060.05340.00280.00190.03880.00150.00180.03880.00150.00180.03980.00162-0.00170.05500.00300.00020.03890.00150.00020.03950.00160.00050.04000.001630.00310.05040.00260.00120.03830.00150.00160.03850.00150.00140.03950.0016La1-0.00340.06570.00437e-040.03610.00134e-040.03610.00138e-040.03710.001420.00430.0720.00520.00140.03880.00150.00140.03870.00150.00130.03980.00163-0.0040.06620.0044-4e-040.03730.0014-7e-040.03730.0014-9e-040.03840.0015Lo10.0010.08870.00793e-040.03790.00146e-040.03810.00154e-040.0390.0015200.08730.00760.00130.03930.00150.00130.03950.00160.00110.04020.001630.00150.08740.007600.03860.00151e-040.03870.0015-3e-040.04020.0016M10.00150.07560.0057-9e-040.03830.0015-6e-040.03840.0015-9e-040.03950.001620.00210.0770.0059-3e-040.03760.001400.03740.0014-4e-040.03840.001538e-040.07580.00580.00180.03810.00150.00130.03830.00150.00180.03910.0015yWithinandREdenotethewithinandrandomestimators,respectively.Table2.8:Bias,SE,andMSEofestimators:isnormalwithn=10andT=5.OLSWithinREm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN16e-040.06710.00450.00170.06730.00450.00120.06740.00450.00160.06690.004520.00250.06630.00444e-040.07080.0050.00150.06890.00487e-040.06990.00493-0.00120.06610.0044-0.00260.06640.0044-0.0020.0680.0046-0.00280.06550.0043La1-0.01380.14050.0199-0.01480.13860.0194-0.01490.13760.0192-0.01530.13830.019420.04870.36090.13260.04610.3590.1310.0470.35880.1310.04680.35880.13093-0.03390.24840.0629-0.02970.24540.0611-0.03080.24570.0613-0.03080.24520.0611Lo10.00550.09060.00820.0020.0660.00440.00230.06690.00450.00130.06590.00432-9e-040.09270.00860.00170.06680.00450.00130.06720.00450.00130.06650.00443-0.00610.09170.0085-9e-040.0660.0044-0.00120.06650.0044-4e-040.06660.0044M1-0.00320.0890.0079-5e-040.07010.0049-0.00110.07070.005-8e-040.06890.004729e-040.09640.00930.00390.06810.00470.00420.06910.00480.00390.0680.00463-0.00120.09090.0083-5e-040.07090.005-5e-040.07090.00500.07120.0051Table2.9:Bias,SE,andMSEofestimators:islogisticwithn=10andT=5.OLSWithinREm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN10.00170.05920.0035-5e-040.05410.0029-6e-040.05420.00294e-040.05060.002625e-040.0590.0035-1e-040.0510.0026-1e-040.05040.0025-4e-040.04880.0024300.05850.0034-8e-040.05180.0027-6e-040.05260.0028-0.00110.04880.0024La1-0.01090.12360.0154-0.01060.11370.0131-0.00990.11380.0131-0.01030.11190.012620.03240.30740.09550.03020.30330.09290.03050.30320.09280.03110.30270.09263-0.01740.21260.0455-0.020.20560.0427-0.01890.20620.0429-0.01960.20490.0424Lo1-0.01380.14050.0199-0.01480.13860.0194-0.01490.13760.0192-0.01530.13830.019420.04870.36090.13260.04610.3590.1310.0470.35880.1310.04680.35880.13093-0.03390.24840.0629-0.02970.24540.0611-0.03080.24570.0613-0.03080.24520.0611M1-0.0050.08080.0066-0.00250.05280.0028-0.00290.05190.0027-0.00150.04870.002420.00390.08250.00680.00190.05610.00320.00210.05610.00310.00190.05220.0027300.08510.0073-0.0020.05410.0029-0.00140.05440.003-0.00190.05010.0025Table2.10:Bias,SE,andMSEofestimators:isLaplacewithn=10andT=5.46OLSWithinREm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN10.00380.07490.00560.00110.08230.00680.00210.08320.00690.00130.05870.00342-9e-040.0770.00592e-040.07710.00591e-040.07790.0061-8e-040.05450.0033-9e-040.07360.0054-0.00350.08180.0067-0.00310.080.0064-3e-040.05720.0033La1-0.00690.09720.0095-0.00360.09090.0083-0.00410.08950.008-0.00270.07130.005120.00670.16010.02570.00570.15610.02440.00670.15630.02450.00460.14560.02123-0.00310.12340.0152-0.00310.11790.0139-0.00280.11730.0138-0.00560.10490.011Lo1-0.01410.13690.0189-0.01050.12560.0159-0.00960.12530.0158-0.00980.11180.012620.02920.30040.09110.03030.29510.0880.02980.29520.0880.02920.28950.08473-0.0210.21360.0461-0.01490.2060.0427-0.01490.20630.0428-0.01650.19740.0392M10.00230.09660.0093-0.00370.08110.0066-0.00170.08010.0064-0.0020.05750.003321e-040.09470.0092e-040.08080.006500.07910.00633e-040.05780.00333-0.00110.09130.00830.00250.07930.00630.00170.07840.00614e-040.05710.0033Table2.11:Bias,SE,andMSEofestimators:isMTNwithn=10andT=5.OLSWithinREm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN1-3e-040.02717e-04-2e-040.01743e-04-2e-040.01743e-04-3e-040.01763e-0427e-040.02838e-04-6e-040.01753e-04-5e-040.01743e-04-6e-040.01773e-043-0.00110.02627e-04-1e-040.01743e-04-2e-040.01743e-0400.0183e-04La10.00280.03750.00142e-040.01693e-042e-040.0173e-043e-040.01743e-0420.00150.03620.00130.0010.01693e-040.0010.01693e-040.00110.01723e-043-6e-040.03790.0014-0.00130.01743e-04-0.00140.01743e-04-0.00140.01793e-04Lo1-4e-040.04460.002-1e-040.01743e-04-1e-040.01743e-04-3e-040.01783e-042-8e-040.04690.002200.01713e-0400.01713e-0400.01773e-0430.00150.04490.002-0.00110.01753e-04-0.00110.01763e-04-0.0010.01793e-04M1-0.01010.12180.0149-0.01260.11480.0133-0.01260.11480.0133-0.01250.11480.013320.04110.34240.11890.03890.34060.11750.03890.34060.11750.03880.34060.11753-0.0270.23050.0539-0.02570.22750.0524-0.02580.22750.0524-0.02590.22750.0524Table2.12:Bias,SE,andMSEofestimators:isnormalwithn=20andT=10.OLSWithinREm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN10.0010.03490.0012-0.00110.03080.001-0.0010.03069e-04-8e-040.03019e-042-0.00240.0360.0013-0.0020.03320.0011-0.00220.0330.0011-0.00190.03220.00130.00120.03310.00111e-040.03250.00112e-040.03220.0011e-040.03120.001La1-0.00650.08810.0078-0.00670.08350.007-0.00680.08340.007-0.00660.08330.00720.01750.23570.05590.01810.23390.05510.01810.23390.05510.01760.23390.0553-0.0140.16020.0259-0.0130.15770.025-0.01310.15770.025-0.01370.15760.025Lo1-0.0020.07420.0055-0.00250.06320.004-0.00250.06320.004-0.00260.06280.00420.010.1720.02970.00790.16730.02810.0080.16730.02810.00760.16720.0283-0.00630.12010.0145-0.00610.11370.013-0.00620.11370.013-0.00660.11350.0129M12e-040.04680.00224e-040.03069e-044e-040.03049e-047e-040.02989e-042-0.00340.04630.0022-0.00330.03240.0011-0.00330.03220.001-0.00310.03180.00130.00310.04680.00226e-040.03250.00117e-040.03240.00118e-040.03190.001Table2.13:Bias,SE,andMSEofestimators:islogisticwithn=20andT=10.47OLSWithinREm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN1-0.00170.0310.001-6e-040.02476e-04-6e-040.02476e-04-1e-040.02165e-0424e-040.03069e-040.00120.02516e-040.00120.02516e-049e-040.02225e-0439e-040.03029e-040.00120.02426e-040.00130.02426e-047e-040.02145e-04La1-0.00650.08810.0078-0.00670.08350.007-0.00680.08340.007-0.00660.08330.00720.01750.23570.05590.01810.23390.05510.01810.23390.05510.01760.23390.0553-0.0140.16020.0259-0.0130.15770.025-0.01310.15770.025-0.01370.15760.025Lo10.00130.08910.0079-1e-040.08240.00680.00170.08370.007-0.00130.05930.003520.00170.08720.00763e-040.08210.00673e-040.08080.00655e-040.05820.00343-7e-040.08240.00680.00160.08520.00730.00270.08320.00690.00120.05920.0035M1-0.00230.04050.0016-4e-040.02356e-04-5e-040.02356e-04-6e-040.02084e-042-5e-040.0440.00193e-040.02386e-044e-040.02376e-0400.02084e-043-0.00120.0440.0019-8e-040.0256e-04-8e-040.02486e-04-0.0010.0225e-04Table2.14:Bias,SE,andMSEofestimators:isLaplacewithn=20andT=10.OLSWithinREm.d.biasSEMSEbiasSEMSEbiasSEMSEbiasSEMSEN10.00180.03770.00140.00120.03740.00140.00110.03710.00147e-040.02335e-042-0.00110.03630.0013-9e-040.03510.0012-0.00130.0350.001200.02175e-0435e-040.03810.0015-9e-040.03550.0013-8e-040.03540.0013-8e-040.02225e-04La1-0.01650.1370.019-0.01670.13450.0184-0.01680.13440.0184-0.01710.13130.017520.05190.39040.15510.04990.39010.15460.050.390.15460.04950.38910.15383-0.03340.26250.07-0.03270.26150.0695-0.03280.26150.0694-0.03370.25990.0687Lo1-0.00190.06960.0048-0.00320.05790.0034-0.0030.05780.0033-0.00260.05020.002520.00790.14430.02090.00530.13910.01940.00550.1390.01940.00580.13610.01863-0.00390.1030.0106-0.00370.09650.0093-0.00360.09650.0093-0.00420.09230.0085M14e-040.04930.00240.00130.03560.00130.00130.03560.00132e-040.02195e-042-7e-040.04890.0024-4e-040.03660.0013-5e-040.03650.001300.02255e-043-4e-040.04970.00251e-040.03870.00151e-040.03860.0015-1e-040.02386e-04Table2.15:Bias,SE,andMSEofestimators:isMTNwithn=20andT=10.482.3ProofsforChapter2ProofofTheorem2.1.2.Section5.5ofKoul(2002)illustratestheclaimholdsforinde-pendenterrors.Proofofthetheorem,therefore,willbesimilartotheoneofTheorem5.5.1inthatsection.fork=1;2;:::;p+1,u2Rp+1;y2R,Jk(y;u):=nXi=1dikF(y+u0Axi);Yk(y;u):=nXi=1dikI"iy+u0Axi;(2.31)Wk(y;u):=Yk(y;u)Jk(y;u);wherexi2Rp+1;i=1;2;:::;nareinthemodel(1.1),anddik=a0kxi.NotethatAPni=1dikxi=ek,whereek2Rp+1iselementaryvectorwhosekthentryis1.There-fore,rewriteT(+Au)=p+1Xk=1ZhfWk(y;u)Wk(y;0)g+fWk(y;u)Wk(y;0)g(2.32)+f(Jk(y;u)Jk(y;0))u0ekf(y)g+f(Jk(y;u)Jk(y;0))u0ekf(y)g+fUk(y;)+2u0ekf(y)gi2dH(y):NotethatthelasttermoftheintegrandisthekthcoordinateofU(y;)+2A1(t)f(y)vectorinQ(t).IfwecanshowthatsupremaofL2Hnormsofthefourtermsoftheintegrandareop(1),thenapplyingCauchy-Schwarz(C-S)inequalityonthecrossproducttermsin(2.32)willcompletetheproof.Thereforetoprovetheoremittoshowthat49forallk=1;2;:::;p+1,EsupZWk(y;u)Wk(y;0)2dH(y)=o(1);(2.33)supZ(Jk(y;u)Jk(y;0))u0ekf(y)2dH(y)=o(1);(2.34)EsupZUk(y;)+2u0ekf(y)2dH(y)=O(1):(2.35)wheresupistakenoverkukb.Firstconsider(2.35).BythesymmetryofFandHandFubini'stheorem,weobtainEZI"iyI"i<y2dH(y)=4Z10(1F)dH:Inaddition,Lemma2.3.1yields,forj>i,EZI"iyI"i<yI"jyI"j<ydH(y)20p21=2l(ji)Z10(1F)1=2dH:Togetherwiththefactthat(1F)(1F)1=2,by(A.1),(A.2),(A.7),andLemma2.3.2,weobtainEZUk(y;)2dH(y)4nmax1ind2ikZ10(1F)dH+40p2nmax1ind2ik(2.36)n1nXi=1nXj=i+11=2l(ji)Z10(1F)1=2dH<1:Therefore,(2.35)immediatelyfollows.50Theproofof(2.34)doesnotinvolvethedependenceoferrors,andhence,itisthesameastheproofof(5.5.11)ofKoul(2002).Thus,weshallprove(2.33).Hereweconsidertheproofforthecase+yonly.Thesimilarfactswillholdforthecasey.TobeginwithletJku(),Yku(),andWku()denoteJk(;u),Yk(;u)andWk(;u)in(2.31)whendikisreplacedwithdiksothatJk=J+kJk,Yk=Y+kYk,andWk=W+kWk.forx2Rp+1;u2Rp+1,y2R,p(y;u;x):=F(y+u0Ax)F(y);Bni:=I"iy+u0AxiI"iyp(y;u;xi):RewriteWkuWk0=nXi=1dikI"iy+u0AxiI"iyp(y;u;xi):NotethatEB2niF(y+ikuk)F(y)(2.37)RecallalemmafromDeo(1973).Lemma2.3.1.Supposeforeachn1,f˘nj;1jngarestronglymixingrandomvariableswithmixingnumberl.SupposeXandYaretworandomvariablesrespectivelymeasurablewithrespectto˙f˘n1;:::;˘nkgand˙f˘nk+m;:::;˘nng,1m;m+kn.Assumep;qandraresuchthatp1+q1+r1,andkXkp1andkYkq1.Thenforeach1m;k+mnjE(XY)E(X)E(Y)j101=rl(m)kXkpkYkq:51ConsequentlyifkXk1=B<1thenforq>1andeach1m;k+mnjE(XY)E(X)E(Y)j1011=ql(m)kYkq:Inaddition,considerfollowinglemma.Lemma2.3.2.For1<r<3,n1n1Xi=1niXk=11=rl(k)=O(1):Proof.Fixan1<r<3.Letpsuchthat1=r+1=p=1.Notethatpr=1r1>12:Therefore,byolder'sinequality,wehaven1n1Xi=1niXk=11=rl(k) n1Xk=1(nk)pnp1k2p=r!1=p n1Xk=1k2l(k)!1=r<1:(2.38)Thelastinequalityfollowsfromtheassumption(A.8),therebycompletingtheproofofthelemma.52Now,weconsiderthecrossproducttermsofEWkuWk02H.ThenEZnXi=1nXj=i+1dikdjkI"iy+u0AxiI"iyp(y;u;xi)I"jy+u0AxjI"jyp(y;u;xj)dH(y)nXi=1nXj=i+1dikdjkZEBniBnjdH10b1=2fnmaxid2ikgfmaxiig1=2n1n1Xi=1niXm=11=2l(m)Zf1=2dH!0:ThesecondinequalityfollowsfromLemma2.3.1,andtheconvergencetozerofollowsfromtheLemma2.3.2withr=2,(A.1),and(A.7).Consequently,byFubini'sTheoremtogetherwith(A.3),weobtain,foreverykukb,limsupn!1EjWkuWk0j2Hlimsupn!1ZananZf(y+s)dH(y)ds=0;wherean=bmaxii!0.Tocompletetheproofof(2.33),ittoshowthatforall>0,thereexistsa>0suchthatforallv2Rp+1,kuvk,limsupn!1EsupkuvjKkuKkvj(2.39)whereKku:=jWkuWk0j2H;u2Rp+1;1kp+1:(2.39)followsfrom(5.5.5)ofKoul(2002),therebycompletingtheproofoftheorem.ProofofCorollary2.1.1.Theproofoftheclaimforindependenterrorscanagainbe53foundinthesection5.5ofKoul(2002).Thebetweentheproofinthesection5.5andoneherearisesonlyinthepartwhichinvolvesthedependenceoftheerror.Thus,wepresentonlytheproofofananalogueof(5.5.27)inKoul(2002).LetLk:=ZhnXi=1dikI"iyI"i<yif(y)dH(y);L:=(L1;:::;Lp+1):NotethatLk=RUk(y;)f(y)dH(y).Therefore,usingC-Sinequalitytogetherwith(2.36),weobtain,forsome0<C<1EkLk2C(p+1)jfj2HZ10(1F)1=2dH:UsingELk=0fork=1;:::;p+1andChebyshevinequality,forall>0thereexistsN1andcsuchthatPkLkc1C(p+1)jfj2HR(1F)1=2dHc12;nN1:TherestoftheproofisthesameasthatofLemma5.5.4ofKoul(2002).ProofsofSection2.1.254ProofofTheorem2.1.6.NotethatbSk(y;r)=Sk(y;r)+n1=2nXi=1"ik"nI˘iy+Z0iv=pn+ni(u;v)I˘iy+Z0iv=pnonI˘iyZ0iv=pnni(u;v)I˘i<yZ0iv=pno#n1=2nXi=1n;ik(u)nI˘iy+Z0iv=pnI˘i<yZ0iv=pnon1=2nXi=1n;ik(u)"nI˘iy+Z0iv=pn+ni(u;v)I˘iy+Z0iv=pnonI˘iyZ0iv=pnni(u;v)I˘i<yZ0iv=pno#=Sr+R1kuvR2kuvR3kuv;(say):Inwhatfollows,theindexkrunsfrom1top+1.Inordertoconservethespace,wewriteiku,iuv,etc.forn;ik(u),ni(u;v),etc.SimilartothewayasthatusedinTheorem2.1.2,inordertoproveTheorem2.1.6,ittoshowthatforj=1;2;3supkub;kvbRjkuv2H=op(1):ConsiderR1kuv.First,defor1kqwk(y;u;v):=n1nXi=1"ikI˘iy+Z0iv=pn+ni(u;v);k(y;u;v):=n1nXi=1"ikF˘(y+Z0iv=pn+ni(u;v));Wk(y;u;v):=pnwk(y;u;v)k(y;u;v):55Alsoconsiderthefollowingprocesses:Tk(y;u;v;):=n1=2nXi=1"ikI˘iy+Z0iv=pn+ni(u;v)dni(;u;v);mk(y;u;v;):=n1=2nXi=1"ikF˘(y+Z0iv=pn+ni(u;v)dni(;u;v));Zk(y;u;v;):=Tk(y;u;v;)mk(y;u;v;):Notethatdni(;u;v)6=dni(;u;v).However,forthepurposeofconservingthespace,letTk(y;u;v;):=n1=2nXi=1"ikI˘iy+Z0iv=pn+ni(u;v)dni(;u;v):Similarly,wemk(y;u;v;)andZk(y;u;v;)bysubtractingtermdni(;u;v).Let"+ik=0_"ikand"ik="+ik"ik.Letanyprocesswithsuperscriptdenotetheonewith"ikreplacedby"ik.Alsodropthesubscriptk,andletwuv;uv;Wuv,etc.standforwk(;u;v);k(;u;v);Wk(;u;v),etc.Now,considers2Rp+1;t2Rqwhereksukandktvk.ThenweobtainZ0iv=pn+ni(u;v)dni(;u;v)Z0it=pn+ni(s;t)Z0iv=pn+ni(u;v)+dni(;u;v):(2.40)NotethatT(y;u;v;0)=pnwuvandm(y;u;v;0)=pnuv.Monotonicityofindicatorfunctiontogetherwith(2.40)inturnimpliesT(y;u;v;)T(y;u;v;0)pn[wstwuv]T(y;u;v;)T(y;u;v;0)56forally2R,kskb,ktkb,ksukandktvk.Usingthefactthata1a2a3impliesja2jja1j+ja3j,appropriatecenteringofTyieldspnwstwuv(2.41)T(y;u;v;)T(y;u;v;0)+T(y;u;v;)T(y;u;v;0)Z(y;u;v;)Z(y;u;v;0)+m(y;u;v;)m(y;u;v;0)+Z(y;u;v;)Z(y;u;v;0)+m(y;u;v;)m(y;u;v;0):Lemma2.3.3.Assumption(B.2)impliesEZZ(y;u;v;)Z(y;u;v;0)2dH(y)=o(1):(2.42)Also,assumption(B.3)implieswithcandin(B.3)liminfn!1PsupZn(y;s;t)(y;u;v)2dH(y)2=1(2.43)wheresupremumistakenoverksukandktvk.Moreover,assumptions(B.1)-(B.5)implysupkub;kvbZn1hnXi=1"ikF˘(y+n1=2v0Zi+ni(u;v))F˘(y)(2.44)f˘(y)fn1=2v0Zi+ni(u;v)gi2dH(y)=op(1):Proof.LetFi1:=˙f˘i1;˘i2;:::g.~p(y;u;v;z):=F˘(y+n1=2v0Zi+ni(u;v)+dni(;u;v))F˘(y+n1=2v0Zi+ni(u;v)):57Observethatithsummandisconditionallycenteredr.v.'s,givenFi1,andhence,covarianceoftwosummandsiszero.Similarto(2.37),theconditionalvarianceofithsummand,givenFi1,isboundedbyE("ik)2j~p(y;u;v;Zi)jE("ik)2jF˘(y+n1=2v0Zi+ni(u;v)+dni(;u;v))F˘(y)j+E("ik)2jF˘(y+n1=2v0Zi+ni(u;v))F˘(y)j:Therefore,(2.42)followsfromFubini'sTheoremandrepeatedapplicationof(B.2)withand=0.Toprove(2.43),observethatthemonotonicityofF˘and(2.40)implym(y;u;v;)m(y;s;t;0)m(y;u;v;);m(y;u;v;)m(y;u;v;0)m(y;u;v;);andhence,m(y;u;v;)m(y;u;v;0)n1=2[stuv]m(y;u;v;)m(y;u;v;0):Usingthefactthata1a2a3impliesja2jja1j+ja3j,weobtainn1=2jstuvjn1=2nXi=1"iknF˘(y+Z0iv=pn+ni(u;v)+dni(;u;v))(2.45)F˘(y+Z0iv=pn+ni(u;v)dni(;u;v))o:58Asaconsequenceof(2.45)andtheassumption(B.3),weobtainliminfn!1Psuppnf(y;s;t)(y;u;v)g2H2liminfn!1P supn1=2nXi=1"iknF˘(y+Z0iv=pn+ni(u;v)+dni(;u;v))F˘(y+Z0iv=pn+ni(u;v)dni(;u;v))o2H2!=1;wherethesupremumsaretakenoverksukandktvk,andhence,(2.43)follows.Next,writemuvandmform(y;u;v;0)andm(y;0;0;0).LetLuv=muvmn1=2nXi=1"ikf˘fn1=2v0Zi+ni(u;v)g2H:Inordertoprove(2.44),byusing(B.4)andcompactnessofkukbandkvkb(seeTheorem9.1.1ofKoul(2002)),ittoshowthatforall>0thereexistsa>0suchthatlimsupn!1PsupLstLuv>=0;(2.46)wheresupistakenoverksk;ktkb,ksuk,andjjtvjj.UsingmonotonicityofF˘,(2.40),triangleinequality,(a1a2)22(a21+a22),and(B.3),weobtainforksk;ktkbwithksukandktvklimsupn!1Psupjmstmuvj2H42=0(2.47)59wheretheequalityfollowsfrom(2.43).Next,observethatLstLuvZh(mstmuv)n1=2nXi=1"ikZ0it=pn+ni(s;t)Z0iv=pn+ni(u;v)f˘(y)ih(mstmuv)+2muvmn1=2nXi=1"ikZ0iv=pn+ni(u;v)f˘(y)n1=2nXi=1"ikZ0it=pn+ni(s;t)Z0iv=pn+ni(u;v)f˘(y)idH(y)2mstmuv2H+2n1=2nXi=1"ikdni(;u;v)f˘2H+2mstmuvHn1=2nXi=1"ikdni(;u;v)H+2muvmn1=2nXi=1"ikZ0iv=pn+ni(u;v)f˘HnmstmuvH+n1=2nXi=1"ikdni(;u;v)f˘Ho:Intheinequality,weusethefactthat(a21a22)=(a1a2)(a1+a2);(2.47)impliesthatsupremumofthetermofthelastequationisop(1);theassumption(B.1)andthefactthatdni(;u;v)=Op(n1=2kZik)implythatsupremumofthesecondtermisOp(1),andhence,supremumofthethirdtermisop(1);theassumption(B.4)ensuresthatsupremumofthelasttermisop(1).Therefore,bymakingassmallasdesired,(2.46)follows,therebycompletingtheproofof(2.44).Lemma2.3.4.supkub;kvbZhn1=2nXi=1"ikni(u;v)f˘(y)i2dH(y)=op(1):60Proof.Notethatsupkub;kvbni(u;v)=O(n1=2).Togetherwiththisfact,Fubini'stheoremyieldslimsupn!1Esupkub;kvbZn1nXi=1nXj=1"ik"jkni(u;v)nj(u;v)f2˘(y)dH(y)limsupn!1Cjf˘j2Hn2nXi=1nXj=11=2l(ji)k"1k24=0;where0<C<1.TheinequalityfollowsfromLemma2.3.1;thelastequalityfollowsfrom(B.1.a)andLemma2.3.2.Consequently,Esupn1=2nXi=1"ikni(u;v)f˘2H!0;wherethesupremumistakenoverkukbandkvkb,therebycompletingtheproofoflemma.Corollary2.3.1.supkub;kvbmuvm0v2H=op(1):Proof.Notethatm(y;u;v;0)m(y;0;v;0)=nm(y;u;v;0)m(y;0;0;0)n1=2nXi=1"ikZ0iv=pn+ni(u;v)f˘(y)onm(y;0;v;0)m(y;0;0;0)v0n1nXi=1"ikZif˘(y)o+n1=2nXi=1"ikni(u;v)f˘(y):HencetheproofofcorollarywillbecompletedbysimplyapplyingC-Sinequalityonthe61crossproducttermsafterweshowthatsupmuvmn1=2nXi=1"ikZ0iv=pn+ni(u;v)f˘2H=op(1);supm0vmv0n1nXi=1"ikZif˘2H=op(1);supn1=2nXi=1"ikni(u;v)f˘2H=op(1);wherethesupremumsaretakenoverkukbandkvkb.Theandthesecondfollowfrom(2.44)ofLemma2.3.3;thelastfollowsfromLemma2.3.4.Lemma2.3.5.supkub;kvbZnW(y;u;v)W(y;0;0)o2dH(y)=op(1):Proof.Similartothesquarerootoftheintegrandin(2.42),pnW(y;u;v)W(y;0;0)issumofconditionallycenteredr.v.'s.Thus,byrepeatedapplicationof(B.2)withand=0,weeasilyobtainforkuk;kvkb,WuvW2H=op(1):(2.48)LetCuv:=WuvW2H.Similartotheproofof(2.46),inordertocompletetheproof,ittoshowthatforeverythereexistssuchthatforeverykuk;kvkb,limsupn!1PsupCstCuv>=0;(2.49)62wheresupistakenoverksk;ktkb,ksuk,andktvk.NotethatCstCuvjWstWuvj2H+2jWstWuvjHjWuvWjH:BythemonotonicityofF˘togetherwith(2.40)weobtainthatjm(y;u;v;)m(y;u;v;0)jm(y;u;v;)m(y;u;v;):RecallofW.ThentriangleinequalityyieldsthatjWstWuvjjW+stW+uvj+jWstWuvj;jWstWuvjpnnjwstwuvj+jstuvjo:Therefore,using(a1+a2)22(a21+a22)repeatedlyand(2.41),wehavejWstWuvj2H16(Z[Z(y;u;v;)Z(y;u;v;0)]2dH(y)+Z[Z(y;u;v;)Z(y;u;v;0)]2dH(y)+Z[m(y;u;v;)m(y;u;v;)]2dH(y)+pn(stuv)2H)forallksk;ktk,ksuk,ktvk.Finally,assumption(B.3),(2.42),(2.43),and63(2.48)prove(2.49),therebycompletingtheproofofthelemma.SSk(y;u;v):=n1=2nXi=1"iknI˘iy+Z0iv=pn+ni(u;v)I˘i<yZ0iv=pni(u;v)o:RewriteR1kuv=hW(y;u;v)W(y;0;0)ihW(y;0;v)W(y;0;0)i+hW(y;u;v)W(y;0;0)ihW(y;0;v)W(y;0;0)i+hm(y;u;v;0)m(y;0;v;0)ihm(y;u;v;0)m(y;0;v;0)i:UsingquadraticexpansionandapplyingC-SinequalityinvolvingL2Hnormonthecrossproduct,theclaimthatsupkuk;kvkR1kuv=op(1)willbetrueifwecanshowthat(a)supkub;kvbjWuvWj2H=op(1);(b)supkvbjW0vWj2H=op(1);(c)supkub;kvbjmuvm0vj2H=op(1):(a)and(b)followfromLemma2.3.5;(c)followsfromCorollary2.3.1.Next,considerR2kuv.Tobeginwith,notethatbyC-SinequalityandFubini'stheoremtogether,weobtainforv2Rq,and0<b<1EZnF˘(y+v0Z1=pn)F˘(yv0Z1=pn)o2dH(y)(2.50)4b2n12b=pn1Zb=pnb=pnZEkZ1kf˘(y+skZ1k)2dH(y)ds:64Moreover,bytheassumptions(B.1.a)and(B.1.c)andboundedconvergencetheorem,wehaveZhEF˘(y+v0Z1=pn)F˘(yv0Z1=pn)i1=2dH(y)(2.51)p2bn1=4Zh(2b=pn)1Zb=pnb=pnEfkZ1kf˘(y+skZ1k)gdsi1=2dH(y)=O(n1=4):Finally,considerv1;v22Rqandletkv1kkv2k.UsingC-SinequalityandFubini'stheoremagain,wehaveEZnF˘(y+v02Z1=pn)F˘(y+v01Z1=pn)o2dH(y)(2.52)n1(kv2k+kv1k)2(kv2k+kv1k)=pn1Zkv2k=pnv1k=pnZEkZ1kf˘(y+skZ1k)2dH(y)ds:q(y;v;z):=F˘(y+n1=2v0z)F˘(yn1=2v0z):Next,rewriteR2kuv=n1=2nXi=1ikunI˘iy+n1=2v0ZiI˘i<yn1=2v0Ziq(y;v;Zi)o+n1=2nXi=1ikuq(y;v;Zi)=R21kuv+R22kuv;say:DuetothemonotonicityofF˘,weobtainsupkvbjq(y;v;Zi)jF˘(y+n1=2bkZik)F˘(yn1=2bkZik):(2.53)65Therefore,byC-Sinequality,(2.53),#n=o(1),and(2.50),wehaveEsupkuk;kvbjR22kuvj2H4b4#2n2b=pn1Zb=pnb=pnZEkZ1kf˘(y+skZ1k)2dH(y)ds!0:Next,considerthecrossproductofR21kuv.Foru;vEZ1nnXi=1nXj=i+1hikujkunI˘iy+n1=2v0ZiI˘i<yn1=2v0Ziq(y;v;Zi)onI˘jy+n1=2v0ZjI˘j<yn1=2v0Zjq(y;v;Zj)oidH(y)b2#2nn1nXi=1nXj=i+1ZEnI˘iy+n1=2v0ZiI˘i<yn1=2v0Ziq(y;v;Zi)onI˘jy+n1=2v0ZjI˘j<yn1=2v0Zjq(y;v;Zj)odH(y)10b2#2nn1nXi=1nXj=i+11=2l(ji)ZhEF˘(y+v0Z1=pn)F˘(yv0Z1=pn)i1=2dH(y)!0:TheinequalityfollowsfromFubini'stheorem;thesecondinequalityfollowsfromLemma2.3.1;theconvergencetozerofollowsfrom#n=0(1),(2.38)withr=2,and(2.51).There-fore,thesumofthecovarianceoftwosummandsinR21kuvwilltendtozero.Consequently,66foruandv,byFubini'stheorem,stationarityof",and(B.5),weobtainlimsupn!1EjR21kuvj2H(2.54)limsupn!12b3#2nn1=2(2b=n1=2)1Zb=n1=2b=n1=2ZEkZ1kf˘(y+skZ1k)dH(y)ds=0:Finally,weshallshowthatEsupjR21kuvj2H=o(1).Letuh=xheuandvl=wlevfor0l;hrwherexh=kuhkandwl=kvlk,andeu;evareunitvectorsinRp+1andRq.Letb=x0x1xr=b;b=w0w1wr=bsothatmaxh(xhxh1)!0andmaxl(wlwl1)!0asr!1.Notethatiku=u0cikandrewriteR21kuv=n1=2nXi=1u0ciknI˘iy+n1=2v0ZiI˘i<yn1=2v0Zion1=2nXi=1u0cikq(y;v;Zi)=T1uvT2uv;say:ik+u:=ikuI(ikuZ0iv>0)=u0cikI(u0cikZ0iv>0),iku=ikuik+u.LetR21h;l,T1h;l,T2h;lstandfortheR21kuhvl,T1uhvl,T2uhvlwithikubeingreplacedbyiku.ObservethatjR21kuvj2H2njT1+uvT2+uvj2H+jT1uvT2uvj2Ho:NotethatR21+uv(R21uv)isaoftwonondecreasing(nonincreasing)functionof67x;w,andhence,forallu=xeu2Rp+1andv=wev2Rqwherexh1xxhandwl1wwl,R21+h1;l1(T2+h;lT2+h1;l1)R21+uvR21+h;l+(T2+h;lT2+h1;l1):Therefore,usingthefactthata1a2a3impliesja2jja1j+ja3jandthat(ab)22(a2+b2)repeatedly,weobtainsupjR21+uvj2H8hmax0h;lrjR21+h;lj2H+2max0h;lrjT2+h;lT2+h1;lj2H(2.55)+2max0h;lrjT2+h1;lT2+h1;l1j2Hi:Usingkik+uk2kikuk2andergodicityof",EjT2+h;l1T2+h1;l1j2H(2.56)16b4#2n2b=pn1Zb=pnb=pnZEkZ1kf˘(y+skZ1k)2dH(y)ds!0;wheretheinequalityfollowsfrom(2.50),andtheconvergencetozerofollowsfromassumption(B.5)and#n=o(1).Similarly,using(2.52),wehaveEjT2+h1;lT2+h1;l1j2H(2.57)b2#2n(jwlj+jwl1j)2(jwlj+jwl1j)=pn1(Zjwlj=pnwl1j=pnZEkZ1kf˘(y+skZ1k)2dH(y)ds+Zjwlj=pnwl1j=pnZEkZ1kf˘(yskZ1k)2dH(y)ds)!0:68Consequently,(2.54),(2.55),(2.56),and(2.57)implyEsupu;vjR21+uvj2H!0:SimilarfactsholdforjR21uvj2H,andthiscompletestheproofofsupR2kuv=op(1).Toshowsupkub;kvbR3kuv2Hisop(1),notethatR3kuvisthesameasR1kuvexceptthefactthatithaso(1)asaweightinsteadofrandomerror.So,theproofissimilar,butmuchsimpler.Hencewedonotpresenthere.69Chapter3GooTesting:KhmaladzeTransformationvsEmpiricalLikelihood3.1KMT&ELmethodsKMTmethodhasnotgainedmuchattentiondespiteasymptoticdistributionfree(ADF)property.KoulprovidesareviewofKMTmethodinthechapter9ofFanandKoul(2006).LetX1;:::;Xnbei.i.d.randomsamplefromalocation-scalefamilywhereFistheerrord.f.,havinganabsolutelycontinuousdensityf.Assumethat_fexistsalmosteverywheresuchthat0<R(_f=f)2dF<1.Letand˙denoteunknownlocationandscaleparameters,respectively.Considertheproblemtotestforallx2R,andsome2R;˙>0H0:F(x)=F0(x)=˙;vs.Ha:H0isnottrue:(3.1)whereF0isaknownd.f.Zi:=(Xi)=˙;bZi:=(Xibn)=b˙n;(3.2)Fn(x):=n1nXi=1I(Zix);bFn(x):=n1nXi=1I(bZix):wherebnandb˙nareconsistentestimatorsofand˙underthenullhypothesis.70Asmentionedintheintroduction,itiswellknownthatthenulldistributionofclassicalK-StestbasedonFndoesnotdependonF0.However,thisfactdoesnotholdanymorewhenthenecessityofestimatingand˙arises,i.e.,atestbasedonbFnisnotdistributionfree.Durbin(1973)showedthatnulldistributionofthetestbasedonbFndependsontheestimators(bn,b˙)aswellasF0.ToobtainADFtest,wepayattentiontoatransformationbasedonbFnwhichwasproposedbyKhmaladze(1979,1980).Let˚0(x):=_f0(x)=f0(x);l(x):=(1;˚0(x);x˚0(x)1)0;(3.3)p0(t):=f0(F10(t));q0(t):=F10(t)f0(F10(t));t=0BBBBB@1tp0(t)q0(t)p0(t)R1t_p20(u)duR1t_p0(u)_q0(u)duq0(t)R1t_p0(u)_q0(u)duR1t_q20(u)du1CCCCCA:martingaletransformedprocessbUn(t):=n1=2nXi=1(I(bZiz)l(bZi)0Zz^bZi1F0(x)l(x)dF0(x));t=F0(z);z2R:(3.4)Then,weakconvergenceofbUntoBrownianmotioninuniformmetricfollowsfromKhmaladze(1981):seethesection4forthedetails.Hence,anytestbasedonT:=sup0t1jbUn(t)jisADF.WhenwetestH0in(3.1)viaKMTmethod,weshalluseTfortheteststatistic.WhenY1;Y2;:::;Ynarei.i.d.observationsfromdistributionK,ELisasL(K)=nYi=1K(fYig)=nYi=1pi(3.5)wherepi=K(fYig)=K(Yi)K(Yi).Itiswell-knownthattheempiricald.f.Kn(x):=n1Pni=1I(Yix)maximizes(3.5).LetR(K):=L(K)=L(Kn)denotetheELratio.LetK0bead.f.withmean0.Owen(1988,1990)usedR(K)andproposed71R(0)=supˆR(K)jZYdK=0˙=sup(nYi=1npijpi0;nXi=1pi=1;nXi=1piYi=0);(3.6)toconstructanonparametricregionandtestforthemeanofY.Owen(1990)showedthatforY˘K02logR(0)!D˜2(1);(3.7)whichisananalogofWilks's(1938)theoremfornonparametriclikelihood.Owen(1990)referredtoR(0)asempiricallikelihoodratio(PELR)sincenuisanceparametersareout."ThePELRhasbeenofinteresttostatisticiansandextendedtovarioussettings.Con-siderthecasewhereunknownKdependsond-dimensionalparameter.QinandLawless(1994)assumedthatthereexistsinformationaboutKand:therearehd\unbiasedestimatingfunctions,"thatis,g1(Y;);:::;gh(Y;)whereEK[gj(Y;)]=0forj=1;2;:::;h.Forexample,letE(Y)=andE(Y2)=()where()isaknownfunction.Then,g1andg2canbewrittenasg1(Y;)=Yandg2(Y;)=Y2(),respectively.Withtheunbiasedestimatingfunctions,theyconsideredananalogof(3:6),i.e.,amaximizationproblemmaxnYi=1npisubjecttopi0;nXi=1pi=1;nXi=1pigj(Yi;)=0;j=1;2;:::;h:Hence,theylinkedPELRwithmanyconstraints.BythemethodofLagrangemulti-pliers,theyobtainedthemaximumandpempiricallog-likelihoodratio(PELLR)lE()=nXi=1log241+hXj=1jgj(Yi;)35;(3.8)wherejistheLagrangemultipliercorrespondingtotheconstraintPni=1pigj(Yi;)=0.Theyshowedj'saredeterminedintermsofandunderH0:=0,722lE(0)2lE(b)!D˜2(d);(3.9)wherebminimizeslE().IncontrasttoQinandLawless(1994),PengandSchick(2013)consideredPELRap-proachcombinedwithmanyconstraints(orunbiasedestimatingfunctions),i.e.,thenumberofconstraintsincreasesasthesamplesizeincreases.Again,letYber.v.whichcomesfromunknowndistributionK.ConsidertestingH0:K=K0whereK0isaknownd.f.'h(x):=p2cos(hˇx);forh=1;2;::::Consequently,wehaveforallhZ[0;1]'h(x)dx=0;Z[0;1]'2h(x)dx=1:NotethatwhenH0:K=K0istrue,K0(Y)willbeauniformr.v.,andhence,E['h(K0(Y))]=0andE['h(K0(Y))]2=1.Withmanyunbiasedestimatingfunctions,'hK0,h=1;2;:::,PengandSchick(2013)proposedananalogof(3.6):Rn(K0)=sup8<:nYi=1npi:pi0;8i;nXi=1pi=1;nXj=1pj'h(K0(Yj))=0;h=1;:::;mn9=;:(3.10)PengandSchick(2013)showedthatunderH0P(2logRn(K0)>˜21(mn))!wheremnandntendtoyandm3n=ntendsto0,and˜21(mn)denotesthe(1)-quantileofthechi-squaredistributionwithmndegreesoffreedom.TheyextendedtheresulttotestingH0whereunderlyingdistributionKdependsonanunknownddimensionalparameter.Withtestimatorb-e.g.,maximumlikelihoodestimator(MLE)-theyderivedRn(Kb)-theteststatisticin(3.10)withK0replacedbyK^-andshowedthatunderthenullhypothesisP(2logRn(K^)>˜21(mnd))!wheremnandntendtoyandm3n=ntendsto0.Sincetheirapproachisfreefromthe73questionofhowmanyconstraintsshouldbeused,weuse2logRn(K^)fortheteststatisticwhenweimplementELmethodtotestH0in(3.1).Atthistime,itshouldbementionedthatELapproachhasonecriticaldrawbackwhenitisemployedforhypothesistesting.NotethatalltheELmethodsintroducedinthissectionsolvethemaximizationproblemsubjecttoormanyconstraints(orunbiasedestimatingfunctions).Assumethatr.v.YcomesfromanunknowndistributionK.ConsidertestingH0:K=K0vsHa:K=KaviatheELmethod.Let0andadenoteparametersassociatedwithK0andKa,respectively.Whenweusetheteststatisticin(3.8)((3.10)),constraintscorrespondingtoH0andHaareEK0[g(Y;0)]=0andEKa[g(Y;a)]=0.WhenK0aresimilartoKa,wewillthereforeobtainthesimilarteststatisticslE(0)andlE(a)(orRn(K0)andRn(Ka)).Asaresult,itisverylikelytomakeatypeIIerrorwhentrued.f.ofYisKa,i.e.,ELmethodwillhaveverypoorpower.ExamplesofsuchK0andKaare;logisticandnormal;andlogisticandStudent'st(STT)wheredegreesoffreedom(df)isgreaterthanorequalto5.ThepoorpowerofELmethodforthesetwocasesareillustratedinthenextsection.See,e.g.,Table3.2and3.3.3.2SimulationstudyLetFifori=N;Ldenoted.f.ofstandardnormalandlogisticr.v.,respectively.NotethatFN(x)=1p2ˇZxexp(y2=2)dy;FL(x)=11+ex;x2R:Inthissimulationstudy,wereporttheobtainedfromgootestfortwolocation-scaledistributions:H0:F(x)=Fi((x)=˙);i=N;L.Table3.1reports74asymptoticcriticalvaluesforKMTandELmethods.ThecriticalvaluesofKMTtestareKMTEL1EL2n=50100200500n=501002005000.052.245.997.819.4912.597.819.4911.0714.070.012.819.2111.3413.2816.8111.3413.2815.0918.48Table3.1:CriticalvalueforKMTandELavailableathomepages.ecs.vuw.ac.nz/ray/BrownianwhichismadebyDr.R.Brownrigg.ForthoseofELmethod,weconsidertwotdf's:mn1=bn1=3c2andmn2=bn1=3c1wherebxcisthelargestintegernotgreaterthanx.LetEL1andEL2denoteELmethodswithdfofmn1andoneofmn2,respectively.Whenwegeneraten=50;100;200,and500samplesfromeachnulldistribution,weuse2and5forlocationandscaleparameters.Werepeatrandomsamplegeneration1,000timesandobtainKMTandELteststatistics.InordertoobtaintwotELteststatistics,weusebn1=3candbn1=3c+1unbiasedestimatingfunctionsfortheconstraintsin(3.10).Empiricallevelsandpowersarethencalculatedfromdividingthenumberofrejectionofnullhypothesisby1,000.WeuseMLEforbandb˙inthesubsequentsections.753.2.1TestingfornormaldistributionInthissection,wecompareKMTandELmethodstotestfornormality.LetFdenotestandardnormald.f.andfbeitsdensity.l(x)andF(x)in(3.3)turnoutl(x)=(1;x;x21)0;F(x)=0BBBBB@1F(x)f(x)xf(x)f(x)xf(x)+(1F(x))(1+x2)f(x)xf(x)(1+x2)f(x)(x3+x)f(x)+2(1F(x))1CCCCCA:LetA11(x):=1F(x);A12(x):=(f(x);xf(x));A21(x):=(f(x);xf(x))0;A22(x):=0B@xf(x)+(1F(x))(1+x2)f(x)(1+x2)f(x)(x+x3)f(x)+2(1F(x))1CA:ThentheinverseofF(x)alsocanbeexpressedinpartitionedform,i.e.,1F(x)=0B@B11(x)B12(x)B21(x)B22(x)1CAwhereB11(x)=(A11(x)A12(x)A22(x)1A21(x))1;(3.11)B12(x)=B11(x)A12(x)A22(x)1;B21(x)=B12(x)0B22(x)=A22(x)1+A22(x)1A21(x)B11(x)A12(x)A22(x)1:76Let~F(x):=1F(x),c1(x):=(x2+1)f2(x)+(x3+3x)f(x)~F(x)+2~F2(x)g1andc2(x):=2~F3(x)+(x3+3x)f(x)~F2(x)(2x2+3)f2(x)~F(x)+xf3(x).Notethatwithr(x):=(x;(x21))0,wehave1F(x)l(x)f(x)=264f(x)B11(x)+f(x)B12(x)r(x)f(x)B21(x)+f(x)B22(x)r(x)375:Finally,withbZiin(3.2),wehavel(bZi)01F(x)l(x)f(x)(3.12)=2f(x)c2(x)h~F2(x)+xf(x)~F(x)f2(x)gi+bZif(x)c1(x)c2(x)h4x~F4(x)+(2x4+8x22)f(x)~F3(x)+(x57x)f2(x)~F2(x)(2x4+3x21)f3(x)~F(x)+(x3+x)f4(x)i+(bZ2i1)f(x)c1(x)c2(x)h2(x21)~F4(x)+(x5+2x39x)f(x)~F3(x)(4x4+9x25)f2(x)~F2(x)+(5x3+9x)f3(x)~F(x)2(x2+1)f4(x)i:Wethenreplacel(bZi)01F(x)l(x)f(x)in(3.4)by(3.12).Table3.2reportsempiricallevelsandpowersofKMTandELmethods.The(second)columnsofKMTandEL'srepresentthosewhenis0.05(0.01).ThevaluecorrespondingtonormalFrepresentstheempiricallevelwhilethosecorrespondingtoothers-logistic,STTwithdfof5,mixtureoftwonormaldistributions(MTN),Cauchy,andLaplace-representthepowers.FortheMTN,weuse0:9N(2;52)+0:1N(2;152);forlogistic,Cauchy,andLaplacedistributions,weuse2and5forlocationandscaleparameters.()impliesthecorrespondingmethodshowstheclosestempiricalleveltotheorhighestpower;e.g.,KMThastheclosest77empiricallevel(0.01)andthehighestpower(0.152)whenn=500with=0:01andn=50with=0:05,respectively.ItishardtojudgethesuperiorityoftwomethodsbytheempiricallevelsinceneitherKMTnorELdemonstratesbetterperformanceconsistently.FnKMTEL1EL2KMTEL1EL2normal500.0260.0880.058()0.012()0.0200.0221000.0320.060()0.0820.0220.016()0.0182000.0320.0440.050()0.0160.0060.008()5000.0240.0620.060()0.010()0.0180.016logistic500.152()0.1480.1000.092()0.0360.0321000.200()0.1480.1680.120()0.0560.0322000.320()0.1760.1680.244()0.0520.0565000.632()0.3720.3840.532()0.1600.160STT500.2440.276()0.2240.208()0.1160.0921000.400()0.3040.2640.292()0.1320.1362000.556()0.5000.4680.480()0.2520.6805000.884()0.8680.8480.804()0.6800.660MTN500.3820.410()0.3460.352()0.2240.1641000.562()0.5280.4620.516()0.2960.2582000.812()0.7560.7060.716()0.5660.5325000.982()0.9660.9580.966()0.9080.886Cauchy500.8320.996()0.9800.7440.984()0.9681000.988()0.9840.9680.9480.984()0.9682001.000()0.9760.9681.000()0.9800.9645001.000()0.9900.9641.000()0.9700.960Laplace500.2760.468()0.4240.1860.188()0.188()1000.4800.788()0.6760.3800.516()0.4442000.7680.932()0.9200.6320.796()0.7565000.9801.000()1.000()0.9641.000()1.000()Table3.2:EmpiricallevelandpowerobtainedfromtestingH0:F=FNHowever,itisnothardtotellwhichofthetwomethodsisbetterintermsofthepower.ForalldistributionsexceptLaplace,KMToutperformsEL:KMTshowsbetterpowerthanELinmostn's.WhenFislogistic,ELdisplayspoorpower(lessthan0.4)asstatedintheprevioussection.ThepowerofKMTismorethanorequaltoalmosttwicethatofELwhennis200or500.ForSTT,MTN,andCauchy,KMTstillmaintainssuperiorityoverELeventhoughthatisweakened.Asshowninthetable,theperformancegapbetweenKMTand78ELiswidenedwhenis0.01.Incontrast,LaplaceFshowstheoppositeresult:ELoutperformsKMTforalln'sand's.ButthesuperiorityofELoverKMTisnotasstrongasthatofKMToverELinthelogisticcase;whennreaches500,theofpowersbetweenELandKMTislessthan0.05whilecounterpartofthelogisticcaseismorethan0.2.3.2.2TestingforlogisticdistributionConsiderlogisticd.f.F(x)=1=(1+ex),anditsdensityf(x)=ex=(1+ex)2.Notethat˚(x)=_f(x)=f(x)=(ex1)=(ex+1).Witht=F(x),wehave_p(t)=1ex1+ex;_q(t)=1+x(1+ex)1+ex:ItiseasytoseethatZ1F(x)_p(u)2du=3e2x+13(ex+1)3;Z1F(x)_p(u)_q(u)du=13ln(1+ex)exfx(3+e2x)+(1+ex)g3(1+ex)3;Z1F(x)_q(u)2du=1(1+ex)+2(1+ex)2xex(1+ex)2+Re(x);whereRe(x)=R1x2s2es(1es)2=(1+es)4ds.NotethatR_q(u)2dudoesnothaveaclosedformsolutionsinceRe(x)doesnothaveone.However,Re(x)isboundedanddecaysto0fastasxgoesto1;itconvergestothevalue(2.43)asxgoesto.Therefore,wegetanumericalapproximationtoRe(x)anduseitwhenwecalculatetheinverseofF.Let79v1(x):=R1F(x)_p(u)_q(u)duandv2(x):=R1F(x)_q2(u)du.Alsoned(x):=h3(1+3e2x)(1+ex)3v2(x)9(1+ex)6v21(x)i1;k1(x):=3(1+ex)3v2(x)3x(1+ex)3v1(x);k2(x):=3(1+ex)3v1(x)+x(1+3e2x)3:Then,usingpartitionedfour22blocksofF(x)asdoneintheprevioussection,weobtainl(bZi)01F(x)l(x)f(x)=3ex(1+ex)B11(x)n13d(x)ex(1ex)k1(x)+k2(x)(1+ex+x(1ex))o+3(1ebZi)(1+ebZi)n9e2x(1+ex)2d(x)B11(x)k1(x)+3exd(x)3(1+ex)3(1ex)v2(x)3(1+ex)3v1(x)f(1+ex)+x(1ex)g+27e3x(1+ex)2B11(x)d2(x)(1ex)k21(x)+f(1+ex)+x(1ex)gk1(x)k2(x)o+ 1+bZi(1ebZi)(1+ebZi)!n9e2x(1+ex)2d(x)B11(x)k2(x)+3exd(x)3(1+ex)3(1ex)v1(x)+(1+3e2x)f(1+ex)+x(1ex)g+27e3x(1+ex)2B11(x)d2(x)(1ex)k1(x)k2(x)+f(1+ex)+x(1ex)gk21(x)o;andhence,useabovequantityforbUnin(3.4).Table3.3reportsempiricallevelsandpowersofKMTandEL'sfortestingH0:F=FL;thevaluecorrespondingtologisticFstandsfortheempiricallevel,andothersrepresentthepowers.ELshowsbetterempiricallevelthanKMTforalln'swhen=0:05.With=0:01,KMT,however,hasbetterempiricallevelthanELexceptn=50.Therefore,80FnKMTEL1EL2KMTEL1EL2logistic500.0320.0760.062()0.0200.018()0.0041000.0300.0620.060()0.014()0.0160.0182000.0260.0600.052()0.010()0.0200.0205000.0540.052()0.052()0.011()0.0060.008normal500.0040.0760.080()0.0000.022()0.0201000.0200.080()0.0660.0000.0180.020()2000.136()0.0980.1040.040()0.0340.0285000.614()0.1760.2040.342()0.0660.084STT500.0600.083()0.0630.040()0.0250.0191000.097()0.0700.0720.070()0.0130.0152000.132()0.0650.0630.101()0.0190.0175000.219()0.0570.0490.173()0.0130.013MTN500.164()0.0890.1010.129()0.0310.0181000.185()0.0850.0970.144()0.0290.0112000.297()0.0760.0870.234()0.0270.0285000.583()0.1150.0810.518()0.0110.009Cauchy500.6490.9650.974()0.5170.9040.930()1000.8551.000()1.000()0.7821.000()1.000()2001.0001.0001.0001.0001.0001.0005001.0001.0001.0001.0001.0001.000Laplace500.1370.1460.208()0.075()0.0250.0541000.1990.350()0.3170.159()0.1260.0932000.3320.547()0.5180.1940.320()0.2965000.6130.915()0.9110.3990.7340.754()Table3.3:EmpiricallevelandpowerobtainedfromtestingH0:F=FLKMTandELtieintermsoftheempiricallevel.ForMTN,normal,andSTT,thefactthatKMToutperformsELintermsofthepowerisevident.IncaseofMTN,KMToverwhelmsELforalln'sand's.TheerenceofpowersbetweenKMTandELincreasesasnincreasesandreachesmorethan0.5forboth's.Inthenormalcase,themaximumpowerELattainsis0.204-whenn=500and=0:05-whilethatofKMTis0.614.Whenn=500and=0:01,bothEL'sdisplaypowerslessthaneven0.1whileKMTshows0.342.WhenFisSTT,ELshowstheextremelypoorpowerwith=0:01;ELobtainsthepowerofonly0.013eventhoughnreaches500.NotethatthecounterpartofKMTis0.173.81WhenFiseitherCauchyorLaplace,ELshowsslightlyorstrictlybetterpowerthanKMT.FortheCauchy,ELattainsthepowergreaterthan0.9forall'sevenwhenn=50whileKMTshowsthepowerlessthan0.8.BothKMTandEL,however,attainthepowersof1whennreaches200;theofperformancesbetweenKMTandELdisappearsasnincreases.ELhasbetterpowerthanKMTwhenFisLaplace,asitdoesintheprevioussection.Whenn=500and=0:05,ELattainsthepowerof0.9;KMTneverattainsthepowergreaterthan0.8.82BIBLIOGRAPHY83BIBLIOGRAPHY[1]Ashenfelter,O.(1978),EstimatingtheofTrainingProgramsonEarningswithLongitudinalData,ReviewofEconomicsandStatistics,60,47-57.[2]Athreya,K.B.andPantula,S.G.(1986),MixingPropertiesofHarrisChanisandAu-toregressiveProcesses,J.Appl.Probab.,23,880-892.[3]Baltagi,B.D.(2001).EconometricAnalysisofPanelData.JohnWileyandSons,Chich-ester.[4]Beran,R.J.(1977).MinimumHellingerdistanceestimatesforparametricmodels.Ann.Statist.,5445-463.[5]Billingsley,P.(1999).ConvergenceofProbabilityMeasures.JohnWileyandSons,NewYork.[6]BloP.(1992),TrendsinGlobalTemperature,ClimaticChange,21,1-16.[7]Deo,C.M.(1973),ANoteonEmpiricalProcessesofStrongMixingSequences,Ann.Probab.,1,870-875.[8]Donoho,D.L.andLiu,R.C.(1988a).The\automatic"robustnessofminimumdistancefunctionals.Ann.Statist.,16552-586.[9]Donoho,D.L.andLiu,R.C.(1988b).Pathologiesofsomeminimumdistanceestimators.Ann.Statist.,16587-608.[10]Dong,L.B.andGiles,D.E.A.(2004).Anempiricallikelihoodratiotestfornormality,EconometricsWorkingPaperEWP0401.[11]Durbin,J.(1973).DistributionTheoryforTestsBasedontheSampleDistributionFunc-tion.SIAM,Philadelphia.[12]Durrett,R.(2005).Probability:TheoryandExamples.,Thomson,BrooksCole.[13]Fan,J.andKoul,H.(2006)FrontiersinStatistics.ImperialCollegePress,London.84[14]Hampel,F.R.(1974).Thecurveanditsroleinrobustestimation.J.Amer.Statist.Assoc.,69383-393.[15]Khmaladze,E.V.(1979).Theuseof!2testsfortestingparametrichypotheses.TheoryProbab.Appl.,24283-301.[16]Khmaladze,E.V.(1981).Amartingaleapproachinthetheoryofgoodness-tests.TheoryProbab.Appl.,26240-257.[17]Khmaladze,E.V.andKoul,H.L.(2004).Martingaletransformsofgootestsinregressionmodels.Ann.Statist.,32995-1034.[18]Kim,J.(2016),KoulMde:AnRPackageforKoul'sMinimumDistanceEstimation,arXiv:1606.04182.[19]Koul,H.L.(1977),BehaviorofRobustEstimatorsintheRegressionModelwithDe-pendentErrors,Ann.Statist.,5,681-699.[20]Koul,H.L.(1985),MinimumDistanceEstimationinLinearRegressionwithUnknownErrorDistributions,Statist.Probab.Lett.,3,1-8.[21]Koul,H.L.(1986),MinimumDistanceEstimationandGoodness-of-FitTestsinFirst-OrderAutoregression,Ann.Statist.,14,1194-1213.[22]Koul,H.L.(2002),WeightedEmpiricalProcessinNonlinearDynamicModels.Springer,Berlin,Vol.166.[23]Koul,H.L.anddeWet,T.(1983),MinimumDistanceEstimationinaLinearRegressionModel,Ann.Statist.,11,921-932.[24]Koul,H.andZhu,X.(2015).Gotestingoferrordistributioninnonparamet-ricARCH(1)models.J.MultivariateAnal,137141-160[25]Mehra,K.L.andRao,M.S.(1975),WeakConvergenceofGeneralizedEmpiricalProcessesRelativetodqunderStrongMixing,Ann.Probab.,3,979-991.[26]Millar,P.W.(1981).Robustestimationviaminimumdistancemethods.ZeitfurWahrscheinlichkeitstheorie.,5573-89.[27]Moulton,B.R.(1986).RandomGroupandthePrecisionofRegressionEsti-mates.JournalofEconometrics.,32385-397.85[28]Noether,G.E.(1949),OnaTheorembyWaldandWolfowitz,Ann.MathStatist.,20,445-458.[29]Parr,W.C.andSchucany,W.R.(1979).Minimumdistanceandrobustestimation.J.Amer.Statist.Assoc.,75616-624.[30]Peng,H.andSchick,A.(2013).Empiricallikelihoodapproachtogoodnessoftesting.Bernoulli,19954-981.[31]Prescott,E.C.(1987),TheoryaheadofBusinessCycleMeasurement,Carnegie-RochesterConferenceonPublicPolicy.[32]Qin,J.andLawless,J.(1994).Empiricallieklihoodandgeneralestimatingequations.Ann.Statist.,22300-325.[33]Stephens,M.A.(1974).EDFstatisticsforgoodnessofandsomecomparisons.J.Amer.Statist.Assoc.,69730-737.[34]Wilks,S.S.(1938)Thelarge-sampledistributionofthelikelihoodratiofortestingcompositehypotheses.Ann.Math.statist.,960-2.86