ESTIMATIONOFSTATISTICALNETWORKANDREGION-WISEVARIABLESELECTIONBySayanChakrabortyADISSERTATIONSubmittedtoMichiganStateUniversityinpartialentoftherequirementsforthedegreeofStatistics-DoctorofPhilosophy2016ABSTRACTESTIMATIONOFSTATISTICALNETWORKANDREGION-WISEVARIABLESELECTIONBySayanChakrabortyNetworkmodelsarewidelyusedtorepresentrelationsbetweenactorsornodes.Recentstudiesofthenetworkliteratureandgraphmodelrevealedvariouscharacteristicsoftheactorsandhowtheythecharacteristicsofneighboringactors.ThemethodologyismotivatedbyformulatingalargenetworkthroughtheExpo-nentialRandomGraphModelandapplyingaBayesianapproachthroughthereferencepriortechniquetocontrolthesensitivityoftheinferenceandtogetthemaximuminformationfromthemodel.WeconsideralargeAmazonproductco-purchasingnetwork(customerswhoboughtthisitemalsoboughtotherproducts),andthepurposeistoshowhowtheblendingoftheExponentialRandomGraphModelandBayesianComputationtlyhandlestheestimationprocedureandcalculatestheprobabilityofcertaingraphstructures.Thesecondmethodologywediscussisanapproachtoanetworkproblemwherethenetworkadjacencystructureremainsunobserved,andinsteadwehaveanodalvariablethatinheritsahiddennetworkstructure.ThekeyassumptioninthismethodisthatthenodesareassumedtohaveasppositioninanEuclideansocialspace.ThemainanalysisisbasedonthreebigU.S.automanufacturersandtheirsuppliers,andrecentresearchhasexploredtheesofthemarketsandanemphasishasbeengiventorevealthestrategicinteractionsamongcompaniesandtheirindustryrivalsandsuppliers,allofwhichhaveimportantimplicationsforsomefundamentalquestionsintheeconomics.Economicshocksaretransmittedthroughthecustomersuppliernetworkandthewholeindustrycouldbebytheseshocksastheycanmovethroughthelinksoftheactorsinanindustry.Wedevelopedanalgorithmthatcapturesthelatentlinkagesbetweenbasedonsalesandcostdatathatvariousdecision-makingissuesandstrategies.Finally,weextendtheproblemofnetworkestimationtoBayesianvariableselectionwherebyanobservedadjacencystructurebetweentregionshasbeenconsidered.Themainideaistoselectrelevantvariablesregion-wise.WeinvestigatethisproblemusingaBayesianapproachbyintroducingtheBayesianGroupLASSOtechniquewithabi-levelse-lectionthatnotonlyselectstherelevantvariablegroupsbutalsoselectstherelevantvariableswithinthatgroup.Weusespikeandslabpriors,alongwiththeConditionalAutoregressivestructureamongthemodelcots,whichvalidatesthespatialinteractionamongthecovariates.Medianthresholdingisusedinsteadofposteriormeantohaveexactzerosforthevariablesthatarenotrelevant.Weimplementtheproblemintheautoindustrydataandincorporatemorevariablestoseewhethertheestimatedadjacencystructurehelpsustoindicatetherelevantvariablesovertmanufacturersandsuppliers.ACKNOWLEDGMENTSAsaPhDstudentoftheDepartmentofStatisticsandProbabilityatMichiganStateUniver-sity,Ifeelveryfortunateinhavingtheopportunitytoworkwithsomeamazingprofessorsinthepastfewyears.First,myadvisorProfessorTapabrataMaiti,whohashelpedmealot,andwithouthisendlesssupportandencouragementinmyresearch,itwouldhavebeenreallytoughtoreachmyresearchgoals.I'mthankfultohimthathehadfaithinme.I'malsothankfultomyresearchcommitteemembers,ProfessorChaeYoungLim,Pro-fessorSrinivasTalluriandProfessorJongeunChoiwhogavemeanopportunitytoworkwiththemandtoprosperinmylearningprocess,whichalsohelpedmealotinlookingtowardsmyresearchcareer.IalsohadopportunitytoworkwithProf.ArnabBhattacharjeefromHariot-WattUniversityandI'mthankfultohimforhisthelpinmyresearch.I'malsothankfultotheDepartmentofStatisticsandProbabilityatMichiganStateUniversityforprovidingmewithfundingsupportsothatIcouldcontinuemyresearch.I'mthankfultotheotherprofessorsinourdepartmentwhocontributedtomywonderfulyearsherethroughtheircoursesandhelpfuldiscussionsessions.I'mthankfultoDellInc.,andsptoDr.ThomasHill,forprovidingmewithaResearchAssistantshipforspring,2016,whichmadeiteasierformefocusonmyresearchasIwasrelievedfromTAduties.IalsohavetomentionthehelpreceivedfromsomeofmyfriendswhoarealsograduatestudentsinthedepartmentasIbfromnicediscussionsandbrain-storming.Person-ally,Ifeelproudtobeapartofsuchaprestigiousdepartment,andI'mthankfultoeveryoneinthedepartmentforsupportingmeinreachingmyobjectives.ivTABLEOFCONTENTSLISTOFTABLES....................................viiLISTOFFIGURES...................................viiiChapter1Introduction...............................11.1SocialNetworks..................................21.1.1ExponentialRandomGraphModel...................31.1.2BayesianERGM.............................51.1.3LatentSpaceModel............................61.2BayesianVariableSelection...........................9Chapter2ReferencePriorDevelopmentinExponentialRandomGraphModel...................................142.1MainIdea.....................................152.2ReferencePriorforOneParameterErdos-ReyniModel............162.2.1'Sampson'sMonkData'Implementation................192.3ReferencePriorforTwoParameterDyadicindependentnetworkModel...222.3.1MethodologyforDerivation.......................232.4SimulationStudyforDyadicIndependentModel................282.4.1Scenario1.................................292.4.2Scenario2.................................332.5Discussion.....................................36Chapter3BigDataApplicationofERGMthroughReferencePrior...373.1BigDataNetwork.................................383.2DataandModel..................................393.3Estimation.....................................403.4Discussion.....................................42Chapter4LatentSpaceNetworkforthreeUSAutoManufacturingGiant454.1MainIdeaandMethodology...........................464.1.1ErrorCorrectionModel.........................504.1.2LatentSpaceModel............................524.2Estimation.....................................534.3DataAnalysis...................................544.4Conclusion.....................................594.5SomePosteriorCalculations...........................594.5.1(s+1)thGibbsStepforupdating(1;1;1):.............604.5.2(s+1)thGibbsStepforupdating(2;2;2):.............614.5.3(s+1)thGibbsStepforupdating˙2z1:.................62v4.5.4(s+1)thGibbsStepforupdating˙2z2:.................624.5.5(s+1)thGibbsStepforupdatingZ1;Z2:...............63Chapter5RegionWiseVariableSelectionwithBayesianGroupLASSO655.1Region-wiseVariableSelection..........................665.2Region-wiseVariableSelectionwithBayesianGroupLASSO...................................695.2.1SpikeandSlabPriorforModelCots...............715.3HellingerConsistencyforthePosteriorDistributionof735.4PosteriorDistributionsandGibbsSamplingforGroupLASSO...................................785.4.1GibbsSampler..............................795.5VariableSelectionforTemporalData......................815.5.1PosteriorDistributionof˚i.......................825.5.2GibbsSampler..............................835.6SimulationStudy.................................835.6.1ASampleSimulationwith..................835.6.2Scenario1:N=7,p=5andT=10..................855.6.3Scenario2:N=14,p=15andT=50.................865.7DataAnalysis...................................885.8Discussion.....................................905.9ProofoftheLemmas...............................915.9.1ProofofLemma1.............................915.9.2ProofofLemma2.............................925.9.3ProofofLemma3.............................945.10SomePosteriorCalculations...........................955.10.1PosteriorCalculationforbeg.......................955.10.2PosteriorCalculationfor˝gj.......................975.10.3PosteriorCalculationfors2.......................985.11SomeDetailsforDataAnalysis.........................98BIBLIOGRAPHY....................................101viLISTOFTABLESTable2.1:TableforPosteriorMeanandStandardDeviation..........35Table3.1:TableforPosteriorMeanandSDforAmazonData.........41Table5.1:RMSE,TPRandFPRcomparisonforBGL-SS,IsingandBGL-SS-CARmodel...............................84Table5.2:BGL-SSandBGL-SS-CARestimatesfor's........84Table5.3:TableforRMSEandTrue/FalsePositiveRates...........87Table5.4:CotestimatesthroughBGL-SS-CARforAutoIndustryData89Table5.5:Companynameswithcorrespondingticker..............99Table5.6:CovariateList..............................100Table5.7:ResponseVariable............................100viiLISTOFFIGURESFigure2.1:ReferencePriorforin1-Parameter(Erdos-Reyni)networkModel.19Figure2.2:PosteriordensityofformonkdatasetwithrespecttoUniformprior,Referencepriorandnormalpriorwith˙=0:01............20Figure2.3:PosteriordensityofformonkdatasetwithrespecttoUniformprior,Referencepriorandnormalpriorwith˙=0:1............21Figure2.4:PosteriordensityofformonkdatasetwithrespecttoUniformprior,Referencepriorandnormalpriorwith˙=1.............21Figure2.5:PosteriordensityofformonkdatasetwithrespecttoUniformprior,Referencepriorandnormalpriorwith˙=100............21Figure2.6:ReferencePriorHeatmapforaTwo-ParameterDyadicIndependentModel..................................27Figure2.7:Scenario1:MCMCiterationsfor1and2withuniformprior...29Figure2.8:Scenario1:Histogramforposteriordistributionof1&2withun-oiformprior...............................30Figure2.9:Scenario1:Auto-correlationplotfortheMCMCiterationof1&2withuniformprior............................30Figure2.10:Scenario1:MCMCiterationsfor1and2withReferenceprior..31Figure2.11:Scenario1:Histogramforposteriordistributionof1&2withref-erenceprior...............................31Figure2.12:Scenario1:Auto-correlationplotfortheMCMCiterationof1&2withreferenceprior...........................32Figure2.13:Scenario2:Histogramforposteriordistributionof1&2withun-oiformprior...............................33Figure2.14:Scenario2:Auto-correlationplotfortheMCMCiterationof1&2withuniformprior............................34viiiFigure2.15:Scenario2:Histogramforposteriordistributionof1&2withref-erenceprior...............................34Figure2.16:Scenario2:Auto-correlationplotfortheMCMCiterationof1&2withreferenceprior...........................35Figure3.1:Adjacencyplotfor20,000nodesfortheobservedAmazonCo-Purchasingnetwork...........................40Figure3.2:DensityPlotforDyadicindependentNetworkModelparameterstoestimatetheAmazonCo-purchasingNetworkthroughReferencePrior41Figure4.1:CorrelationMatrix...........................56Figure4.2:ProbabilityoflinkagesforChrysler,Ford,andGMwiththeirSuppliers57Figure4.3:Positionofthe24companiesinthelatentspace...........58Figure5.1:Gibbsiterationsof'sforthescenariounderBGL-SS-CARwhen˙=0:5..................................85Figure5.2:PosteriorDistributionof˙2forthescenariounderBGL-SS-CARwhenthetrue˙2is0:25........................85Figure5.3:PosteriorDistributionof˙2forthesecondscenariounderBGL-SS-CARwhenthetrue˙2is0:25.....................86Figure5.4:PosteriorDistributionof˙2fortheData...............90ixChapter1Introduction11.1SocialNetworksSocialnertworkmodelinghasbecomeincreasinglypopularinpastfewyearsdueit'sabilitytocausallinksbetweennodesandforexplainingthoselinksinprobabilisticterms.Itisveryimportanttomodelirregularsocialbehaviorthatliesbeyondtheregularvariabilityandbringsstochasticityinthemodel.Moreoverinacomplexsocialenvironment,itisveryimportanttonotonlyhaveaprobabilisticexplanationoftheedgesbutalsotoexplainsomespstructuretoexploresomeinterestingsocialinteractions.Astatisticalnetworkisarepresentationofrelationaldataintheformofamathematicalgraphwhereeachnoderepresentsanindividualandarelationbetweenapairofnodesisrepresentedbyanedgebetweenthosetwonodes.NetworkdatatypicallyconsistofasetofNnodesandarelationaltieyijmeasuredoneachorderedpairofnodes.Thisframeworkhasmanyapplicationsinsocialnetworkliterature.Thesimplestsituationiswhenyijisadichotomousvariablethatindicatesthepresenseorabsenseofsomerelationofinterest.ThedataareoftenrepresentedbyanNNsociomatrixortheadjacencymatrixY.Variousprobabilisticmodelsofnetworkrelationshavebeendevelopedwithinpastfewyears.AstatisticalnetworkisagraphconsistsofasetofNnodes(orVertices)=fn1;n2;-:::;nNg.andasetofLedges(orconnections)=fl1;l2;:::;lLgthatdenotesthelinksbetweennodes.AnAdjacencyorSociomatrixYofdimensionNNcanbeusedtorepresentthenetworkby,yij=8>>>>>>>><>>>>>>>>:1ifedgeexistsfromnodenitonodenj;0Otherwise.(1.1)2HollandandLeinhardt(1981)includestheparametersforthepropensityoftiestobereciprocal,aswellasparametersforthenumberoftiesandindividualtendenciestogiveorreceiveties.Althoughthismodelassumesthen2dyadstobeindependentandknownasp1model.FrankandStrauss(1986),PattisonandWasserman(1999)andWassermanandPattison(1996)havegeneralizedtheideaofp1modeltopmodelthroughdyaddependencyassumption.WangandWong(1987)developedastochasticblockmodelwherenodesbelongtosomeprespgroups.NowickiandSnijders(2001)presentamodelwheregroupmembershipisunobservedandthedyadsinasocialnetworkareconditionallyindependentgiventhelatentclassmembershipofeachactor.Inthespatialcontext,Castroetal.(2015)developedamodelwherelatentgroupmembershipisinferredusingspatialclusteringwithanunknownnumberofclusters.Likewise,intheclassicalspatialeconometricsliterature,BhattacharjeeandHolly(2013)developGMMmethodstoinferonalatentnetworkofmembersinacommittee;forrelatedclassicalinferencesonlatentspatialnetworks,seealsoBhattacharjeeandJensen-Butler(2013),Baileyetal.(2015)andBhattacharjeeetal.(2015).1.1.1ExponentialRandomGraphModelFrankandStrauss(1986)characterizedtheexponentialrandomgraphmodel(ERGM)thatallowsanestimationofvariousgraphicalstructuresthroughanassumptionofdyaddepen-dence.Thetypicalformofexponentialrandomgraphmodel(ERGM)isgivenby,P(Y=y)=etS(y)c()(1.2)3where,S(y)isaknownvectorofgraphistheparametercorrespondingtotheS(y),c()isthenormalizingconstant.ERGMisveryimportantinthesensethatitgoesbeyondtheideaofdiscoveringthelinkprobabilitybetweenapairofnodesbyconsideringsomegraphicalcharacteristicamongasetofnodes.Thatis,S(y)canrepresenttnetworkifweobservefy34;y43gandfy12;y21g,asmeanswecanexpecttohavesomereciprocatingcharacteristicbetweenthenodes,whichmeansifnodeiislinkedwithnodej,thenwecanexpectjwillalsobelinkedwithi.Atypicalexampleofsuchlinkscanbeafriendshipnetwork.Butthismightnotbetrueinallinstances.Forexample,wecanthinkanelectricitypowersupplynetworkthatisunusualtobereciprocated.Thecorrespondingparameterestimatesthefrequencyofappearanceofthespationpresentinthenetwork.ThemainissuethattheMaximumLikelihoodEstimationoftheERGMmodelfacesistocalculatec().SupposeGdenotesallpossiblegraphsofY.Hence,c()=XGetS(y).NowGconsistsof2(n2)possibleundirectedgraphsanditisextremelytoevaluatethenormalizingconstantevenformoderatelysmallgraphs.TodealwiththecomplexissuesofcomputationintensitywithERGMforevenmoderatesizednetwork,Besag(2000),Handcock(2000),Snijders(2002)havedevelopedalikelihood-basedinferencebasedonMCMCalgorithms.Although,MonteCarlomaximumlikelihoodestimationfromtheproblemofmodeldegeneracyaswegetaverypoorestimateofthenormlizingconstantiftheinitiatialvalueofliesinthedegenerateregion.Approxi-matemaximumlikelihoodapproacheshavebeendevelopedbyFrankandStrauss(1986).ApseudolikehoodapproachisproposedbyStraussandIkeda(1990)andWassarmanandPat-terson(1996).ButthestatisticalpropertiesofpseudolikelihoodestimatorsinthiscontexthavebeencriticizedbyBesag(2000)andSnijders(2002).RecentdevelopmentonERGM4hasledtonewspthathavebeendiscussedbyHunterandHandcock(2006),calledthecurvedERGM.1.1.2BayesianERGMABayesianextentiontotheExponentialRandomGraphModelhasbeendiscussedinCaimoandFriel(2011)wheretheyhaveconsideredˇ(jy)=P(y)ˇ(),whereapriordistributionˇ()isplacedonandinterestisintheposteriorˇ(jy).SuchaBayesiantreatmenteasilysolvestheproblemofevaluatingthevalueofnormalizingconstantinthelikelihoodestimationcase.ABayesiantreatmentaslosolvestheproblemofmodeldegeneracyforMCMCmaximumlikeloodtechnique.Although,theposteriorofthisBayesianproblembecomes\doublyintractable"duetotheintractibilityofsamplingdirectlyfromtheposteriordistributionbutalsoduetotheintractibilityofthelikelihoodwithintheposterior.AsimpleimplementationofMetropolis-Hastingsalgorithmproposingtomovefromtowouldrequirethecalculationoftheratio,e0S(y)ˇ()e0S(y)ˇ()c()c()whichisunworkableduetothenormalizingconstantc()andc().Tohandlethe\doublyintractable"posterior,Murrayetal.(2006)andCaimoandFriel(2011)proposedanexchangealgorithmwithsamplesfromanaugmenteddistribution.ˇ(;y;jy)/P(y)ˇ()h(j)P(y)(1.3)whereP(y)isthesamedistributionastheoriginaldistributiononwhichthedatayis5h(j)istheproposaldistribution.Clearlymarginaldistributionofistheposteriordistributionofinterest.Thestepsforexchangealgorithmareasfollows:1.Draw˘h(j)2.Drawy˘P()3.Proposetheexchangemovefromtowithprobability=min 1;e0S(y)ˇ()h(j)e0S(y)e0S(y)ˇ()h(j)e0S(y)!1.1.3LatentSpaceModelInahighlytialpaper,etal.(2002)developedalatentvariablemodelwherenodeisassignedwithalatentpositionziinthesocialspace.Theideaisthattheprobabilityofarelationaltiebetweentwoindividuals(ornodes)arehigheriftheseindividualsaresimilarintheunobservedcharacteristicspace.Inthiscontextthesocialspacereferstoaspaceofunobservedlatentcharacteristicsthatrepresentpotentialtransitivetendenciesinnetworkrelations.Theresultingnetworksareprobabilisticallytransitivesincei!jandj!ksuggestsiandkareprobablynotfarapartinthesocialspace.Mostrecently,handcockandRaftery(2007)developedamodelbasedclusteringofsocialnetworkswheretheymodeledthelatentpositionsasamixturesofmultivariatenormals.Thelatentspacemodeltakesaconditionalindependenceapproachtomodelingbyas-sumingthepresenceorabsenceofatiebetweentwonodesthatindependentofallotherties,6giventheunobservedpositionsinthelatentspaceofthetwonodes.P(YjZ;X;)=Yi;jP(yi;jjzi;zj;xi;j;)HereXandxi;jareobservedcharacteristicthataredyadspandmaybevectorvaluedandandZarerespectivelyparametersandtheunknownlatentpositions.Consideralogisticregressionmodelasbelow,i;j=logodds(yi;j=1jzi;zj;xi;j;;)=+0xi;jf(zi;zj)Thefunctionfischosentobesimplewhichrepresentstheformsofnetworkdependence.Hereweassume,z1,...,zn˘Normal(0;˙z2)Thelatentspacemodelisinherentlyreciprocalandtransitive.Ifi!jihenitmeansthedistancebetweennodeiandnodejissmall,whichmakesj!imoreprobable.Againi!jandj!kimpliythedistancebetweennodejandnodekisnottwolarge,whichmakestheeventj!kmoreprobable.f(zi;zj)canbereplacedbyanyarbitrarysetofdistancesdi;jsatisfyingthetriangleinequality.Ingeneral,weprefertomodelthedi;j'sasdistancesinsomelow-dimentionalEuclideanspaceforreasonsofparsimonyandeaseofmodelinterpretablity.7Wesayasetofdistancesdi;jrepresentsthenetworkYiffdi;j>c8i;j:yi;j=0gandfdi;j<c8i;j:yi;j=1g(1.4)Wesaythatanetworkisdkrepresentableif9pointszi2<suchthatthedistancesdi;jsatisfy(1.4).Hence,dkrepresentabilityisequivalenttobeingabletoasetofpointsfortheactorssuchthati˘jiandjliewithinkdimnetionalunitballscenteredaroundeachother.GivenanetworkdataY=yi;jandpossiblecovariatesofthemodelX=xi;j,thegoalistoestimatetheunknownparametersofthemodel,denotedas.Theparameterincludestheregressorcots,andthevarianceoftherandompositionsofthenodesinthelatentspace.WetakeaBayesianapproachforestimationusingapriorprobabilitydistributionp().Conditionaldistributionoftheparametersgiventheinformationinthedataisp(jY)=p(Yj)p()=p(Y)TheMCMCbasedinferenceconstructsadependentsequenceofvaluesasfollows:‹Sampleaparameterfromaproposaldistributionh(jk);8‹Computetheacceptanceprobabilityr=max1;p(Yj)p()h(kj)p(Yj)p()h(jk);‹Setk+1=withprobabilityrandk+1=kwithprobability1r.Thisalgorithmproducesasequenceofvalueshavingadistributionwhichisapproxi-matelyequaltothetargetdistributionp(jY).Apointestimateofisoftentakentobetheposteriormean,whichisapproximtedbytheaverageofthesampledvalues.1.2BayesianVariableSelectionRecentdevelopmentsinstatisticalliteratureputahugeemphasisonvariableselectionfortheexplosivesamplespaceduetotheincreaseindimention,niceframeworkshavebeendevelopedtohandlethevariableselectionprocedureintheBayesianframework.Liang,SongandYu(2013)introducedtheideaofBayesianSubsetRegression(BSR)startingwithasubsetmodelandtakingGaussianpriorsonthemodelcots0isandtheyshowedthatifthetruemodelbecomessparse,i.e.,limn!1PPni=1jij<1wherePnisthemodeldimention,BSRreducestoEBIC.Undersomemildconditions,theyhavealsoshowntheposteriorconsistencyofthemodel.TheyhavealsoproposedavariablescreeningprocedurebasedonthemarginalinclusionprobabilityofthepredictorsandtheyhaveshownthatithasthesamepropertyofSureIndependenceScreening(SIS)wherewerankpredictorsaccordingtotheirmarginalutilityandthenselectsasubsetofthepredictorsofthemarginalutilityexceedingsomepredthreshold.BondellandReich(2012)proposedavariableselectioncriterionbasedontheposterior9credibleregion.Heretheythefullmodelusingallpredictorsandthenusedthehighestposteriordensityregionoftohaveasparseestimate.Thissparsevectorthendeterminestheselectedmodel.Penalizedregressionisamethodthatnotonlyselectstherelevantvariablesbutalsoestimatestheregressioncotssimultaneously.LASSOregression(Tibshirani,1996)providesadecentsolutionbyputtingupperboundontheL1-norm.Suitablyselectingthepenaltyparametercanprovideanexactzeroestimateforthecorrespondingirrelavantvari-ables.Tibshirani(1996)pointedoutthattheLASSOestimatorcanbeinterpretedasthemaximumapostiriori(MAP)estimatorwhentheregressionparametershaveindependentandidenticalLaplacepriors.LeastAngleRegression(LARS)providesmoreattractiveso-lutionsinceitfollowsthefullLASSOsolutionpathwiththecostofonlyoneleastsquareestimation(Efronetal.,2004).ParkandCasella(2008)introducedtheLASSOinasimilarBayesiancontextwheretheyintroducedalaplaceprioronthepenaltyparameterthatboilsdownthewholeproblemintoaBayesiancontext.TheyhavealsousedthefactthatthelaplacedistributioncanbeexpressedasagammascaledmixtureofNormalthatfacilitatestheposteriorcomputaion.AmajoradvantageofusingBayesianLASSOoverFrequetistLASSOisitprovidesreliablestandarderroroverthenon-Bayesianmethod(KnightandFu,2000;ChatterjeeandLahiri,2001;Tibshirani,1996).Moresp,theLASSOestimatorisequivalenttotheposteriormodewithindependentlaplacepriorforthecots.Usingthefactthatlaplacedistributioncanberepresentedasascalemixtureofnormals,ParkandCasella(2008)developedafullyBayesianhierarchicalmodelandntGibbssamplerfortheposteriorcompuations.ForlargensmallPregression,Liang,Truong,Wong(2001)establishedanexplicitre-lationshipbetweentheBayesianapproachandthepenalizedlikelihoodapproachforlinear10regression.TheyshowedempiricallythatBayesianSubsetRegression(BSR)thatischoosingpriorssuchthattheresultingnegativelog-posteriorprobabilityofthesubsetmodelcanbeapproximatelyreducedtofrequentistssubsetmodelselectionstatisticuptoamultiplicativeconstant.Selectingrelevantvariablesinahigh-dimentionalsetupisaverycommonfeatureinvariousBayesianandeconometricapplications.Inanadditivemodel,asetofcontinuouspredictormayberepresentedasgroupofpredictors.Huangetal.(2012)providesaniceinsightfortheapplicationofgroupvariableselection.YuanandLin(2006)proposedagroupLASSOmethodthatprovidesagroupvariableselection.BayesiangroupLASSOtechniquehasbeendevelopedbyKyungetal.(2010)andRamenetal.(2009)thathandlestheproblemofselectingthevariablesatthegrouplevelonly.Ifweconsideralinearregressionmodelofthefollowingform,yei=pXg=1Xgeg+ewheree˘N(0;˙2IN),egisacotvectorandXgisthecovariatematrixforthecorrespondinggthgroupandconsidertheminimizationproblemas(Simonetal;2012),mine jjyeipXg=1Xgegjj22+1jjejj1+2pXg=1jjegjj2!thenthesecondpenaltyterminducesavariableselectioninthegrouplevelandthepenaltyterminducesavariableselectionwithingrouplevel.ItcaneasilyshownthatthelaplacepriorcorrespondingtoaboveminimizationproblemcanbeexpressedasascalemixtureofnormalsandhenceafullBayesianimplementationwouldbeeasy.11Amixturepriorwithapointmassatzeroisaveryetooltohaveacontroledamountofshrinkageinthevariableselectiontechnique.SpikeandSlabpriors(IshwaranandRao,2005)andzeromixturepriors(MitchellandBeauchamp,1988)areveryetechniqueforBayesianVariableSelection.Wesetupthefollowinghierarchicalmodelas:yijxei;e;˙2˘N(xe0ie;˙2)kj˚k;˝2k˘N(0;˚k˝2k)˚kjˆ;v0˘(1ˆ)v0(:)+1(:)˝2kjb1;b2˘Gamma((a1;a2)ˆ˘U[0;1]˙2˘Gamma(b1;b2)Hence,theabovehiearchicalmodelimplies:kj˝2k;ˆ˘(1ˆ)N(0;v0˝2k)+ˆN(0;˝2k)(1.5)Theselectionofkcanbecontroledbythetwonormalsastheshrinkagev0pullsthenormalaround0(spike)andtheofkiscontroledthroughthesecondnormal.Zeronormalmixturepriorsinthehierarchicalformulationforvariableselectionhavebeenusedinthelinearregressionmodel(GeorgeandMcCulloch,1997).PointmassmixturepriorsarealsostudiedbyJohnstoneandSilverman(2004)andXuandGhosh(2015)forestimationofpossiblysparsesequenceofGaussianobservationswithanemphasis12onusingtheposteriormedianinsteadofthePosteriormean,whichhasbeenproventobemoree.AverystrongresulthasbeenshowninJohnstoneandSilverman(2004)andYuanandLin(2005)wheretheyhavecombinedthepowerofpointmassmixturepriorsanddoubleexponentialdistributionandtheresultingempiricalbayesestimatoriscloselyrelatedtoLASSOestimator.LykouandNtzoufras(2013)proposedasimilarapproachbyspecifyingashrinkageparameterthroughBayesfactorandZhangetal.(2014)generalizesthispriorforgroupLASSOtechniqueandproposedahierarchicalstructuredvariableselectiontechnique.13Chapter2ReferencePriorDevelopmentinExponentialRandomGraphModel142.1MainIdeaThekeyinBayesianliteraturefromthefrequentististhatBayesianusespriorinformationonthemodelparametersthat,insomesense,makesthemodelmorerobustandprotectsitfrombeingcarriedoutbysamplingerrorsorthroughalackofsamples.SomecomplexcomputationalissuesthatarisesinthefrequesntistapproachcanalsobeovercomedthroughtheimplementationBayesiantechniques.Itisthereforeimportanttohaveastrongknowledgeaboutthepriordistributionofthemodel'sparameters.Inmanysituations,wemaynothaveastrongholdonthepriorsandastringentinformativepriormaydrivetheproblemtoproducesomeunrealisticestimatesoftheparameters.Soitissometimesnecessarytobenon-informativeaboutthepriorknowledge.ThekeyideaofaBayesianproblemistochooseapriorinsuchawaythatitdoesnotbecomeverystringentandcanproducetheestimatesinadata-drivenfashion.Theproblemisthentooutanon-informativepriorthatcanextractthemaximizedinformationtobuildtheposterior.(Bernardo1979),BergerandBernardo(1989,1991a,1991b)havedevelopedtheideaofreferencepriorthatcanmaximizetheinformationbetweenthepriorandtheposteriorforagivenproblemthroughKullbeck-Libeler(KL)divergence.SupposewehavedataYparameterizedbywithtstatisticT=T(Y).AreferencepriorisapriorthatmaximizesKLdivergencefromtheposteriorˇ(jt)averagedoverthedistributionofTi.e.,wewanttomaximizeI;T)=ZZp(t)ˇ(jt)logˇ(jt)ˇ()dt(2.1)overallˇ().15Thereferencepriorsatisfyingequation(2:1)maximizestheposteriorinformationob-tainedfromtheclassofallthedefaultpriors.Inthiscase,wearelookingforsuchapriorsothatourdatacanhaveitsmaximumimpactontheposteriorestimates.Moreover,posteriordistributionbecomeshighlysensitivewiththechoiceoftheinformativeprior.Ifweintro-ducemorecovariatesinthemodel,thesensitivityincreases.Wewouldthenliketousethevirtueofthedefaultpriorsthatdonotputanypriorinformationontheparameterestimatesandreliesonthefactthatitoptimizestheposteriorestimatesinadata-driveninformationtheoretictechnique.ERGMmodelfacesakeytheoriticalissueofworkingwithasinglesample.InatypicalERGMproblem,weworkwithoneobservedinstanceoftheadjacencystructure.Hence,technically,ERGMoperatesinalimiteddataatmospherewherethedatadriveninformationisverylimited.Hence,itisveryimportanttohaveaprocedurethatcanoptimizetheextractionoftheavailableinformation.2.2ReferencePriorforOneParameterErdos-ReyniModelThemainpurposeofthischapteristoimplementthereferencepriortechniqueforBayesianERGM.NotethatthemainpurposeofintroducingthereferencepriorforanERGMmodelisthatwecanresolvetheproblemofthesensitivityoftheposteriorestimatescausedthroughtheinformativepriorsaswellasprovidetheposteriorestimationsthatareoptimizedinainformationtheoreticsense.ThesimplestexampleofoneparameterERGMmodelistheErdos-Reyninetworkmodel.Inthismodelingscenario,weassumetheedgesareindependentwithedgeprobability16(ErdosandReyni,1959).Themodelcanbewrittenas,f(yj)=Yi6=jyij(1)1yij;0<<1(2.2)Thismodelconcentratesontheexistenceofthesinglepossibleedgetionyij,whichisparameterizedby.Hereiscalledtheedgedensityparameter.Reparameterizationof(2.2)gives,f(yj)˘expfXi6=jyijg(2.3)SothisisanExponentialRandomGraphModel.Referencepriorisintermsofmutualinformation,andsincethemutualinformationisitselfinvariant,thereferencepriorsbecomeinvariantunderreparameterization.Sowecancalculatereferenceprioreitherfororfor.Supposexebethevectorofobservationsfromthemodelandxe(k)=(xe1;xe2;:::;xek)beavectorofindependentreplicatesofthevectorobservationsfromthemodel.Lettk=tk(xe1;xe2;:::;xek)2˝kbeanytstatisticsofthereplicatedobservations.Letusˇ(jtk)=p(tkj)ˇ()Rp(tkj)ˇ()and,fk()=expZ˝kp(tkj)log[ˇ(jtk)]dtk17Theorem(Berger,Bernardo,andSun;2009).AssumeastandardmodelMfp(xej);xe2X;2ˆRgandPisthestandardclassofcandidatepriors.Letˇ()beacontinuousstrictlypositivefunctionsuchthatthecorrespondingformalposteriorˇ(jtk)isproperandasymptoticallyconsistent.foranyinteriorpoint0of,f()=limk!1fk()fk(0)1.Ifeachfk()iscontinuousand,foranydandlargek,ffk()=fk(0)giseithermonotonicinkorboundedabovebysomeh()whichisinte-gratableonanycompactset,and,2.f()isapermissiblepriorfunction,thenf()isareferencepriorforthismodelMandpriorclassP.Sincetheanalyticalderivationofareferencepriormaybetechnicallydemandingduetoacomplexmodel,wecanusethefollowingalgorithmforthe1-parameterfamilyofdistribu-tions.Algorithm(Berger,Bernardo,Sun,2009).1.Initialvalues:Chooseamoderatevaluefork;Chooseanarbitrarypositivefunctionˇ();Choosethenumbermofsamplestobesimulated.2.Foranygiven,repeat,forj=1;2;:::;m:Simulatearandomsamplefx1j;x2j;:::;xkjgofsizekfromp(xj);Computenumericallytheintegralcj=RQki=1p(xijj)ˇ();evaluaterj()=log(Qki=1p(xijj)ˇ()=cj).183.Computeˇ()=expfm1Pmj=1rj()gandstorethepairf;ˇ()g.4.Repeatroutines(2)and(3)forallvaluesforwhichthepairf;ˇ()gisrequired.Forthe1-parameterErdos-ReyniERGMmodel,wecancomputethereferencebyusingthenotionofmaximizingtheKLdivergenceprinciple.Figure(2.1)showsthereferencepriorforintheErdos-Reynimodel.NowthepriorforintheoneparameterErdos-ReynimodelisgivenbyI()=qn(1).Itistheninterestingtoseetheresemblancebetweenthepriorandthereferencepriorinthe1-parametermodel.Itisalsoaniceexampletoseethatthereferencepriorinoneparametermodelisnothingbutpriorundercertainregularityconditions.Figure2.1:ReferencePriorforin1-Parameter(Erdos-Reyni)networkModel2.2.1'Sampson'sMonkData'ImplementationWeimplementthereferencepriorapproachforoneparameter,Erdos-ReyniModeltoSamp-son'sMonkdataset,whichprovidesanadjacencystructurerepresentingtheinteractionbe-tween18monksinamonastery.Ourtargetistogettheposteriorestimateofthemodel19parameter.Sincethepurposeofthispaperistocatchthesensitivityduetoinformativepriors,wecomparetheposteriorthroughthereferencepriorwithanon-informativeuniformpriorandwithnormalpriorswitha0meanandfourtvaluesforthestandardde-viation.Theoutputshowsthesensitivityoftheposteriorduetotheinformativenormalpriors.Itisimportanttoaddresspriorsensitivityinsuchasimpleone-parametermodelingsituation.Ifweincorporatemoreparametersinthemodelbyintroducingmorecomplexstructuresinthenetwork,weexpectthesensitivitytoincreasefurther.Theuseofnon-informativepriors,suchasauniformprior,willresolvetheissueofpriorsensitivity,butitisnotguaranteedtoprovidethemaximizedinformationwithrespecttotheprior.Figure2.2:PosteriordensityofformonkdatasetwithrespecttoUniformprior,Referencepriorandnormalpriorwith˙=0:0120Figure2.3:PosteriordensityofformonkdatasetwithrespecttoUniformprior,Referencepriorandnormalpriorwith˙=0:1Figure2.4:PosteriordensityofformonkdatasetwithrespecttoUniformprior,Referencepriorandnormalpriorwith˙=1Figure2.5:PosteriordensityofformonkdatasetwithrespecttoUniformprior,Referencepriorandnormalpriorwith˙=100212.3ReferencePriorforTwoParameterDyadicinde-pendentnetworkModelTointroduceatwo-parameterdyadicindependentnetworkmodel,wedecomposeYinton2dyadsofpairs,Dij=(yij;yji)fori<j.Todescribethejointdistribution,weextendourindependenceassumptionfromtheedgeindependentErdos-Reynimodeluptodyads.Here,weconsiderreciprocatededgesalongwithsingleedges.Wecanthenwriteourmodelas,f(yj1;2)˘exp(1Xi6=jyij+2Xijyijyji)(2.4)Here,Xi6=jyijdenotesthenumberofedgesinthenetworkandXijyijyjidenotesthenumberofmutualties.Inthismodelingscenario,we1astheedgedensityparameterand2asthereciprocityparameter.Wecanextendtheone-parameterreferencepriorideatoatwo-parametercasethroughthesequentialscheme(BergerandBernardo,1992).Thatmeanswearrangeourparametersintermsoftheirinferentialimportance;inparticular,theparametershouldbetheparameterofinterest.Forthedyadicindependentmodel,obviouslyourparameterofinterestwouldbe2.Sincethenormalizingconstantisunknownandhardtocalculateforthedyadicindependentmodel,wetrytosolvetheprobleminaBayesiansetup.NowthecomputationofareferencepriorrequiresaclosedformexpressionofthelikelihoodfunctionthatisintractableinaERGMsetupevenforamoderatelylargenetwork.Therefore,tofacilitatethereferencepriorcomputation,weapproximatethelikelihoodfunctionthroughapseudo-likehoodthat22isgivenby,PL(1;2jY)=Yi6=jP(yij=1jyijC)=Yi6=je1+2yij1+e1+2yij(2.5)whereyijCdenotesthenetworkY,exceptnodesiandj.AtheoreticalvalidationofthisapproximationcanbefollowedfromStrauss&Ikeda(1990)andBesag(1974,1975)wheretheyhaveshownthatlikelihoodmaximizationof(2.4)isequivalenttothemaximizationof(2.5).2.3.1MethodologyforDerivationWeconsiderthefactthatatwo-parameterdyadicERGMisasymptoticallynormallydistributedsinceitisnothingbutatwo-parameterexponentialfamily.Suppose=f1;2:11;21gistheparameterspaceofthetwo-parameterdyadicindependentmodel.LetusconsideranestedsequenceofflgofcompactsubsetsofsuchthatS1l=1l=Thekeytechniquereliesonthefactthatwearrangethemodelparametersbyimportancewithrespecttothemodel.Wecalculatethedensityˇl2(1j2),asˇl2(1j2)/expnXYp(Yj1;2)logp(1jY;2)o(2.6)23and,ˇl1(1;2)/ˇ(2j1)expnXYp(Yj2)logp(2jY)o(2.7)Thekeyadvantagehereisthepseudo-likelihoodapproximationandsincewehavetheassumptionofconditionalindependenceoverthedyads,thesingleobservednetworkofsizeninducesadyadicmodelwithn2replications.Inasequentialsetup,wewillbeconsideringthedistributionof1for2andalsodistributionof2.Fora2,andeachtermof(2:5)isanexponentialfunctionof1andhenceitistwicecontinuouslyditiablew.r.t1.Hencewehave,lnYi;jPL(1+1jyij;2)ˇlnYi;jPL(1jyij;2)+1pnXi;j@lnPL(1+1jyij;2)@1+12nnXi;j@2lnPL(1+1jyij;2)@1@where,=1pnNowthesecondtermisasymptoticallynormalbyCentralLimitTheoremandthethirdtermconvergestoinformationmatrixI(e)inprobability.Hence,byLeCam(1960)wehavethepseudolikelihoodislocallyasymptoticallynormalandhenceregularwhichmeanspseudolikehoodisasymptoticallynormal.Similarly,eachtermof(2:5)istwicetiableasafunctionof2.24SupposeI()betheFisherinformationmatrixwhere,Iij(e)=E@2@i@jlogf(Yj1;2)Now,@2PL@21=Xi6=jny2ije1+2yij(1+e1+2yij)2oandE@2PL@21=n(n1)e1+21+e1+2ne11+e1+e1+21+e1+2oSimilarly,E@2PL@22=n(n1)ne1(1+e1)2+e1+2(1+e1+2)2one11+e1+e1+21+e1+2on(n1)1and,E@2PL@2@1=n(n1)e1+2(1+e1+2)2ne11+e1+e1+21+e1+2on(n1)ThereforewegetjI(e)j=n2(n1)2ne11+e1+e1+21+e1+2o2n(n1)2e1+2(1+e1+2)2e1(1+e1)225Nowweset,S()=(I())1.Inatwo-parametersetup,wecansetS()=f((aij)):1i2;1j2g.NowsupposeSjbetheupper-leftjjcornersofSandsetHj=S1j.Nowifhj()bethelower-rightjjcornerofHj,thenwehaveh1()=1a11=n(n1)ne11+e1+e1+21+e1+2on(n1)11+e1+22e1+2+1+e12e1andh2()=a11=n(n1)ne11+e12+e1+21+e1+22one11+e1+e1+21+e1+2on(n1)1Theaboveformulationgivestheexplicitresultfortheprobabilitiesgivenbyequations(2.5)and(2.7)by,ˇl2(1j2)=jh2()j1=2Il(2)(1)Rl(2)jh2()j1=21(2.8)and,ˇl1(1;2)=ˇl2(1j2)expf12El[(logjh1()j)j2]gIl(12)(2)Rl(12)expf12El1[(logjh1()j)j2]2gwhere,E1[g()j2]=Zf1:(12)2lgg()ˇl2(1j2)126Typically,wedeterminethereferencepriorforthetwo-parameterdyadicindependentmodelas,ˇ()=limlˇl1()ˇl1()(2.9)where,isanypointonforwhich,thefollowingconditionEYlD(ˇl1(jY);ˇ(jY))!0asl!1whereD(g;h)theKLdivergencebetweendensitiesgandh.Figure2.6:ReferencePriorHeatmapforaTwo-ParameterDyadicIndependentModel272.4SimulationStudyforDyadicIndependentModelInthissection,wewillbeusingasimulatednetworkwithnodesizeN=50.Werestrictoursimulationtoasparsenetworkasmostofthereal-worldexamplesgeneratesparsenetworksandtheideaistocheckhowthereferencepriorcanutilizeitsvirtueofmaximizingtheinformationfromthedatatogettheposteriorestimates.ThekeyfeatureofExponentialRandomGraphModelisthatthespecstructureisinherentinthemodelthroughspparameters.Inourapplication,wewilltrytoseeagraphwherewebelievethatithasadyadicfeature,whichmeansthenetworkisdirectedandwearenotgoingtoseethenetworkfeaturesbeyondtwonodes.Inthecurrentprogressinnetworkliteratureandcomputerscience,thekeyissueistoestimatealargesparsenetworkwheremostofthedyadsareempty.ThiscreatesdiyinthemixingoftheMCMCchainsincethechaininvolvesmostofitstimetolinkingtheemptydyadsincaseofasparsenetwork.Theusualremedy(seeHunteretal.,2008)istouseadefault\tienotie"(TNT)samplerwherewedividethewholesetofdyadsintotwosets,onewithallthelinksandanotherwithnolinks,andthenwepre-assignprobabilitiesforthesetwosetsanddrawdyadsfromthesetwosetsinsteadofdrawingrandomlyfromthewholenetwork.Theadvantageofthistechniqueisthatmostofthereal-worldnetworksaresparseandthesamplergivesmorechancestothesetwithedgestoplaymostofthepartintheMCMCchain.Moreover,wecantunetheprobabilityofthetwosetstogainabettermixingsincewetrytocontrolthedegreeofsparsityoftheestimatednetwork.Inthistechnique,ourcomputationdoesnotdependonthesizeNofthewholenetwork,butinsteadourcomputationalcomplexitybecomesO(m)wheremisthesamplesizetobeselectedfromthetwosetsateachiterationstep.282.4.1Scenario1Inthescenario,Weset1=3and2=3.Werantheiteration40,000timeswithaburn-inperiodof20,000.WetookaproposaldistributionofN(0;(0:25)2),whichmakestheacceptancerateoftheMCMCsamplerstayaround20%.TheposterioroutputoftheMCMCchain,alongwiththeauto-correlationplotandtheMCMCiterations,isshownfromFigure2.7toFigure2.12b.Wecomputetheposteriorusingtwotdefaultpriors,withU(;1)andthesecondwiththereferencepriorforthetwo-parameterdyadicindependentmodel.FordecreasingtheautocorrelationbetweentheMCMCsamples,weapplyathinningprocesswithn=5.Figure2.7:Scenario1:MCMCiterationsfor1and2withuniformprior29(a)1(b)2Figure2.8:Scenario1:Histogramforposteriordistributionof1&2withunoiformprior(a)1(b)2Figure2.9:Scenario1:Auto-correlationplotfortheMCMCiterationof1&2withuniformprior30Figure2.10:Scenario1:MCMCiterationsfor1and2withReferenceprior(a)1(b)2Figure2.11:Scenario1:Histogramforposteriordistributionof1&2withreferenceprior31(a)1(b)2Figure2.12:Scenario1:Auto-correlationplotfortheMCMCiterationof1&2withreferenceprior32(a)1(b)2Figure2.13:Scenario2:Histogramforposteriordistributionof1&2withunoiformprior2.4.2Scenario2Inthesecondscenario,weset1=5and2=5.Werantheiteration40,000timeswithaburn-inperiodof20,000.WetookaproposaldistributionofN(0;(0:3)2),whichmakestheacceptancerateoftheMCMCsamplerstayaround20%.TheposterioroutputoftheMCMCchain,alongwiththeautocorrelationplotandtheMCMCiterations,isshownfromFigure2.13atoFigure2.16b.Wecomputetheposteriorusingtwotdefaultpriors,withU(;1)andthesecondwiththereferencepriorforthetwo-parameterdyadicindependentmodel.FordecreasingtheautocorrelationbetweentheMCMCsamples,weapplyathinningprocesswithn=5.ItcanbeseencomparingthetpriorsetupinthetwotscenariosthattheautocorrelationfortheMCMCiterationsafterthinningisdecreasingfaster(ataroundlag50inthescenarioandataround100inthesecondscenario)inthereferencepriorsetupthantheuniformsetup(ataroundlag100atthescenarioandataround150in33(a)1(b)2Figure2.14:Scenario2:Auto-correlationplotfortheMCMCiterationof1&2withuniformprior(a)1(b)2Figure2.15:Scenario2:Histogramforposteriordistributionof1&2withreferenceprior34(a)1(b)2Figure2.16:Scenario2:Auto-correlationplotfortheMCMCiterationof1&2withreferencepriorTable2.1:TableforPosteriorMeanandStandardDeviationScenario'sParametersPosterior-MeanPosterior-SDSCENARIO1UniformPrior1-3.6720.98223.6190.974ReferencePrior1-2.5300.92222.5480.936SCENARIO2UniformPrior1-5.2291.56025.2201.551ReferencePrior1-4.4901.46424.5101.448thesecondscenario).Thus,weneedmoreiterationscomparedtoreferencepriorsetupinuniformsetuptoachieveelyindependentdraws.352.5DiscussionThispaperisdedicatedtodevelopingadefaultpriorsetupinaBayesianmodelingscenariothatnotonlyovercomestheissueofsensitivityoftheinformativepriorsbutalsointegratestooptimizetheposteriorestimatesthroughaninformationtheoreticsetup.Althoughthepaperhasdevelopedatwo-parameterdyadicindependentnetworkmodel,itcanalsobeextendedtomorecomplicatednetworkmodels.TheMetropolis-HastingsalgorithminERGMsetupismuchmorechallengingsinceERGMfromtheproblemofmodeldegeneracywheretheiterationscanconvergetoafulloremptygraph.Poorchoiceoftheinitialvaluesofthemodelparameterscanleadtheiterationstoafulloremptynetwork.Toavoidthissituation,wehaveusedaweakthresholdingforthenumberofedgesofthenetwork.WehavediscussedthattheBayesiantechniqueappliedtoalargenetworkdatacanbeesincewecaneasilyavoidtheproblemofobtainingthenormalizingconstant.Moreover,theapplicationoftheTie-No-Tiealgorithminalargenetworkbecomesesinceitusesarandomsamplefromthesetofedgesandemptydyadsthatareassumedtobeasub-sampleoftheobservednetworkandaredistributedastheproposedmodel.SoTNTprovidesastrongtheoreticalbackgroundofanalternativetothetraditionalMCMCiteration.Alongwiththat,itmakestheMCMCiterationswayfastersinceitalwaysconsidersasubsampletoperformeachiterationsnomatterhowbigthenetworkis.Ouranalysisalsoshowsthatuseofareferencepriorprovidesanaddedbbygeneratingtheindependentsamplesfromtheposteriordistributionfasterthantheotherdefaultpriors.36Chapter3BigDataApplicationofERGMthroughReferencePrior373.1BigDataNetworkRecentadvancementsincomputer,Internetandsocialmediahighlighttheveryimportantaspectsofaccessingandanalyzingtheuserdataforfuturedevelopments.Networkliteraturehasbroadenedtheareaofuserinterfaceswherenotonlycanauserobtainhispreferredproductsbasedonhisinputsbutalsothemanufacturersgetanideaaboutwhatpaththeyneedtofollowtogettheirproductclosertoeachindividualcustomer.Forexample,anAmazonpurchasercanseetherecommendedproductbasedontheproducthehasviewedorpurchased.Aprobabilisticdeterminationofthelinksisthereforeimportanttooptimizethesalesofcertaincompaniesaswellastooptimizetheutilityoftheproductsofthepurchasers.Developmentsinonlinetrading,shopping,andmediaservicesinthepastfewyearshaveopenedagatewaytoanalyzethecharacteristicsofindividualusersthroughamassivedataatmosphere.Carefulanalysisandhandlingofsomuchdataisachallengingtaskthatrequiresmassivespaceandtimeandisstillnotapplicableinmanycircumstances.Networkestimationthroughgoodstatisticalpropertiesoftheestimatesisapopularofstudyinrecentyears.OneoftheimportantnetworkmodelsthatcanstructurallymodelthenetworkthroughnicestatisticalpropertiesistheExponentialRandomGraphModel(ERGM).AlthoughamoderatelylargenetworkcancreatetroubleinestimationofparametersinERGMmodel,Bayesianestimationcanovercomethisissuethroughbypass-ingthecalculationofthenormalizingconstant.ButsincetheestimationprocedureinvolvesMCMCiterationsthroughtheMetropolis-Hastingstechnique,theestimationprocedurebe-comesveryslowandagoodmixingbecomeschallengingevenforasparsenetworkofsize500500.NotmuchworkhasbeendonetohandlelargesparsenetworksinanERGMsetup.38ThiemichenandKauermann(2016)haveproposedanERGMmodelwithnon-parametriccomponentstoestimatelargenetworksthroughsubsamplingtechniques,butthismethodislimitedtodensenetworksonly.HeandZheng(2015)proposedanestimatingtechniqueoflargesocialnetworkthroughgraphlimitsforERGM,buttheirmethodalsofailsforlargesparsenetworkswhereitisshownthatthegraphlimittendstozeroinasparsesituation.ThischapterisdedicatedtoestimateasparseBigDataAmazonco-purchasingnetworkwith262111nodesthroughtheapplicationofTNTprocedureandthroughtuningtheproba-bilityofedgesetandemptydyadset.WeimplementthereferencepriortechniqueaswehaveseenthattheACFfunctiondropsfasterinthereferencepriorscenarioandsoweobtainedtherandomsamplesfromtheposteriorsfasterthantheotherpriorcase.3.2DataandModelForaBigDataimplementationofERGM,weconsideranAmazonco-purchasingnetworkfromtheStanfordSNAPdatarepositorywherethenetworkisbasedonthe"CustomersWhoBoughtThisItemAlsoBought"featureoftheAmazonwebsite.Weconsider262111nodes,anditcanbeconsideredasasparsenetworksincetheobservednetworkhas1048575directedlinksand562240reciprocatedlinks.ThemainideaoftheAmazonrecommendationsistopickbuyersforsimilartypesofitems.Forthepurposeofthedataanalysis,wetoassignaprobabilitytoeachpossiblelinkbasedontheobservedadjacencystructure.Thatmeansifabuyerpurchasesorvisitssomeitemwebpage,thenthesystemwouldincorporatetheinformationfromtheotherbuyerstopickanitemsimilartowhattheyhavepurchasedtoshowintherecommendationlist.FortheimplementationpurposesoftheERGMmodel,weintroducethedyadicindepen-39dentERGMmodelwheretheonedirectionalandthereciprocatedlinkarebeingcaptured.ThatmeansapairofitemslistedonAmazontobeconsideredundermodelinganddirec-tionalandreciprocatedlinksbetweenthemisgoingtobeestimated.Theofanythirditemwillbeconsideredtobeconditionallyindependentofthesetwo.Henceconsiderthemodel,f(Yj1;2)˘exp(1Xi6=jyij+2Xijyijyji)(3.1)whereyijbeingtheobservedadjacencybetweenithandjthitemlistedonAmazon.1isthesinglelinkparameterand2isthereciprocationparameter.Hence,Xi6=jyij=1048575andXijyijyji=562240.3.3EstimationFigure3.1:Adjacencyplotfor20,000nodesfortheobservedAmazonCo-Purchasingnetwork40Table3.1:TableforPosteriorMeanandSDforAmazonDataParametersPosterior-MeanPosterior-SD1-6.5810.22926.5820.231ThekeyadvantageoftheTNTprocedureisthatitusesasmallsubsetoftheobservednetworkateachMCMCiterationinsteadofobservingthewholenetwork,whichmakesthewholecomputationtimemuchfasterthanthetraditionalMCMC.Thismethodisapplicabletolargenetworksandfacilitatesgoodestimatesbasedoninformationtheoreticsensealongwithfastercomputation.Wedividedthedyadsintotwosets,aswediscussedbefore,andassignedequalprobabilitytothesetwithedgesandtothesetofemptydyads.Thisspassignmentgivesadecentlevelofmixingwithasparseestimateofthenetwork.Wetake5,000MCMCiterationswithaburn-inperiodof4,000.WesetourproposaldistributiontobeN(0;(0:075)2).Theposteriormeanof1isgivenby6:581andtheposteriormeanof2is6:582.Figure3.2:DensityPlotforDyadicindependentNetworkModelparameterstoestimatetheAmazonCo-purchasingNetworkthroughReferencePrior41AkeyfeatureoftheERGMmodelisthenodesareconsideredequivalent.Tocheckwhetherthesparsenessoftheobservednetworkissustainedintheestimate,theideaistorandomlypickdyadsfromthenetworkandtrytore-assignordroptheedgesbasedontheestimatedmodel.Forthatpurpose,wepicked100000dyadsandtriedtoreassignthedirectedandthereciprocatededges.Whenwetriedtoreassignthoseedges,wesawthattheerrorrateintermsofsparsitywas5:233%forthedirectededgesand5:226%forthereciprocatededges.Althoughthenetworkhas262111nodes,whichmeanstheadjacencymatrixcanhave262111262111edges,thecomputationtimewasveryfastcomparedtothesizeoftheadjacencymatrix.Althoughwehadtochoosethesizeofthesample,weneededtoselectateachiterationstep,asamoderatesizeofsamplesgivesagoodmixalongwithfastcomputationtime.Inourexample,werantheiterationsbytaking1000randomsampleofdyadsateachstepandassigningequalprobabilitytotheedgessetandtheemptydyadset.Wealsocantunetheprobabilitybasedonthedegreeofsparsity,alongwithourvalueofthesampletobechosenateachiterationthatmighthavebettermixing.3.4DiscussionItisimportanttoobservethatsuchalowestimateof1showsthesparsitypropertyofthenetworkisvalidfortheassumedfamily.Alsoweobservedahighvaluefortheestimateof2,whichmeantmostoftheestimateddirectionallinkswereessentiallyreciprocatingandifiththeitemisbeingshownintherecommendationforthejthitem,thenitistrueintheotherway,whichmakessenseinaproductrecommendationscenario.Thisideacanbeextendedtomorecomplexparametersituationwherewebringmore42structuraltiesinthemodel.Bringingsomeinterestingtiesbetweenthenodecanexploreadetailedsocialcharacteristicsofthebuyerandhelptodecidetheoptimizedrecommen-dationsforhim.Inourmodelingscenario,alowvalueofthedirectionaledgeparameterindicatessomesptendenciesofthebuyersanditrevealsthatasetofbuyercaneasilybeseparatedintovariouscategoriesoftheproductsandrandomrecommendationscanbeirrelevantaccordingtothenuyerschoice.Forexample,recommendingkidstoyscouldbeirrelevanttoastudentwhoboughtbooksoralaptopinhislastcoupleofpurchases.Againaveryhighvalueofreciprocatingparameterisverymeaningfulinthesensethatitcanhelpclusteringagroupofbuyerswhoarebuyingtkindofproductsofsimilarcategories.Forexample,Buyer1whoboughtadesktopcomputerinhislastpurchasemayendupbuyingaprinterofit.Also,Buyer2whoboughtaprinterinhislastpurchasemighthaveanoldcomputerandhemaybeinterestedinreplacinghisoldcomputerandmaybeinterestedinthecomputerthatBuyer1purchased.HenceBuyer1andBuyer2maybepurchasingtproductsonthatdaybutthenetworkcharacteristicrevealsthattheybelongtothesameclusterofbuyerswhoarebuyingcomputerrelatedproducts.Hence,averysimpletwoparameterdyadicindependentmodelgivesusadirectionforthecompaniesneedtomaketooptimizetheirproductrecommendation.Hence,inclusionofmorecomplexgraphicalstructurecouldbebforoptimizingthesellsforthemanufacturersaswellastheproductutilitiesforthebuyers.ThekeyadvantageofapplyingtheTNTalgorithmintheERGMandreferenceprioristhatitmakesthecomputationfastsince,evenforalargesparsematrix,thecomputationmostlydependsthesizeoftherandomsamplesdrawnfromthetwosetsaswellasthepre-spprobabilitiesforthesetwosets.AllowingustotunethesetprobabilitieshelpswiththecontrolingthemixingoftheMCMCchainsincethechaindoesnotspendmostofitstime43ontheemptysetsbybeingsparse.AlthoughitisverychallangingtoobtaintheoptimizedvalueofthesamplesizeofthedyadsateachiterationstepsalongwiththeoptimizedvalueofthetunningprobabilitythatmakestheposteriorMCMCsamplesclosesttotheactualposteriorandmakesthecomputationveryfast.AlthoughthenodesareconsideredequivalentinERGM,thekeyadvantageoftheim-plementationoftheERGMtechniqueisthatwecanpre-specifythegraphstructureweareinterestedinandcandetermineifaspstructureisdominatingorhasnointheformationofthelinks.44Chapter4LatentSpaceNetworkforthreeUSAutoManufacturingGiant454.1MainIdeaandMethodologyThereisgrowinginterestintheuseofnetworkmodelstorepresentrelationsbetweenactorsornodes.Insocialnetworks,theseactorsareindividuals,whileintheinlinkagescontext,theyareSometimesthetwocometogether,forexample,intheliteratureondirectornetworks;forarecentdiscussionfromamoregeneralcontext,seeBorgattietal.(2009).ThispaperexaminesinnetworksintheUSautomobileindustry,focusingonthe3manufacturinggiants-Chrysler,FordandGeneralMotors-and21ofthemostprominentintermediariesinthesector.Theanalysisisbasedonamodelofoperatingleverage(Bhattacharjeeetal.2014),askingthequestion:howmuchdob(orlose)fromnetworkexternalitiesinsharingm-sprisk?Theresultspointtoacombinationofexplanations:corporategovernancelinkages,supplychainnetworksandpotentiallydemandsidelinkagesaswell.Therefore,thischaptersuggestsawholisticapproachintheanalysisifinnetworks,combininganumberofchannels(ordrivers)anddisciplinaryapproaches.Recentstudiesoninnetworkshaverevealedvariouscharacteristicsoftheitsmanagersanditsownership,andhowthisit'sneighboringInturn,neighborhoodhasbeencapturedbylinkagesalongthesupplychain(Hertzeletal.,2008;Wang,2012;AhernandHarford,2014;Itzkowitz,2015),directornetworks(RenneboogandZhao,2011,2014),orjointventuresandinvestmentsyndication(WangandWang,2012).Questionshavebeenaskedaboutwhethersupply-sideordemand-sidelinkagesdriveinter-actionsbetween(EllisandMcGuire,1993;Venables1996),andaboutthenatureofthenetworksthemselves-whethercohesivenetworksofsociallyembeddedtiesorsparsenetworksrichinstructuralholes(GrandoriandSoda1995;HiteandHesterly2001).Recent46researchinnancehasexploredthefunctioningofthemarketplacingemphasisonstrategicinteractionsbetweenandtheirindustryrivalsandsuppliers.Inthissetting,economicorshocksaretransmittedthroughthecustomersuppliernetworkandthewholeindustrycanbebythoseshockssincethesecanmovethroughthelinksofinthatindustry,andbeyond;see,forexample,AhernandHarford(2014).Inpartic-ular,Vickeryetal.(1999)andNarasimhanandJayram(1998)arguethatsalesvolatility,disruptionsandopportunitiesinthesupplyanddemandenvironmentsareimportantaspectsofthemarket.Intermsofmethodology,networkmethodsbasedongraphtheoreticframeworkshavehelpeddocumentapositivelinkbetweennetworkstructuresandperformance(GeletkanyczandBoyd2011;Larckeretal.,2013).Thekeymechanismisthatastrongnetworkprovidesbetteraccesstoinformationwhichthenbringsbetoainitsdecisionmak-ing(LarckerandTayan,2010;Omeretal.,2012).Then,thisframeworkhelpedresearchersrevealpreviouslyhiddenrelationshipsbetweentheconnectionofthecorporateeliteandboardroomissuessuchasdecisionmakingonmanagerialcompensation,investments,andhiringandoftopmanagement.RebbeboogandZhao(2011)andHortonetal.(2012)demonstratedthataCEO'sdirectandindirectconnectionshispowerandthevalueofhisinformation-connections,whichisinhigherremuneration.Indeed,arelationshipbetweeninlinkagesandbusinessperformancehasbeenthefocusofmanystudies.Vickeryetal.(1999)foundtrelationshipsbetweensupplychainyandtmeasuresofperformanceintheUSfurnitureindustry.Likewise,VonderembseandTracy(1999)observedalinkbetweensupplierselectioncriteriaandmanufacturingperformance.NarasimhanandJayram(1998)arguedthatresearchinsupplychainmanagementtendstofocusontheindividualfunctionsandfailstoexamine47thecausallinkagesthatcomprisethesupplysideoftheeconomy.Swaminathanetal.(1998)emphasizedtheimportanceofdemandforcastinginthesupplychaindynamics.Inthispaperwedevelopamethodbasedonnetworkdatathatcapturesthelatentlinkagesbetweenbasedonamodelofoperatingleverage,andwhichthenvariousdecisionmakingissuesandstrategiesforthoseNetworkdatatypicallyconsistofasetofNnodesandarelationaltieyijmeasuredoneachorderedpairofnodes.Thisframeworkhasmanyapplicationsinsocialnetworkliterature.Themostsimplestsituationiswhenyijisadichotomousvariableindicatingthepresenseorabsenseofsomeconnectionbetweennodes(inourcase,iandj.ThedataareoftenrepresentedbyanNNsocial-interactionmatrixortheadjacencymatrixW.Inourproposedmodel,theadjacencystructureofthenetworkwithNisnotobservedbutitdependsonthelatentpositionsZ=(z1;z2;:::;zN).FollowingBhattacharjeeetal.(2014),weapplyamodelofalleveragewhereacananticipatepartofthevariationinitssalesturnoverandthereactionofcoststothesesinsales.Thisisbecausethereisanequilibriummarginandanerrorcorrectionmodelthatcapturespartialadjustmenttothisequilibrium.Weestimatethispanelerrorcorrectionmodelandextractresidualswhichinturncapturethereactiontoanunanticipatedchangeinsales.WhatremainsintheerroraftersystematichavebeenremovedaretheofinlinkagesthatbringpositiveornegativeriskmanagementexternalitiestotheThen,inthesecondstep,weusethecovariancestructureoftheseerrorsbetweentoimplementthelatentspacealgorithm.WeimplementourmodeltoestimatethenetworkstructurebetweentheUSautoindustrywhereourdataconstitutethecostsandsalesturnoverforFord,GMandChrysler,togetherwith21suppliers.Theobjectiveistoestimatetheoperationallinkagesbetweenthethreeautomobilemanufacturersandtheir48mainindependentsuppliers.Inct,wegobeyondtheregressionmodellingapproachofRamcharran(2001)toinferoninteractionsbetweenusingtheirlatentpositionsintwodimentionalEuclideanspace.TheresultsprovideexcitingnewevidencesoninnetworksthatcanthenbeinterpretedbasedontheoperatingorganizationalandgovernancestructuresoftheseTheUSautomobileindustryislargeandconsistsofhundredsofasmallnuberofwhichareauto-manufacturers,andthevastmajorityauto-ancilliariesthatsupplyvariouscomponents.Ourempiricalobjectiveistoanalyzethestructureandinteractionbetweenintheindustryfocusingonthelatentinnetwork.FollowingRamcharran(2001),wefocusattentionon3majormanufacturers-Chrysler,FordandGeneralMotors,togetherwiththetop21suppliersthatwerelistedmostfrequentlyinthevariousissuesofWard'sAutomotiveYearbook.Annualdataonsalesturnoverandcosts(ofsales)arecollectedfortheperiodof1950through2013fromtheCompustatdatabase.Theseconstitutethebasicdataforourempiricalwork.Inaddition,wealsouseinformationfromBloombergSPLCdatabaseonthesupplychainnetworkforFord.Economiclinksofmanufacturestotheirsuppliersandcustomersconstitutesourbaselinecharacterizationofinnetworks.Inlinkagestheactionsofsuppliersandcustomersofindistress(Hertzeletal.,2008;Wang,2012).Supplierscanimposecostsbyfailingsupplytradecredit,backingawayfromenteringintolongtermcontracts,delayingshipments,sourcingnewcustomersorshiftingsalesawayfromthedistressedandexistingcustomers.Likewise,inlinksthroughcorporategovernancechannels(forexample,directornetworks)canenhanceorreducecreditconstraintson(Ren-neboogandZhao,2011,2014).Manyimportantaspectsoftheautoindustrymarketcanbebythecorporatepoliciesbythewhichcouldbedrivenbythelatentlinkages49betweentheFirmsdecisionscanbeadirectconsequencebyanegativeorpositivelinkstheyhavewiththeotherInterlinkagescanalsothesupplierstoswitchtotcustomersandalsocanhaveatstockpricewhenindustryrivalshaveapositivelinkwiththesamecustomer.Finally,innetworkscanberelatedtodividendpolicies(Wang,2012)andrelationshipspinvestment(WangandWang,2012).4.1.1ErrorCorrectionModelWehavethecostofgoodssolddataYitobservedfori=1;2;:::;Ncompaniesandt=1;2;:::;Kyears.Xitisthesalesturnoverforthecompanyifort=1;2;:::;Kyears.Tohandlethenon-stationarityofthedata,considerthepanelerrorcorrectionmodel,Yit=i+iXit+(1i)(Yi;t1iXi;t1)+it(4.1)te=we0t+te(4.2)whereW=(we01;we02;:::;we0N)0isasymmetricadjacencymatrixwithwij=1ifnodeiandjhasedgebetweenthemand0otherwise,withwii=0for1i;jNand(IW)beingsingular.Theparameter(1i)determinesthespeedatwhichthesystemcorrectsbacktotheequilibriumrelationshipYi;t1iXi;t1afterasuddenshock.Weassumetheerrorsitareiidacrosstime.Underthenon-singularityassumptionof(IW)wehave,E(ee0)=(IW)1IW)10whereE(0)=diag(˙12;˙22;:::;˙N2).50Underthespatialerrormodeldescribedby(4.1)and(4.2),thepopulationspatialau-tocovariancematrixE(0)isunknownandpositivelywithprobabilityone.Thereexistsaconsistentestimator,^ofthepopulationspatialautocovariancematrixE(0).Thus,BhattacharjeeandJensen-Butler(2013)hasmadeanestimationmethodfortheunderlyingregressionmodel.Basedontheresidualfromtheseestimates,aconsistentesti-matorforthespatialauto-covariancematrixisobtained.Thisestimatoristhenusedtoestimatethespatialweightmatrix.Theyshowedthatunderthepreviousassumptions,thematrixV=(IW)0_ 1˙1;1˙2;:::;1˙N!isconsistentlyestimateduptoanorthogonaltransformationby^1=2=^E^1=2^E0where^Eand^containtheeigenvectorsandtheeigenvaluesrespectively,oftheestimatedspatialautocovariancematrix^Aftergettinganestimateofthespatialweightmatrix,weuseittogenerateBootstrapsamplesforthespatialweightmatrixtoperformthefollowingmultipletestingproblem:H0ij:jwijj=0vsH1ij:jwijj=181i<jN:(4.3)WeusethisasaninitialestimateofWforimplementingthelatentspacemodel.514.1.2LatentSpaceModelInthesecondstage,toassignalatentpositionforeachcompanyintheEuclideanspace,wetakeaconditionalindependenceapproachtomodelingbyassumingthepresenceorabsenceofatiebetweentwonodesisindependentofallotherties,giventheunobservedpositionsinthelatentspaceofthetwonodes.P(WjZ1;1;1;1)=Yi;jP(wi;jjz1i;z1j;1;1;1)HereZistheunknownlatentpositions.Wealogisticregressionmodelasbelow,logodds(wi;j=1jz1i;z1j;1;1;1)=11f(ie;je)1f(z1i;z1j)f(zi;zj)canbereplacedbyanyarbitrarysetofdistancesdi;jsatisfyingthetriangleinequality.Ingeneral,weprefertomodelthedi;j'sasdistancesinsomelow-dimensionalEuclideanspaceforreasonsofparsimonyandeaseofmodelinterpretability.Thelatentpositionmodelisinherentlyreciprocalandtransitive.Ifi!jandj!kthendijanddjkarenottoolarge,whichimpliestheeventj!i(reciprocity)andi!k(transitivity).Herewesetourdistancemodelas,logodds(jwi;jj=1jz1i;z1j;1;1;1)=11jjiejejj1jz1iz1jj(4.4)Here1istheadditiveconstant.Themodelimpliesthatreducingthelatentdistance52betweennodeiandjwillincreasethelogoddsofnodeiandjtobeconnected.Similarly,reducingtheEuclideandistancebetweenthemodelerrorsofnodeiandjwillincreasethelogoddsofnodeiandjtobeconnected.1and1arethescalingfactorsforthecorrespondingdistancemeasures.Toidentifythepositiveandthenegativelinkagesamongthetlinks,wesetupanestedmodelwhereweassumehavingapositiveandnegativeconnectionbetweentwonodesisindependentofallotherties,giventhelatentpositionsofthetwonodesandontheofthelinkages.Wecanwrite,logodds(wi;j=1jz2i;z2j;2;2;2;jwi;jj=1)=22jjiejejj2jz2iz2jj(4.5)4.2EstimationDistancesbetweenasetofpointsinEuclideanspaceareinvariantunderrotation,andtransition.Hence,annumberoflatentpositionsZgivethesamelog-likelihood.Sp,logP(WjZ;)=logP(WjZ;)foranyZthatisequivalenttoZundertheoperationsofrotation,ortransition.Thiscreatesanidenyissueofforthelatentpositions.Ourmodelinvolvesaone-dimensionalEuclideanspace,butsinceeachdimensionisbeingidentseparatelythroughtwotmodels,weneedtotwozi'stoovercometherotation,ortransitionissueforthelatentpositions.Weassume,zi1,...,zin˘i:i:dN(0;˙zi)8i=1;253Weformulatemutuallyindependentpriorsfor1,2,1,2,1,2,˙z12,˙z22as,ˇ(i)=1p2ˇei2˙2i1<i<18i=1;2ˇ(i)=121p2ˇei2˙2i0i<18i=1;2ˇ(i)=121p2ˇei2˙2i0i<18i=1;2ˇ(˙zi2)=21212)e12˙zi2˙zi12˙zi2>08i=1;2ThedetailedGibbsstepsfortheposteriorcomputationsaregiveninsection4.5.4.3DataAnalysisWeconsiderCOMPUSTATdataforpast64yearsstartingfrom1950to2013.ThedataconsistsofCostofGoodsSoldandSalesTurnoverforthreemajorU.S.-basedautomanu-facturers:(1)GM,(2)Ford,and(3)Chryslerandtheir20majorsuppliers.Wedividethetwovariableswiththecorrespondingconsumerpriceindextoremovethescalefactorsofdollarvaluesovertheyears.Wethenimplementtheerrorcorrectionmodel(4.1)andextractthemodelerrorsandusetheerrorstocalculatethecorrelationmatrix(showninFigure4.1),andthenweusethecorrelationmatrixtocalculateabootstrapsamplefromW.WeneedtogetthebootstrapsincewehavetotestforeachelementoftheWmatrix.Weassign0towijifwefailtorejecttheijthtestand1otherwise.WeusethisestimatedWastheadjacencystructurebetweenthecompaniesforourlatentspace54model.Fortheissuesofrotation,andtransition,wekeepthelatentpositionofGMandFordtobetosomevalues.WeRuntheGibbsupdatefortheparametervaluesfor3,000timeswithaburn-inperiodof2,000.Weachievedanoverallniceconvergenceataround1,000iterations.Figure4.2showstheprobabilityofhavingnoconnection(showninblue),apositiveconnection(showningreen),andanegativeconnection(showninred)bythethreemajorautomanufacturersandtheirkeysuppliers.Herealinkbetweencompanyiandjthebusinessimpactofithcompanyandjthcompanyeitherwayswithrespecttotheautoindustrymarket.ItisevidentfromFigure4.2thatthenetworkthatthethreebigman-ufacturershavewiththeirsuppliersismoderatelydensewithahighprobabilityofbeingconnectedwiththeirsuppliers.Also,asimilaritybetweenChryslerandFordcanbeseenwithrespecttotheirconnectedness.ThisisevidentinFigure4.3,whichprovidesthelatentpositionsofthecompanies,anditcanbeseenthatFordandChryslerareclosetoeachotherinthelatentspace.TheprobabilitiesforGMimplythat,althoughhavingthesamesetofsuppliersasFordorChrysler,GMreliesonaselectnumberofsupplierswithrespecttotheautoindustrymarketwithaverylargeprobabilityofbeingdisconnectedfromafewofthesesuppliers.Acarefulinspectionofthelatentpositionsofthesuppliersandthreemajorcompaniesrevealsanimportantaspectoftheautoindustrymarket.IfwelookatthepositionofFordandChrysler,theyaresittinginthemiddleoftheirsuppliersandmoderatelydependonalmostallofthemwithrespecttotheirbusiness.Again,thepositionofGMinthelatentspacerevealsthatGMhasaspsubsetofsuppliersthatitreliesonforitsbusiness.5556Figure4.1:CorrelationMatrix57Figure4.2:ProbabilityoflinkagesforChrysler,Ford,andGMwiththeirSuppliers58Figure4.3:Positionofthe24companiesinthelatentspace4.4ConclusionThecurrentliteratureincorporateehighlightstheimportanceofinnetworksinancialmanagementofSuchnetworkscanbebasedonsupplychains,butequallymaydirectornetworks,jointventuresordemandsidelinkages.Hence,analysisofnetworkstructurerequiresabroadperspectivethatallowseachofthesepotentialdriversofnetworkinteractionstoactandinteract.WedevelopnewBayesianmethodologytoanalyzelatentinnetworks.AppliedtodataontheUSautoindustry,theestimatedinnetworksastrongofthesupplychain,butalsogovernancelinksbetweenImportantly,theestimatednetworksalsopointtobothpositive(complementary)andnegative(competitive)interactuionsbetweentheAlotofinterestingquestionsemerge,relatingtotheimpactofinnetworksoncorporateissues.4.5SomePosteriorCalculationsWesetinitialvaluesfortheparametersas01;02;01;02;01;02;˙0z12;˙0z22andweupdatetheparametersaccordingtothefollowingGibbssteps.594.5.1(s+1)thGibbsStepforupdating(1;1;1):Thefullconditionaldistributionof1;1;1isgivenby:P(1;1;1j˙2z1;Z1;W;)/ Yi>je(11jjieje1jz1iz1jj)jwijj1+e11jjieje1jz1iz1jj!e(212˙21+212˙21+212˙21)=K111j˙2z1;Z1;Wwhichisnotaclosedformexpressionofanydistribution.HenceweneedtoperformMetropolis-Hastingswithasymmetricproposaldistributionfor1;1;1.Ther-thstepfortheMetropolis-Hastingsalgorithmisgivenby,1.Generate1srfromN(1sr1;˙met),1srfromN(1sr1;˙met)and1srfromN(1sr1;˙met).2.Calculate:u=Kr1sr1sr1sj˙2r1z1s;Z1s;WKr11sr11sr11sj˙2r1z1s;Z1s;W3.Setr1s=r1s,r1s=r1s,r1s=r1swithprobabilityuotherwisecontinuewiththevalueofstepr1withprobability1u.4.Repeattheabovestepsnmettimes.Hence,wegetnmetsimulatedsamplesfromthedistributionof1;1;1j˙2z1s;Z1s;60W;.Wecanusethissimulateddistributionupdate1s;1s;1sto1(s+1);1(s+1);1(s+1).4.5.2(s+1)thGibbsStepforupdating(2;2;2):Thefullconditionaldistributionof2;2;2isgivenby:P(2;2;2j˙2z2;Z2;fwij:jwijj=1g;)/ Yi>je(22jjieje2jz2iz2jj)(1+wij)=21+e22jjieje2jz2iz2jj!e(222˙22+222˙22+222˙22)=K222j˙2z2;Z2;Wwhichisnotaclosedformexpressionofanydistribution.HenceweneedtoperformMetropolis-Hastingswithasymmetricproposaldistributionfor2;2;2.Ther-thstepfortheMetropolis-Hastingsalgorithmisgivenby,1.Generater2sfromN(r12s;˙met),rs2fromN(r12s;˙met)andr2sfromN(r12s;˙met).2.Calculate:u=Kk2sr2sr2sj˙2r1z2s;Z2;WKr12sr12sr12sj˙2r1z2s;Z2s;W3.Setr2s=r2s,r2s=r2s,r2s=r2swithprobabilityuotherwisecontinuewiththevalueofstepr1withprobability1u.614.Repeattheabovestepsnmettimes.Hence,wegetnmetsimulatedsamplesfromthedistributionof2;2;2j˙2z2s;Z2s;W;.Wecanusethissimulateddistributiontoupdate2s;2s;2sto2(s+1);2(s+1);2(s+1).4.5.3(s+1)thGibbsStepforupdating˙2z1:Theposteriordistributionof˙2z1isgivenby,˙2z1jZ1˘(NXi=1z1i2+1)inv˜21+4˙2z1(s+1)canbegeneratedfromtheconditional˙2z1jZ1s4.5.4(s+1)thGibbsStepforupdating˙2z2:Theposteriordistributionof˙2z2isgivenby,˙2z2jZ2˘(NXi=1z2i2+1)inv˜21+4˙2z2(s+1)canbegeneratedfromtheconditional˙2z2jZ2s624.5.5(s+1)thGibbsStepforupdatingZ1;Z2:Thefullconditionaldistributionofz1iisgivenby:P(z1ij1;1;1;˙z12;W;)/ Yj6=ie(11jjieje1jz1iz1jj)jwijj1+e11jjieje1jz1iz1jj!ez2i˙2z1=Kz1ij111;˙z12;W)whichisnotaclosedformexpressionofanydistribution.HenceweneedtoperformMetropolis-Hastingswithasymmetricproposaldistributionforz1i.Ther-thstepfortheMetropolis-Hastingsalgorithmisgivenby,1.Generatezr1isfromN(zr11is;˙2met).2.Calculate:u=Kzr1isj1(s+1)1(s+1)1(s+1);˙2z1(s+1);W)Kzr11isj1(s+1)1(s+1)1(s+1);˙2z1(s+1);W)3.Setzr1is=zr1iswithprobabilityuotherwisecontinuewiththevalueofstepr1withprobability1u.4.Repeattheabovestepsnmettimes.Hence,wegetnmetsimulatedsamplesfromthedistributionof,z1ij1(s+1);1(s+1);1(s+1);˙2z1(s+1);W;63Wecanusethissimulateddistributionupdatez1istoz1i(s+1).Wecanupdatez2issimilarly.64Chapter5RegionWiseVariableSelectionwithBayesianGroupLASSO655.1Region-wiseVariableSelectionInvariousspatial-economicanalyses,theproblemistoselectvariablesthatarelocatedspatially.SometimestheinterestliesonestimatingtherelevanceofindividualvariableovertlocationswhereaBayesianframeworkcanbereallyeprovidedthefactthatwehavetheideaoftheadjacencystructureorratherthenetworkstructurebetweenthelocation.Hence,themainideaistoincorporatetheadjacencystructurebetweenthenodes(locations)sothatwecanincorporatethisfactthatrelevanceofavariableinacertainlocationisonitsstatusontheadjacentlocations.Forexample,ifwecanassumethatannualsnowfallrateisakeyfactorondecidingtheautoinsurancepremiumratesinMichigan,thenitwouldalsoberelevantorwouldhavesomeimpactondecidingtheautoinsuranceratesinOhioorIndiana.Itisveryimportanttoselectvariablesthatarerelevanttoeachlocationandthetheproblembecomesabi-levelselectionwherewenotonlyselectthevariableoverallbutwealsoinspectwhetheritisttoeachlocation.Spatialorcross-sectionaldependencyisacommonfeatureinpresenteconometricappli-cations.Tocapturespatialdependency,apopularapproachistointroduceaspatialweightmatrixWcontainingthespatialweightsoveritselements(GiacominiandGranger,2004).Therearevariouswaystoretrievethespatialweights:fromgeographicdistances,notionsofeconomicdistances(Conley,1999;Pesaran,2004;Hollyetal.,2010),socio-culturaldistances(ConleyandTopa,2002;BhattacharjeeandJensen-Butler,2005)etc.Analternativeandincreasinglypopularapproachistoestimatespatialpanelregressionmodelsundermulti-factorerrorstructures.Factormodelsarepotentiallypowerfulinthesensethattheydonotrequirestrongandunvassumptionsonthenatureofspatialdependence.Inalocationvariableselectionproblem,itisimportanttoconsiderthevariableselection66proceduretobedependentspatially.Considerthefollowingmodelforeachlocationi2f1;2;:::;Ng:yei=Xiei+ei(5.1)whereyeiisa(R1)vectorofresponsevariables,Xiisthe(Rp)designmatrixcontainingpvariables,eiisthe(p1)cotvectorandei˘N(0;˙2I).SmithandKohn(1996),SmithandFahrmeir(2015)haveconsideredthevariableselectionproblembyattachinganindicatorvectorei=(i1;i2;:::;ip)0correspondingtoeiwherewesetij=0ifij=0andsetij6=0ifij=1.Theabovemodelcanalternativelybeexpressedas:yei=Xi(ie)ei(ie)+eiToundertaketheposteriorcomputation,Kohnetal.(2001)haveconsideredapropercon-ditionalpriorbysettingitproportionaltothelikelihood:ei(ie)jyei;˙2;ei˘N(^ei(ei;R˙2(Xi(ie)0Xi(ie))1)where,^ei(ei)=(Xi(ie)0Xi(ie))1Xi(ie)0yeiIfweassume,P(˙2jei)/1˙267thenbySmithandKohn((1996),wecanshowthatP(eijyei)/P(yeijei)P(ei)Weneedtosetapriordistributiononietoestimatetheabovemodel.Afterwedecideonthepriorknowledge,wecanruntheMCMCsamplingschemes(SmithandKohn,1996)andtheMetropilos-Hastingstechniquetoouttheposteriorestimates.SmithandFahrmeir(2015)haveconsideredthefactthatthevariableselectionprocedureshouldhaveaspatialimpact,andtheyaddressedthisissuebyintroducingtheIsingpriortechniquewherefore(j)=(1j;2j;:::;Nj),theyhaveconsideredthepriorknowledgeonas,P()=Qpj=1P(e(j)),where,P(e(j))/expnNXi=1ijij+Xi˘kikjwikI(ij=kj)oHere,I()isanindicatorfunction,wijisthepre-specweightduetoadjacencybe-tweenlocationiandj.ThetermPi˘kikjwikI(ij=kj)evaluatestheinteractionbetweentheoftheelementse(j)forallpairwiseneighboringsites.AcriticalissueofthistechniqueistospecifytheexternalPNi=1ijijwheretheparameterijisapriori.Theusualtechniqueistouseapre-estimateofijwhichdependsonthetypeoftheproblem,andtheposteriorestimatesaremuchsensitiveoverthechoiceofij's(seeSmithandFahrmeir,2015).Thispaperisfocusedonproposinganalternativetechniqueofthelocation-wisevariableselectionthatnotonlyinvolvestheimpactoftheadjacencystructureonthevariableselectionbutalsoovercomestheambiguityofpre-spnofthehyper-parameters.ThevariableselectionprocessiscarriedoutbyimplementingtheBayesianGroupLassotechniquewhereweputanemphasisonasimilarbi-levelvariableselectionapproachthatincorporatesacross-sectionaldependencyamongthecotsoverthevariouslocations.Weusethe68spikeandslabprioronthegrouplevelandwithinthegrouplevelwhereagroupmeansthemodelcovariatesoverseverallocations.Ourpurposeistoselectwithingrouplevel,whilekeepinginmindthattherelevanceofacovariateinacertainlocationdependsonitsrelevanceontheotherlocations.Weintroduceaconditionalautoregressivestructureamongthemodelcovariatestoincorporatesthisfact.Themedianthresholdingtechnique(XuandGhosh,2015)facilitateshavingexactzeroestimatesofthenon-relevantvariablesforthecorrespondinglocationssinceithasaslightlybettermodelselectionaccuracyaswellasabetterpredictionperformancethanthetraditionalLASSOmethod.ThekeyfactorofthetechniqueintroducedbyXuandGhosh(2015)istousetheposteriormedianestimatorthatderivesthatunderanorthogonaldesignandworksasasoftthresholdingestimator,andthemedianthresholdingisconsistentinmodelselectionandhasanoptimalasymptoticestimationrate.5.2Region-wiseVariableSelectionwithBayesianGroupLASSOSupposeweobserveresponsesyeironi=1;2;:::;Nlocationsandonr=1;2;:::;Rindepen-dentreplications.Wesetupthefollowinglinearregressionmodelas:yir=Xeirei+iri=1;2;:::;Nr=1;2;:::;R:(5.2)whereXeirisap1vectorofpredictorsfortheithlocationandforrthreplicate.ei=(i1;i2;:::;ip)0isavectorofmodelcotscorrespondingtotheithlocation.Weassumethespatialerrorsareindependentlyandidenticallydistributed(i:i:d)over69timeandahomoscedasticstructureacrosslocationsasE(ere0r)=˙2In.Nowtodividethesetofcotsintotgroup,wecanrewriteourmodel(5.2)as,yer=pXg=1Xegreg+er;r=1;2;:::;R(5.3)whereye0r=(y1r;y2r;:::;yNr)istheresponsevectoratreplicateroverNlocations,Xegr=(x1gr;x2gr;:::;xngr)0andeg=(1g;2g;:::;Ng)08g=1;2;:::;pand8r=1;2;:::;R.Thepurposeofthispaperistoperformavariableselectionwherespatialdependenceisdrivenbyobservedstructuralinteractions.Sinceourmodelinvolvesavariableselectionoverasetofcovariatesinmultiplelocations,weproposethegroupLASSOmethodthatgeneralizestheLASSOinordertoselectthegroupedvariablesforaccuratepredictionofre-gression.ThegroupLASSOestimatorcanbeobtainedbysolvingthefollowingminimizationproblem,min RXr=1(jjyerpXg=1Xegregjj+1jjejj1+2pXg=1jjegjj2)!(5.4)TheBayesianformulationprovidesshrinkageofthecotsinthegroupandwithinthegroup'slevel.ButtheclassicalgroupLASSOtechniquedoesnotprovideexactzeroestimatesforthecotsthatarenotrelevant.Thus,weintroducesparsityatthegroupandwithinthegrouplevelbyassumingspikeandslabpriorforthemodelcovariatethatbringssparsityinthemodelcots.JohnstoneandSilverman(2004)showedthatposteriormedianwitharandomthresholdingestimatorprovidesgoodestimatealongwithsomedesirablepropertiesunderspikeandslabpriorsfornormalmeans.Weusetheposteriormedianinsteadoftheposteriormeanasourposteriorestimatesofthemodelcots.705.2.1SpikeandSlabPriorforModelCotsWeproposethefollowingBayesianhierarchicalmodelthatwerefertoasBayesianSparsegroupLASSOtoenableshrinkagebothatgrouplevelandwithinagroup.yerje1;e2;:::;eg;˙˘N(pXg=1Xegreg;˙2IN)(5.5)egj˝eg;˙˘N(0;˙2Vg)g=1;2;:::;p(5.6)HereV1=2g=diagf˝g1;:::;˝gNg,˝gj0,g=1;2;:::;p;j=1;2;:::;N.XuandGhosh(2015)haveintroducedasparsegroupLASSOmodelingwheretheyhaverepresentedthemodelcotsasascaledversionofasparsediagonalmatrixthathelpstoselectvariableswithingrouplevelalongwiththegroupselection.Tointroducesparsityinthemodelandtoselectrelevantvariablesatthegroupandwithinthegrouplevel,wereparametrizethecotvectorsas,eg=V1=2gbeg(5.7)Herebeg,whennonzerohasa0meananddispersionmatrixIN.ThediagonalelementsofV1=2gcontrolthemagnitudeofelementsofeg.AhierarchicalBayesianmodelingusingSpikeandSlabpriorhavebeenintroducedbyInswaranandRao(2005)inwhichtheyhaveconsideredanprobabilitystructureat0thathelpsbringingsparsityinthemodel.OnekeyadvantageoftheSpikeSlabmodeliswecanshowthatthepriorvariancecanbedependentonthesamplesizeandhenceanappropriateshrinkagelevelcanbeachievedandastrongselectionconsistencycanbeshown(Narisetty,2014).Tohaveasparseestimateofthemodelcots,wethefollowingmultivariate71spikeandslabpriortoselectingvariablesatgrouplevel:begiid˘(1ˇ0)Nn(0;In)+ˇ00(beg);g=1;2;:::;p(5.8)Notethatwhen˝gj=0,gjisdroppedoutofthemodelevenwhenbgj6=0,whichmeans˝egdrivesawithingrouplevelselectionforaselectedgroupofeg.NowselectingtheelementsofegmeansselectingthegthcovariateoverNtlocations.Here,weusethefactthatimportanceofthegthvariableontheithlocationshoulddependontherelevanceofthegthvariableontheadjacentlocation.ConsideraspatialadjacencystructureamongtheNspatiallocationsthatcanberepresentedthroughaknownspatialweightmatrixW=((wij));i=1;2;:::;Nandi=1;2;:::;N.Here,wijistheweightcorrespondingtothestrengthofadjacencybetweenlocationiandj.Forthepriorselectionofthewithingroup,weassumeaspatialcross-sectionaldependenceisconvolutedwithinthecovariatestructureofthemodel.Wewouldusethisfactlaterontothepriorstructureofthemodel'scots.Toperformagrouplassovariableselectionintheabovemodel,weneedtoconsideraproperpriorforthebetathatconsidersthespatialrelationshipsamongthecovariates.WethereforeassumedaConditionalAutoregressivePriorfortheprioron˝as,˝gjj˝gi:i6=j˘(1ˇ1)N+(NXi=1;i6=jwijwj+˝gi;s2wi+)+ˇ0(˝gj);g=1;2;:::;p;j=1;2;:::;N(5.9)Herewj+=PNi=1wij.whereN+denotesafoldednormaltowardsthepositivesideoftherealline.Remarks.Inchapter4,wehaveconsideredaSpatialErrorCorrectionModelwherethe72adjacencystructureWisunobserved.Thechaptershowsanestimationtechniqueofthelinkprobabilitiesinatwo-stepprocedurewhereanerrorcorrectionmodelisconsideredtohaveapre-estimateoftheWmartixandthenthelatentmetworkmodelisincorporatedwhereaBayesianestimateofaconnection(positiveornegative)ornoconnectionisobtained.TheabovemodelingtechniquecanalsobecarriedoutwithanunobservedstructureoftheWandthenatwo-stepvariableselectiontechniquebyestimatingWandusingthatWasobservedinourmodeltoconsideritforthevariableselectionpurpose.WewillbefollowingasimilartechniqueinthedataanalysispartofthispaperwherewewillconsidertheWtobepre-estimated.Insteadofspecifyingvaluesforhyperparameters,weset,˙2˘IG(;);=0:1;=0:1(5.10)ˇ0˘Beta(a1;a2);ˇ1˘Beta(c1;c2)(5.11)s2˘IG(1;k)(5.12)5.3HellingerConsistencyforthePosteriorDistribu-tionofInthissection,wewillshowthattheposteriordensityofgji.e.(gjjrest)=(˝gjjrest)(bgjjrest)isHellingerconsistentunderatruedensityisP0.Supposethetruevalueoftheijthmodelcoientis0ij.WewillapplytheSchwartztheoremtoshowthat73(gjjrest)isconsistentforthetruedensityunder0.Theorem(Dueto'Schwartz(1965)').LetthemodelP=nf(j;˙):2RNp;˙>0obetotallyboundedrelativetotheHellingermetricHandletyi1;yi2;:::;yiRbeiidP0=nf(j0;˙):˙>0oforsomeP02P.IfisaKullback-Leiblerprior,i.e.,forall>0P2P:P0logdPdP0<>0thentheposteriorisHellingerconsistentatP0,thatis,H(P;P0)>jyi1;yi2;:::;yiR!P0a:s0:AnequivalentformulationoftotallyboundedmodelcanbefoundinLeCam(1986)whereithasbeenshowninvolvinganunbiasedtestfortestingH0:=000vsH1:2UcforeveryneighbourhoodUof0.Moregenerally,existanceofuniformlyconsistenttestforH0:=000vsH1:2UcimplysHellingerconsistencyat000.Letusassumeˇ(K(000))>08>0,whereK(000)istheKLneighborhoodof000denotedbyf:K(000;)<gandK(;)istheKLdivergence.Lemma1.SupposeUbetheneighbourhoodaround000.If,1.000isintheKLsupportofˇ.2.Rqf(y)f000(y)dy<82Uc.3.sup2UcRsf(y)f000(y)dyRqf(y)f000(y)dy!0a:s:underthejointdistributionofyi1;yi2;:::;yiRasR!1.74then,thereexistsauniformlyconsistenttestforH0:=000vsH1:2Uc(seeVanDeGeer(1993);ChoiandRamamoorthi(2008)).TheproofofLemma1isgiveninsection5.9.Lemma2.SupposeBisanormalwithmeanBandastandarddeviationof˙B,andsupposeTisapositivenormalwithmeanTandastandarddeviationof˙T.ThenthedistributionofZ=TBisgivenbyf(Z)=eˆ21+ˆ222ˇˆ2)"0K0+(ˆ21+ˆ22)jZjZ!2K1+(ˆ41+ˆ42)jZj24!4K2+:::#where,K(Z)=12(Z2)R10eyZ24yy+1dyˆ1=B˙Bandˆ1=T˙T,r(ˆ1ˆ2Z)=1+ˆ1ˆ2Zr+1+(ˆ1ˆ2Z)2(r+2)(2)2!+:::with(r+k)(k)=(r+k)(r+k1):::(r+1).Here,K(Z)iscalledthemodBesselfunctionofsecondkind.TheproofofLemma2isgiveninsection5.9.SparsityAssumption.Ifwehaveasparsityassumptionforalargenetwork,aswellasonthemodel,thenwecanapproximatethepriormeanof˝gjcloseto0.ThiswouldbeourkeyassumptiontoshowtheHellingerconsistency.Thatmeanswewouldassumethateachnodehasaverysmallnumberofedgeswithrespecttothewholesetofnodes,andwesetmostofthe˝gj'sat0endingupwithaverysmallnumberofantcovariates.Inotherwords,ifwesetlimN!1PNi=1;i6=jwijwj+˝gi=0,thenwehavealargeN,thepriormeanforbgj=0,andpriormeanfor˝gjˇ0.ItisshownthatthepriordistributionofgjisK0(jgjj)ˇˆ2).75Lemma3.SupposeBisanormalwithmeanBandstandarddeviation˙BandsupposeTisapositivenormalwithmeanTandstandarddeviation˙T.ThenthemeanofZ=TBisgivenbyZ=(˚(T˙T)T˙T)˙TB+BT)T˙TT˙T)exp(122T˙2T)TheProofofLemma3isgiveninsection5.9.Thus,itiseasytoobservethatthepriordistributionofgjhasameanof0andsoasparsityassumptionof˝gjgivesthepriormeanof˝gjˇ0,andsowehavegj=0.AsthepriordistributionofbetaisamoBesselfunctionofsecondkind,theLemma2showsitissymmetricaround0.Proof(Theorem).AccordingtoLemma1,priordensityofgj=˝gjbgjisgivenbyˇ(ij)=l0(gj)+(1l)K0(jijj)ˇˆ2)(5.13)wherel=1(1ˇ0)(1ˇ1)andˆ2=Pi6=jwij˝gis2.Sete0ibethetruevalueofei.AccordingtoSchwartz'sTheorem,wecalculate,NYi=1fe0i;˙logQNi=1fe0i;˙QNi=1fe;i˙=1˙p2ˇexp(12NXi=1yirxe0ire0i˙)212NXi=1( yirxe0irei˙!2 yirxe0ire0i˙!2)76Supposethereexistsan1forwhichtheKLconditiondoesnotholds,i.e.,ˇ(:e12PRt=1PNi=1yirxe0ire0i˙2"RXr=1NXi=1 yirxe0irei˙!2RXr=1NXi=1 yirxe0ire0i˙!2#<2)=0Here;2=12(˙)R(p2ˇ)R=2>0=)ˇ(:RXr=1NXi=1yirxe0irei˙02yirxe0ire0i˙02o<3)=0Here;3=2exp"12RXr=1NXi=1 yirxe0ire0i˙!2#=)ˇ(:2RXr=1NXi=1yitpXj=1xijr(0ijij)RXr=1NXi=1pXj=1x2ijr(0ij2ij2)<3)=0=)ˇ(:NXi=1pXj=1h2ijRXr=1x2ijr+2p(0ijij)RXr=1yirxijri<4)=0(Where;4=3+NXi=1pXj=10ij2RXr=1x2ijr>0)=)ˇ(:NXi=1pXi=1(ij0ij)aij)<5)=0where,5=p24<0andaij=PRr=1yirxijr.Theaboveinequalityinsideˇeiei:holdssincePNi=1Ppj=12ijPRr=1x2ijr>0.Hence,ˇ(:NXi=1pXj=1ijaij>5+NXi=1pXj=10ijaij)=077=)NYi=1pYj=1ˇij(ij:ijaij>6+0ijaij)=0(5.14)where6=5=Np<0.Equation(5.14)istruesince(ijaij>6+0ijaij;8i1;j1)=)(NXi=1pXj=1ijaij>5+NXi=1pXj=10ijaij)Equation(5.14)meansthat8i2f1;2;:::;Ngandj2f1;2;:::;pgandfor>0thereexistsanaij:aij=(OR,aij=)where6=M(OR,6=M)whereMisalargepositiveintegerwithjMj>0ij(wecanassume)suchthat,ˇij(ij:ij>M+0ij)=0(5.15) OR;ˇij(ij:ij<M+0ij)=0!Now(5.15)isacontradictionsinceij˘l0(ij)+(1l)K0(jijj)ˇˆ2),0<l<1andLemma2showsthedensityofijissymmetricwithrespecttozeroandl>0.5.4PosteriorDistributionsandGibbsSamplingforGroupLASSOLetusdenoteXr=(Xe1r;Xe2r;:::;Xepr).Thejointposteriorofb=fbei:i=1;2;:::;pg,˝˝˝2=f˝2ij:i=1;2;:::;p;j=1;2;:::;Ng,78˙2,ˇ0,ˇ1;s2conditionalontheobserveddatais:P(b;˝˝˝2;˙2;ˇ0;ˇ1;s2jyer;Xr;r=1;2;:::;R)/(˙2)NR=2exp(12˙2RXr=1jjyetpXi=1XegrV1=2gbegjj22)pYg=1"(1ˇ0)(2ˇ)N=2exp(12be0gbeg)I(beg6=0e)+ˇ00(beg)#pYg=1NYj=1"(1ˇ1)2(22)1=2exp((˝gjwijwi+˝gi)22s2wi+)I(˝gj>0)+ˇ00(˝gj)#(˙2)1exp2ˇa110(1ˇ0)a21ˇc111(1ˇ0)c21k(s2)2exp(ks2)5.4.1GibbsSampler‹Theposteriordistributionofbegconditionaloneverythingelseisgivenby:begjrest˘lg0(beg)+(1lg)N(eg;g)(5.17)wherelgistheposteriorprobabilityofbgbeingequalto0egiventheotherparameters,i.e.lg=P(beg=0jrest)=ˇ0ˇ0+(1ˇ0)jgj1=2exp12˙4jj1=2g(PRr=1V1=2gXe0tg(yerXr(g)V1=2(g)b(g)))jj2279where,Xr(g)=(Xer1;:::;Xer(g1);Xer(g+1);:::;Xerp),b(g)=(be01;:::;be0g1;be0g+1;:::;be0p)0.SimilarlyV(g)=diag(V1;:::;Vg1;Vg+1;:::;Vp)matrixafterdeletingthegthrowandgthcolumn.Also,eg=1˙2gRXr=1(V1=2gXe0rg(yerXr(g)V1=2(g)b(g)))g= IN+1˙2RXr=1(V1=2gXe0rgXergV1=2g)!1‹Theconditionalposteriorof˝gjisgivenby,˝gjjrest˘qgj0(˝gj)+(1qgj)N+(ugj;v2gj);g=1;2;:::;p;j=1;2;:::;N(5.15)where,ugj=v2gj˙2RXr=1(yerXr(gj)V1=2(gj)be(gj))xrgjbgj+v2gjwi+s2Xi6=jwijwi+˝giv2gj=(wi+s2+b2gj˙2RXr=1x2rgj)1qgj=ˇ1ˇ1+2(1ˇ1)vgjpwi+sexp122gjv2gjwi+2s2(Pi6=jwijwi+˝gi)2ugjvgj)HereweXr(gj);V(gj);be(gj)similarlybyremovingthecorrespondinggjthele-ment.‹˙2jrest˘IG(NR2+;12PRr=1jjyerXrjj22+)HereXr=(Xer1;;:::;Xerp),=(e01:::;e0p)0.80‹ˇ0jrest˘beta(#(bg=0)+a1;#(bg6=0)+a2)‹ˇ1jrest˘beta(#(˝gj=0)+c1;#(˝gj6=0)+c2)‹s2jrest˘IG(1+12#(˝gj=0);t+12Ppg=1PNj=1˝gjPi6=jwijwi+˝gi2).WeconsiderourposteriorvaluesofthemodelcotsovertheGibbssamplertobe^gj=(gjjrest)(˝gjjrest).Tohaveourposteriorestimateofthegj,weusethesameapproachfollowedbyXuandGhosh(2015)whohaveusedtheposteriormedianinsteadoftheposteriormean.TheyhaveshowninapaperthattheposteriormedianworksasarandomthresholdingestimatorthatsattheoraclepropertywithafasterconvergencethanthegeneralgroupLASSOestimatorunderanorthogonaldesign.5.5VariableSelectionforTemporalDataVariableselectionforspatiallydependentdatacanbecarriedoutalongthemethodwediscussedinthelastfewsections.Butwehaveignoredthefactthatthetemporaldependencemightalsointhesenseofhavingdependencyovertheresponsesthatarecloserwithrespecttotime.AssumewehaveresponsevectorforNlocationsyetovertimepointst=1;2;:::;T.Considerthespatial-temporalregressionmodelas,yet=yet1+pXg=1Xegteg+et(5.13)where,=diag(˚1;˚2;:::;˚N)81Themodelweconsideredunderequation(5.13)isnothingbuttheAR(1)process.Station-arityoftheAR(1)processrequirestheassumption:j˚ij1,81iN.ThereasonthatwehavechosenAR(1)overhigher-orderautoregressiveprocessesisthatwewanttoavoidtheabstractrestrictionsontheautoregressivemodelparametersduetothestationarityoftheprocess.AR(1)processallowsthetemporaldependencetodecreasegraduallyasthetimelagincreases.Weconsidertheautoregressivecomponentparameters˚itobeindependentlyuniformbetween1and1,i.e.,ˇ(˚i)=NYi=1I(1˚i1)5.5.1PosteriorDistributionof˚iWecanwritetheposteriorprobabilityof˚i;i=1(1)Njrestas,P(˚i;i=1(1)Njrest)/expn12˙2TXt=2jjyetyet1pXg=1XegtV1=2gbegjj2oNYi=1U(1;1)/expn12˙2TXt=2(yetpXg=1XegtV1=2gbeg)0(yetpXg=1XegtV1=2gbeg)2NXi=1˚imeityt1i+NXi=1˚2iy2t1ioHere,meitistheithrowof(yetPpg=1XegtV1=2gbeg).Hence,82P(˚i;i=1(1)Njrest)/expn12˙2NXi=12˚iTXt=2meityet1i+˚2iTXt=2ye2t1io/expnPTt=2ye2t1i˙2NXi=1˚iPTt=2meityet1iPTt=2ye2t1i2oHence,˚ijrest˘NPTt=2meityet1iPTt=2ye2t1i;˙2PTt=2ye2t1i81iN5.5.2GibbsSamplerGibbs'samplingstepsforthespatio-temporalmodelwouldbesimilartothesituationforthespatialmodeling.WewouldfollowthesameupdateprocedurealongwiththeupdateofautoregressivecomponentmatrixWewouldreplaceyetwithyetyet1fortheGibbsSamplerstepsinsection(5.4.1).5.6SimulationStudy5.6.1ASampleSimulationwithInthissetting,wepreselectsomevaluesforijsandwecomparetheBGL-SS-CARwiththesimpleBGL-SSandthevariableselectionwithISINGpriorbasedontheRMSEandtheTPR/FPRvalues.Wesetp=5,T=10,andN=7.Thecomparisonisdonebasedononlyonesimulation.Wewillincreasethenumberofsimulationsinthesubsequentsectiontohaveabetterviewofthepredictionerrormeasurement.Itisevidentfromtable5.1andtable5.2thatwhenaCARstructureisbeingconsidered83Table5.1:RMSE,TPRandFPRcomparisonforBGL-SS,IsingandBGL-SS-CARmodelMethodsBGL-SSISINGBGL-SS-CARRMSE0.8681.121.06TPR10.830.81FPR0.3680.260.21Table5.2:BGL-SSandBGL-SS-CARestimatesfor'sMethodsTrueBGL-SSISINGBGL-SS-CAR1100.150.300.38212.51.631.111.9231-2.25-0.70-1.06-0.844100.19005132.301.330.816100.330.100.3671-1-1.570.3601200.17002200003200004200005200006200.990.3707200001320.161.280.342322.892.231.5033-3-0.96-2.58-2.434331.311.781.01531.500.960.716310.160073-2-1.790.8801400002400003400004400005400006400007400001500.190.640.382521.050.880.2535-1-2.12-1.92-1.3545-3-2.081.680.6555000.470.23652001.8275-1.5-1.750-2.6584withinthemodelcots,itisclearlyout-performingtheregularSparseGroupLASSOandISINGmodelingtechniquebothintermsofRMSEaswellasintermsofFPRorTPR.5.6.2Scenario1:N=7,p=5andT=10Inthescenario,wetakeasimulateddataover7spatiallocationwithapre-spadjacencystructure(W).Wetake5variablesforeachofthelocations,andwetakethedataover10timepoints.Weset˙intothreespvalues:˙=0:5,˙=1and˙=3.WerantheGibbsiterationsfor10,000timesandwetooktheburn-inperiodtobe8,000.Wereplicatedthesimulation20timestohaveabettermeasureoferrors.Figure5.1:Gibbsiterationsof'sforthescenariounderBGL-SS-CARwhen˙=0:5Figure5.2:PosteriorDistributionof˙2forthescenariounderBGL-SS-CARwhenthetrue˙2is0:25855.6.3Scenario2:N=14,p=15andT=50Inthesecondscenario,wetakeasimulateddataover14spatiallocationswithapre-spadjacencystructure(W).Wetake10variablesforeachofthelocations,andwetakethedataover50timepoints.Weset˙tothreespvalues:˙=0:5,˙=1and˙=3.WerantheGibbsiterations10,000times,andwetooktheburningperiodtobe8,000.Wereplicatethesimulation20times.Figure5.3:PosteriorDistributionof˙2forthesecondscenariounderBGL-SS-CARwhenthetrue˙2is0:25Intable5.3,wearecomparingsimpleBayesianSparseGroupLassoandISINGmodelvstheBayesianSparseGroupLassowithaCARstructure.ThecomparisonisbeingdoneusingtheFalsePositiveRatesandtheTruePositiveRatesasthetwomethodsbasedonthetwoscenariosweconsideredbefore.ItcanbeobservedfromthetablesaboveisBayesiansparsegroupLASSOtechniqueismostlyoutperformingthesimplesparsegroupLASSOaswellastheISINGmodelintermsoftheRMSEandTPR.Whichmeanswhenaspatialdataisconsideredandwhenitisknownorexpectedforthevariablestohaveadependencystructureconvolutedinthejointdistributionofthecovariates,wecanexpectthataCARstructureancatchtherelevantvariablemore86Table5.3:TableforRMSEandTrue/FalsePositiveRatesScenario1Scenario2MethodsBGL-SSISINGBGL-SS-CARBGL-SSISINGBGL-SS-CAR˙=0:5RMSE1.020.920.800.420.310.28TPR0.560.700.620.710.780.77FPR0.080.060.010.130.150.04˙=1RMSE0.700.720.600.710.760.51TPR0.610.660.710.680.680.72FPR0.200.100.030.230.070.02˙=2RMSE1.030.750.780.951.080.92TPR0.690.610.730.720.820.88FPR0.180.150.030.310.100.07tlythantheothercompetativemethodsandisbetterintermsofloweringthemodelerrors.AnadvantageofusingBGL-SS-CARoverISINGmodelintermsofcomputationsisthatunliketheISINGmodelitisfreefromtheambiguityofprespecifyingvaluesforsomehyperparameter.Also,abi-levelshrinkagebringsmorecontrolonthesparsityofthemodelthroughthetwoactingvariabilities,onebetweengrouplevelsandanotherwithingrouplevels.ItisalsoveryinterestingtoobservethatincorporatingtheCARstructureinthepriorsetupoftheBayesiansparsegroupLASSOtechniqueresultsinaverytvariableselectionintermsoftheFalsepositiverates.Wecanseefromtable5.3thatFPRisclosetozeroinallthesimulationscenariosandalsoverylowcomparedtotheothertwocompetativemethods.ThismeansbringingCARstructureinthemodelingscenarioforaspatiallyrelateddataallowsthemodeltobeverytinidentifyingthecovariateswhicharenotrelevantforagivenlocation.875.7DataAnalysisInthissection,weconsidercompustatdataovertheU.S.autoindustrymarket.Thedataconsistof20U.S.automanufacturersandthesuppliers,includingthree3U.S.automanu-facturinggiantsGM,Ford,andChrysler.Thedataspanfrom1960to1987andincludedataforninevariablesthatconsistofsomekeyfactorsofthemanufacturingindustrylikeactualcosts,costofgoodssold,totalsalesrevenuetotaletc.Inchapter4,wehaveconsid-eredthisdatatodeterminethelatentnetworkstructurewithintheU.S.automanufacturingindustry.Themethoddiscussedinchapter4isdedicatedtoobtainingtheprobabilityofacon-nectionornoconnectionsbetweenthecompanies,i.e.,existenceofaconnectionbetweencompanyiandcompanyjmeanswij=1andwij=0standsfornoconnectionbetweencompanyiandj.Inourapplication,wewillconsidertheestimatedadjacencymatrixfromchapter4tobetheobservedWandthepurposeistorunavariableselectionamongtheavailablemodelcovariates.Akeythingtonoteisthatinchapter4,wehaveusedrevenuetotalastheresponseandsaleasthecovariatesincethosetwoaretheoreticallythekeyfactorsfortheactualrelationshipsamongthecompanies.Inourproblemhere,weconsider8covariates,andweranavariableselectiontoseewhichvariablesaremostimportant.Thedatamighthavetheissueofnon-stationarityovertimesincewearenotconsideringatimeparametertohandlethedependenceovertime.Instead,weusetheoftheresponseandthecovariateasoutactualmodelresponseandthecovariates,i.e.,ln(yit)=ln(yet1)+ln(Xeit)ei+iti=1;2;:::;Nt=2;3;:::;T:88whereln(yit)=ln(yit)ln(yit1)andln(xitj)=ln(xitj)ln(xit1j).Table5.4:CotestimatesthroughBGL-SS-CARforAutoIndustryDataTickeractatcogsgplctltppegtsaleAL:10(0.03)0(0.12)0.28(0.13)0.08(0.07)0(0.03)0(0.10)0(0.11)0.62(0.26)HON0(0.11)0(0.05)0.16(0.06)0.04(0.03)0(0.03)0(0.11)0(0.08)0.81(0.38)ARV0(0.03)0(0.4)0.50(0.44)0.12(0.10)0(0.03)0(0.02)0(0.01)0.37(0.22)C:30(0.11)0(0.04)0.40(0.33)0.03(0.03)0(0.02)0(0.01)0(0.09)0.51(0.32)CTB0(0.01)0(0.06)0.61(0.34)0.08(0.12)0(0.05)0(0.06)0(0.05)0.28(0.13)DAN0(0.09)0(0.09)0.41(0.19)0.12(0.11)0(0.04)0(0.08)0(0.03)0.43(0.26)DE0(0.05)0(0.04)0.27(0.15)0.06(0.04)0(0.10)0(0.07)0(0.05)0.66(0.51)ETN0(0.03)0(0.05)0.26(0.19)0.11(0.09)0(0.05)0(0.04)0(0.11)0.63(0.46)F0(0.18)0(0.07)0.41(0.28)0.08(0.13)0(0.06)0(0.11)0(0.04)0.41(0.20)GE0(0.11)0(0.09)0.30(0.23)0.14(0.9)0(0.10)0(0.05)0(0.04)0.57(0.56)GM0(0.11)0(0.09)0.35(0.17)0.08(0.06)0(0.08)0(0.12)0(0.08)0.55(0.07)SPXC0(0.07)0(0.13)0.33(0.17)0.09(0.11)0(0.05)0(0.08)0(0.09)0.55(0.54)GR0(0.13)0(0.10)0.20(0.14)0.07(0.05)0(0.06)0(0.09)0(0.03)0.73(0.47)GT0(0.10)0(0.06)0.19(0.11)0.07(0.05)0(0.02)0(0.10)0(0.05)0.72(0.41)JCL0(0.07)0(0.05)0.33(0.35)0.14(0.12)0(0.13)0(0.08)0(0.12)0.53(0.45)ANV:10(0.07)0(0.06)0.36(0.29)0.21(0.09)0(0.11)0(0.08)0(0.12)0.37(0.21)OC0(0.11)0(0.06)0.25(0.17)0.10(0.05)0(0.01)0(0.04)0(0.16)0.64(0.49)PPG0(0.05)0(0.07)0.32(0.25)0.13(0.09)0(0.08)0(0.13)0(0.10)0.52(0.45)AOS0(0.06)0(0.09)0.47(0.23)0.07(0.05)0(0.10)0(0.07)0(0.10)0.43(0.45)UTX0(0.05)0(0.09)0.31(0.14)0.12(0.10)0(0.03)0(0.12)0(0.04)0.56(0.40)Weconsidermodelingthedatausing0revt0(RevenueTotal)asourresponseandusetheother8variablesasourmodelcovariatestoperformavariableselection.Weconsiderthespatio-temporalmodeling(BGL-SS-CAR)byconsideringanAR(1)processovertime.Table5.4showsthevariableselectionalongwiththeposteriormedianestimatesofthecots.ThetablecorrespondstotheBGL-SS-CARmodelingscenario.The'Ticker'symbolshowsthecompanytickersintheUSstockmarket.Ourmodelincludes8covariatesare'act','at','cogs'etc.ThethreeU.S.automanufacturinggiantsaregivenbyC:3=Chrysler,F=Ford,GM=GeneralMotors.Valuesinthebracketsshowingthestandarderrorsoftheestimates.ItiscleartoseefromthetableaboveaccordingtoourBayesianGroupLASSOmodelthat0cogs0=CostofGoodsSold,0gp0=GrossLossProperty,PlantandEquipment-Total(Gross)and0Sale0=Salestotaliscomingouttobetvariables.89whichareheuristicallyandtheoreticallymakesenseandgoconsistentlywiththeselectedcovariatesinchapter4.Figure5.4:PosteriorDistributionof˙2fortheData5.8DiscussionThetopicpresentedinthischapterusesavariableselectiontechniquethattakesinformationfromthespatiallylocatedcovariatesaswellasthespatialadjacencystructureamongthenodesthatfacilitateinselectingthecovariatesoverseveralspatiallocations.Sincethispaperusesthespatialadjacencystructureamongthenodes,itisimportanttohavereliableinformationontheadjacencystructureamongthenodes.Sincethedataisobservedinaspatio-temporalfashion,itisimportanttoundergothetestifthereisanyspatialortemporalnon-stationarity.Inpractice,wetaketheoftheresponsestobeourinitialdatatoremovethenon-stationarity.AssumptionofaCARstructureamongthecovariatesisanimportantassumptioninthispaper.ThisnotonlyfacilitatesthevariableselectiontechniquethroughabettermentoftheRMSEortheTPRbutitalsoprovidesaprofoundheuristicandtheoreticalvalidationsince90itismustbeexpectedthatthevariableselectioninaspatialsituationmustdependonitsneighboringspatiallocations.Sincetheposteriormeandoesnotprovideanexact0estimateforthenon-relevantcovari-ates,Geweke(1994),KuoandMallick(1998),andGeorgeandMcCullough(1997)suggestedthehighestposteriorprobabilityModelviaGibbssamplingcalculatesthehighestposte-riorprobabilityaround0andrejectsthosevariablesthathaveaverytposteriorprobabilityaround0.FDR-basedvariableselectionhasbeenproposedtoselectvariablesifmarginalinclusionprobabilityislargerthansomepre-controlledthreshold.Theposteriorestimationisdistinctiveinthesensethatitdirectlygivesthezeroornon-zeroestimateswithoutgoingtoasecond-stepestimation.5.9ProofoftheLemmas5.9.1ProofofLemma1Letusconsider,w.l.g,y˘N(;˙2).Letusassume2(00+)cNowwehave,Zqf(y)f0(y)dyexpn18x2(0)2o<expn18x22o=Forthesecondpartoftheproof,letusassumey2(M;M)whereMisalargepositiveintegersothatwecantruncatethedistributionin(M;M).LetustakeRreplicatesofy91asyi1;yi2;:::;yiR.So,wehave,Zsf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dyˇ˙2x(0)ReMeMReR2x2(202)Hencesup2UcRsf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dy=˙2ReMeMReR2x2(02+20)forR.So,foreach,ifweassumeMˇ()1˙2expnx22˙(02+20)o,thenwecanshow,sup2UcZsf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dy!0asR!1Again,Zqf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dy=expnR8x2(0)2oSo,sup2UcRqf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dy=expnR8x22oforR.Nowfromtriangleinequality,sup2UcZsf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dyZqf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dy=sup2Ucsf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dy+sup2UcZqf(yi1;yi2;:::;yiR)f0(yi1;yi2;:::;yiR)dy!0asR!15.9.2ProofofLemma2ConsiderarandomvariableZ=TBandsetZ0=B.TheJacobianoftransformationisJ(T;BZ;Z0)=1Z0.92HencethepdfofZisgivenby,f(Z)=Z1exp(12Z0B˙B2+Z0ZT˙T2i)2ˇ˙B˙TT˙T)1Z0dZ0=exp(122B˙2B+2T˙2T)2ˇ˙B˙TT˙T)Z1exp(12"Z022Z0B˙2B+Z2Z022ZZ0T˙2T#)1Z0dZ0Set,Z;Z0)=exp(12"Z022Z0B˙2B+Z2Z022ZZ0T˙2T#)Hence,f(Z)=exp(122B˙2B+2T˙2T)2ˇ˙B˙TT˙T)"Z10Z;Z0)dZ0Z0+Z0Z;Z0)dZ0Z0#HencewehavethesimilarsettingasofCraig(1936).HenceusingthesametechniquewecanshowthattheabovefunctionhasaclosedformexpressionwithanpolynomialfunctionofjZjwiththecotsarebeingascaledversionoftheBesselfunctionofsecondkind.f(Z)=eˆ21+ˆ222ˇˆ2)"0K0+(ˆ21+ˆ22)jZjZ!2K1+(ˆ41+ˆ42)jZj24!4K2+:::#93where,K(Z)=12(Z2)R10eyZ24yy+1dyˆ1=B˙Bandˆ1=T˙T,r(ˆ1ˆ2Z)=1+ˆ1ˆ2Zr+1+(ˆ1ˆ2Z)2(r+2)(2)2!+:::with(r+k)(k)=(r+k)(r+k1):::(r+1).5.9.3ProofofLemma3MTB(˘)=Z1Z10e˘tb1˙Bp2ˇe12bB˙B21T˙T)˙Tp2ˇe12tT˙T2dtdb=12ˇT˙T)˙T˙BZ1Z10e12bB˙B2+tT˙T22˘tbNow,bB˙B2+tT˙T22˘tb=˙2Tb22b˙2TB+2B˙2T2˙2B˙2T˘tb+˙2B(t2+2T+2T)˙2B˙2T=1˙2B˙2Tf˙2Tb2+2b(˙2TB+˙B2˙2T˘t)+(˙2TB+˙B2˙2T˘t)2(˙2TB+˙B2˙2T˘t)2+2B˙2T+˙2B(t2+2T+2T)g=hb(B+˙2B˘t)˙2Bi2(˙TB+˙2B˙T˘t)22B˙2T˙2B(t22T+2T)˙2B˙2T=hb(B+˙2B˘t)˙Bi2+2T˙2T+ht(˙2TB˘+T1˙2B˙2T˘2)˙Tq1˙2B˙22˘2i2(˙2TB˘+T)2˙2T(1˙2B˙2T˘2)94(5.10)Hence,MTB(˘)=˙2TB˘+T˙Tq1˙2B˙2T˘2)T˙T)exp(12(˙2TB˘+T)2˙2T(1˙2B˙2T˘2)122B˙2B)(1˙2B˙2T˘2)1=2NowthemomentofZcanbeobtainedfromthederivativeofMTB(˘)at˘=0:@@˘MTB(˘)j˘=0=(˚(T˙T)T˙T)˙TB+BT)T˙TT˙T)exp(122T˙2T)5.10SomePosteriorCalculations5.10.1PosteriorCalculationforbegP(begjrest)/expˆ12˙2RXr=1jjyerpXg=1XegrV1=2gbegjj22˙pYg=1(1ˇ0)(2ˇ)N=2exp12be0gbeggI(beg6=0e)+ˇ00(beg)=expˆ12˙2RXr=1(yerXrV1=2be)(yerXrV1=2be)˙(1ˇ0)(2ˇ)p=2expˆ12be0gbeg˙I(beg6=0e)+expˆ12˙2RXr=1(yerXrV1=2be)(yerXrV1=2be)˙ˇ00(beg)95where,Xr=(xe1r;xe2r;:::;xepr)np;be=(be01;be02;:::;be0p)np1;V=diag(V1;V2;:::;Vp)npnpHence,P(begjrest)=expˆ12˙2RXr=1jjyerXr(g)V1=2(g)be(g)jj22+12˙2RXr=1(yerXr(g)V1=2(g)be(g))0XrgV1=2gbeg+12˙2RXr=1begV1=2g0Xe0rg(yerXr(g)V(g)be(g))012begIp1˙2XgV1=2gXe0rgXergV1=2gbe0g˙(1ˇ0)(2ˇ)p=2I(beg6=0e)+expˆ12˙2RXr=1jjyerXrV1=2bejj22ˇ00(beg)I(beg6=0e)˙Setg=Ip1˙2PgV1=2gXe0rgXergV1=2g&g=1˙2gPRr=1V1=2g0Xe0rgyerXr(g)V1=2(g)be(g)So,P(begjrest)=(1ˇ0)jgj1=2N(g;g)expˆ12˙2jjyerXr(g)V(g)begjj22+12˙4jj1=2gRXr=1V1=2g0Xe0rg(yerXr(g)V1=2gbeg)jj22˙I(beg6=0e)+ˇ0expˆ12˙2RXr=1jjyerXrV1=2bejj22˙ˇ00(beg)I(beg6=0e)965.10.2PosteriorCalculationfor˝gjP(˝gjjrest)/expˆ12˙2RXr=1jjyerpXi=1XrgV1=2gbegjj22˙pYi=1NYj=1(1ˇ1)2(2ˇs2)1=2expˆ12(˝gjPi6=jwijwi+˝gi)2s2=wi+˙I(˝gj>0)+ˇ10(˝gj)=expˆ12˙2RXr=1(yerXr(gj)V1=2gjbe(gj))0(yerXr(gj)V1=2gjbe(gj))˙expˆ12˙2RXr=1(yerXr(gj)V1=2gjbe(gj))Xergj˝gjbgj+12˙2RXr=1Xergj˝gjbgj(yerXr(gj)V1=2gjbe(gj))˝2gj2b2gj˙2RXr=1x2rgj+wi+s2+˝gjPi6=jwijwi+˝gis2=wi+(Pi6=jwijwi+˝gj)22s2=wi+˙(1ˇ1)2(2ˇs2wi+)1=2I(˝gj>0)+expˆ12˙2RXr=1(yerXrV1=2be)0(yerXrV1=2be)˙ˇ10(˝gj)Here,Xergj=(0;0;:::;0;xrgj;0;:::;0)0.Set,v2gj=wi+s2+b2gj˙2PRr=1x2rgj1ugj=v2gj˙2PRr=1(yerXer(gj)V1=2gjbgj)Xergjbgj+v2gjwi+s2Pi6=jwijwi+˝gj97Hence,P(˝gjjrest)=expˆ12˙2RXr=1(yerXr(gj)V1=2(gj)be(gj))0(yerXr(gj)V1=2(gj)be(gj))˙expˆ12˙2(Xi6=jwijwi+˝gi)2+12u2gjv2gj˙ugjvgj)N+(ugj;v2gj)(2ˇv2gj)1=2(1ˇ1)2(2ˇ2)1=2I(˝gj>0)+expˆ12˙2RXr=1(yerXrV1=2be)0(yerXrV1=2be)˙ˇ10(˝gj)5.10.3PosteriorCalculationfors2P(s2jrest)/pYg=1NYj=1(1ˇ1)2(2ˇs2)1=2exp(˝gjPi6=jwij˝gj)22s2I(˝gj>0)+ˇ10(˝gj)t(s2)2expts2/(s2)(M2+2)expˆ1s2t+12pXg=1NXj=1(˝gjXi6=jwijwi+˝gj)2˙where,M=#(˝gj6=0)8g=1;2;:::;p&j=1;2;:::;N.5.11SomeDetailsforDataAnalysis98Table5.5:CompanynameswithcorrespondingtickerTickerCompanyNamesAL:1ALCANINC(RIOTINTO)HONHONEYWELLINTERNATIONALINCARVARVININDUSTRISINC(MERITOR)C:3CHRYSLERCTBCOOPERTIRE&RUBBERCOMPANYDANDANAHOLDINGCORPDEDEERE&COETNEATONCORPPLCFFORDGEGENERALELETRICCOGMGENERALMOTORSSPXCSPXCORPGRGOODRICHCORPGTGOODYEARTIRE&RUBBERCOJCLJOHNSONCONTROLSINCANV:1AEROQUIP-VICKERSINCOCOWENSCORNINGPPGPPGINDUSTRIESINCAOSSMITH(AO)CORPUTXUNITEDTECHNOLOGIESCORP99Table5.6:CovariateListCovariateCodeCovariateNameactCURRENTASSETS-TOTALatASSETS-TOTALcogsCOSTOFGOODSSOLDgpGROSSPROFIT(LOSS)lctCURRENTLIABILITIES-TOTALltLIABILITIES-TOTALppegtPROPERTY,PLANTANDEQUIPMENT-TOTAL(GROSS)saleSALES/TURNOVER(NET)Table5.7:ResponseVariableResponseCodeResponseNamerevtREVENUETOTAL100BIBLIOGRAPHY101BIBLIOGRAPHY[1]Anselin,L.,Bera,A.K.(1998)\Spatialdependenceinlinearregressionmodelswithanintroductiontospatialecono-metrics".StatisticsTextbooksandMonographs,155237-290[2]Banerjee,S.,Carlin,B.P.,Gelfand,A.E.(2004)\HierarchicalModelingandAnalysisforSpatialData".MonographonStatisticsandAppliedProbability101.Chapman&Hall/CRC[3]Berger,J.O.,Bernardo,J.M.(1992).\OnTheDevelopmentofthereferencePriorMethod".BayesianStatistics4:ProceedingsoftheFourthValenciaInternationalMeeting,4,35-60.[4]Berger,J.O.,Bernardo,J.M.,Sun,D.(2009).\TheformalnitionofReferencePriors".TheAnnalsofStatistics.905-938.[5]Besag,J.(1974).SpatialInteractionandtheStatisticalAnalysisofLatticeSystems,JournaloftheRoyalStatisticalSociety.SeriesB(Methodological).36,192-236.[6]Bhattacharjee,A.,Castro,E.,Maiti,T.,Marques,J.(2016).\Endogenousspatialregressionanddelineationofsubmarkets:anewframeworkwithapplicationtohousingmarkets".JournalofAppliedEconometrics31,32-57[7]Bhattacharjee,A.,Holly,S.(2013)\Understandinginteractionsinsocialnetworksandcommittees".SpatialEconomicAnal-ysis8,23-53[8]Bhattacharjee,A.Jensen-Butler,C.(2013)\EstimationoftheSpatialWeightsAMtrixunderStructuralConstraints".RegionalSci-enceandUrbanEconomics,43,617-634[9]Bondell,H.D.andReich,B.J.(2012)\Consistenthigh-dimensionalBayesianvariableselectionviapenalizedcredibleregions".JournalofAmericanStatisticalAssociation,107:1610-1624102[10]Borgatti,S.P.,Mehra,A.,Brass,D.J.,Labianca,G.(2009)\Networkanalysisinthesocialsciences".science,323,892-895[11]Caimo,A.,Friel,N.(2011).Bayesianinferenceforexponentialrandomgraphmodels,SocialNetworks,33,41-55[12]Castro,E.A.,Zhang,Z.,Bhattacharjee,A.,Martins,J.M.andMaiti,T.(2015)\RegionalFertilitydataAnalysis:AsmallareaBayesianApproach".CurrentTrendsinBayesianMethodologywithApplications,203-224[13]Casella,G.,Giron,F.J.,Martinez,M.L.andMoreno,E.(2009)\ConsistencyofBayesianProceduresforVariableSelection".TheAnnalsofStatistics,37,1207-1228[14]Castillo,T.,Schmidt-Heiber,J.andVanDerVaart,A.(2015)\BayesianLinearRegressionwithSparsePriors".TheAnnalsofStatistics,43,1986-2018[15]Choi,T.,Ramamoorthi,R.V.(2008)\Remarksonconsistencyofposteriordistributions.InPushingthelimitsofcontempo-rarystatistics:contributionsinhonorofJayantaK.Ghosh".InstituteofMathematicalStatistics,170-186[16]Craig,C.C.(1936)\OntheFrequencyFunctionFunctionofxy".TheAnnalsofStatistics,7,1-15[17]Erds,P.,Rnyi,A.(1959).\Onrandomgraphs,I".PublicationesMathematicae(Debrecen),6,290-297[18]Frank,O.,Strauss,D.(1986).\MarkovGraphs".JournaloftheAmericanStatisticalAssociation,.81,832-842[19]Ghosal,S.(1997).\AReviewofConsistencyandConvergenceofPosteriorDistribution".InVaranashiSymposiuminBayesianInference[20]Ghosal,S.,Ghosh,J.K.,VanDerVart,A.W.(2000)\ConvergenceRatesofPosteriorDistributions".TheAnnalsofStatistics,28,500-531[21]Goodreau,S.M.,Handcock,M.S.,Hunter,D.R.,Butts,C.T.,Morris,M.(2008).\AstatnetTutorial".Journalofstatisticalsoftware,24,nihpa54860103[22]Grandori,A.,Soda,G.(1995).\Innetworks:antecedents,mechanismsandforms".Organizationstudies,16,183-214[23]Holland,P.W.andLeinhardt,S.(1981).\Anexponentialfamilyofprobabilitydistributionsfordirectedgraphs".JournaloftheAmericanstatisticalAssociation,.76,33-50.[24],P.D.(2003)RandomModelsforNetworkData.[25].P.D.,Raftery,A.E.,Handcock,A.E.(2002).\LatentSpaceApproachestoSocialNetworkAnalysis".JournaloftheAmericanstatis-ticalAssociation.97,1090-1098.[26]Handcock,M.S.andRaftery,A.E.(2007).\ModelBasedClusteringforSocialNetworks".J.R.Statist.Soc.A.170,1-22.[27]Hertzel,M.G,Li,Z,M.S,Rodgers,K.J.(2008)Inlinkagesandthewealthofdistressalongthesupplychain.JournalofCorporateFinance87,374-387.[28]Hunter,D.R.,Handcock,M.S.(2012).\Inferenceincurvedexponentialfamilymodelsfornetworks".JournalofComputationalandGraphicalStatistics,15,565-583[29]Ishwaran,H.andRao,J.S.(2005)\SpikeandSlabVariableSelection:FrequentistandBayesianStratigies".TheAnnalsofStatistics,33,730-773[30]Jiang,W.(2007)\BayesianVariableSelectionforHighDimentionalGeneralizedLinearModels:Conver-genceRatesoftheFittedDensities".TheAnnalsofStatistics,35,1487-1511[31]Jiang,W.(2005)OntheConsistencyofBayesianVariableSelectionforHighDimentionalBinaryRegres-sionandNeuralComputation,18,2762-2776[32]Kleijn,B.J.K.(2013).\CriteriaforPosteriorConsistency".arXiv:1308.1263104[33]Koskinen,J.H.,Snijders,T.A.B.(2007).\Bayesianinferencefordynamicsocialnetworkdata",JournalofStatisticalplanningandInference,.137,3930-3938[34]Kraft,C.(1955).\Someconditionsforconsistencyanduniformconsistencyofstatisticalprocedures",125-142[35]LeCam,L.(1960).\Locallyasymptoticallynormalfamiliesofdistributions".UniversityofCaliforniaPub-licationsinStatistics,3,37-98[36]Lee,K.J.,Jones,G.L.,B.S.,Bassett,S.S.(2014).\SpatialBayesianVariableSelectionModelonFunctionalMagnaticResonanceImagingTime-seriesData".BayesianAnalysis,9,699-732[37]Leskovec,J.,Krevl,A.(2015).SNAPDatasets:Stanford.LargeNetworkDatasetCollection.[38]Mur,J.,Angulo,A.(2007)\TheSpatialDurbinModelandtheCommonFactorTests".SpatialEconomicAnalysis,1,207-226[39]Murray,I.,Ghahramani,Z.,MacKay,D.(2012).\MCMCfordoubly-intractabledistributions".arXivpreprintarXiv:1206.6848[40]Narasimhan,R.,Jayaram,J.(1998).\Causallinkagesinsupplychainmanagement:anexploratorystudyofNorthAmericanmanufacturingDecisionsciences,29,579-605[41]Narisetty,N.N.,He,X(2014)\Bayesianvariableselectionwithshrinkingandpriors".AnnalsofStatistics,42,789-817[42]Nowicki,Krzysztof,andTomA.B.Snijders(2001)\Estimationandpredictionforstochasticblockstructures".JournaloftheAmericanSta-tisticalAssociation,96.1077-1087[43]Park,T.,Casella,G.(2008)\Thebayesianlasso".JournaloftheAmericanStatisticalAssociation,103,681-686105[44]Pesaran,M.H.(2004).\Generaldiagnostictestsforcrosssectiondependenceinpanels".[45]Pattison,P.,Wasserman,S(1998)\Logitmodelsandlogisticregressionsforsocialnetworks:II.Multivariaterelations".BritishJournalofMathematicalandStatisticalPsychology,42,789-817[46]Ramcharran,H.(2001)\Inter-FirmLinkagesandilityintheAutomobileIndustry:TheimplicationforSupplyChainManagement".TheJournalofSupplyChainManagement,37,11-17[47]Robins,G.,Pattison,P.,Kalish,Y.,Lusher,D.(2007).\Anintroductiontoexponentialrandomgraph(p*)".SocialNetworks,29,173-191.[48]Scutari,M.(2013).\OnthePriorandPosteriorDistributionsUsedinGraphicalModelling".BayesianAnal-ysis,.8,505-532[49]Schwartz,L.(1965).\Onbayesprocedures".ZeitschriftfrWahrscheinlichkeitstheorieundverwandteGebiete,.4,10-26[50]Smith,M.andFahrmeir,L.(2007)\SpatialBayesianVariableSelectionwithApplicationtoFunctionalMagnaticResonanceImaging".JournalofAmericanStatisticalAssociation,102,417-431[51]Smith,M.,Kohn,R.(1996)\NonparametricregressionusingBayesianvariableselection".JournalofEconometrics,75,317-343[52]Shalizi,C.S.,Rinaldo,A.(2013)\ConsistencyUnderSamplingofExponentialRandomGraphModels".TheAnnalsofStatistics,.41,508-535[53]Strauss,D.,Ikeda,M.(1990)\PseudolikelihoodEstimationforSocialNetworks".JournaloftheAmericanStatisticalAssociation,.85,204-212[54]Tibshirani,R.(1996)\Regressionshrinkageandselectionviathelasso".JournaloftheRoyalStatisticalSoci-ety.SeriesB(Methodological),.58,267-288106[55]VandeGeer,S.(1993)\Hellinger-consistencyofcertainnonparametricmaximumlikelihoodestimatorslikeli-hoodestimationofexponentialfamilyrandomgraphmodels".TheAnnalsofStatistics,.21,14-44[56]VanDuijn,M.A.,Gile,K.J.,Handcock,M.S.(2009)\Aframeworkforthecomparisonofmaximumpseudo-likelihoodandmaximumlike-lihoodestimationofexponentialfamilyrandomgraphmodels".SocialNetworks,.31,52-62[57]Wang,J.(2012)\DoFirms'RelationshipwithPrincipalCustomer/SuppliersShareholdersIn-come?".JournalofCorporateFinance,18,860-878[58]Weisstein,E.W.(2003)\NormalProductDistribution".FromMathWorld-AWolframWebResource[59]Xu,X.andGhosh,M.(2015)\BayesianVariableSelectionandEstimationforGroupLASSO".BayesianAnalysis,10,909-936[60]Zou,H.(2006)\TheAdaptiveLASSOandIt'sOracleProperties".JournalofAmericanStatisticalAs-sociation,101,1418-1429107