PREDICTIVEMODELSFORROBOTICSANDBIOMEDICALAPPLICATIONSByHuanNDoADissertationSubmittedtoMichiganStateUniversityinpartialful˝llmentoftherequirementsforthedegreeofMechanicalEngineeringDoctorofPhilosophy2017ABSTRACTPREDICTIVEMODELSFORROBOTICSANDBIOMEDICALAPPLICATIONSByHuanNDoDatasciencehasbeentransforminganenormousnumberofresearchareas.Ithasopenedthedoortonewmeasurestoanalyzeandextractusefulinformationfromrawdata.However,whileithasbeenappliedextensivelyincomputersciencecommunity,therehasbeenamodestnumberofsuchapplicationsinthe˝eldofroboticsandbiomedicalengineering.Inthisdissertation,weconsidertheapplicationsofdataanalysisandmachinelearningtoolsintworesearchtopics:mobilerobotlocalizationandcardiovascularpredictivemodels.Inthe˝rstpartofthedissertation,wetackleaproblemoffeatureselectionforappearance-basedlocalization.Rawimageisahigh-dimensionalsourceofdata,andastheresolutionofvisualsensorhasbeenimprovedrapidly,weareequippedwithevenhigherdimensionalandrichervisualinformation.Todealwiththehighdimensionalityproblem,acommonandstraightforwardstrategyistoselectthemoste˙ectivevisualfeaturesforthelocalizationtask,i.e.,featureselection.Inthisdissertation,weproposetwomethodsoffeatureselection.Firstofall,wemodeleachdimensionofthefeaturevectorasaGaussianprocessrandom˝eldwiththeindependentvariablesasthecoordinatesoftherobot.Thus,thelocationsoftherobotcanbeinferredbyapplyingamaximumlikelihoodestimator.Theoptimalsetoffeaturesarechosenbybackwardeliminationscheme.Secondly,tominimizethelocalizationerrorinspatialspaceandtoselecttheoptimalsubsetoffeatures,weformu-lateamultivariateversionoftheLeastAbsoluteSelectionandShrinkageOperator(LASSO)regressionmodel.Underthisformulation,wedevelopacombinedlocalizationschemethatconsistsoftheregressionanda˝lteringestimator.Inthelaterpartofthisdissertation,weexploretheuseofpredictivemodelstopredictthegrowthofanabdominalarteryundertheprogressionofadisease,AbdominalAorticAneurysm(AAA).AsapatientwhoisdiagnosedwithAAA,his/herarterymaylocallybeenlargedinpathologicalconditionsand˝nallyruptures,wedeveloptwopredictionap-proachesusingtwocommontypesofAAAgeometricaldata:3Dshapesfromcomputertomography(CT)scansand2Dpro˝leofmaximaldiametersovercenterline.Firstofall,wedevelopourDynamicalGaussianProcessImplicitSurface(DGPIS)for3Dshapeprediction.Inthismethod,weconsidera3Dsurfaceasamanifoldembeddedinascalar˝eldoverthe3Ddimensionalspace,thechangesofwhichpropagatethechangesinthesurface.Thus,byutilizingadynamicmodeltorepresenttheevolutionofthe˝eldovertime,wecanmakeaninferenceabouttheAAAsurfaceinafuturetime.Secondly,maximalAAAdiameterisacrucialcriterionformakingasurgeryinterventiondecisioninclinicalpractice.Thus,weinvestigateaDeepBeliefNetwork(DBN)modelthatistrainedonarti˝cialdatacreatedfromProbabilisticCollocationMethod(PCM)andrealpatientsbasedreconstructeddata.SincethemeritofDBNanddeepstructureingeneraldependsonamassivesizeoftrainingdata,whichiscommonlyrareinthisapplication,weovercometheshortagebypre-trainingtheDBNonsimulateddatageneratedfromPCM,then˝ne-tuningtheneuralnetonrecon-structeddatafromtherealpatients.Theexperimentalresultsillustratethee˙ectivenessofourproposedmethods.CopyrightbyHuanNDo2017AcknowledgmentsFirstofall,Iwouldliketoexpressmysinceregratitudetowardtwoofmyadvisers,Dr.JongeunChoiandDr.SeungikBaek,whohavebeennotonlymyprofessionalmentorsbutalsomyrespectedcompanionsalongmyPhDcareer.Withouttheirconstantencouragementandsupport,Icertainlywouldnotbeabletoaccomplishthisdissertation.Additionally,IwouldliketothankmyPhDcommitteemembers,Dr.GuomingZhuandDr.Pang-NingTan,fortheirsuggestionsandadvices.Secondofall,Iwouldliketothankmycollages,Mr.DavidYork,Mr.AlexanderRobinson,andMs.TamLe,whohavebeenkindlyhelpingmetocollectexperimentaldata.Inaddition,IwouldliketoappreciatetheVietnamEducationFoundation,whichhasgrantedmethisopportunitytopursuehighereducationintheUS.Finally,Iamgratefultomyfamily:myparents,mywife,andmylittledaughter.Myloveiswiththemforever.vTableofContentsLISTOFTABLES....................................viiiLISTOFFIGURES...................................ixChapter1Introduction................................11.1Part1:appearance-basedlocalization........................11.1.1Background.....................................21.2Part2:predictivemodelsofAbdominalAorticAneurysm.............31.2.1Background.....................................41.3Contribution.....................................51.4Publication......................................71.4.1Journalarticles...................................81.4.2Conferenceproceedings..............................8Chapter2Featureselectioninfeaturespace...................102.1Imagefeatures....................................122.2Gaussianprocess(GP)model............................142.2.1Theˆ-thrandom˝eld...............................162.3Localizationandfeatureselection..........................182.4Indoorandoutdoorexperiments...........................192.4.1Experimentalsetups................................202.4.2LearningofGPmodelsinanempiricalBayesapproach.............222.4.3LocalizationutilizingtheBagofWords(BOW).................242.4.4Experimentalresults................................27Chapter3Featureselectioninspatialspace...................323.1Imagefeatures....................................353.2TheLASSO......................................363.3ThegroupLASSO..................................373.4TheextendedKalman˝lter.............................393.5Experimentalstudy..................................413.5.1IndoorExperiment.................................413.5.2OutdoorExperiment................................45Chapter4AbdominalAorticAneurysm'sshapeprediction.........544.1DataAdaptation...................................564.2Modelandmethod..................................574.2.1Gaussianprocessregression............................574.2.2DiscretizedGaussianprocessspace........................594.2.3Spatio-temporalGaussianprocess.........................604.2.4Implicitsurfaces..................................614.2.5Dynamicimplicitsurfacemodel..........................62vi4.3Surfaceextractionandpost-processing.......................654.4Experimentalresults.................................654.4.1Constantgrowthrateassumption.........................664.4.2Uncertaintyquali˝cation..............................684.5Discussion.......................................694.5.1Decisionmakingviapredictionandcon˝denceregions.............704.5.2Estimatedhyper-parametersandmodelparametersasinformativefeatures..714.5.3Limitationsandfutureresearchdirections....................72Chapter5AbdominalAorticAneurysm'smaximumdiametersprediction795.1Data..........................................815.1.1Realpatientbasedreconstructeddata......................815.1.2Arti˝ciallygenerateddata.............................825.2Predictionofmaximumdiametercurve.......................845.3ProbabilisticCollocationMethod..........................865.3.1Deterministicinput.................................865.3.2Stochasticinput..................................875.3.3Selectionofbasicfunctionsandcollocationpoints................885.3.4Maximumdiametercurvetransformation.....................905.4DeepLearning....................................915.4.1RestrictedBoltzmannMachine..........................925.4.2ContrastiveDivergence...............................945.5Realcasestudy....................................965.5.1Experimentalset-up................................965.5.2Experimentalresults................................98Chapter6Conclusionandfutureworks......................103APPENDICES......................................106AppendixAE-Malgorithm................................107AppendixBSurfaceextraction..............................112AppendixCProofofCorollary2.............................116BIBLIOGRAPHY....................................117viiListofTablesTable2.1ThelocalizationperformanceforCase1...................28Table2.2ThelocalizationperformanceforCase2...................28Table2.3ThelocalizationperformancecomparisonbetweentheproposedapproachandtheBOWinCase2.TheRMSEisfromthetestset............28Table3.1Localizationperformancecomparison....................45Table4.1DemographicDataofPatientsfrom[34]...................56Table4.2HyperparametersandHausdor˙DistancewithdetailsofeachCase....66Table5.1DemographicDataofPatientsadaptedfrom[34]..............82Table5.2E˙ectofnumberoflayersontheprediction..................97Table5.3E˙ectofnumberofnodesina2-layerDBNontheprediction........97Table5.4Comparisonamongdi˙erentpredictivemodelsintermsofRMSEandnor-malizedunit....................................100viiiListofFiguresFigure2.1(a)and(b)showthewrappedomnidirectionalimageandtheunwrappedpanoramicimage,respectively.(c)and(d)showthereducedsizegrayscaleimageandthetwo-dimensionalFFTmagnitudeplot,respectively.12Figure2.2OutdoortrajectorycollectedfromaGPSunit...............21Figure2.3DotiPhonePanoramalens(left)andtheindoorenvironment(right)forCase1.....................................22Figure2.4(a)Dataacquisitioncircuit,(b)panoramiccameraand(c)vehicleusedinCase2...................................23Figure2.5GPmodelsforeachof˝rstthreeFFTfeaturesfromtheoutdoordataset.The˝rstrowshowsthemeansandthesecondrowshowsthevariancesoftheGPmodels...............................25Figure2.6TrainingdatasetassignmentfortheBOW.Thetestpoints,thetrainingpoints(withnewlabels),andthe5mradiiareplottedinbluedots,reddiamonds,andbluecircles,respectively.Thetrainingpointsthatdonotbelongtoanytestgroupsareeliminated..................26Figure2.7PredictionresultforCase2withthehistogram.ThetestpathandthepredictionareplottedovertheGoogleMapimageinreddiamondsandgreendots,respectively............................31Figure3.1(a)Shrinkageofestimatevectorentrieswithrespecttodi˙erentvaluesof.(b)errorcurveofdi˙erentestimatevectorsbappliedonvalidationdataset....................................37Figure3.2(a)Indoorexperimentenvironmentandthezoomed-inpictureofthemobilerobotshownintheupperleftcorner.(b)OutdoorexperimentinthecampusofMichiganStateUniversityonGooglemap.........43Figure3.3PlotofP[1;1](dashedline)andP[2;2](solidline)foriterationsfrom20to80forthegroupLASSO-basedwithEKFlocalization...........44Figure3.4Indoorexperiment:TheevolutionoftheentriesofestimatematrixBversusthepenalty.Eachpairoffeaturesthatassociatewithsamecoordinateisplottedinsamecolor.....................46Figure3.5Indoorexperiment:Thetruetrajectory(blacksolidline),groupLASSO(greensquares),groupLASSO+PF(dottedblackline)andgroupLASSO+EKF(reddotteddashedline)predictions........47ixFigure3.6(a)Dataacquisitioncircuit,(b)panoramiccameraand(c)vehicleequippedwiththecameraontop......................48Figure3.7Thetraining(dashed)andtestingpaths(solidlines)areplottedinmeters.49Figure3.8Outdoorexperiment:Thetruetrajectory(blacksolidline),groupLASSO(greensquares),groupLASSO+PF(blackdottedline),andgroupLASSO+EKF(reddotteddashedline)predictionsareplottedinmeters...................................52Figure3.9Outdoorexperiment:(a)TheevolutionofentriesoftheestimatematrixBversusthepenalty.(b)Overall200entriesoftheoptimalmatrixBareplottedinbluebars...........................53Figure3.10ThesnapshotoftheParticleFilteratt=50:Thetruetrajectory(redsolidline),groupLASSO(greensquares)andgroupLASSO+PF(bluedashedline)predictionsareplottedinmeters.Eachparticleisplottedwiththecoloringrayscalecorrespondingtoitsprobabilityweight.Thesumoftheweightsofallparticlesis1....................53Figure4.1Summaryofourproposedmethod:pointclouddataisinsertedintothespatio-temporalGaussianprocessaszero-valueobservationstogeneratethe˝eld.Then,thetemporalevolutionofthe˝eldisinferredthroughthedynamicmodelin(4.8).Finally,the˝nalpredictioniscomputedbyutilizingtheKalmanFilterintheE-Malgorithm.............58Figure4.2Exampleofanestimated˝eld:crosssectionviewsofthe3D˝eldatthesameheightinz-axisareshownatdi˙erenttimest.Theon-surfacepoints(wherethe˝eldiszero)arelabeledinwhitesolidlines.ThegrowthoftheAAAintheradialdirectioncanbevisualizedfromtoptobottom˝gures................................61Figure4.3Convergenceofthedistanceof(a)Aand(b)wtotheirequilibriumvalues(A1andw1),startedwithrespecttodi˙erentinitialvaluesofAandw...................................64Figure4.4Therelativetimesofthescanscomparedtothe˝rstscanareplottedforeachpatientindays.............................67Figure4.5Fullytrainedcaseusingallavailablelongitudinaldata:3-Drenderingfromtheestimated(^Di;j)andtrue(Di;j)pointclouds:Thepredictedandtruepointcloudsareusedtorenderthe3-DsurfacesusingthePoissonreconstructioninMeshlab..........................74xFigure4.6Uncertaintyquanti˝cationfor7patientsforfullytrainedcase:Theim-plicitsurfacesareregeneratedonthegridbyrealizingthemulti-variateGaussiandistributionwiththemeanvectorandcovariancematrixcom-putedin(4.5).Then,weapplythesameproceduredescribedinSec-tion4.3toestimatetheAAAsurfacesthatarecorrespondingtotherealizedimplicitsurfaces.EachrealizationoftheAAAsurfacefromtheposteriordistributionisplottedwith2%occupancy.Therefore,spatialregionsthatappearbrighteraremorelikelytobeoccupiedbytheAAAsurface.Thetruecloudpointsareplottedinreddots...........75Figure4.7Last-3trainedcaseusingthemostrecentthreelongitudinaldatapoints:3-Drenderingfromtheestimated(^Di;j)andtrue(Di;j)pointclouds:Thepredictedandtruepointcloudsareusedtorenderthe3-DsurfacesusingthePoissonreconstructioninMeshlab.NotethatweruntheLast-3trainedcaseforpatientP1,P2,P3,P4andP6,sinceothercasesalreadyhave4scansintotal.............................76Figure4.8Uncertaintyquanti˝cationfor5patientsforLast-3trainedcase:Theim-plicitsurfacesareregeneratedonthegridbyrealizingthemulti-variateGaussiandistributionwiththemeanvectorandcovariancematrixcom-putedin(4.5).Then,weapplythesameproceduredescribedinSec-tion4.3toestimatetheAAAsurfacesthatarecorrespondingtotherealizedimplicitsurfaces.EachrealizationoftheAAAsurfacefromtheposteriordistributionisplottedwith2%occupancy.Therefore,spatialregionsthatappearbrighteraremorelikelytobeoccupiedbytheAAAsurface.Thetruecloudpointsareplottedinreddots.NotethatweruntheLast-3trainedcaseforpatientP1,P2,P3,P4andP6,sinceothercasesalreadyhave4scansintotal.....................77Figure4.9AnexampleofusageofA(x)asaninformativefeature.Twosuccessivelyshapes,namelypreviousandcurrentshapes,areplottedinblacksquaredandcircledpointcloudsinstandardizedunit,respectively.Additionally,theintersectionsofthemwiththeplanez=0areplottedinsolid(previ-ous)anddashed(current)whitelinesonthez=0plane.Furthermore,thecross-sectionviewsoftheA(x)˝eldatz=0andz=1arecolorplottedonthetwoplanes.Themigrationofthesurfaceisshownclearlyinthedirectionindicatedbytheredarrow.Then,thevelocityofthemigrationintheindicateddirectioncanbecomputedby(4.11).....78Figure5.1Exampleof2Dpro˝lecurvesofmaximaldiametersoverthecenterlineobtainedfrom(a)realpatient,(b)theG&Rcomputationalmodel,and(c)thePCMapproximation.NotethatthetwoG&RandPCMcurvesinthisexamplearenotbasedontherealcenterline,sotheirunitsarenotincentimeters,butinspatialsiteunit.Thespatialsitecanbeseenasa1DmeshintheFEMcode.......................82xiFigure5.2Overalldiagramoftheproposedmethod..................85Figure5.3Twotransformationsofamaximumdiametercurve:theoriginal,trans-lated,andsecondpeakmodi˝edcurvesareshowninsolidblue,dashedred,anddottedblacklines,respectively.Notethatthe˝rsthalvesoftheoriginalandsecondpeakmodi˝edcurvesoverlapeachother.......91Figure5.4DeeparchitectureoftheDBN.TwolayersofRBMaretrainedinanunsupervisedmanner(pre-trained)usingCD-1algorithm.Thetoplayerutilizesaneuralnetworksigmoidregressionforprediction........95Figure5.5E˙ectofnumberofepochstothepredictionerror.TheRMSEand˝ne-tunetrainingtimeareplottedinsolidblueanddashedredlines,respectively.Thetrainingincreaseslinearlywiththenumberofepochs,whiletheRMSErapidlydecreasesforthebeginningthenstartstoin-creaseagainforthenumberofepochsthatislargerthan500.......99Figure5.6Thetrue,predictedbyourproposedmethod,andpredictedbythemixed-e˙ectsmodelareshowninsolidblackdashedbluepredic-anddotteddashedredlines,respectively...101FigureA.1ThresholddeterminationforpatientP5:the˝eldofpointsonthelatticeforthetrainingandtest˝eldsareplottedinblueplusesandreddots,respectively.Thethresholdsfortrainingandtestdataareshowninsolidanddashedlines,correspondingly......................113FigureA.2CrossviewofpredictedsurfaceoftheAAAattwodi˙erentverticalheights:(a)z=86:9(mm)and(b)z=76:21(mm).Thepointcloudsandtwoclustersareplottedincirclesandred/bluestars,respectively.Thesurfacepointsthatareselectedbytheconvexhullareconnectedwiththeblackdashedline.Asshownin(b),withouttheclustering,twobrancheswillbefalselymergedintoone..................115xiiChapter1Introduction1.1Part1:appearance-basedlocalizationInrecentyears,buildingacompleteself-drivingcarhasbeenabrutalraceamongawiderangeofcompetitors1,frominformation-technologycompaniessuchasGoogletoprolongautomobilemakerssuchasGeneralMotors,orstart-up-basedcompanieslikeTesla.Self-drivingisafuturistictechnologythatwithnodoubtwillbeagamechangerintermsofnotonlypro˝tabilitybutalsohumanityasmorehumanlivescanbesavedfromtra˚caccidents.However,theadvanceofself-drivingtechnology,whichcriticallydependsonthelocaliza-tioncapability,nowishinderedbythelimitationofGPSacccuracy2orbeinginterruptedatGPS-deniedregions.Hence,therearestrongmotivationsfordevelopinglocalnavigationtechnologiesthatdonotsolelydependonGPS.Therefore,wedevelopalocalizationschemethatisbuiltbasedonvisualsensormeasurements,i.e.,appearance-basedlocalization.Byincludingthewholedimensionsofvisualdataandprovidingacompletetreatmentofassoci-atingthefeatureselectionoutcomesandthedynamicsofthesystem,ourapproachesyieldexcellenttrackingperformance,whicharevalidatedinanumberofexperimentalresults.1ThelistofcompaniesthatapplyfortestinglicensefromCaliforniaDepartmentofMotorVehicles:https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/testing2Thestate-of-the-artGPSnowhasaccuracyof1˘2meters,whichisobviouslyinadequatetopreventhighspeedcollision.1Thus,theymaybecombinedwithlocalizationwithGPSastheinformationiscollectedbyself-drivingvehicles.Furthermore,asGPSisdisabledinacertainGPS-deniedregions,ouralgorithmsareabletoserveasaback-upunittomainthereliabletracking.1.1.1BackgroundInordertomakeafastandpreciseestimation,mostoftheexistinglocalizationalgorithmsextractasmallsetofimportantfeaturesfromtheroboticsensormeasurements.Thefeaturesusedindi˙erentapproachesforroboticlocalizationrangefromC.1:arti˝cialmarkerssuchascolortags[56]andbarcodes(thatneedtobeinstalled)[97],C.2:geometricfeaturessuchasstraightwallsegmentsandcorners[57],andtoC.3:naturalfeaturessuchaslightandcolorhistograms[5].Mostofthelandmark-basedlocalizationalgorithmsareclassi˝edinC.1andC.2.Itisshownin[94]thatautonomousnavigationispossibleforoutdoorenvironmentswiththeuseofasinglecameraandnaturallandmarks.Inasimilarattempt,[23]addressedthechallengingproblemofindoorplacerecognitionfromwearablevideorecordingdevices.Thelocalizationmethodswhichrelyonarti˝cialmarkers(orstaticlandmarks)havedisadvantagessuchaslackof˛exibilityandlackofautonomy.Amethodisdescribedin[109]thatenablesrobotstolearnlandmarksforlocalization.Arti˝cialneuralnetworksareusedforextractinglandmarks.However,thelocalizationmethodswhichrelyondynamiclandmarks[109]havedisadvantagessuchaslackofstability.Furthermore,therearereasonstoavoidthegeometricmodelaswell,evenwhenageometricmodeldoesexist.Suchcasesmayinclude:1)thedi˚cultyofreliablyextractingsparse,stablefeaturesusinggeometricalmodels,2)theabilitytouseallsensorydatadirectlyratherthanarelativelysmallamountofabstracteddiscreteinformationobtainedfromfeatureextractionalgorithms,and3)highcomputationalandstoragecostsofdealingwithdensegeometricfeatures.Incontrasttothelocalizationproblemwitharti˝cialmarkersorpopulargeometricalmodels,thereisagrowingnumberofpracticalscenariosinwhichglobalstatisticalinforma-tionisusedinstead.Someworksillustratelocalizationusingvariousspatiallydistributed2(continuous)signalssuchasdistributedwirelessEthernetsignalstrength[30],ormulti-dimensionalmagnetic˝elds[113].In[117],aneuralnetworkisusedtolearntheimplicitrelationshipbetweentheposedisplacementsofa6-DOFrobotandtheobservedvariationsinglobaldescriptorsoftheimagesuchasgeometricmomentsandFourierdescriptors.Insimilarstudies,gradientorientationhistograms[67]andlowdimensionalrepresentationofthevisiondata[110]areusedtolocalizemobilerobots.In[83],analgorithmisdevelopedfornavigatingamobilerobotusingavisualpotential.Thevisualpotentialiscomputedfromtheimageappearancesequencecapturedbyacameramountedontherobot.Amethodforrecognizingscenecategoriesbycomparingthehistogramsoflocalfeaturesispresentedin[70].Withoutexplicitobjectmodels,byusingglobalcuesasindirectevidenceaboutthepresenceofanobject,theyconsistentlyachieveanimprovementoveranorderlessimagerepresentation[70].1.2Part2:predictivemodelsofAbdominalAorticAneurysmComputer-aideddiagnosis(CAD)hasbeenstudiedandapplied3sincethe1960stohelpphysicianstomakedecisionsfasterandmoreaccurate,especiallywithtimepressure.Thetechnology,whenbeingmature,willcertainlyplaypivotalroleindiseaseprognosisandclinicalmanagement.CADo˙erstwomainanalysisfunctionsthatoutperformthoseofhumans.Firstly,itprovidessyntheticinformationfromvarioussourcesofdata,e.g.,CT,MRIorPETscans,whichisnotobvioustoseewithinformationfromasinglesource.Secondly,givenmeasurementsforonepatient,itcanprocessthroughanavailabledatabaseto˝ndsimilarcasesandtheirpathology.Thesefeaturesaimaphysicianwithmoreinformationthathe/shecouldobtainbyviewingatoneimageormedicalrecordalone,thusyieldbetter3Themarketofmedicalimageanalysissoftwareisforecastedtoreach4.5billionUSAin2024:http://www.grandviewresearch.com/industry-analysis3informedtreatmentdecisions.OneoftheaspectofCADthathasbeenreceivingattentionfromresearchersisitspredictivecapabilityofbiomedicalvariablesthatevolveintime.Inthisstudy,weparticularlyconsideradiseasenamedAbdominalAorticAneurysms(AAAs),whichisalocalenlargementoftheaortacomingfromthehearttocarrybloodtotheabdominalbody.Anundetectedaneurysmwillkeepgrowinguntilthepointofrupture.BecauserisksfromopensurgeryorendovascularrepairoutweightheriskofAAArupture,surgicaltreatmentsarenotrecommendedwithAAAslessthan5.5cmindiameter[85].SincemonitoringthegrowthofAAAtodecidewhenasurgeryisnecessaryiscriticalforthetreatmentofthedisease,apredictivemodeltopredictthegrowthhasbeenheavilyinvestigated.1.2.1BackgroundTopredictthetemporalgrowthofthe3DshapeofanAAA,therehavebeenresearche˙ortstoadoptstatisticalregressiontechniquesinpredictingAAAshapegrowth.Ijazetal.[54]hasappliedasimilarapproachofspatio-temporalGaussianprocessregressiontoinferthegeometricalgrowthofaparameterizedAAA'ssurface.Additionally,thereisagrowthandremodeling(G&R)modelthatusesaFiniteElementMethod(FEM)-basedstress-mediateddiseaseprogression[27].Finally,surfaceparameterizationbasedmethodshavebeencommonlydeployedinapplicationsdealingwithAAAs[54,88].Thesurfaceparameterizationwithcross-sectionalcontoursandalongitudinallinearenaturallyfavoredduetothetubularformofanaorta[88].Ontheotherhand,besidethe3Dshape,therearealsovariousbiomechanicalanalysesusingothergeometricalfactors(e.g.,di˙erentdiametermeasurements[34,65],tortuosity[84],morphologicalparameters[91],presenceofthrombus[6,76,125],in˛uenceofthespine[27])havebeenproposedtoreliablypredicttheriskofrupture.Amongthosegeometricalfactors,maximumdiameterisacriticalcriterionforscreening,surveillanceandinterventiondecisionmaking[73].Inparticular,agroupofauthors[34]proposesamethodtomodelAAA's4geometricalshape,givenbyaseriesofmaximalspheresalongthecenterlinewithinthelesion.Thegeometricalmodelisplottedasa2Dpro˝lecurve(diametersversusaxialdirection)andisusedtoassesstheriskoflocalruptureofanaorta.Sweetingetal.[107]utilizeamultilevelmodelstomakepredictionsaboutthefuturesizeofaneurysmofindividualwithlongitudinalAAAscanningdata.Thehierarchicallineargrowthmodelutilizesazero-meanGaussiandistributedrandom-e˙ectstermtosimulatethegrowinge˙ectsofaneurysms.Additionally,thelinearandquadratichierarchicalgrowthmodelshavebeenalsoheavilyusedtomakepredictionsaboutthefuturesizeofaneurysms[11,26].1.3ContributionInthissection,wediscussanoverviewofthedistributionofthisdissertationintheorderofchapters.Inchapter2,weproposeapositionestimationmethodusinganomnidirectionalcamera.WepresentanapproachtobuildamapfromoptimallyselectedvisualfeaturesusingGPre-gression.First,wedescribehowweextractsomerobustpropertiesfromvisiondatacapturedbyanomnidirectionalcamera.Inparticular,wedescribehowdi˙erenttransformationsareappliedtothepanoramicimagetocalculateasetofimageproperties.Wethentransformthehighdimensionalvisiondatatoasetofuncorrelatedfeaturecandidates.AmultivariateGPregressionwithunknownhyperparametersisformulatedtoconnectthesetofselectedfeaturestotheircorrespondingsamplingpositions.AnempiricalBayesmethodusingapointestimateisusedtopredictthefeaturemap.Next,afeaturereductionapproachisdevel-opedusingthebackwardsequentialeliminationmethodsuchthatanoptimalsubsetofthefeaturesisselectedtominimizetheRootMeanSquareError(RMSE)andcompressthefea-turesize.Thee˙ectivenessoftheproposedalgorithmsisillustratedbyexperimentalresultsunderindoorandoutdoorconditions.Additionally,wecompareourresultswithanotherappearance-basedlocalizationmethodutilizingthebagofwords(BOW)algorithm[9].5Inchapter3,wefocusonbuildingadirectmappingfromfeaturesspacetothecoordinatesinsteadofsolvingforlocationsfromthefeaturemapviaMLE.Therefore,thetestphaseisexecutedinashortperiodoftimesinceourmethoddoesnotrequireanyimagematchingoroptimizationsteps.Secondly,ourmethodisrobusttovisualnoiseanddoesnotneedhighresolutionimages,whichallowsittobeapplicableinlowcostembeddedsystems.Forinstance,weuserelativelylowqualityandnoise-proneimagescapturedfromaregularwebcam(LogitechC270,Logitech,Newark,CA,U.S.A.),yetthesystemyieldsremarkablyaccurateresults.Furthermore,incontrasttoapproachin[99],ourproposedmethoddoesnotrequiredepthmeasurementsoftheSURFpoints.Finally,asthedimensionalitydecreasesinthedata,therequiredstorageinitsdatabasealsoreduces.Unliketheworkin[8]whenthequantityofdata(i.e.,numberofimagestobestored)isreducedbasedonremovingimagesthathavesimilarvisualfeatures,ourmethodeliminatesredundancyinthequalityofdata.Inaddition,itsapplicationisnotlimitedtolocalization,butisversatiletoothercomputervisionproblemssuchasobjectrecognition,movementtracking,etc.Inchapter4,weproposeamethodtomodeladynamicallychanging3DshapeofanAAAgivenitslongitudinalsurfacedataforitspredictioninfuturetime.Webrie˛ydiscussourDynamicalGaussianProcessImplicitSurface(DGPIS)modeltotackleourproblemasfollows.Ourapproachbuildsontheconceptofimplicitsurfaces[24].AnimplicitsurfacedescribestheshapeofanAAAbyafunctionthatmapscoordinatesofeachpointinthespacetoascalarvalue(z),whichmayindicatethepoint'srelativepositionwithrespecttothesurfaceoftheAAA,i.e.,inside(z<0),outside(z>0),oronthesurface(z=0).NotethattheAAAcanhaveanarbitrarilycomplexshape.Whenweconsiderdynamicallychangingshape(e.g.,AAA),theoutcomeoftheimplicitsurfacefunctionisatime-varyingrandom˝eldoverthe3Dspatialspace.Thesurfaceisvisiblewhilethe˝eldishiddenexceptthenoisymeasurementsatz=0.Inotherwords,thesurfaceisembeddedintherandom˝eld.Thus,thechangesinthesurfacearedrivenbythecorrespondingchangesinthehidden˝eld.We˝rstestimatethehidden˝eldfromthepointclouddatabyutilizingthespatio-temporal6Gaussianprocessregressionasanobservationprocessfromthepointcloudsdata.Wethenfurtherre˝nethespatio-temporal˝eldbyassumingthateachpointinthe˝eldfollowslineardynamicsintime,keymodelparametersofwhichwillbeestimatedviatheExpectation-Maximization(E-M)algorithm[101].Finally,there˝nedspatio-temporal˝eldallowsustoestimatetheembeddedsurface.There˝nedspatio-temporalrandom˝eldusinglineardynamicsprovidesawaytopredictafutureshapeoftheAAAanditsgrowinguncertaintyasthepredictiontimehorizonincreasesotherwisenotpossiblewiththeGaussianprocessregressiontechnique.ThemodelinthisstudycanbeviewedasasteptowardsaBayesianapproachthatwillbecapableofincorporatingvariousuncertainties,patient-speci˝cdata,andcomputationalmodelsforaneurysmgrowth.Inchapter5,weintroduceadeeplearningnetworktothesolvetheproblemofmaximumdiametercurvepredictionofanAAAbasedonalimiteddatasizecollectedfromrealpatients.Inparticular,weprovideacompleteframeworktoconnectthecomputationalmodelwithadeepbeliefnetwork.Computationalmodelshavebeenservingasaninvestigationtooltosimulatee˙ectsofexternalstructuretotheprogressionoftheAAA[28].Moreover,duetothesimpli˝edgeometry,thecomputationalmodelshavenotbeenusedas(orapartof)apredictivetool.Uptodatethecomputationalmodelanddeepstructurearedevelopedintwoseparatedandindependentparadigms.Inthisstudy,thecomputationalmodelprovidesdeterministicgrowingtrendwhilewhilethedeepstructurecanlearnthevariationinthedata.1.4PublicationInthissection,Iwouldliketopresentthelistofpublished(aswellaswillbepublished)journalsarticlesandconferenceproceedingsthatarehighlyrelatedtothisdissertation.71.4.1Journalarticles(J1)HuanN.Do,J.ChoiandS.ofmaximumdiametersofAbdom-inalAorticAneurysmsbasedonDeepBeliefNetworkandProbabilisticCollocationMetho(submissionpending).(J2)HuanN.DoandJ.Choi,earance-basedLocalizationusingGroupLASSOre-IEEETransactionsonVehicularTechnology,(submitted).(J3)HuanN.Do,A.Ijaz,H.Gharahi,B.Zambrano,J.Choi,W.LeeandS.Baek,tionofAbdominalAorticAneurysmsusingGaussianProcessImplicitIEEETransactionsonBiomedicalEngineering,(submitted).(J4)HuanN.Do,M.Jadaliha,J.ChoiandC.Y.Lim,eatureselectionforpositionestimationusinganomnidirectionalImageandVisionComputing,vol.39,pp.1-9,2015.(J5)HuanN.Do,M.Jadaliha,M.TemelandJ.Choi,ullyBayesianFieldSLAMusingGaussianMarkovRandomFAsianJournalofControl,vol.18,Issue4,2015.1.4.2Conferenceproceedings(C1)HuanN.Do,JongeunChoi,andSeungikBaek."PredictionofAbdominalAorticAneurysmshapeevolutionusingGaussianprocessimplicitsurfaces."SummerBiome-chanics,BioengineeringandBiotransportConference(SB3C),2016.July2016,Na-tionalHarbor,Maryland,USA.(C2)HuanN.Do,JongeunChoi,andChaeYoungLim."VisualfeatureselectionforGP-basedlocalizationusinganomnidirectionalcamera."AmericanControlConference(ACC),IEEE,2015.(PDF)8(C3)HuanN.Do,J.Choi,C.Y.LimandT.Maiti,"Appearance-basedlocalizationusingGroupLASSOregressionwithanindoorexperiment",IEEEInternationalConferenceonAdvancedIntelligentMechatronics(AIM),2015.(C4)HuanN.Do,J.Choi,C.Y.LimandT.Maiti,"Appearance-basedoutdoorlocaliza-tionusinggroupLASSOregression",DynamicsSystemandControl(DSCC),2015.9Chapter2FeatureselectioninfeaturespaceMinimizinglevelsoflocationuncertaintiesinsensornetworksorroboticsensorsisimportantforregressionproblems,e.g.,predictionofenvironmental˝elds[14,55].Localizationofarobotrelativetoitsenvironmentusingvisioninformation(i.e.,appearance-basedlocaliza-tion)hasreceivedextensiveattentionoverpastfewdecadesfromtheroboticandcomputervisioncommunities[7,19].Vision-basedrobotpositioningmayinvolvetwosteps.The˝rststepinvolveslearningsomepropertiesofvisiondata(features)withrespecttothespatialpositionwhereobservationismade,so-calledmapping.Thesecondstepisto˝ndthebestmatchforthenewspatialpositioncorrespondingtothenewlyobservedfeatures,so-calledmatching.Themappingfromthesevisualfeaturestothedomainoftheassociatedspatialpositionishighlynonlinearandsensitivetothetypeofselectedfeatures.Inmostcases,itisverydi˚culttoderivethemapanalytically.Thefeaturesshallvaryasmuchaspossi-bleoverthespatialdomainwhilevaryingassmallaspossibleforagivenpositionoverthedisturbance.Forexample,theyshouldbeinsensitivetochangesinilluminationandpartialobstruction.Motivatedbytheaforementionedsituations,weconsidertheproblemofselectingfeaturesfromtheoriginalfeaturesetinordertoimprovethelocalizationperformanceofarobot.Thecentralassumptionwhenusingafeatureselectiontechniqueisthattheoriginalfeature10setcontainsmanyredundantorirrelevantfeatures.Tofacilitatefurtherdiscussion,letusconsideracon˝gurationwheretheinputvectorXistherobotpositionandtheoutputfeaturevectorYisthecollectionofextractedfeaturesfromthevisiondata.We˝rstbuildafeaturemapFatarobotlocationXsuchthatF(X)=Y.Inordertoreducepositionestimationerror,theidealsubsetisde˝nedasfollows.Yopt=argmin^YkXF1(^Y)k2;where^YisavectorthatconsistsoftheselectedentriesoftheoriginalvectorY.Howeverwithahighcardinalityoftheoriginalfeatureset,theoptimalsolutionreliesonthecombinatorialoptimizationwhichisnotfeasible.OneexampleofthefunctionF(X)canbethemutualinformationcriterion,inwhichFandF1couldbechosenasfollows:F(X)=argmaxYI(X;Y);F1(Y)=argmaxXI(X;Y);whereI(X;Y)=RRP(X;Y)logP(X;Y)P(X)P(Y)isthemutualinformationofXandY.Notethat,inthecasewhereP(X)andP(Y)areconstantthenF(X)obtainedbymaximizingthelog-likelihoodfunction.Guoetal.[39]showbyusingmutualinformation,onecanachievearecognitionratehigherthan90%whilejustusing0:61%offeaturespaceforaclassi˝cationproblem.However,theapproachbasedonmutualinformationcouldsu˙erfromitscomputationalcomplexity[86].Therecentresearche˙ortsthatarecloselyrelatedtoourproblemaresummarizedasfollows.Thelocationforasetofimagefeaturesfromnewobservationsisinferredbycom-paringnewfeatureswiththecalculatedmap[12,78,79].In[116],aneuralnetworkisusedtolearnthemappingbetweenimagefeaturesandrobotmovements.Similarly,thereexistse˙ortonautomatically˝ndingthetransformationthatmaximizesthemutualinformationbetweentworandomvariables[115].UsingGaussianprocess(GP)regression,theauthors11Figure2.1(a)and(b)showthewrappedomnidirectionalimageandtheunwrappedpanoramicimage,respectively.(c)and(d)showthereducedsizegrayscaleimageandthetwo-dimensionalFFTmagnitudeplot,respectively.of[12,96]presente˙ectiveapproachestobuildamapfromasparsesetofnoisyobservationstakenfromknownlocationsusinganomnidirectionalcamera.Whiletheselectionofvisualfeaturesforsuchapplicationsdeterminestheultimateperformanceofthealgorithms,suchatopichasnotbeeninvestigatedtodate.Therefore,buildingonBrook'sapproach[12]ourworkexpandsitmoreonthefeatureextractionandselectioninordertoimprovethequalityoflocalization.ABayesianpointofviewistakentomakethemapusingaGPframework.2.1ImagefeaturesConventionalvideocameraswithprojectivelenshaverestricted˝eldsofview.Withdi˙erentmirrors,360panoramicviewscanbeachievedinasingleimage[29].Inthisstudy,tomakelocalizationinsensitivetotheheadingangle,anomni-directionalcameraisusedtocapturea360viewfromtheenvironmentofarobot.Beforeanomnidirectionalimageisprocessed,itis˝rstunwrapped.Whenitcomesfromthecamera,theimageisanonlinearmappingofa360panoramicviewontoadonutshape.12Recoveringthepanoramicviewfromthewrappedviewrequiresthereversemappingofpixelsfromthewrappedviewontoapanoramicview[17,78].Figs.2.1-(a)and2.1-(b)showthewrappedomnidirectionalimageandtheunwrappedpanoramicimage,respectively.Wewillusethenotationy[i]generallyforalltypesofimagepropertiesthatwillbeextractedfromimagei.Inparticular,wewillusetheFFTcoe˚cients,thehistogram,andtheSteerablePyramid(SP)decomposition[103]asimageproperties[70].Thesefeaturetypesandtheirproperties(indicatedbyy[i])arebrie˛yexplainedasfollows.FFT(128):ThefastFouriertransform(FFT)isappliedtothepanoramicimagetocalcu-lateasetofimagepropertiesy.ForasquareimageofsizeNN,thetwo-dimensionalFFTisgivenbyF[i](ˆ;l)=N1Xa=0N1Xb=0f[i](a;b)ej2ˇ(ˆaN+lbN);wheref[i]isthei-thtwo-dimensionalrealizedimage,andjistheimaginaryunit.TouseFFT,weconvertpanoramiccolorimagestograyscale128128pixelimages,i.e.,[f(a;b)].Fig.2.1-(c)and(d)showthereducedsizegrayscaleimageanditstwo-dimensionalFFTmagnitudeplot,respectively.Ofteninimageprocessing,onlythemagnitudeoftheFouriertransformedimageisutilized,asitcontainsmostofthein-formationofthegeometricstructureofthespatialdomainimage[81].Additionally,themagnitudeoftheFouriertransformedpanoramicimageisnota˙ectedbytherotationinyawangle.In[78],itwasshownthatthe˝rst15componentsofFFTcarryenoughinformationtocorrectlymatchapairofimages.Wespecifythe˝rst64FFTcompo-nentsofeachaxis,e.g.,y[i]=fF[i](1;0);;F[i](64;0);F[i](0;1);;F[i](0;64)gtobeour128-dimensionalimagepropertiesoftheFFTfeatures.Histogram(156):Theimagehistogram[40]isatypeofahistogramthatactsasagraph-icalrepresentationofthetonaldistributioninadigitalimage.Thenumberofpixelsineachtonalbinofthehistogramfortheimageisusedasaimagepropertyfromthe13histogram.Thus,thenumberofdi˙erenttonalbins,(whichis156)correspondstothenumberofimagepropertiesfromthehistogramoftheimage.SP(72):TheSteerablePyramid(SP)[103]isamulti-scalewaveletdecompositioninwhichtheimageislinearlydecomposedintoscaleandorientationsubbands,andthentheband-pass˝ltersareappliedtoeachsubbandindividually.Usingthemethodfrom[12],animageisdecomposedby4scaleand6orientations,whichyields24subbands.Eachsubbandisrepresentedbythreevalues,viz.,theaverage˝ltersresponsesfromtop,middle,andbottomoftheimagesuchthatwehave72imagepropertiesfortheSPdecomposition.Themulti-scalewaveletdecompositionisalsousedwidelybyappearance-basedplacerecognitionmethods[12,110].SURF(64):TheSpeeded-UpRobustFeatures(SURF)[45]isapowerfulscale-androtation-invariantthatutilizesHaarwaveletresponsestoproducea64dimensionaldescriptorvectorforpointsofinterestinanimage.Furthermore,theSURFfeatureofeachpointofinterestiscalculatedlocallybasedontheneighborhoodregionaroundit.Ingeneral,speci˝cimageprocessingtogenerateoriginalfeatureswilla˙ecttheoverallperformanceofthelocalization.Thesefeaturesarerobusttochangesintheyawangleofthevehicle,whichresultsinhorizontalshiftsofthepixelsofthepanoramicimages.Additionally,imagesareconvertedintogray-scaleforalltypesoffeaturessincethegray-scaleimagesarelesslikelytobea˙ectedbyillumination[59].ThepresenceofmovingobjectsandocclusionsaretreatedbymodelingimagefeaturesasGaussianprocessesviaverticalvariabilityandmeasurementnoise,respectively.2.2Gaussianprocess(GP)modelWeproposeamultivariateGPasamodelforthecollectionofimagefeatures.AGPde˝nesadistributionoveraspaceoffunctionsanditiscompletelyspeci˝edbyitsmeanfunction14andcovariancefunction.Wedenotethaty[i]ˆ:=yˆs[i]2Risthei-threalizationoftheˆ-thimagepropertyands[i]2Sistheassociatedpositionwheretherealizationoccurs.HereSdenotesthesurveillanceregion,whichisacompactset.Then,theaccumulativeimagepropertiesyisarandomvectorde˝nedbyy=yT1;;yTmT2Rnm,andyˆ=y[1]ˆ;;y[n]ˆ2Rncontainsnrealizationsoftheˆ-thimageproperty.WeassumethattheaccumulativeimagepropertiescanbemodeledbyamultivariateGP,i.e.y˘GP;,where:Sn!Rmnand:Sn!Rmnmnarethemeanfunctionandthecovariancefunction,respectively.However,thesizeandmultivariatenatureofthedataleadtocomputationalchallengesinimplementingtheframework.Formodelswithmultivariateoutput,acommonpracticeistospecifyaseparablecovari-ancestructurefortheGPfore˚cientcomputation.Forexample,Higdon[46]calibratedaGPsimulatorwiththehighdimensionalmultivariateoutput,usingprincipalcomponentstoreducethedimensionality.Followingsuchmodelreductiontechniques,wetransformthevectorytoavectorzsuchthatitselementsfzˆjˆ2mg,wherem=f1;;mgarei.i.d.Thestatisticsofycanbecomputedfromthelearningdataset.y=1nnXi=1y[i];y=1n1nXi=1jjy[i]yjj2:Thesingularvaluedecomposition(SVD)ofyisafactorizationoftheformy=USUT,whereUisarealunitarymatrixandSisarectangulardiagonalmatrixwithnonnegativerealnumbersonthediagonal.Insummary,thetransformationwillbeperformedbythefollowingformula.z[i]=S1=2UT(y[i]y):(2.1)Fromnowon,weassumethatweappliedthetransformationgivenby(2.1)tothevisualdata.Hence,wehavethezero-meanmultivariateGP:z(s)˘GP(0;K(s;s0)),whichconsistsofmultiplescalarGPsthatareindependentofeachother.152.2.1Theˆ-thrandom˝eldInthissubsection,weonlyconsidertheˆ-thrandom˝eld(visualfeature).Otherscalarrandom˝eldscanbetreatedinthesameway.Arandomvectorx,whichhasamul-tivariatenormaldistributionofmeanvectorandcovariancematrix,isdenotedbyx˘N(.Thecollectionofnrealizedvaluesoftheˆ-thrandom˝eldisdenotedbyzˆ:=(z[1]ˆ;;z[n]ˆ)T2Rn,wherez[i]ˆ:=zˆ(s[i])isthei-threalizationoftheˆ-thrandom˝eldands[i]=(s[i]1;s[i]2)2SˆR2istheassociatedpositionwheretherealizationoccurs.Wethenhavezˆ(s)˘N(0;ˆ);whereˆ2Rnnisthecovariancematrix.Thei;j-thelementofˆisde˝nedas[ij]ˆ=Cov(z[i]ˆ;z[j]ˆ):Inthisdissertation,weconsiderthesquaredexponentialcovariancefunction[90]de˝nedas[ij]ˆ=˙2f;ˆexp 122X`=1(s[i]`s[j]`)2˙2`;ˆ!:(2.2)Ingeneral,themeanandthecovariancefunctionsofaGPcanbeestimatedaprioribymaximizingthelikelihoodfunction[123].ThepriordistributionofzˆisgivenbyN(0;ˆ).Anoisecorruptedmeasurement~z[i]ˆatitscorrespondinglocations[i]isde˝nedasfollows.~z[i]ˆ=z[i]ˆ+[i]ˆ;(2.3)wherethemeasurementerrorsf[i]ˆgareassumedtobeanindependentandidenticallydis-tributed(i.i.d.)Gaussianwhitenoise,i.e.,[i]ˆi:i:d:˘N(0;˙2).Thus,wehavethat~zˆ˘N(0;Rˆ);whereRˆ=ˆ+˙2I.Thelog-likelihoodfunctionisde˝nedbyL;ˆ=12~zTˆR1ˆ~zˆ12logdetRˆn2log2ˇ;(2.4)16wherenisthesizeof~zˆ.Thehyperparametervectoroftheˆ-thrandom˝eldisde˝nedasˆ=(˙f;ˆ;˙;˙1;ˆ;˙2;ˆ)2R4>0.Usingthelikelihoodfunctionin(2.4)thehyperparametervectorcanbecomputedbytheMLestimatorˆ=argmaxL;ˆ;(2.5)whichwillbepluggedinpredictionasinanempiricalBayesway.Allparametersarelearnedsimultaneously.Ifnopriorinformationisgiven,thenthemaximumaposterioriprobability(MAP)estimatorisequaltotheMLestimator[123].InaGP,every˝nitecollectionofrandomvariableshasamultivariatenormaldistribution.Considerarealizedvalueoftheˆ-thrandom˝eldz?ˆbeingtakenfromtheassociatedlocations?.TheprobabilitydistributionP(z?ˆjs?;s;~zˆ)isanormaldistributionwiththefollowingmeanandvariance.ˆ(s?)=CTˆR1ˆ~zˆ;˙2ˆ(s?)=˙2f;ˆCTˆR1ˆCˆ;(2.6)wherethecovarianceCˆ:=Cov(z?ˆ;zˆ)2R1nisde˝nedsimilarto(2.2).Inordertoestimatelocations?,usingtheMAPestimator,weneedtocomputeP(s?j~z?ˆ;s;~zˆ),wherethenoisyobservation~z?ˆisthesummationoftherealizedvaluesoftherandom˝eldz?ˆandanoiseprocess.P(s?j~z?ˆ;s;~zˆ)=P(~z?ˆjs?;s;~zˆ)P(s?js;~zˆ)P(~z?ˆjs;~zˆ):(2.7)AMAPestimatorgiventhecollectionofobservations~zˆisamodeoftheposteriordistribution.s?ˆ=argmaxs?2SP(s?j~z?ˆ;s;~zˆ):(2.8)IfP(s?js;~zˆ)andP(~z?ˆjs;~zˆ)areuniformprobabilities,thentheMAPestimatorisequal17totheMLestimator,givenbys?ˆ=argmaxs?2SLˆ(s?);(2.9)wheretheˆ-thlog-likelihoodfunction,i.e.,Lˆ(s?),isde˝nedasfollows.Lˆ(s?)=12j~z?ˆˆ(s?)j2˙2+˙2ˆ(s?)+log˙2+˙2ˆ(s?)+log2ˇ:(2.10)2.3LocalizationandfeatureselectionLetbethecollectionofindicesthatareassociatedtothemultiplescalarrandom˝elds(ofthemultivariateGP).Providedthatallscalarrandom˝elds(ofthemultivariateGP)areindependentofeachother,wethenobtainacomputationallye˚cientMLestimateofthelocationgiventheobservationsofallscalarrandom˝eldsf~zˆjˆ2gasfollows.s?=argmaxs?2SXˆ2Lˆ(s?);(2.11)whereLˆ(s?)istheˆ-thlog-likelihoodfunctionasgivenin(2.10).Inthisdissertation,abackwardsequentialeliminationtechnique[61]isusedforthemodelselection.Itismainlyusedinsettingswherethegoalisprediction,andonewantstoestimatehowaccuratelyapredictivemodelwillperforminpractice.Tothisend,wedividethedatasetintotwosegments:oneusedtolearnortraintheGPmodelandtheotherusedtovalidatethemodel.TheRMSEisusedtomeasuretheperformanceofGPmodels.Itisde˝nedbythefollowingequation.RMSE=vuut1ncncXi=1s[i]cs?2;(2.12)18wherekkistheEuclideannormofavector.Inthecasethat=;,wede˝nethefollowing.RMSE(;)=vuut1ncncXi=1s[i]cmedian(sc)2;wheremedian()isthemedianofarandomvector.Assumethatm=f1;;mgisthesetofallfeatures.Dupuisetal.[25]reportedthatthebackwardsequentialeliminationoutperformstheforwardsequentialselection.Thus,weuseabackwardsequentialeliminationalgorithmasfollows.`1=`argminˆ2`RMSE`ˆ);8`2m;(2.13)where`ˆ=fpjp2`;p6=ˆg.Finallyasubsetoffeaturesisselectedasfollows.opt=argmin1;;mRMSE:(2.14)TheoptimumsubsetopthastheminimumRMSEamongf1;;mg.ThemappingandmatchingstepsoftheproposedapproachesinthisdisesrtationaresummarizedinAlgorithm1andAlgorithm2,respectively.2.4IndoorandoutdoorexperimentsInthissection,wedemonstratethee˙ectivenessoftheproposedlocalizationalgorithmswithexperimentsusingdi˙erentimagefeatureswediscussed.Wereportresultsontwodi˙erentdatasetscollectedindoors(Case1)andoutdoors(Case2).19Algorithm1learningmapsfromasparsesetofpanoramicimagesobservedinknownlocationsInput:#1.trainingdatasetincludesasetofpanoramicimagescapturedfromknownspatialsites,Output:#1.alineartransformationfromimagepropertiestouncorrelatedvisualfeatures,#2.theestimatedhyperparameter,theestimatedmeanandtheestimatedvariancefunctionofeachindependentvisualfeature,1:extractimagepropertiesy[i]intheavailablelearningdataset.2:useSVDtomakeasetofuncorrelatedvisualfeaturesz[i]using(2.1)3:foreachindependentvisualfeatureestimatehyperparametersusing(2.5)4:computethemeanfunctionandvariancefunctionforeachofindependentfeaturesusing(2.6)5:chooseoptimalsubsetofvisualfeaturesusing(2.14)toeliminatesomeofthevisualfeaturesthatareworthlessforthelocalizationgoal.Algorithm2localizationpredictiveinferenceusinglearnedmapofvisualfeatures.Input:#1.alineartransformationfromimagepropertiestouncorrelatedvisualfeatures,#2.theestimatedhyperparameterandtheestimatedmeanandvariancefunctionofselectedvisualfeatures,Output:#1.positionofnewlycapturedimages.1:capturenewimagesandobtainimagepropertiesy?.2:computetheselectedvisualfeaturesz?using(2.1)3:computethelikelihoodfunctionofselectedfeaturesˆ2optoverpossiblesamplingposi-tionsusing(2.10)4:determinetheestimatedpositions?optusing(2.11)2.4.1ExperimentalsetupsInCase1,theKogetopanoramalenswasusedtocapture360images.Intotal,207pairsofexactsamplingpositionswererecordedmanuallyandcorrespondingcapturedpanoramicimagesonaregularlattice(72:7m2)werecollected.InCase2,weuseavisionandGPSdataacquisitioncircuitwhichconsistsofanArduinomicrocontroller(ArduinoMEGAboard,OpenSourceHardwareplatform,Italy),aXsensGPSunit(MTi-G-700,XsensTechnologiesB.V.,Netherlands),aRaspberryPimicrocon-troller(RaspberryPimodelB+,RaspberryPiFoundation,UnitedKingdom)andawebcam(LogitechHDwebcamC310,Logitech,Newark,CA,U.S.A.)gluedtoa360lens(KogetoPanoramicDotOpticLens,Kogeto,U.S.A.).Thedataacquisitioncircuitwassecuredinside20Figure2.2OutdoortrajectorycollectedfromaGPSunit.thevehiclewhiletheomni-directionalcamerawas˝xedontheroofofthevehicle.Thevehiclewasdriventhroughthesurveillancearea(Fig.2.2).ThesurroundingsceneswererecordedbytheRaspberryPiunitwhilethetruthlocationsmeasuredbytheXsensGPSunitwerestoredontheArduinomicrocontroller.Wecollected378datapoints,ona6186meterareaonthecampusofMichiganStateUniversity,EastLansing,MI,U.S.A(seeFig.2.2).Figs.2.3andFig.2.4showsthesetupsforCase1andCase2,respectively.Thedatasetsaredividedinto50%learning,25%backwardsequentialelimination(orvalidation)and25%testingdatasubsets.ThelearningdatasetisusedtoestimatethemeanfunctionsandthehyperparametersforthecovariancefunctionstobuildGPmodels.Thevalidationdatasetisusedtoselectthebestfeaturesinordertominimizethelocalization21Figure2.3DotiPhonePanoramalens(left)andtheindoorenvironment(right)forCase1.estimationRMSEandcompressthefeature.Afterthetrainingandfeatureselection,weevaluatetheperformanceoftheselectedmodelusingthetestingdataset,whichwasnotusedfortrainingorfeatureselection.Toanalyzeourresultsinastatisticallymeaningfulway,wecalculatetheBayesianInfor-mationCriteria(BIC)indexforthemodelwithallfeaturesandtheonewithonlyselectedfeaturesinadditiontotheRMSE.TheBICisacriterionformodelselectionbasedontheloglikelihoodwithapenaltyonthenumberofparameterstopenalizeover-˝tting.ThemodelwithasmallerBICindexislesslikelytobeover-˝tted[64].2.4.2LearningofGPmodelsinanempiricalBayesapproachAsillustrativeexamplesforthecaseofutilizingFFTfeaturesoflength128,weapplytheproposedalgorithmtobothdatasets.Thevarianceoftherandom˝eld˙2f,thespatialbandwidth˙2`;ˆ,andthenoisevariance˙2areestimatedforeachfeatureindependently.Thus,fortheFFTcase,1284=512hyperparametersneedtobeestimatedintotalforeachexperimentalsetup.ThehyperparametersforCase2areestimatedinthesamemanner.22Figure2.4(a)Dataacquisitioncircuit,(b)panoramiccameraand(c)vehicleusedinCase2.23The3Dplotsofthemeansandvariancesofthe˝rstthreeGPmodelsforthecaseof128FFTfeaturesareshowninFig.2.5.Tostudythee˙ectoftheturningangleofthevehicle(ortheyawangle)onthefeatures,werunthealgorithmwithanotherdatasetinwhichthecollectedpanoramicimagesarepre-processedsothattheheadingofthepanoramicimageiskeptconstantusingtheyawanglesfromtheGPSunit,denotedas(˝xedangle)inTable2.1.AllinferentialalgorithmsareimplementedusingMatlabR2013a(TheMathWorksInc.,Natick,MA,U.S.A.)onaPC(3.2GHzInteli7Processor).2.4.3LocalizationutilizingtheBagofWords(BOW)WealsocomparetheGP-basedapproachwithalocalizationschemebasedontheBOW.Tohaveafaircomparison,wefeedanidenticaldatasettobothofthemethods.WeutilizetheSURFastheimagedescriptorforourBOW.Wede˝nethenotationy[i]asasetofSURFpointsextractedfromimagei.NoticethatthenumberofSURFpointsvariesfordi˙erentimages.TheregionaroundeachSURFpointisrepresentedbyadescriptorvectorof64length.TheSURFpointsfromthewholedatasetareaccumulatedandputintothek-meansclustering[41].Eachcentroidisde˝nedasacodewordandthecollectionofcentroidsisde˝nedasthecodebook.EachSURFpointismappedintotheindexofthenearestcentroidinthecodebook.Therefore,weobtainahistogramofcodewordsforeachimagethatindicatestheappearancefrequencyofallcodewordsintheimage.Lastly,thetestsetisclassi˝edbyapplyingthek-nearest-neighborclassi˝er[60]basedonthehistogramofcodewords.Wesubsample25%ofthedatatobethetestset(thesametestsubsetusedinTable2.1),whichisassociatedwithanewlyde˝nedlabelset,i.e.,IT:=f1;;nTg.Thelabelofeachtestdatapointsisassignedtothenon-testdatapointswithina5meterradiuswithrespecttos(seeFig.2.6).Suchrelabelednon-testdatapointsareusedfortrainingtheBOW.SincetheBOWismostlyusedforclassi˝cationsuchasidenticalscenesrecognition[9],wede˝ne24Figure2.5GPmodelsforeachof˝rstthreeFFTfeaturesfromtheoutdoordataset.The˝rstrowshowsthemeansandthesecondrowshowsthevariancesoftheGPmodels.25Figure2.6TrainingdatasetassignmentfortheBOW.Thetestpoints,thetrainingpoints(withnewlabels),andthe5mradiiareplottedinbluedots,reddiamonds,andbluecircles,respectively.Thetrainingpointsthatdonotbelongtoanytestgroupsareeliminated.thelocalizationerrortocomputetheRMSEasfollows.Letst(i)2Sbethelocationofthetestpointiforalli2IT.Leth?(i)bethepredictedlabelforthetestdatapointi.Thenwede˝netheerrorattestpointiasfollows.errori=8><>:kst(i)st(h?(i))kifi6=h?(i);0ifi=h?(i);(2.15)foralli2IT.262.4.4ExperimentalresultsOurmethodoverdi˙erentfeatures:Theindoorandoutdoorperformancesunderdi˙erentimagefeaturesaresummarizedinTable2.1andTable2.2,respectively.Weconsiderthreedi˙erenttypesofappearance-basedfeaturessuchastheFFT[117],thehistogram[40],andtheSPdecomposition[103].WecalculatetheRMSEofourlocalizationestimationfromthemodelonthesamevalidationsetandonaseparatedtestset,denotedbyandinthetables,respectively.Tocomparethereductioninthenumberoffeatures,weusethecompressionratio[95].Thecompressionratioisde˝nedas:Compressionratio:=numberoforiginalfeaturesnumberofselectedfeatures:Table2.1andTable2.2showtheappearance-basedfeaturetype(column1),thetotalnumberoffeatures(column2),theoptimumnumberoffeaturesalongwiththecompressionratioonthevalidationset(column3),thelocalizationRMSEobtainedusingthetotalnumberoffeatures(column4),andthelocalizationRMSEfromtheoptimumnumberoffeaturesselectedbythebackwardsequentialelimination(denotedasE")implementedonthetestset(column5),andthelocalizationRMSEfromvalidationset(column6),themaximumlocalizationerror,i.e.,emaxtakenoverthetestset(column7),theBICindexesofthemodelusingthetotalnumberoffeatures(column8),andtheBICindexesfromtheoptimumnumberoffeatures(column9).ForCase2,theFFTandSPfeaturesaretestedwiththedataintwosituationswhentheyawanglesarevaryingandwhentheyare˝xed(denotedas(˝xed)inTable2.2)togaugethee˙ectofyawangles.Performanceamongfeatures:ForCase1,theSPshowsthelowestlocalizationRMSEwith4.8compressionratio.ForCase2,thehistogramshowsthelowestlocalizationRMSEwith2.7compressionratio.ThepredictedtrajectorythatutilizesthehistogramforCase2isshowninFig.2.7.Forallexperiments,BICindexesbeforeandafterthefeatureselection27#offeaturesRMSE(meters)emaxBICindexfeaturetotaloptAllBE(meters)AllopttypeVTTVT103103FFT12877(1.7)1.481.680.947.216.29.5FFT(noisy)12820(6.4)1.781.890.617.116.72.2Hist1569(17.3)1.691.710.557.118.52.2Hist(noisy)15650(3.1)1.581.641.146.020.56.4SP7215(4.8)0.670.850.273.422.24.1SP(noisy)7262(1.2)1.631.450.595.89.17.9Table2.1ThelocalizationperformanceforCase1#offeaturesRMSE(meters)emaxBICindexfeaturetotaloptAllBE(meters)AllopttypeVTTVT103103FFT12841(3.1)14.510.94.358.528.48.1FFT(˝xed)12825(5.12)13.713.64.856.728.44.7Hist15658(2.7)7.56.93.742.433.811.4SP7235(2.1)21.0818.727.8663.415.67.1SP(˝xed)7228(2.6)14.5214.7413.1951.915.65.5Table2.2ThelocalizationperformanceforCase2FeaturetypeNumberoffeaturesRMSE(meters)totalOptimumGPBOWun˝xed˝xedun˝xed˝xedun˝xed˝xedFFT128412510.9713.68--Histogram15658-6.91---SP72352818.7214.74--SURF-----23.1522.07Table2.3ThelocalizationperformancecomparisonbetweentheproposedapproachandtheBOWinCase2.TheRMSEisfromthetestset.28showthesigni˝cantimprovementthatmakestheselectedmodellesslikelysusceptibletoover-˝tting.FromtheRMSEonthetestdataset,allfeaturetypesseemtoberobusttothevaryingyawangles.E˙ectoflocalizationnoise:Toinvestigatethee˙ectofnoisysamplingpositionsonthemethod,weadded˝ctitiouslocalizationnoisegeneratedbyaGaussianwhitenoiseprocesswithstandarddeviationof0.3048meters(i.e.,1foot)tothesampledlocationsofCase1,whichisdenotedby(noisy)inTable.2.1.Asexpected,theresultsshowdegradationwhennoisysamplingpositionsareusedduetothesamplinguncertaintyintheGPlearningandpredictionprocesses.ComparisonofCases1and2:NotethatsamplingpositionsofCase1(non-noisydata)wererecordedexactlywhilethoseofCase2werenoisyduetotheuncertaintyintheGPSunit.Ontheotherhand,itisclearthatCase2hasthelargerRMSEduetothelargerscaleofthesurveillancesitecomparedtoCase1.Together,theresultsofCase1areshowntooutperformthoseofCase2.ComparisonwiththeBOW:WecomparetheperformancebetweenourproposedGP-basedapproachandtheBOWinTable2.3.Table2.3showsthefeaturetypes(column1),thetotalnumberoffeatures(column2),theoptimumselectednumberoffeaturesfromtheangle-varyingdataset(column3)andthe˝xedangledataset(column4),thelocalizationRMSEoftheGP-basedmethodfromtheangle-varyingdataset(column5)andthe˝xedangledataset(column6),thelocalizationRMSEoftheBOWfromtheangle-varying(column7)andthe˝xedangle(column8)datasets.SincetheperformanceoftheBOWhighlydependsontheclusteringresults,weruntheBOWwithdi˙erentsizesofclusters,andtheonethatyieldsthehighestclassi˝cationpercentage(80-90%)ischosentocalculatetheRMSEusingtheerrorde˝nedin(2.15).Asdiscussed,thechangeinyawangledoesnot29showsigni˝cante˙ectontheSURF.Table2.3showsthatourapproachoutperformstheBOW.Insummary,weachievesigni˝cantreductioninthenumberoffeatureswhileimprovingtheRMSEwhenappliedtothevalidationset.Furthermore,wemaintainapproximatelythesameRMSElevelswhenappliedtoanewtestdataset.30Figure2.7PredictionresultforCase2withthehistogram.ThetestpathandthepredictionareplottedovertheGoogleMapimageinreddiamondsandgreendots,respectively.31Chapter3FeatureselectioninspatialspaceRecallthatinchapter2,wehaveusedtheerrorebetweenthelocationXandtheinversemappingofthereconstructedvisualfeaturevector^Y,i.e.,e=kXF1(^Y)k2.Then,thelocalizationisdoneby˝ndingthelocationXthatminimizestheerrore.Inthischapter,weapproachtheproblemintheoppositerespective:theerrorfunctionnowischangedtoe=kXF(Y)k2.Usingadirectmappingfromvisualfeaturetolocationhasbeenemergedrecentlyundertheterminologyearance-basedloRecentstudyhasshownthatappearance-basedlocalizationtechniquesaremorerobusttonoise,viewobstruction,andimagequalityinconsistencyincomparisonwithgeometricalmarkers[102].Localizationofarobotrelativetoitsenvironmentusingvisioninformationhasreceivedextensiveattentionoverpastfewdecadesfromtheroboticandcomputervisioncommunities[7,19].Sinceitprovidesmoreaccurateestimationofpositions,itsapplicationsarenotlimitedinlocalization.Forinstance,minimizinglevelsoflocationuncertaintiesinsensornetworksorroboticsensorsisimportantforregressionbasedpredictionofenvironmental˝elds[14,55].Theappearance-basedmethodisbasedonasupervisedlearningframeworkthatuti-lizescollectionsoflocationcoordinatesandvisualobservations.Fundamentally,modelsarelearnedfromalargenumberofimagesatthetrainingstep,thennewlycollectedimagesarerecognized[43].Visualfeatureshavebeenheavilyusedrecentlyinroboticapplications.32In[117],aneuralnetworkisusedtolearntheimplicitrelationshipbetweentheposedis-placementsofa6-DOFrobotandtheobservedvariationsinglobaldescriptorsoftheimagesuchasgeometricmomentsandFourierdescriptors.Insimilarstudies,gradientorientationhistograms[67]andlowdimensionalrepresentationofthevisiondata[110]areusedtolo-calizemobilerobots.In[83],analgorithmisdevelopedfornavigatingamobilerobotusingavisualpotential.Thevisualpotentialiscomputedfromtheimageappearancesequencecapturedbyacameramountedontherobot.Amethodforrecognizingscenecategoriesbycomparingthehistogramsoflocalfeaturesispresentedin[70].Withoutexplicitobjectmodels,byusingglobalcuesasindirectevidenceaboutthepresenceofanobject,theyconsistentlyachieveanimprovementoveranorderlessimagerepresentation.Additionally,theSURFfeaturehasbeenoftenusedforclassi˝cationtypeproblemssuchastopologicallocalization(imagematching)[112]andplacerecognition[111].Seetal.[99]investigatedacloselyrelatedproblembyusingtheego-motiontechniquetodeterminetheposeofcamerabasedontheSIFTfeatures[75].In[18],theauthorsdevelopedanx-rayvisionsystemtobuildamapofatotallyunseenterritorybyutilizingthewi˝signal.Whiletheselectionofvisualfeaturesforsuchapplicationsdeterminestheultimateperformanceofthealgorithms,thistopichasnotbeenfullyinvestigatedtodate.Inmostofthecases,utilizingthefullsetoffeaturesisunlikelytoyieldoptimaloutcomesduetothepresentofredundantfeatures.Additionally,anexhaustivesearchbyexecutingtaskswithallpossiblecombinationsoffeaturesiscomputationallyunfeasible.Therefore,selectingthemoste˙ectivefeaturesiscriticalintwocategories:(1)tomaximizetheper-formanceofthetasksand(2)toeliminateredundantfeatures.Forexample,inlocalizationtasks,theselectedfeaturesmustberobusttolocalnoiseandsensitivetodi˙erentlocationsoverthespatial˝eld.OneexampleofwidelyusedfeatureselectiontechniqueisthePrincipalcomponentanalysis(PCA)[58],whichisutilizedto˝ndthemappingfromimagefeaturestotherobotlocations[98].Thereareanumberofmethodstodeterminethesigni˝cantvari-ablesinPCA.Accordingto[87],themeritofthosemethodshighlydependsontheextent33ofcorrelationofvariables.Anotherexampleisreportedin[8],theauthorutilizedaiterativegraphtheoryalgorithmtoobtainaConnectedDominatingGraphtoreducethenumberofimagesneededtobestoredintrainingdataset,aswellasretainsalientSIFTfeatures.However,sincethosemethodsareunsupervisedlearning,theselectionprocessmightnotbeoptimizedforthelocalizationperformance.Thus,wefocusourinvestigationtodevelopafeatureselectionmethodbasedonasupervisedlearningframework.Therecentresearche˙ortsthatarecloselyrelatedtoourproblemaresummarizedasfollows.Thelocationforasetofimagefeaturesfromnewobservationsisinferredbycom-paringnewfeatureswiththecalculatedmap[12,78,79].In[116],aneuralnetworkisusedtolearnthemappingbetweenimagefeaturesandrobotmovements.Similarly,thereexistse˙ortonautomatically˝ndingthetransformationthatmaximizesthemutualinformationbetweentworandomvariables[115].We˝rstunwraptheomni-directionalimagesintopanoramiconesandextractthreetypesofvisualfeatures,whichareFastFourierTransform,colorhistogram,andSUFT.Wecon-catenatealldi˙erenttypesoffeaturestogetherandnormalizethefeaturevectorstoeliminatedominancyinvalueamongdi˙erentfeatures.Then,wetreattherobot'scoordinatesasmul-tivariatetargetsandtrainthegroupLASSO[82]regressionmodelonthetrainingdataset.Oncethemodelistrained,wefeedthenewlycapturedimageintothemodeltoobtainanestimatedposition.Next,wetreattheestimatedpositionasanoisyobservationanddeploya˝lteringestimator,e.g.,EKForPF,toremovethenoiseinthegroupLASSOregressionoutcomes.Preliminaryresultsforindoorandoutdoorexperimentswerereportedin[20]and[21],respectively.Theoverallstructureofthischapterisasfollows.WewillintroducethevisualfeaturesthatareusedinthisstudyinSection3.1.WewillthendiscusstheLASSOregressioninSection3.2andthegroupLASSOregressioninSection3.3.InSection3.4,wewillshowhowtointegratetheEKFintotheLASSObasedlocalizationperformance.ExperimentresultsarediscussedinSection3.5.Fornotations,weuseA[i;j]fortheentryofrowi,columnj,34A[i;:]forthei-throwofthematrixA,andN(0;˙2)asthenormaldistributionwithzeromeanandvariance˙2.3.1ImagefeaturesInspiredbyabiologicalobservationthatinsectsandarthropodsdeveloptheirnervoussystembasedonawideviewsenseofsight,recentlyomni-directionalcamerashavebeenheavilystudiedfornavigation[119].Incontrasttoconventionalcameraswithrestricted˝eldsofview,panoramiccamerascanprovidecompleteviewsofthesurroundingenvironmentinasingleimage.Inordertoextractusefulfeaturesfromanomni-directionalimage,we˝rstunwrapitasfollows.Therawimagecapturedbytheomni-directionalcameraisanonlinearmappingofa360panoramicviewontoadoughnutshape.Werecoverthepanoramicviewbyapplyingareversemappingofthepixelsfromthewrappedviewontoapanoramicview[17,78].Fromthepanoramicimages,weextractthreetypesofvisualfeatures.Inthischapter,wewillusethenotationxigenerallyforthecollectionofalltypesoftheimagefeaturesthatareextractedfromimagei.Inparticular,weusetheFastFourierTransform(FFT)coe˚cients,thehistogram,andSURFastheimagefeatures[70].ThesefeaturetypesandtheirpropertiesarediscussedinSection2.1.Fortheindoorenvironment,weutilizetheFFTandHistogram,sincetheyaremorerobusttothedynamicchangeofthesurroundingcomparedtotheSURF.Fortheoutdoorexperiment,sincealargeportionoftheimageiscoveredbyuniformtonalpixelsofthesky,theFFTandHistogramfeaturesarenotsensitivetothechangeinsurroundingscenes.TheSURFfeaturescapturethelocalpropertiesofpointsofinterestinanimageotherthantheglobalpropertiesliketheothertwo.Therefore,weusetheSURFfortheoutdoorexperiment.Featuresfromdi˙erenttypesareconcatenatedandnormalized.353.2TheLASSOSupposethatoneimageiscapturedforeverysamplingtimei.Thus,wehavethedata(xi;yi);i2f1;;Ngwherexi=[x[1]i;;x[p]i]Tandyiaretheinputvariablevectorandthetarget,respectively.De˝neX2RNpandy2RN1asthecollectionsofNobservations.Weassumethattheinputvectorandthetargetarestandardized.Let^b=[^b1;;^bp]Tbethelinear˝ttingestimatevectorthatprovidestheestimatedtargetvector^y,i.e.,^y=X^b:(3.1)Thevector^bcanbecomputedbytheleastsquaresminimizationas:^b=argminbNXi=1kyXbk22;(3.2)wherek:k2isthe`2norm.TheLASSOregressionintroducesapenaltyterminto(3.2)asfollows.^b=argminb 12NkyXbk22+pXi=1jbij!;(3.3)where>0isapenaltyparameter.Thelevelofshrinkageappliedtotheestimatedb,i.e.,thenumberofzeroentriesofb,isdeterminedby.Forinstance,Figure3.1-(a)showstheevolutionofentriesofthevector^bwithfrom0(allentriesarenon-zero)toa˝nalvalue(allentriesarezeros).Noticethatasincreasesfrom0,weremuneratetheleastsquaresestimationaccuracyforthesparsenessofvectorb.Thecollectionofvector^bthatiscorrespondingtodi˙erentvaluesofisobtainedfromthetrainingdataset.Then,itisappliedtothevalidationdatasettoformtheerrorcurve.AnexampleofanerrorcurveisshowninFigure3.1-(b).Finally,theoptimalischosenattheminimumoftheerrorcurve.Inthisstudy,weusetheRootMeanSquaredError(RMSE)tocomparetwotrajectoriessotheoptimalischosenattheminimumoftheRMSEcurve.36(a)(b)Figure3.1(a)Shrinkageofestimatevectorentrieswithrespecttodi˙erentvaluesof.(b)errorcurveofdi˙erentestimatevectorsbappliedonvalidationdataset.3.3ThegroupLASSOInSection3.2,theestimatedtargetyiisascalarquantity.However,sincethetwocoordinatesoftherobot'slocationneedtobeestimatedsimultaneously,weshowhowweextendtheLASSOregressiontoestimateamultivariatetarget,i.e.,thegroupLASSOregression[82].LetY2RNkasthecollectionofNtargetvectorsofkdimensionsyi.De˝ne^B2Rpkastheestimatematrixofpestimatevectorofkdimensionsthatestimatesthetarget^Y:^Y=X^B:(3.4)Notethatk=2inthisstudy,whichiscorrespondingtotwospatialcoordinates.Wede˝nethe`1=`2normas:kBk`1=`2=pXi=1 kXj=1B2ij!12=pXi=1kB[i;:]k2;(3.5)whereB[i;:]isthei-throwofthematrixB.Theestimatematrix^Bcanbecalculatedasfollows.^B=argminB12NkYXBk2F+kBk`1=`2;(3.6)37wherek:kFdenotestheFrobeniusnorm1.AccordingtoObozinskietal.[82],(3.6)isequiv-alenttothesecond-orderconeprogram(SOCP):^B=argminB;2Rp112NkYXBk2F+pPi=1i;subjecttokB[i;:]k2i:(3.7)De˝nevec(:)operatoras:vec(B)=266666664B[:;1]B[:;2]...B[:;k]377777775:Applyingthevec(:)operatortobothsidesof(3.7)yields:vec(^B)=argminvec(B); 12Nkvec(Y)(IX)vec(B)k22+pXi=1i!;subjecttokWivec(B)k2i;(3.8)whereistheKroneckerproduct[36]andWi2Rkkpistheweightmatrix2,i.e.,W1=[IkOkpk],W2=[OkIkOkp2k];.Notethatthe`1=`2regularizationpenalizesthenormoftherowsofestimatematrixBin(3.6),whicharecorrespondingtothetwocoordinates.Thusfeaturesthatarenotsigni˝cantforthelocalizationofbothcoordinatesaremorelikelytobeeliminated.1TheFrobeniusnormofamatrixAis:kAkF:=qPi;jA2i;j2IkandOkaretheidentityandzeromatricesofsizekk383.4TheextendedKalman˝lterInthissection,weshowhowtointegratetheEKFintothegroupLASSO-basedlocalizationasapost-processingstep.Forthesakeofsimplicity,wedescribeinthissectionthebicyclekinematicsmodel[16]ofafrontwheeldrivenvehiclefortheoutdoorexperiment.Fortheindoorexperiment,weutilizetheunicyclekinematicsmodelthate˙ectstheequation(3.9),andtherestoftheEKFshallstayunchanged.Weutilizethebicyclekinematicsforafrontwheeldrivenvehicle.Leti,uiandLbethefrontwheelangle,therearwheellinearspeedattimeiterationiandthewheelbase(distancebetweenthetwowheelaxes)ofthevehicle,respectively.De˝nexi2R3asthestatevectorthatincludesthepositionandtheheadingangleoftherobot,e.g.,xi=[q[1]iq[2]i i]T.Therefore,thestatetransitionequationofthevehiclecanbedescribedasfollows.xi+1=xi+i266664uicos iuisin itan(i)Lui377775+!i=f(xi;ui;i)+!i;(3.9)where!i˘N(0;!i)isthewhitenoiseonthelocationandorientation.Let^xi+1;Pi+1betheestimatedstateandthecovariancematrixoftherobotbeforetakingthepositionmeasurement,respectively.Assumethatlocalizationerrorsforx,yaxesandtheorientation areindependent,thus:Pi=266664˙21;i000˙22;i000˙2 i377775:(3.10)39Wehavethepredictionasfollows.^xi+1=f(xi;ui;i)Pi+1=rfPirfT+!i;(3.11)whererfistheJacobianmatrixofthefunctionfwithrespecttothestatevectorxi,e.g.,rfxi=26666410iuisin i01iuicos i001377775;(3.12)whereiisthesamplingperiod.Sincewedonottakethemeasurementofthevehicle'sheadingangle,theobservationhastheform:~qi=Mxi+ei;(3.13)whereM=264100010375;andei˘N(0;˙2eI).Notethat~qi=^yiin(3.4)sincewetaketheestimationofgroupLASSO-basedlocalizationasthenoisymeasurementfortheEKF,andtheobservationnoiselevel˙eissettobetheRMSEofthegroupLASSO-basedlocalization.Themeasurementupdatecanbecomputedasfollows.^xi+1=^xi+1+Ki+1(~qi+1M^xi+1)Pi+1=(IKi+1M)Pi+1;(3.14)wheretheKalmangainKi+1is:Ki+1=Pi+1MT(MPi+1MT+˙2eI)1:(3.15)40Thepredictionandupdatemeasurementcomputations,i.e.,equations(3.11)and(3.14),arerecursivelyexecutedinthelocalizationprocess.Notethatforthetwowheelsmobilerobot,(3.9)becomes:xi+1=xi+i266664uicos iuisin ii377775+!i;=f(xi;ui;i)+!i;whereiistherobot'sorientation.3.5Experimentalstudy3.5.1IndoorExperimentInthissection,weshowhowweimplementourproposedmethodinanindoorexperimentalset-up.ThemobilerobotandthetestingenvironmentisshowninFigure3.2-(a).Wecontroltherobotremotelyviawirelesscommunicationunitswhilethepanoramicvisionofthesurroundingsceneisrecorded.Inordertotrackthetruepositionsoftherobot,weutilizeanoverheadcamera(LogitechHDwebcamC910,Logitech,Newark,CA,U.S.A.)thatismountedontheceiling.Notethatthepositionsobtainedfromtheceilingcameraareonlyusedforevaluatingthemethod'sperformance.Insummary,wecollectthefollowingthreedatasetsallsampledat0:5Hz.‹Thecommandinputs,turnratesandcollapsedtimebetweensamplingsfui;i;ig.‹Thesurroundingscenerecordedinthepanoramicvision.‹Thepositionsofthemobilerobot(groundtruth).Wecomparethelocalizationresultsperformedbythethreefollowingtechniques:41‹Open-loopbasedlocalization:Wenaivelyapplytherecordedcommandinputstotherobot'skinematics,whichisdescribedin(3.9)toobtaintherobot'sposition.‹GroupLASSO-basedlocalization:Werandomlysamplewithoutreplacementthewholedatasetandcategorizethemintothreelabels:training,validationandtestingbycorrespondingpercentages:50%,25%and25%.Then,weapplythegroupLASSOregressiontothelabeleddataandcomparetheperformanceonthetestdatawithothertechniques.‹GroupLASSO-basedandEKFlocalization:WeapplytheEKFasapost-processingtothetestresultofthegroupLASSO-basedlocalizationasdescribedinSection3.4.‹GroupLASSO-basedandPFlocalization:WeapplytheParticleFilter(PF)[35]asapost-processingtothetestresultofthegroupLASSO-basedlocalization.Notethatfortheindoorexperiment,thetestdataisthelocationsthatarerandomlychosenfromtheoriginaltrajectory.Therefore,whenweruntheEKFandthePFtoestimatethetrajectoryoftherobot,weassumethattherobotdonottakemeasurementatthenon-testlocations,i.e.,estimationwithmissingobservations[104].Ifanobservationismade,theEKFexecutestheupdatestepby(3.14).Otherwise,itpredictsthestatevectorbyopen-loopkinematicsandpropagatesthecovariancematrixby(3.11).Similarly,forthePF,theweightsofthe˝lterdensityareupdatedonceanobservationismade.Otherwise,auniformprobabilityisassignedforallparticles[35].ToillustratetheestimationwithmissingobservationbytheEKF,Figure3.3showstheevolutionsofthe˝rstandsecondentriesofthediagonalofmatrixPi(P[1;1]andP[2;2])thatrepresenttheuncertaintiesinxandyaxes.Noticethatthevarianceofpositioncoordinatesdropswheneveranobservation,whichisindicatedbyaverticaldashedlineinFigure3.3,ismade.Thissimulatesapracticalsituationwherethelocalizationobservationsaremostlymissingandverysparseduetoline-of-sightorGPS-deniedenvironments.42(a)(b)Figure3.2(a)Indoorexperimentenvironmentandthezoomed-inpictureofthemobilerobotshownintheupperleftcorner.(b)OutdoorexperimentinthecampusofMichiganStateUniversityonGooglemap.InTable3.1wecomparethelocalizationresultsperformedbyopen-looplocalization,groupLASSO-basedlocalization,groupLASSO-basedwithEKFlocalizationandgroupLASSO-basedwithPFlocalizationwhicharedenotedasen-loLASSO+EKandrespectively.Figure3.4showstheevolutionoftheelementsofgroupLASSOestimatematrixB.EachrowofBconsistsoftwoelementsthatcorrespondafeaturetothelocationwithtwocoordinates.EachpairofelementsisplottedinthesamecolorinFigure3.4.Noticethatelementsfromapair(samecolors)arebotheithereliminatedorpreserved.Table3.1showstheRMSEsofthefourmethods.Theopen-looppredictionyieldstheworstperformance,sincethereisnofeedbackintheprediction,theerrorisaccumulated.ThegroupLASSO-basedlocalizationreduces50%3numberoffeaturesandyields29%lowerRMSEcomparedtotheopen-loopmethod.ByapplyingtheEKF,thegroupLASSO-based3Theelementsoftheestimatematrix^Bthataresmallerthan1103areconsideredtobe0.43Figure3.3PlotofP[1;1](dashedline)andP[2;2](solidline)foriterationsfrom20to80forthegroupLASSO-basedwithEKFlocalization.44MethodRMSENumberoffeaturesRuntime(sec)(meters)TrainTestIndoorOpen-loop0.7039--0.033GroupLASSO0.499327/602061.70.005GroupLASSO+EKF0.315827/602061.71:271GroupLASSO+PF(MC)(0:39;0:01)27/602061.70:532OutdoorOpen-loop7.32--0.686GroupLASSO22.1349/2001861.829.376GroupLASSO+EKF5.0849/2001861.829:388GroupLASSO+PF(MC)(11:87;2:93)49/2001861.829:543Table3.1LocalizationperformancecomparisonandEKFlocalizationresultsinthelowestRMSEwith55%reductioncomparedtotheopen-loopcase.Figure3.5showsthetruetrajectory(solidline),groupLASSO-basedlo-calization(squaredots),groupLASSOandEKFlocalization(reddotteddashedline),andgroupLASSOandPFlocalization(blackdottedline).Table3.1hasindicatedtheexcellentperformanceofourproposedmethod.3.5.2OutdoorExperimentInthissection,wedemonstratethee˙ectivenessofourproposedmethodusinganoutdoorexperimentalstudy.Figure3.6showsthedataacquisitioncircuitandthesurveillancevehicle.Wedrivethevehicle(AcuraRSX2003)inMichiganStateUniversitycampusandrecordthepanoramicsceneviaanomni-directionalcamerainstalledonthetopofthevehicle.Wecollectthreedatasets,whicharesampledat1Hz:(1)Theestimatedcontrolinputsandsamplingtimesfi;ui;ig,(2)thescenerecordedbytheomnidirectionalcamerathatisstoredintheRasp-berryPi(RaspberryPimodelB+,RaspberryPiFoundation,UnitedKingdom),(3)thepositionsofthevehiclethataretrackedbytheXsensGPS(MTi-G-700,XsensTechnologiesB.V.,Netherlands).Wecomparethelocalizationperformancesbasedonthefollowingthreetechniques:‹Open-loopbasedlocalization:Wenaivelyapplytheopen-looppredictionbyusing45Figure3.4Indoorexperiment:TheevolutionoftheentriesofestimatematrixBversusthepenalty.Eachpairoffeaturesthatassociatewithsamecoordinateisplottedinsamecolor.46Figure3.5Indoorexperiment:Thetruetrajectory(blacksolidline),groupLASSO(greensquares),groupLASSO+PF(dottedblackline)andgroupLASSO+EKF(reddotteddashedline)predictions.47Figure3.6(a)Dataacquisitioncircuit,(b)panoramiccameraand(c)vehicleequippedwiththecameraontop.48Figure3.7Thetraining(dashed)andtestingpaths(solidlines)areplottedinmeters.49thekinematicsdescribedin(3.9)andthecommandinputs.‹GroupLASSO-basedlocalization:Werecordtwoseparatedtrajectoriesfortrain-ingandtesting,asshowninFigure3.7.Theoriginaldataisdivided50-50fortrainingandvalidation.ThegroupLASSOregressionusesthetrainingdatatotraintheesti-matematrixB,thevalidationdatatocomputetheoptimalmatrix^Bandthenapplies^Btoestimatethetestdata^Y.‹GroupLASSOandEKFlocalization:WeapplytheEKFasapost-processingtothetestresultofthegroupLASSO-basedlocalizationasdescribedinSection3.4.‹GroupLASSOandPFlocalization:WeapplythePFasapost-processingtothetestresultofthegroupLASSO-basedlocalization.Forthisoutdoorexperiment,wealsousetheRMSEtoevaluatetheperformanceofthethreetechniquesdescribedabove.Table3.1showstheRMSEsofthethreemethods(column2)andthenumberofusedfeaturesovertheinitialtotal(column3).ThegroupLASSO-basedlocalizationreduces75:5%4ofthewholefeatures.ByapplyingtheEKF,thegroupLASSO-basedlocalizationwiththeEKFresultsinthebestRMSEwith30:6%reductioncomparedtotheopen-loopmethod.Figure3.8showsthetruetrajectory(blacksolidline),groupLASSO-basedlocal-ization(greensquaredots),groupLASSOandEKFlocalization(reddotteddashedline),andgroupLASSOandPFlocalization(blackdottedline).FundamentallytheestimationstepofthegroupLASSO-basedlocalizationshownin(3.4)isalinearregression.Thus,itsestimationerrorhasasmallerbiascomparedtotheopen-loopprediction,whichgraduallydeviatesfromthetruetrajectory,butexhibitshighvariance.Therefore,weutilizetheEKF(aswellasPF)asalow-pass˝ltertosmoothoftheestimatedtrajectorybyprojectingthenoisyoutputsofthegroupLASSO-basedlocalizationontothe4Theentriesoftheestimatematrix^Bthataresmallerthan1103areconsideredtobe0.50vehiclekinematics.Figure3.9-(a)showstheevolutionofthegroupLASSOregressionesti-matematrixB.Eachpairoffeaturesthatiscorrespondingtoonelocation(twocoordinates)isplottedinthesamecolor.TheoptimalBisplottedinFigure3.9-(b).Notethatthematrixissparsewithalargenumberofentrieshavingvaluesofzero.Forcomparison,wealsoimplementtheparticle˝lter(PF)[1]thattakesthegroupLASSOoutputasmeasurements.SincethePFinvolvesrandomlygenerationoftheparticles,adapt-ingtheevaluationcriteriafrom[1],weevaluatetheperformanceofthePFbytheMonteCarlo(MC)simulation[80]with100runs.ThestatisticsoftheMCarereportedinTa-ble.3.1withthemeanandstandardvariationdenotedby(˙)".TheestimatedpathbythegroupLASSOandPFlocalizationwithlowestRMSEamongtheMCsimulationsisplottedinblackdottedlineinFigure3.8.AsnapshotofthePFatsamplingtimet=50isshowninFigure3.10.Theweightdensitiesof800particlesareplottedingray-scalecoloreddots.Notethatthesumofweightdensitiesoverallparticlesequalsto1.ThePFestimatedlocationiscomputedbyaveragingthelocationsofallparticleswiththecorrespondingweightdensities.ThecomputationaltimeisreportedinTable.3.1inseconds.SincetheEKFandPFareappliedasthepost-processingafterthegroupLASSO,theirtestphasecomputationtimesarereportedasthesumofthegroupLASSOandtheadditionalrunningtimetoperformtheEKForPF.Generallythetrainphaserequires30minutesforbothindoorandoutdoordata,whilethetestphaseissigni˝cantlyshorter.ThepredictionprocessofthegroupLASSOincludesonematrixmultiplicationin(3.4)andthevisualfeaturesextraction.SincetheSURFfeatureinvolvesclustering,itrequiresmoretimetoextractcomparedtotheFFTandhistogram.Therefore,thetestphasecomputationaltimeintheoutdoorexperimentislongerthanintheindoorcase.51Figure3.8Outdoorexperiment:Thetruetrajectory(blacksolidline),groupLASSO(greensquares),groupLASSO+PF(blackdottedline),andgroupLASSO+EKF(reddotteddashedline)predictionsareplottedinmeters.52(a)(b)Figure3.9Outdoorexperiment:(a)TheevolutionofentriesoftheestimatematrixBversusthepenalty.(b)Overall200entriesoftheoptimalmatrixBareplottedinbluebars.Figure3.10ThesnapshotoftheParticleFilteratt=50:Thetruetrajectory(redsolidline),groupLASSO(greensquares)andgroupLASSO+PF(bluedashedline)predictionsareplottedinmeters.Eachparticleisplottedwiththecoloringrayscalecorrespondingtoitsprobabilityweight.Thesumoftheweightsofallparticlesis1.53Chapter4AbdominalAorticAneurysm'sshapepredictionTheaortaisamajorarteryinwhichbloodcirculatesthroughtheheart.Anaorticaneurysmisidenti˝edasanenlargementoftheaortagreaterthan50%ofthenormaldiameter.Thevastmajorityofaorticaneurysmsareintheabdominalregion,amongthese,over90%occurwithintheinfrarenalaorta[89][126].Theinfraneralaortaliesbetweentherenalbranchesandtheiliacbifurcation.Inthisregion,adiametergreaterthan3cmisconsideredasanAAA.Inmostofthecases,AAAshavenosymptomsandarefoundincidentally,butifonerupturesthepatientmortalityratecanbemorethan90%[62][63].BecauserisksfromopensurgeryorendovascularrepairoutweightheriskofAAArupture,surgicaltreatmentsarenotrecommendedwithAAAslessthan5.5cmindiameter[85].However,themaximumtransversediameterofanAAAisnotareliableindicatorofrupturepotentialonitsown.Therefore,variousbiomechanicalanalysesusinggeometricalfactors(e.g.,di˙erentdiametermeasurements[34,65],tortuosity[84],morphologicalparameters[91],presenceofthrombus[6,76,125],in˛uenceofthespine[27])havebeenproposedtoreliablypredicttheriskofrupture.Althoughrecentadvancesinbiomechanicshaveprovidedstate-of-the-artspatialestimatesofstressdistributionsofAAAs,butthesestudiesdonotaddresstheproblem54ofpredictingtheshapeatafuturetimeandarelimitedinassessingthetimeevolutionanduncertaintyquali˝cation.Forclinicaltreatmentsandrecommendations,apatient-speci˝cpredictivetoolisrequiredtoincorporatetheadvancesincomputationalmodeling.Thedevelopmentofsuchtoolrequiresamajorparadigm-shiftsinceclinicalmeasurementsareassociatedwithlimitedinformation,uncertaintyandincompletenessofthemodel.Inshort,thischaptertacklesthefollowingproblems:‹Data-drivendynamicgrowthmodeldevelopment:WedevelopadynamicmodeltosimulatetheAAA'sgrowththatistrainedonpatient'sspeci˝cdata.‹PredictionofAAAgeometricalchangesandtheirvalidation:Comparisonsofpredic-tionsandthetrue(notincludedinthelearningphase)scanimagesareprovidedtoevaluatetheaccuracyoftheproposedscheme.‹Predictionuncertaintyquanti˝cation:Thepoint-wisecon˝denceintervalisprovidedforthepredictedAAAsurfacealongwiththeestimationerror.‹Possibleutilityofthemethodology:Possibleutilityoftheproposedmethodisdiscussedfromhelpingdecisionmakingtofeatureextractionapplications.Tothebestofourknowledge,thisisthe˝rststudythatpredictsthe3DshapeoftheAAAgrowthusingavailable(patient-speci˝c)pointcloudsdatainastatisticalperspective,whichallowsuncertaintyquanti˝cationintheprediction.Standardnotationswillbeusedthroughoutthispaper.LetR;R0;R>0,andZbede˝nedasthesetsofreal,non-negativereal,positivereal,andintegernumbers,respectively.Indenotestheidentitymatrixofsizen.Forcolumnvectorsva2Ra,vb2Rb,andvc2Rc,col(va;vb;vc):=[vavbvc]2Ra+b+cstacksallvectorstocreateonecolumnvector,andkvakdenotestheEuclideannorm(orvector2-norm)ofva.LetE(z)andVar(z)denotetheexpectationandthevarianceofrandomvariablez,respectively.Arandomvectorz2Rq,whichisdistributedbyamultivariateGaussiandistributionofamean2Rqandacovariance2Rqq,isdenotedbyz˘N(.55Thechapterisorganizedasfollows.InSection4.1,wediscussthedataacquisitionthatisfollowedbythepredictionmodelinSection4.2.Section4.3describesourpost-processproceduresfortheestimatedpointcloud.GeometricalpredictionresultsfromourmethodologyareillustratedinSection4.4.Finally,wediscusstheresultsinSection4.5.4.1DataAdaptationInthiswork,weadaptthepointcloudsdatathatisreconstructedbytheworkreportedin[34].Inparticular,fromthelongitudinalpatient-speci˝cdatacomprises37computer-tomographyscanimagesoftheAAAsobtainedfrom7di˙erentpatients,Gharahietal.[34]describedthesegmentationprocessoftheoutersurfaceandimageregistration.Then,theyextractpointcloudsbyrandomlysamplingfromthesegmentedsurfaces.Inthiswork,weutilizethepointcloudsastheinputstoourmethod.PatientIDNumberGenderAgeTimeofScans(Years)ofScansP17Male68[0,1.07,5.76,6.70,7.68,8.64,9.10]P26Male66[0,1.02,2.04,2.94,3.94,5.85]P35Male54[0,1.06,2.07,3.07,3.54]P45Male62[0,0.63,1.85,2.87,3.84]P54Male73[0,0.27,0.74,1.57]P66Male70[0,2.12,2.58,3.28,3.99,4.31]P74Male54[0,1.11,2.15,3.20]Table4.1DemographicDataofPatientsfrom[34]Thecollectionofdatasets,whichareintheformofpointclouds,obtainedfrom7patientsisdenotedasfDi;`ji2I;`2Tg,whereI:=fP1;P2;P3;P4;P5;P6;P7gandT:=f1;;7g.IrepresentsthecollectionofpatientIDsandTcontainstheavailablescansofaparticularpatient.Forexample,thesecondscandataofpatientFisdenotedbyDF;2.Ademographicdataofpatientsthatisreportedin[34]isshownagaininTable4.1.Asshowninthetable,notallofthepatientshavesevendatapointsandtheremaining,non-existingscansaretakenasemptydatasetsintherepresentationabove.564.2ModelandmethodAHiddenMarkovmodel(HMM)isaspecialcaseofastatespacemodel,whichhasbeenstudiedextensivelyincontrolcommunity[31]andcomputerscience[93].Itisextraordinarilye˚cienttorepresentsequentialdatasuchasspeechrecognition,naturallanguagemodeling,andanalysisofbiologicalsequences[4].AstatespacemodelisastructurethatconsistsofaMarkovchainoflatent(hidden)variablesandobservations(visibleunits).Eachobservedunithasaconditionaldistributionconditionedonthecorrespondinglatentvariable.Whenthelatentvariablesarediscrete,wehaveaHiddenMarkovmodel[4].InspiredbytheHMM,weutilizeasimilargraphicstructure.Inourmodel,weconsideraperspectivethatateachdatapoint,weobservethepointcloud,whichhavedistributionconditionedonthepotential˝eld.Also,thestatisticsofthepotential˝eld(whichishiddenfromus)canbeinferredfromitsdistributioninthepast.Therefore,ourgeneralschemeisthatbasedontheobservedpointcloud,wereconstructthehiddenpotential˝eld.Then,weinferthestatisticsofthe˝eldatthefuturetime.Finally,weobtainthepredictedpointcloudfromthepredicted˝eld.ThedetailedstructureofourproposedmethodisshowninFig.4.1.4.2.1GaussianprocessregressionInthissection,webrie˛yreviewGaussianprocessregression.AGaussianprocessisformallyde˝nedasfollows[90].De˝nition1:AGaussianprocessisacollectionofrandomvariables,any˝nitenumberofwhichhaveajointGaussiandistribution.AGaussianprocessiscompletelyspeci˝edbyitsmeanandcovariancefunctions.Let˘2Q:=RTˆRddenotetheindexvector,where˘:=xTtTcontainsthesamplinglocationx2RˆRd1andthesamplingtimet2TˆR0.Foranillustrativepurpose,weconsideraGaussianprocessz(x)˘GP((˘);K(˘;˘0)):57Figure4.1Summaryofourproposedmethod:pointclouddataisinsertedintothespatio-temporalGaussianprocessaszero-valueobservationstogeneratethe˝eld.Then,thetem-poralevolutionofthe˝eldisinferredthroughthedynamicmodelin(4.8).Finally,the˝nalpredictioniscomputedbyutilizingtheKalmanFilterintheE-Malgorithm.Ingeneral,themean(:)andthecovariancefunctionsK(;)ofaGaussianprocesscanbeestimatedaprioribymaximizingthelikelihoodfunction[124].Suppose,wehavepnoisecorruptedobservationswithDe=(˘(i);z(i))ji=1;;p.Assumethat~z(i)=z(i)+(i);(4.1)where(i)isindependentandidenticallydistributed(i.i.d.)whiteGaussiannoisewithvari-ance˙2.˘isde˝nedas˘=col(˘(1);˘(2);:::;˘(p)).Thecollectionsoftherealizationsz=z(1);:::;z(p)T2Rpandtheobservations~z=~z(1);:::;~z(p)T2RphavetheGaussiandistributionsz˘N((˘);K(˘));~z˘N(˘);K(˘)+˙2Ip;whereK(˘)2RppisthecovariancematrixofzandisobtainedbyKij(˘)=K(˘(i);˘(j))andIp2Rppistheidentitymatrix.WecanpredictthevaluezoftheGaussianprocess58atapoint˘[90]aszjDe˘N(˘);˙2(˘);(4.2)wherethepredictivemeanE(zjDe)is(˘)=(˘)+kT(˘)K(˘)+˙2Ip1(~z(˘))(4.3)andthepredictivevarianceisgivenby˙2(˘)=Var(zjDe)=˙2kT(˘)K(˘)+˙2Ip1k(˘):(4.4)Herek(˘)2Rpisthecovariancematrixbetweenzandzobtainedbykj(˘)=K(˘(j);˘)and˙2=K(˘;˘)2Risthevarianceat˘.4.2.2DiscretizedGaussianprocessspaceWeinvestigatethe˝eldonagridthatisde˝nedbythediscretizationofthespace.LetSc:=[x[1]min;x[1]max][x[2]min;x[2]max][x[3]min;x[3]max]bethecontinuousspatialdomaininR3,wherethespatiallimitationsxmaxandxminaredeterminedbycoordinatesofpointcloudsinthetrainingdataset.Wediscretizedthe3-DcontinuousspaceintonspatialsitesS:=fs[1];;s[n]g2Rn3,wheren=h(x[1]maxx[1]min)h(x[2]maxx[2]min)h(x[3]maxx[3]min).hischosensuchthatn2Z>0.Thecollectionoftherealizedvalueoftheimplicitsurfaceonthelatticeisdenotedasz:=(z[1];;z[n])T,wherez[i]:=z(s[i]).Thepriordistributionofzischosensuchthatz˘N(1;0),where1isacolumnvectorof1and0isthepriorcovariancematrix,i.e.,[i;j]0:=K(z[i];z[j]).Recallthatwetakepnoisymeasurementwiththemodelasdescribedin(4.1).UsingtheGPregression,theposterior59distributionofzonthediscretelatticeSisaGaussiandistribution,i.e.,z˘N(;)wheretheposteriormeanvectorandthecovariancematrixarede˝nedas:=KTC1~z;=0KTC1K;(4.5)whereK:=K(~z;z)2RpnandC:=K(~z;~z)2Rpp.Inthisstudy,weconsideradiscretizedgridpointsSˆR3onthe3-Dspaceofthesize404040.Therangeofeachdimensionwillbechosensuchthatthegridcompletelycoversthepointclouddatafromallavailablescansofapatient.4.2.3Spatio-temporalGaussianprocessForpatienti,weusethecovariancefunctionintheformofanexponentialkernelfunctionasfollows.K(x;x0;t;t0)=˙2f;iexpjtt0j22˙2t;iexp 123Xˆ=1jx[ˆ]x0[ˆ]j2˙2ˆ;i!;(4.6)wherex=[x[1]x[2]x[3]]Tisthecoordinatesvectorinthe3-Dspace.Thehyper-parametervectori:=[˙f;i˙t;i˙1;i˙2;i˙3;i]Tconsistsofthefunctionbandwidth,timebandwidthandthreespatialbandwidths,respectively.Thehyper-parameterscanbedeterminedbymaximizingthelikelihoodfunction.Noticethatsincethetimebandwidthisobtainedinadata-drivenmanner,thevalueof˙t;icouldprovideaninsightoftheevolutionoftheAAAwithrespecttotime.Forinstance,smaller˙t;iimpliesthattheaneurysmchangesmorerapidlyandviceversa.Thespatio-temporalGaussianprocessregressionprovidesawaytocollectimplicitsurface˝eldobservationsasshowninFig.4.1.604.2.4ImplicitsurfacesInthissection,wedescribethede˝nitionofanimplicitsurface[24].Animplicitsurfacedescribesanobjectinaspacebymappingcoordinatesofalocationinthespaceontoascalarvalue.Thevaluewillindicateifthelocationbelongstothesurfaceoftheobject.Inparticular,letx2R3beacoordinatesvectorina3-Dspace.Then,de˝neafunctionf:R3!Rastheimplicitsurface(IS):f(x)8>>>><>>>>:=0;ifxonthesurface,>0;ifxoutsidetheobject,<0;ifxinsidetheobject.(4.7)Figure4.2Exampleofanestimated˝eld:crosssectionviewsofthe3D˝eldatthesameheightinz-axisareshownatdi˙erenttimest.Theon-surfacepoints(wherethe˝eldiszero)arelabeledinwhitesolidlines.ThegrowthoftheAAAintheradialdirectioncanbevisualizedfromtoptobottom˝gures.Forexample,Fig.4.2showsanestimated3D˝eldfrompatientP1atthreedi˙erenttimesusingthespatio-temporalGaussianprocessregression.Fromtoptobottom,the˝guresshowcrosssectionviewsofthe˝eldatthesameheightinz-axisatthreesamplingpoints:atthe˝rstscan,after5:76years,andafter8:68years.Thosepointsonthe˝eldthatareoutsideofthesurfacehavepositivevalues(brightestyellow)andthosethatarecompletelyinsidethesurfacehavevalueslessthan0(darkestblue).Asshowninthe˝gure,theradialgrowthoftheAAAisre˛exedbythespreadingoftheblueregionasthetimeelapses.Notethatapointcloudprovidesonlythepointsthatbelongtothesurface,i.e.,havevaluesofzerointhe61IS˝eld.Therefore,itimplieszero-valueobservations[24]forthespatio-temporalGaussianprocessregressiondiscussedinSection4.2.3.Toimplementthespatio-temporalGaussianprocess,weutilizeanumberofbuilt-infunctionsintheGPMLpackage[90].4.2.5DynamicimplicitsurfacemodelLetf(x;t)betheIS˝eldthatrepresentstheaortasurfaceatthesamplingtimetwiththedata(CTscans)availablefort=f1;;tg.Then,wecande˝nethetemporalgrowthmodeloftheIS˝eldasfollows.f(x;t)=f(x;t1)+t(A(x)+W(x))(4.8)whereA(x)isthegrowthrateoftheIS˝eldandW(x)isthezero-meanprocessnoise:W(x)i:i:d˘N(0;w).Weassumethegrowthrateistime-invariantforthelastseveralob-servations.tisthegapbetweentwosuccessivescantimes,i.e.,t:=˝t˝t1.Notethattheuncertaintyintheprocessincreaseslinearlywiththegapbetweenobservations,i.e.,tW(x)˘N(0;2tw).Weassumethattheinitial˝eldf(x;0)hasanormaldistributionwithmean0andcovariance0,i.e.,f(x;0)˘N(0;0).TheobservationsoftheIS˝eldismodeledastheevolutionofaspatio-temporalGP.Theposteriordistributionofthe˝eldcanbetreatedinanobservationmodelasfollows.y(x;t)=f(x;t)+V(x;t);(4.9)whereV(x;t)istheobservationnoise:V(x;t)i:i:d˘N(0;v(t)).Thenoisecovariancematrixv(t)canbeestimatedastheposteriorcovarianceoftheGP.WeutilizetheExpectation-Maximization(E-M)algorithmtoestimatew(x)andA(x).AdetailedderivationfortheKalmanFilterE-MalgorithmisprovidedinAppendix6.Inanutshell,wede˝nealikelihoodfunctionforthemodelconditionedonthedata.WethenderivetheexpectationofthelikelihoodfunctionwithrespecttoA(x)andw,which62isdenotedasA;w)(seeAppendix6).Hence,foreachiterationroftheE-MalgorithmtheoptimizedvaluesforA(x)andwaretheonesthatmaximizeA;w),whichcanbefoundfromthe˝rstderivativeofA;w)(seeAppendix6).SinceA;w)isacontinuousfunctionofAandw,byTheorem2in[120],A;w)willconvergetoastationaryvalue.Furthermore,sincethe˝rstderivativeofA;w)iscontinuous,(A;w)convergestoastationarypoint[120].Toillustratethispoint,Fig.4.3-(a)showstheevolutionofthedistancebetweenA(r)anditsequilibriumvalueA1intermsoftheFrobeniusnorm,i.e.,kA(r)A1kF,convergesto0after10iterations.ASimilarevolutionofwisshowninFig.4.3-(b).TheembeddedsurfacedistributionoftheIS˝eldisstraightforwardfromthepredictionstepoftheKalmanFilterasfollows.E[f(x;L)jD1:L1]=E[f(x;L1)jD1:L1]+tA(x);Cov(f(x;L)jD1:L1)=Cov(f(x;L1)jD1:L1)+w2t:(4.10)Notethatonecanalternativelyavoidadynamicmodelin(4.8)anddirectlyapplyaspatio-temporalGaussianprocesswithakernelfunctiondescribedin(4.6)toobtainaninferenceofthe˝eldinfuturetime[54].However,therearetwocriticalrationalsthatinte-gratingobservationregressionanddynamicmodelsoutperformstheaforementionedmethod.Firstofall,sincetheposteriorcovarianceofaGaussianprocessisboundedbypriorcovari-ance0in(4.5),theuncertaintyquanti˝cationwilleventuallyreachalimitasweincreasetheinferencetimetfurther.Incontrast,ourdynamicmodeldoesnotimposeanylimitontheposteriorcovarianceastheterm2tisnotboundedin(4.10),indicatingthefactthataswetrytoinferatafuturetime,theuncertaintyinthepredictionwillescalateac-cordingly.Secondly,naivelyapplyingtheGaussianprocessregression(aswellasanyotherregressionmodel)wouldoverlookthegrowingtrendofAAAs.ThisproblemisresolvedbyincorporatingthetermA(x)inthedynamicmodel(4.8).63Figure4.3Convergenceofthedistanceof(a)Aand(b)wtotheirequilibriumvalues(A1andw1),startedwithrespecttodi˙erentinitialvaluesofAandw.Therearetwomaincomputationalchallengesinourmethod.Firstofall,adrawbackoftheGaussianprocessregressioniscomputationalcomplexitywithrespecttothenumberofmeasurements.Itcanbeseenfrom(4.3)and(4.4)thatthecalculationofboththepredictivemeanandpredictivevariancerequirestheinversionofcovariancematriceswhosesizesdependonthenumberofobservationsp,i.e.,complexityisO(p3).Dependingonthedensityandthenumberofpointclouds,thesizeofpointsthatareobservedinourstudycanbeexceptionallylarge.ThesecondmajorsourceofcomputationalconsumptionistheinversionofcovariancematrixintheKalmanFilteralgorithm.TheKalmanFiltercovariancematrixhasadimensionofnnwithnisthenumberofspatialsitesthatisdiscussedinSection4.2.2.Therefore,theinversionhascomplexityofO(n3).Inthisstudy,weusen=64;000andforthosepatientswithlargetrainingdataset(morethan4datapoints),theE-Malgorithmiscomputationallyprohibitive.Toresolvetheseproblems,wedividethedensespatialgridSintosparsersub-gridsandapplyourmethodinadistributedmanner.The˝naloutcomeonSismergedfrompredictiveresultsonindividualsub-grids.Thedetailofthee˚cientKalmanFilteralgorithmisreportedinAppendixA.644.3Surfaceextractionandpost-processingIntheprevioussection,wehaveobtainedthestatisticsofpredictedIS˝eldthatisprovidedin(4.10).Inthissection,webrie˛yshowhowtoextractthepredictedsurfacefromthepredictedIS˝eld.NotethatthepredictedsurfaceisanembeddedmanifoldofthepredictedIS˝eldthatsatis˝es(4.7).However,duetotheerrorincomputationanduncertaintyintheGaussianprocessregression,achievingexactzerovaluesoftheIS˝eldforon-surfacepointstunedouttobenotpracticalfromourpreliminarystudy.Therefore,werelaxtherestrictedequalityconditionin(4.7)bysettinganear-zerothresholdofthe˝eldunderwhichthecorrespondingpointisconsideredtobeon-surface.ThethresholdofthetrainingphaseisdeterminedbyminimizingtheEuclidiandistancebetweentheestimatedandthetruepointclouds(seeAppendixB).To˝ndthethresholdofthetestphase,weutilizeafactthateachparticularvalueofthethresholdclassi˝esthewhole3DlatticeSintoaparticularspatialbinarypatternoftwocategories:on-surfaceando˙-surface.Therefore,weassignthetestphasethresholdtothevaluethatyieldsthemostsimilarspatialpatterntothetrainphasethreshold(seeAppendixB).4.4ExperimentalresultsInthissection,wearrangeanexperimentalstudytoillustratethee˙ectivenessofourap-proachinrealisticscenariosusingthereconstructedpointcloudsD(id;scans)thatareadaptedfrom[34].Tovalidateourapproach,thelastavailabledatapointforeachpatientD(id;L)isusedasthegroundtruthwhereasthepreviousdatapointsfD(id;j):j=f1;;L1ggofthatpatientareusedtotrainthemodel.Then,wecomparethepredictionresultswiththeexistingtruedatasetD(id;L)foreachofthe7patientsusingtheHausdor˙distance[53].Thus,wehave7di˙erentpatientspeci˝ccaseswithdi˙erentlongitudinallengthsandmor-phologicalpropertiesalongwithinter-scantimeinterval.Forexample,weusethelast665datapointsofpatientH,fDH;1;;DH;6g,fortraining,andpredictthemostcurrentstateoftheaneurysm,^DH;7.TheHausdor˙distanceisaquantitativemeasurebetweentwopointclouds.However,itdoesnotprovidestheinsightofthespatialshape.Inordertoobtainavisualestimationoftheprediction,weutilizethethePoissonReconstructionfunctioninMeshLab(NationalResearchCouncil,Rome,Italy)torenderthe3-Dstructureofthesurfaces.TheresultsareshowninFig.4.5.EachpairofestimatedandtruescansarelabeledasDi;jand^Di;j,respectively.PatientNumberHyperparameters˙tHausdor˙DistanceIDofScansi:=[˙t;i˙1;i˙2;i˙3;i](days)(Full)(Last-3)P17[722:96;13:69;12:82;29:75]722.9617.8616.16P26[1085:38;7:02;5:89;12:47]1085.3811.2811.53P35[683:49;3:07;3:15;7:41]683.498.326.26P45[835:32;5:52;5:04;11:85]835.329.812.86P54[475:18;6:09;5:63;13:28]475.189.85-P66[597:75;2:05;2:06;4:91]597.756.686.40P74[553:98;2:40;2:43;5:39]553.988.32-Table4.2HyperparametersandHausdor˙DistancewithdetailsofeachCaseTable4.2showsthecaseIDs(column1),patientIDs(column2),numberofavailablescans(column3),estimatedhyper-parametervectors(column4),estimatedtimebandwidth˙t(column5),andtheHausdor˙distances(column6and7).Foreachcase,theHausdor˙distancebetweenthepredictedandestimatedpointcloudsiscalculated.ThemeanandstandarddeviationoftheoverallHausdor˙distancesare10:30(mm)and3:64(mm).Thesmallstandarddeviationimplieshighprecisionofourapproach.4.4.1ConstantgrowthrateassumptionInwhatfollows,weinvestigatethee˙ectoftheassumptionontheconstantgrowthrateA(x)in(4.8).Weobservethatincreasingnumberofscansinthetrainingdataresultsindecreasingpredictionerrors,butonlyuptoacertainpoint.Then,theerrorstartstoincrease.This66Figure4.4Therelativetimesofthescanscomparedtothe˝rstscanareplottedforeachpatientindays.67iscontrarytothecommonexpectationthatlargeramountoftrainingdatayieldslowererrors.Asimilarinvestigationisreportedin[107]:thenumberofmeasurementsdoesnotapparentlye˙ecttheprecisionofthepredictedresults.Thisobservationmaybeexplainedbyanabruptincreaseintheenlargementrateoftheaneurysmcausedbyover-stretcheddamageofelastin[118].Totestthishypothesis,weconductaseparatestudyinwhichwelimitthetrainingdatasettothethreemostrecentdatapoints.InTable4.2,theresultsofthisstudyarereferredtoaswhereas,thelabelisusedwhenalltheavailabledatapiontsareusedfortraining.Thepredictedsurfacesandcon˝denceregionsareshowninFig.4.7andFig.4.8,respectively.TheHausdor˙distancesintheLcaseprovidesimilarresultsforthosepatientswithsmallaneurysms.Incontrast,thecaseyieldsdecreasingdistancesforthosepatientswhohavelargeraneurysms.4.4.2Uncertaintyquali˝cationOurapproachprovidesquanti˝cationofuncertaintyintheprediction,i.e.,con˝denceregions.Inordertocomputethecon˝denceregions,wesampletheIS˝eldsbyrealizingtheposterioriGaussiandistributionwiththeupdatedmeanvectorandcovariancematrixcomputedby(4.10)inaMonteCarlosimulationwith100runs.WethenapplythesameprocedureasdescribedinSection4.3toestimatetheAAAsurfacesfromthesampledIS˝elds.Withasu˚cientnumberofrealizations,thesetofsurfacesformsthecon˝denceregionoftheprediction.InFig.4.6,wevisualizethecon˝denceregionasfollows.First,wesetthetransparencyof2%foreachrealizedAAAsurfaceinthesurfacerenderingprocess.Wethenoverlaptheminspace.Therefore,spatialregionsthatappearbrighteraremorelikelytobeoccupiedbytheAAAsurface.Thetruepointcloudsareplottedinreddots.684.5DiscussionInthissection,wediscusstheresults,thepossibleutilitiesandlimitationsofourapproachalongwithfutureresearchdirections.PatientP6hasahighnumberofscanswithregularscanningtimesandshowstohavethelowestHausdor˙distance,i.e.,thehighestaccuracy.PatientP3yieldsthesecondlowestHausdor˙distance.DespitehavingthesamenumberofscansaspatientP6,patientP2showsaremarkablyhighererrorinprediction.Thiscouldbeduetoanunprecedentedlongertime(687days)betweenthelatestadjacentdatapointsforpatientP2,ascomparedtothelatestadjacentdatapointsofP6(116days)asshowninFig.4.4.WecompareourresultswiththoseinPowellsetal.[11,26]whomuselinearandquadratichierarchicalgrowthmodelspredictingthefuturesizeofaneurysms.Thehierarchicallineargrowthmodelutilizesazero-meanGaussiandistributedrandom-e˙ectstermtosimulatethegrowinge˙ectsofaneurysms.Thisapproach,however,overlooksincreasinggrowth,anditwasreportedinsomecasesthatthemodelspredictAAAdiametertodecreaseinafuturetime[107]whichisnotrealistic.Incontrast,thegrowinge˙ectinourmodelisobtainedbypatient-speci˝c˝ttingbasedonlongitudinaldata.Secondly,thehierarchicalmodelfocusesonlyonAAAmaximaldiametersanddoesnotprovidethegeometricalshapeanalysis.Ontheotherhand,ourapproachaddressesacompletetreatmentofAAAgeometricalshapeprediction.Theimplicitsurfaceissimilartothewell-knownlevelset[130](alsoknownasimplicitcontour)methodinthesensethatbothmethodsutilizetheevolutionofanimplicitscalarfunctiontosimulatethechangesoftheinterfacewherethescalarfunctioniszero.However,thelevelsetmethodreliesontheprior(observer's)knowledgeinanimagesuchasimageintensity,oranatomicalmodel,asadrivingforcetoevolvethefunctionandpropagatetheinterface.Thus,ithasnotbeenusedsofarasapredictivemodelasproposed.Comparedtotheoutcomesofotherpreviousworks,ourresultsyieldfollowinginnovativemerits.Firstofall,therehasbeenresearche˙orttopredictaneurysmgrowthandits69uncertaintymerelybasedonadata-drivenstatisticalframeworksuchasnon-linearregression[54].However,naivelyapplyingsuchregressionmodelsoverlooksacriticalfactaboutAAAsthatistheirunrelentingincreasesindiameters[68]byneglectingthedeterministicgrowthdynamics.Inthisstudy,weproposealineardynamicmodelthatsimulatesanAAA'sgrowth,whichwillbecalibratedbyaspeci˝cpatient'sdata.Secondly,ourmethodprovidesacompletepredictionofAAA'sgeometricalstructure,unlikerecentmethodsthatreducethedatadimensionssuchascenterlineparameterizationbasedapproaches[34].Moreover,eachtypeofparameterizationsisassociatedwithaparticulartopology,whichsigni˝cantlyrestrictsitsversatility[88].Toovercomethisdisadvantage,wedevelopourmodeltobefreeofgeometricalparameterization.4.5.1Decisionmakingviapredictionandcon˝denceregionsThemajorpossibleutilityofouralgorithmsisinhelpingcliniciansinconductingmedicaltreatmentofanAAA,e.g.,monitoring,opensurgeryorendovascularrepair,byprovidingaccesstoapredictedAAAatafuturetime.Moreover,theclinicianswillalsohaveaccesstothecon˝denceregionoftheprediction.Thiswillhelptheminmakingamoreinformeddecisionforthetreatmentbasedontheprediction.WecanobservefromFig.4.6thatthecon˝denceregioncoversthetrueAAAsurfacebetterforthecasewithlowerHausdor˙distance.Thus,clinicianscangaugethereliabilityofthepredictedAAA.NotethatstandardG&Rcomputationalmodelsdonotprovidetheuncertaintyintheirprediction[38,100,128].Therefore,ourmodelshowsansigni˝cantimprovementoverthem.704.5.2Estimatedhyper-parametersandmodelparametersasinfor-mativefeaturesNotethatbesidesthe˝nalprediction,thehyper-parametersestimatedbythespatio-temporalGaussianprocess,asshowninTable4.2,alsomayprovideinsightfulinformation.Thespatialbandwidthsf˙1;i;˙2;i;˙3;igin(4.6)provideinformationofspatialvariationofanAAA's3Dstructure.Forexample,ahighvaluefor˙1;iimpliesthattheAAAsurfacevariessmoothlywhereasalowervalueindicatesthatthesurfacehashighvarianceinthedirectionofthe˝rstaxis.Theestimatedspatialbandwidthsbymaximizingthelikelihoodfunctionareshownincolumn4inTable4.2.Noticethattheestimatedspatialbandwidthsforthedirectionofz-axis,i.e.,˙3;iarenormallylargerthanthoseforthedirectionsofxandyaxes.SincetheAAAhasatubularstructure,therearelessspatialvariationalongthez-axiscomparedtotheothertwocoordinates.Additionally,thetimebandwidth˙trepresentsthetemporalchanges.Smaller˙timpliesthattheAAAshapeislikelytochangemorerapidlywithrespecttotime.Theestimatedhyper-parametersandlinearmodelparametersmaybeviewedasfeaturesthatmayencodetheinformationabouttheevolutionofanAAA.Thehyper-parametersestimatedfortheregressionprovideauniquepatient-speci˝cfeaturevectorwhichmaycap-tureboththetemporalandspatialvariationpatternsoftheAAAsurface.Collectivefeaturevectorsobtainedfrommorepatientscouldbeusefulinbuildingaclassi˝cationmoduleca-pableofdetectingpatientswithimminentdangerofrupture[91].Inaddition,theestimatedlineargrowthrateA(x)providesinformationofthemigrationofthesurfaceina3Dspace.Inthiswork,weutilizethevalueofA(x)merelyforupdatingtheIS˝eld.However,ifweviewA(x)asascalar˝eldasshowninFig.4.9,itspartialderivativeswithrespecttospatialandtimemayprovideaninsightfulinformationaboutthemovingsurface.ThisanalysiscanbeobtainedbyapplyingatechniquethatissimilartoLucas-Kanade(LK)optical˛ow[10]asfollows.Assumethatwewouldliketocomputethevelocityofthemovementoftheon-surfacepointsx(t)overtimet,i.e.,dxdt.Sinceon-surfacepointshavezerovaluesofIS˝eld,71wehavef(x(t);t)=0,hence@f(x(t);t)@t=0.Applyingthechainrule:@f(x;t)@xdxdt+@f(x;t)@t=0(4.11)Notethat@f(x;t)@xisthespatialderivativeoftheIS˝eldattimetandcanbeestimatedby@E(f(x;t)jD1:t)@xwithE(f(x;t)jD1:t)givenin(4.10).Furthermore,@f(x;t)@tisthederivativeofthe˝eldwithrespecttotimeandcanbeapproximatedasA(x).Thus,from(4.11),wewillbeabletocomputethevelocityofthemovementoftheon-surfacepointsas:dxdt=A(x)@f(x;t)@x:4.5.3LimitationsandfutureresearchdirectionsThemethodisbasedonanempiricalBayesianmethod,hyper-parametersareobtaineda-prioribymaximizingthelikelihoodfunction[90]fortheIS˝eldobservations.Therefore,theuncertaintiesassociatedwiththehyper-parametersarenottakenintoaccount.IncontrasttotheempiricalBayesianframework,thefullyBayesianapproachmarginalizestheuncertaintiesinthehyper-parameterswithincreasedcomplexity.Hence,onefutureresearchdirectionistodevelopafullyBayesianversionofourproposedschemetakingintoaccountuncertaintiesinhyper-parameters[22].Therefore,asafutureworkweinvestigatetheimplicationofusingtheconstantgrowthrateA(x)inSection5.2,whichcouldbealimitation.Thetime-varyingorswitchingA(x)canbeexplainedasaresultofmechanicaldamageofelastinthatcausesthegrowthratetosuddenlyescalate.However,ourdynamicmodelassumesthatwiththreemostrecentscans,thegrowthrateA(x)isconstant.Thus,adisruptivegrowthduetoelastindamageisnotcapturedbyourdynamicmodel.Inotherwords,forthosecasesthatexperiencetheabruptincrement,themodelwithconstantA(x)trainedonthewholetrainingdata,underestimatesthegrowthrateafterthepointofelastindegradation.Therefore,thelineardynamicmodelin72(4.8)canbeimprovedbyestimatingtwogrowthrates,e.g.,A1(x)andA2(x)andaswitchingtime˝stocopewiththeswitchedgrowthrate.WehaveshownthatourmodelisabletoprovideareliablepredictionofAAAevolution,withoutincludingtheG&Rcomputationalmodel[121,127].However,inordertoextenditbeyondsurfacepredictiontopredicttherupturepotential(mechanicalstress,thee˙ectofthrombus,etc.),wehavetocombineourapproachwiththeG&Rcomputationalmodelsothatbothshapeevolutionandmechanicalstressarecombinedtoestimatetherupturepotential.ThecombinationofourapproachandtheG&Rcomputationalmodelwillbeacomputationallyandtheoreticallychallengingtaskgiventhecomputationalcomplexityofthemodelanditsunknownparameters.However,abiomechanics-basedcomputationmodelstructurewillprovideaconstraintinspaceandtime,whichwillhelpinreducingthesizeofthecon˝denceregionofthepredictedAAAatfuturetime.Therefore,ourfutureresearchwouldfocusontheincorporationofthecomputationalG&RmodelinourBayesianframework.73^DP1;7^DP2;6^DP3;5^DP4;5DP1;7DP2;6DP3;5DP4;5^DP5;4^DP6;6^DP7;4DP5;4DP6;6DP7;4Figure4.5Fullytrainedcaseusingallavailablelongitudinaldata:3-Drenderingfromtheestimated(^Di;j)andtrue(Di;j)pointclouds:Thepredictedandtruepointcloudsareusedtorenderthe3-DsurfacesusingthePoissonreconstructioninMeshlab.74(P1)(P2)(P3)(P4)(P5)(P6)(P7)Figure4.6Uncertaintyquanti˝cationfor7patientsforfullytrainedcase:Theimplicitsurfacesareregeneratedonthegridbyrealizingthemulti-variateGaussiandistributionwiththemeanvectorandcovariancematrixcomputedin(4.5).Then,weapplythesameproceduredescribedinSection4.3toestimatetheAAAsurfacesthatarecorrespondingtotherealizedimplicitsurfaces.EachrealizationoftheAAAsurfacefromtheposteriordistributionisplottedwith2%occupancy.Therefore,spatialregionsthatappearbrighteraremorelikelytobeoccupiedbytheAAAsurface.Thetruecloudpointsareplottedinreddots.75^DP1;4^DP2;4^DP3;4^DP4;4DP1;4DP2;4DP3;4DP4;4^DP6;4DP6;4Figure4.7Last-3trainedcaseusingthemostrecentthreelongitudinaldatapoints:3-Drenderingfromtheestimated(^Di;j)andtrue(Di;j)pointclouds:Thepredictedandtruepointcloudsareusedtorenderthe3-DsurfacesusingthePoissonreconstructioninMeshlab.NotethatweruntheLast-3trainedcaseforpatientP1,P2,P3,P4andP6,sinceothercasesalreadyhave4scansintotal.76(P1)(P2)(P3)(P4)(P6)Figure4.8Uncertaintyquanti˝cationfor5patientsforLast-3trainedcase:Theimplicitsurfacesareregeneratedonthegridbyrealizingthemulti-variateGaussiandistributionwiththemeanvectorandcovariancematrixcomputedin(4.5).Then,weapplythesameproceduredescribedinSection4.3toestimatetheAAAsurfacesthatarecorrespondingtotherealizedimplicitsurfaces.EachrealizationoftheAAAsurfacefromtheposteriordistributionisplottedwith2%occupancy.Therefore,spatialregionsthatappearbrighteraremorelikelytobeoccupiedbytheAAAsurface.Thetruecloudpointsareplottedinreddots.NotethatweruntheLast-3trainedcaseforpatientP1,P2,P3,P4andP6,sinceothercasesalreadyhave4scansintotal.77Figure4.9AnexampleofusageofA(x)asaninformativefeature.Twosuccessivelyshapes,namelypreviousandcurrentshapes,areplottedinblacksquaredandcircledpointcloudsinstandardizedunit,respectively.Additionally,theintersectionsofthemwiththeplanez=0areplottedinsolid(previous)anddashed(current)whitelinesonthez=0plane.Furthermore,thecross-sectionviewsoftheA(x)˝eldatz=0andz=1arecolorplottedonthetwoplanes.Themigrationofthesurfaceisshownclearlyinthedirectionindicatedbytheredarrow.Then,thevelocityofthemigrationintheindicateddirectioncanbecomputedby(4.11).78Chapter5AbdominalAorticAneurysm'smaximumdiameterspredictionAsdiscussedinchapter4,maximumdiameterisanimperativecriteriontodetermineforsurgicalplaningofAAAs.Onecommonwaytomonitorthemaximumdiameteristhemaximumdiametercurve(MDC),whichisthecollectionofmaximumdiametersalongthecenterlineofanAAA.Tobeconsideredasananeurysm,arelativecriterioncanbeusedsuchthattheenlargementoftheaortaisgreaterthan50%ofthenormaldiameter.Ontheotherhand,anabsolutecriterioncanalsobeused.Forexample,intheinfraneralaorta,whichistheregionliesbetweentherenalbranchesandtheiliacbifurcation,adiametergreaterthan3cmisconsideredasanAAA.Duetotheseveree˙ectsandhighmortalityrateofthedisease,massivedatabasesofbothheathyandpositivelydiagnosedhavebeencollectedoverthedecades.Whiletherehavebeenlarge-scalepopulation-basedscreeningstudies,suchasMulticenterAneurysmScreeningStudy(MASS)intheUK[108],analysismodelsthatisbasedonlongitudinaldatahasnotbeenproperlyinvestigated.Incontrasttopopulation-basedmodels,whenlongitudinalpatient-speci˝canalysismodelsarecarefullycalibratedbasedonindividualcasesthosearebeabletoaidphysiciansindetectinganeurysmsandmakingdecisions.Forclinical79treatmentsandrecommendations,apatient-speci˝cpredictivetoolisrequiredtoincorporatetheadvancesincomputationalmodelingandcomputationalcapacity.Thedevelopmentofsuchtoolrequiresamajorparadigm-shiftsinceclinicalmeasurementsareassociatedwithlimitedinformation,uncertaintyandincompletenessofthemodel.Inthisstudy,weadoptdeeplearningtotackletheproblem.Deeplearninganddeeparchitecturesingeneralhavebeentransforminganenormousnumberofresearchareas,majorityincomputervision[77]andnaturallanguageprocess-ing[15].Furthermore,deeplearningalsohasbeenheavilyappliedinriskpredictionbasedonelectronichealthrecord[13],imagelabeling[44],tra˚c˛owprediction[52],imageseg-mentation[74],medicalimagesegmentation[69],andmanyother˝elds.Deeparchitecturehasbeeninvestigatedsince1980[32]andhasbeenprovedtobemoree˙ectiveandrequireslessresourcecomparedtoashallowstructureofthesamesize,i.e.,samenumberofnodes.Themeritsofdeepstructurecomefromitsabilitytoreduceredundantworksbydistributingthetasksthroughitslayers[69].Forinstance,thelowlayerscanperformlowleveltaskslikegradientscomputationoredgedetectionwhilethehigherlayerscanperformclassi˝cationorregression.However,asthenetworksareconstructedindeeperlayers,thetrainingbecomesprohibitivelyslowduetotheproblemofanishinggradien[3].Inparticular,whentheerrorisback-propagatedfromtheoutputlayer,itismultipliedbythederivativesofacti-vationfunction,whichisnearzeroforthosesaturationnodes.Consequently,theerrorasthedrivenforceforthegradientdecentalgorithmisdramaticallydissipatedthatresultsinextremelyslowtrainingrateforthosenodesbehindthesaturatednode.Thetrainingproblemhadremaineduntil2007whenHintonproposedatwo-stagelearningscheme[49].Firstly,thenetworkistrainedinanunsupervisedandlayer-wisemannerintheformofarestrictedBoltzmannMachine(RBM),i.e.,pre-training.Then,thenetworkistrainedagainwithlabeleddata,i.e.,˝ne-tuning.However,itisshownthatthedeepstructureonlyyieldshighlevelofgeneralizationandlowtesterrorwhenitistrainedonalargeavailabletrainingset.Forinstance,LeCunetal.[71]utilizeaMNIST[72]datasetof80hand-writtennumberswith60;000samplestotraina3-layerdeepnetwork.Unfortunately,inmostofcasessuchlargedatasetofAAAisnotavailableortime-consumingmanualsegmentationisrequired.Thesimilarsituationexistsforothertypesofmedicaldataset.Thus,uptodate,theapplicationsofdeepstructureinmedicaldataarelimitedtomedicalimagesegmentation.Inthisstudy,weinvestigatetheuseofaDeepBeliefNetwork(DBN)tosolvetheproblemofpredictionofMDCatafuturetimeinaregressionframework.Oneofthemainobstacletoapplydeepstructuretosuchabiomedicalproblemistheshortageoftrainingdatasetduetovarietyofreasonssuchashighcostofdataacquisitionordisruptioninpatients'visitings.Tocopewiththeproblemofsmalllabeleddataset,weproposeamethodtouseasmallsizedataandgeneratealargearti˝cialdatathatrequiresashorttimeandlesscomputationalresource.Tothebestofourknowledge,thisisthe˝rste˙orttoadaptadeepstructureintheproblemofpredictionofAAA.InSection5.1,wedescribethetypesofdatathatourmodelbaseson.InSection5.3,wediscusshowweusethePCMmodeltogeneralourarti˝cialdataset.InSection5.4,wediscussourdeepnetworkstructureandtheoverallpredictionmodel.TheresultsareshowninSection5.5.5.1Data5.1.1RealpatientbasedreconstructeddataWhiletherearevariouswaystomeasurethemaximumdiameterofanAAA[66],weutilizetheresultsfromtheworkofGharahietal.[34].Themethodstartswitha3DpointcloudoftheAAAwall,thenamaximallyinscribedsphereisde˝nedasthelargestspherewithintheouterarterialwallsurface.TheinscribedsphereismovedfromthebottomtothetopofanAAA.Then,thetraceofthemovementofsuchsphere'scenterpointformsthecenterline.Finally,themaximumdiametersareplottedversusthecenterlineasshowninFigure5.1-(a).81PatientIDNumberGenderAgeTimeofScans(Years)ofScansP17Male68[0,1.07,5.76,6.70,7.68,8.64,9.10]P26Male66[0,1.02,2.04,2.94,3.94,5.85]P35Male54[0,1.06,2.07,3.07,3.54]P44Male73[0,0.27,0.74,1.57]P56Male70[0,2.12,2.58,3.28,3.99,4.31]P64Male54[0,1.11,2.15,3.20]Table5.1DemographicDataofPatientsadaptedfrom[34]Weadaptthecollectionofmaximumdiametercurvesfromtheauthorsof[34]asusethemastheinputtoourmethod.Accordingto[34],thelongitudinaldataiscollectedfrom6patientsintotal,thedemog-raphyofwhichisshownagaininTable5.1inthisdissertation.(a)(b)(c)Figure5.1Exampleof2Dpro˝lecurvesofmaximaldiametersoverthecenterlineobtainedfrom(a)realpatient,(b)theG&Rcomputationalmodel,and(c)thePCMapproximation.NotethatthetwoG&RandPCMcurvesinthisexamplearenotbasedontherealcenterline,sotheirunitsarenotincentimeters,butinspatialsiteunit.Thespatialsitecanbeseenasa1DmeshintheFEMcode.5.1.2Arti˝ciallygenerateddataG&Rcomputationalmodel:TheG&Rcomputationalmodel[27]isa˝niteelementmethod(FEM)basedvascularmodelthatsimulatesthemechanicalandgeometricalstateofanaorticaneurysmatagiventime.TheG&Rmodelutilizesindividualmicrostructuralpropertiesofdi˙erentconstituentstocomputethestress-stretchstate.IntheG&Rmodel,theaorta82isconsistofthreestress-bearingconstituents:elastin,collagen˝berfamilies,andvasoactivesmoothmusclecells.Eachconstituenthasitsownpropertiesandcontributiontothestrengthoftheartery'swall.TheG&Rmodelconnectstheconstituentstothestress-stretchstateofthearteryandcalculatestheevolutionofthosemicrostructuralpropertieswithastress-mediatedfeedbackapproach.Elastincontributesresilienceandelasticitytotheaortictissue;however,itdegeneratesovertimeandbeirreplaceable.Thedegenerationinelastincausesalocalizeddilationoftheaorta,leadingtotheweakeningofthewall.Consequently,itresultsinanincreaseofthediameterandthewallstressoftheaneurysm.Thedegenerationoftheelastinisspeci˝edbytheelastindamagefunctionisde˝nedasamodi˝edGaussianfunction:d(s)=kdexp(sd)2˙2d;(5.1)wheresisthecoordinatede˝nedonthecenterline.dandareestimatedforeachpa-tientbasedontheirdataastheyhavespeci˝ce˙ectsontheshapeofthedamagefunction;therefore,onthestress-stretchandgeometricalstateoftheAAA.Sincethemaximumaorticdiameterlocatesatacloseproximityofthemaximumdamage,dismostlikelytobeinthevicinityofthecenterlinelocationswherethelocalenlargementisthemostsevere.Weassignthevaluesof2or4to,whichisdeterminedbyminimizingtheerrorbetweenthesimulationandtherealdata.Additionally,kdisascalingfactor,i.e.,kd2[0;1),suchthataskdapproaches1thedegradationofelastinincreasesleadingtothedilatationoftheartery.Ontheotherhand,kd=0meansnodegradationoftheelastin,whichresultsintheretainoftheartery.˙disthestandarddeviationoftheGaussianfunction;hence,itde˝nesthewidthofthe2Dpro˝lecurveoftheaneurysm.Thesethreeparameterskd,˙d,andddirectlya˙ectthetimeevolutionoftheaneurysm;thus,eachuniquegroupofthethreeparametersyieldsanuniqueoutcomeoftheG&Rcode.OneexampleisshowninFigure5.1-(b).PCMapproximationmodel:OnecommondisadvantageoftheG&Rmodelisthatitis83extremelytime-andcomputationalresource-consuming.Therefore,togeneratealargedatasetforthedeepnetwork,itisnottheoptimaloption.Alternatively,weuseasmallnumberofG&RmodelsimulationstotrainthePCMandgeneratealargearti˝cialdatasetusingthePCM.ThedetailofthePCMisdiscussedinSection5.3andanexampleisshowninFigure5.1-(c).5.2PredictionofmaximumdiametercurveAssumethatwehaveanAAAevolvingandforeveryoneyear,weareabletoobtainanMDCfromthatparticularAAA.Letft;ibeoneMDCoftheAAAiatscantimetandacollectionofft;iforaspanoftprovidesusthetimelineevolutionoftheAAAi.Wede˝neourregressionproblemasfollows.Foreachdatapoint,wedeterminethefeaturevectorxitobethecollectionofthreemostrecentMDCs,andthepredictiontargetyiistheMDCforthenextscaninafuturetime:xi=[ft2;i;ft1;i;ft;i];yi=ft+1;i:IntheG&Rcode,thecenterlineisdiscretizedintoagridsizeof221,sothedimensionsofourdataandlabelare663and221,respectively.Givenasetofparameter=[kd;˙d;d]andatimespan,theG&Rmodel(aswellasthePCMapproximationcode)willproduceatimeseriesofMDCstosimulatethegrowthofAAAduringthetimespan,i.e.,anarti˝ciallygeneratedcollectionoffxi;yig.Next,wenormalizeeachdatapointandsavethenormalizationscaleasaseparateddataset.Then,thenormalizationscaledatasetisusedtopredictthescaleofthefutureMDCbyutilizingtheGaussianprocessregression.Ithasbeenshownthatitisdi˚culttolearnthevarianceforeachvisibleunit[47].Thus,bynormalizationprocess,wetransformtheoriginaldatatobethenewonewithzeromeanandunitvariance.Additionally,wenormalizethedata84Figure5.2Overalldiagramoftheproposedmethod.sincetheoutputregressionlayerutilizesthelogisticfunction,sotheoutcomeisintheformofprobability,i.e.,intherange[0;1].Then,thenormalizeddataispluggedintoaDeepBeliefNetworkwithaneuralnetwork(NN)regressionastheoutputlayer.Afterbeingpre-trainedwiththenormalizedarti˝cialdata,theNNis˝ne-tunedagainwiththerealdatasetbyback-propagation.Finally,thepredictedMDCisthentransformedbacktothenormalscaletobethe˝nalpredictionoutcome.TheoverallmethodisdepictedinFigure5.2.TocomparetwoMDCs,weusethestandardRootMeanSquaredError(RMSE).Predictionofnormalizationscale:asaforementioned,weutilizeGaussianprocessregres-siontopredictthenormalizationscaleofthefuturescan.ThedetailofGaussianprocessisdiscussedinSection4.2.1.Webrie˛ydescribeourmodelhere.Foradatapointi,letui=[max(ft2;i);max(ft1;i);max(ft;i)]andvi=max(ft+1;i)bethepredictorvectorandlabelvalue,respectively.Thus,wehaveaGaussianprocessv(u)˘GP((u);K(u;u0));85where(:)andK(:;:)arethemeanandcovariancefunctions,respectively.Thus,thepre-dictedvalueofvhasaGaussiandistributionv˘GP((u);˙2(u))withthemeanandvariancearecomputedin(4.3)and(4.4).5.3ProbabilisticCollocationMethodInthissection,weshowhowwemakeanapproximatedoutcomeoftheG&RusingthePCMmodel.5.3.1DeterministicinputConsidertheG&Rmodelthattakesthegroupofparameters=fkd;˙d;dgastheinputandproducesthesimulatedMDCyastheoutput:y=();(5.2)where(:)istheG&Rcomputationalcode.Duetothehighdemandofcomputationalresourceandtimeofthecode,weshallapproximate(:)byutilizingasetofNbasisfunctionsfgi()g,withi=1;;N,suchthat:^y=NXi=0igi();(5.3)whereNandiaretheorderoftheapproximationandregressioncoe˚cients.Fornow,weassumethatthesetoffunctionsfgi()gisknown.Theregressioncoe˚cientsfigcanbesolvedasfollows.Wede˝netheresidualbetweenthetruthandtheapproximationasfollows.R(fig;)=^y()y():(5.4)86ApplyingtheOrdinaryLeastSquaresestimationto(5.3),wecanstraightforwardlyobtainthattheoptimalsetofcoe˚cients^iasfollows:hgi();R(fig;)i=Zgi()R(fig;)=0;(5.5)wherei=1;;Nandh:;:irepresentsthedotproductbetweentwodeterministicfunc-tions.(5.5)canbesolvedbyusingasimilarideafromtheGaussianquadrature[106]byapproximatingtheintegralas:ZR(fig;)gi()'NXj=1vjR(fig;~j)gi(~j)=0;(5.6)wherevjand~jaretheweightsandabscissas,respectively.IftheweightsandthebasisfunctionsarechosensuchthatQi;jvjgi(~j)>0foralliandj,thesummationin(5.6)canbefurtherapproximatedas:R(fjg;~j)=0;j=0;;N:(5.7)Notethatthequadraturepoints~jarealsothecollocationpoints.(5.7)canbeusedto˝ndthecoe˚cientsfjgbyrunningthemodelatN+1di˙erentcollocationpointsandsolvingasystemofN+1equations.5.3.2StochasticinputSupposethattheinputnowisarandomvectorwithaknownprobabilitydensityfunction(PDF)ˇ().Thus,(5.5)istransformedintotheprobabilityspaceasfollows.Zˇ()R(fig;)gi()=0:87Similarly,withtheproperchoiceofvjandgi(),(5.7)becomes:ˇ()R(fjg;~j)=0;wherej=0;;N.SincethePDFfunctionˇ()isalwayspositive,(5.7)canstillbeusedto˝ndthecoe˚cientsinthestochasticcase.5.3.3SelectionofbasicfunctionsandcollocationpointsUptothispoint,wehaveassumedthatthebasicfunctionsandthecollocationpointsaregiven.Inthissection,weshowhowtodeterminethemtosatisfytheassumptionswehavemadeintheprevioussection.Theorem1.Consideraquadratureformula:ZzW(z)F(z)dz'NXj=1wjF(zj);wherewjandzjaretheweightsandabscissas.ThenforaweightfunctionW(z)=z(1z),thereexistsanoptimalchoiceofNquadraturepointsinthesensethatthehighestpossiblepowerofzinapowerseriesexpansionofF(z)iscorrectlyintegratedwhenthesez-valuesareusedasquadraturepoints.Theoptimalquadraturepointsarethezerosofthepolynomialofdegree(N+1),i.e.,P()N+1(x),thatsatis˝esthefollowingorthogonalitycondition:ZzW(z)zjP()N+1(z)dz=0;forj=0;;N:(5.8)ThedetailproofofTheorem1isprovidedinChap3of[114].Inshort,thechoiceofcollocationpointsastherootsofthenextorderorthogonalpolynomialwillmakethecolloca-88tionmethodapproximationclosettoGalerkin'smethod,whichyieldsthebestperformanceamongtheMethodsofWeightedResidual(MWR)[114].Corollary2.ConsiderthesamequadratureformulainTheorem1,andasetofN+1orthogonalpolynomialfunctionsfgi(z)g.Ifthesetoffunctionssatis˝esthecondition:Zzˇ(z)gi(z)gN+1(z)dz=0;i=1;;N;(5.9)then,italsosatisfycondition(5.8)andthezerosofgN+1(z)aretheoptimalquadraturepoints.TheproofofCorollary2isstraightforwardandprovidedinAppendixC.IfwechoosetheweightfunctiontobethePDFof,i.e.,W()=ˇ(),(5.9)canbeusedtogeneratethesetfgi()ginarecursivemannerasfollows.Inpractice,wede˝netheinitialconditions:g1=0;g0=1;andtheorthogonalpolynomialscanbeobtainedrecursivelybysolvingtheequations:Zˇ()gi()gi+1()=0;i=1;;N:However,forhighorderpolynomials,solving(5.9)manuallyistime-consuminganderror-prone.Thus,thesetofbasisfunctionscanbecomputedalternativelyande˚cientlybyusingFavardtheoremasfollows.Theorem3.(FavardTheorem)IfasequenceofpolynomialsfPi(z)g,wherei=1;;N,89satis˝estherecurrencerelation:Pi(z)=(zi)Pi1(z)iPi2(z);P1=0;P1=1;(5.10)whereiandiarerealnumbers.Then,fPi(z)gareorthogonalpolynomials.Inthisstudy,weadapttheworkofZhouetal.[129]thatutilizes1i=hgi1;gi1iandi=phgi1;gi1i.TheoverallPCMalgorithmisshowninAlgorithm3.Notethatevenhereweusethesameorderforallrandomvariables(N-thorder),ingeneraltheorderscanbedi˙erent.Furthermore,eventhePCMhasbeenwidelyusedforapproximationofunivariatepredictiontarget,i.e.,yisscalar,inourapproachweextendittobeamultivariateapproximation.Theextensionisstraightforwardsincein(5.11)nowshallbeamatrixinsteadofavector.Asarelativecomparison,theG&Rtakes2daystorun512setsofparameters,whilePCMtakes20secondstoproducethesameresults.5.3.4MaximumdiametercurvetransformationItisacommonpracticeintrainingdeepstructurethattheoriginaltrainingdatasetisaugmentedtoincreaseitsvariationandgeneralization.Forinstance,themultiplesourcesoftrainingdatasetscanbemixedtogethertoformanewone[71]ordi˙erentviewanglesareachievedtogeneralmoreposesofa3Dobject[122].WeobservethatsomeoftherealMDCsdonotstartat2(cm)duetosometruncationinthepre-processingstep.Therefore,wegeneralizethetrainingbyaugmentingtheoutcomesofPCMbyapplyingalineartransitiontothecurve,anexampleofwhichisshownindashedredlineinFigure5.3.Furthermore,sincetheG&Rhasonlyonedamagefunctionasshownin(5.1),itcannotgeneraldatawithasecondarylocalenlargement,whichisobservedina1ThedotproductbetweentwofunctionsA(z)andB(z)withrespecttoarandomvariablezwiththeprobabilitydensityfunctionw(z)isde˝nedashA(z);B(z)i=Rzw(z)A(z)B(z)dz.2Forthesakeofnotationalsimplicity,wedenotefkd;˙d;dgasf[1];[2];[3]ginAlgorithm3.90Figure5.3Twotransformationsofamaximumdiametercurve:theoriginal,translated,andsecondpeakmodi˝edcurvesareshowninsolidblue,dashedred,anddottedblacklines,respectively.Notethatthe˝rsthalvesoftheoriginalandsecondpeakmodi˝edcurvesoverlapeachother.numberofourpatients.Thus,wecreateamergedMDCbycombiningtwodi˙erentMDCstogether.AnexampleofamergedMDCisshownindottedblacklineinFigure5.3.5.4DeepLearningInthissection,weshowhowweusethearti˝ciallygenerateddatafromPCMtotraintheDBNandpredicttheMDCinafuturetime.WeutilizeastandardstructureoftheDBN[50]withtwolayersofRestrictedBoltzmannMachine(RBM)[47]asshowninFigure5.4.Firstofall,thetwolayersofRBMarepre-trainedinanunsupervisedmanner.Then,weunfold91thetwoRBMsintoanNN.Finally,we˝ne-tunetheNNwiththegroundtruthfromboththerealandarti˝ciallygenerateddatabyback-propagation.5.4.1RestrictedBoltzmannMachineAnRBMisabuildingblockoftheDBNandwepre-traintheRBMlayersasfollows.Assumethatwehavetwotypesofvariables:theobservedone(x)andthehiddenone(h).ThetwovariablesaregovernedbyanenergyfunctionE(x;h).Assumethatbothvisibleandhiddenunitshavebinomialdistributions,aBoltzmannMachineisanenergy-basedmodelthathastheenergyfunctionasasecond-orderpolynomial[2]:E(x;hj)=bTxcThhTWxxTUxhTVh;whereisthecollectionoftheo˙setsbandc,andtheweightsW,U,andV.Thus,anyprobabilisticdensityfunctionP(x)(aswellasjointandconditionalPDFs)canbeeasilyrepresentedbyanormalizedformoftheenergyfunction.Forinstance,thePDFofxcanbecomputedas:P(x)=XheE(x;h)Z;(5.13)whereZ=P~xPhE(~x;hj)isthenormalizationfactorand~xisallpossiblevaluesofthevisiblevectorx.Therealizationof~xcanbeconsideredasreconstructedvisibleunits.Inordertomaximizethelikelihoodfunctionto˝tthemodeltoatrainingdata,wecanusethelog-likelihoodgradient@logP(x)@tolearntheparameter.However,whenthisenergy-basedPDFisusedtocomputethegradientofthelog-likelihood,itrequiressamplingoftwoconditionalprobabilities:P(hjx)andP(x;h).ThiscanbedoneviatheMonteCarloMarkovChain(MCMC)sampling[51],whichishighlycomputationallyexpensive.Therefore,aRestrictedBoltzmannMachinelearntbyusingContrastiveDivergenceisnormallyutilizedasamoree˚cientalternativesolution.TheRestrictedBoltzmannMachineisintroducedbypostinganadditionalcondition:92U=0andV=0.Inotherword,therearenoconnectionbetweenunitsinthesamelayer,eithervisibleorhidden.NotethatthereisnolinksamongunitsinthesamelayerinFigure5.4.Furthermore,weassumethatthevisibleunithasGaussiandistribution,i.e.,vi˘N(ai;˙i),andthehiddenunithasbinomialdistribution,i.e.,hj2f0;1g.Then,wecande˝neamodi˝edenergyfunctionas[47]:E(x;hj)=VXi=1(xiai)22˙2iHXj=1cjhjVXi=1HXj=1wijxi˙ihj(5.14)TheconditionalPDFofthevisibleunitsgiventhehiddenonescanbecomputedas:P(xjh;)=eE(x;hj)PxeE(x;hj):NotethatP(hjx;)canbecomputedinthesamemanner.UsingE(x;hj)in(5.14),wehave:P(hj=1jx;)=sigm VXi=1wijxi+cj!;P(xi=xjh;)=N ai+˙iHXj=1hjwij;˙2i!;(5.15)wheresigm(x)=11+exp(x)isthesigmoidfunction.ThelikelihoodgradientcanbecomputedbytakingthederivativeofP(x)in(5.13)with93respectto,wehave:logP(x)@=1PheE(x;h)XheE(x;h)@E(x;h)@+1ZX~xXheE(~x;h)@E(~x;h)@=XhP(hjx)@E(x;h)@+X~xXhP(~x;h)@E(~x;h)@=EP(hjx)@E(x;h)@+EP(~x;h)@E(~x;h)@:(5.16)TheexpectationEP(hjx)[:]isalsoreferredaspositivephasedistributionordatadistribu-tionwhiletheotherexpectationEP(~x;h)[:]isre˙eredasnegativephaseormodeldistribu-tion[2].Optimizationof(5.16)involvessamplingfromP(~x;h)anditcanbedonebyrunningGibbssamplinguntilitreachestheequilibriumdistributionforeachparameterlearningupdateiteration,whichisextremelytimeconsuming.Alternatively,Hinton[48]suggestedtheContrastiveDivergence(CD)learningthatminimizesthedi˙erencebetweenthedatadistributionandtheone-step(ora˝nitenumberofsteps)reconstructeddistributioninsteadofminimizesthedi˙erencebetweenthedataandmodeldistributiondirectly.5.4.2ContrastiveDivergenceInaCD-klearning,thegradientoftheparameteriisapproximatedby:i=logP(x)@i=EP(hjx)@E(x;h)@iEPk(~x;h)@E(x;h)@i;(5.17)wherePk(~x;h)isthedistributionofthereconstructedvisibledataxafterkstepsofGibbssamplingandisthelearnrate.Inthisstudy,weutilizeCD-1thatinvolvesonefullstepof94Figure5.4DeeparchitectureoftheDBN.TwolayersofRBMaretrainedinanunsupervisedmanner(pre-trained)usingCD-1algorithm.Thetoplayerutilizesaneuralnetworksigmoidregressionforprediction.Gibbssampling,i.e.,P1(~x;h).Applying(5.17)to(5.14),wehavethelearningupdaterulesforeachRBMlayerasfollows.wij wij+EP(hjx)hjxi˙iEP1(~x;h)hjxi˙i;cj cj+EP(hjx)[hj]EP1(~x;h)[hj];ai ai+EP(hjx)viai˙2iEP1(~x;h)viai˙2i;(5.18)95whereEP(hjx)hjxi˙i=1NNXt=1~h[t]jx[t]i˙i;EP1(~x;h)hjxi˙i=1NNXt=1P(h[t]j=1j~x[t]i;)~x[t]i˙i;EP(hjx)[hj]=1NNXt=1~h[t]j;EP1(~x;h)[hj]=1NNXt=1P(h[t]j=1j~x[t]i;);EP(hjx)viai˙2i=1NNXt=1 x[t]iai˙2i!;EP1(~x;h)viai˙2i=1NNXt=1 ~x[t]iai˙2i!;whereNisnumberofsamplesand~hjand~xiaresampledwiththedistributionsin(5.15).5.5RealcasestudyInthissection,wedemonstratethee˙ectivenessofourproposedpredictivemodelwithexperimentusingtherealMDCobtainedfrom[34].5.5.1Experimentalset-upForadeepstructure,theselectionofthenumberofhiddenunits,i.e.,layersandnodes,isanimportantfactorthatdeterminestheperformanceofthemodel.Asageneralrule,ifthenumberofparametersismorethanthenumberoftrainingcases,themodelwillbeover˝tted.Therefore,asaruleofthumbforgenerativemodelofhigh-dimensionaldata,thenumberofparametersisconstrainedbyseveralordersofmagnitudegreaterthanthedimensionalityofthedata[47].Inthisstudy,ourdatahasthedimensionalityof663.Inordertodeterminetheoptimalnumberoflayers,westudythee˙ectofnumberofhidden96NumberofWeightsTrainingRMSE(cm)hiddenlayerstime(s)16706313.464.9627726317.225.0838746320.396.2849766323.659.76510786326.799.79Table5.2E˙ectofnumberoflayersontheprediction.NumberofnodesWeightsTrainingRMSE(cm)RBM-1RBM-2time(s)9003663159968.785.0440010030676338.834.611001007726317.225.0810040010756323.115.123690025680324.487.02Table5.3E˙ectofnumberofnodesina2-layerDBNontheprediction.layersonthe˝nalprediction,whichisshowninTable5.2.Inthisanalysis,wecontinuouslyaddonemoreoflayerof100hiddennodestothestructure,startingwith1.Asshowninthetable,theoptimalnumberofweightsisforoneortwolayers,whenitisapproximatelytwoordersofmagnitudeofthedimensionalityofthedata.Inordertotestthee˙ectofnumberofnodeswithineachlayer,weconstructanumberof2-layerDBNswithdi˙erentsetsofhiddennodesineachlayer.Inparticular,sincethereisaremarkabledi˙erenceinthedimensionsofthedataandthelabel,i.e.,663versus221,wetesttheDBNwiththreedistributionsofnodes:highlyconcentratedneartheinputlayer,equallydistributed,andhighlyconcentratedneartheoutputlayer.AsshowninTable5.3,asthenodesneartheinputlayerdecreases,themodelfailstocapturetherepresentativefeaturesinthedata,leadingtohigherpredictionerror.Oneoftheprobleminourdataisthatwepre-traintheDBNwithalargearti˝ciallygenerateddata(51200sample)whiletherealdata(8samples)for˝ne-tuningisextremelylimitedcomparedtothearti˝cialone.Totacklethisproblem,we˝xtheepoch3ofthe3Thenumberoftimesthatthemodelistrainedthroughthewholetrainingset.97pre-trainingprocesstobe1andincreasethenumberofepochsofthe˝ne-tuningprocess.Bydoingthis,weimprovetheportionofgeneralizationcapabilitythatiscontributedbytherealdatasetovertheonefromthearti˝cialdataasacounter-measurementforthelargegapinthenumbersofdatapointsbetweentwosourcesofdata.TheRMSEandtrainingtimefordi˙erentepochsfrom1to900isshowninFigure5.5.Astheepochsislargerthan500,theerrorstartstoincreaseagainasthemodelstartstoover˝tthe˝ne-tuningdataandthegeneralizationcapacityisreduced.Mixed-e˙ectmodel:Wecomparetheperformanceofourproposedmethodtothenonlin-earmixed-e˙ects,whichhasbeenusedextensivelyasapowerfulgrowthhierarchicalmodeloverthedecades[11,26,107].Forthemixed-e˙ectsmodel,weutilizeabasicformofthegrowthfunctionas:yi;j=0+(1+b1)ti;j+(2+b2)t2i;j+i;j;whereyi;jandti:jarethediametercapturedatthetimejofpatienti,b=[b1;b2]istherandom-e˙ectstermsandb˘N(0;b),=[0;;2]isthevectorofparameters,andi;jistheindependenterrorterm,i.e.,i;j˘N(0;˙2w).bandare˝ttedtothedataviathefminsearchfunctioninMATLAB.Neuralnetworkwithdropouts:dropoutisasimplebute˙ectivetechniquetoregularizeaneuralnetworktoobtainaonethatislesslikelytobeover˝tting[105].Whendropoutisapplied,werandomlyremoveasetofunits(frombothvisibleandhiddenlayers)temporarilyfromtheoriginalnetwork.Inthisstudy,we˝xeachunitwithaprobabilityofp=0:9.Then,fortrainingphase,aunitwillbepresentedwiththeprobabilityp.Duringthetestphase,theweightofaunitwillbemultipliedbythevaluep.5.5.2ExperimentalresultsThepredictionerrorfor6casesisshowninTable5.4andtheplotsofpredictionsareshowninFigure5.6.Thetrue,predictedbyourproposedmethod,andpredictedbythe98Figure5.5E˙ectofnumberofepochstothepredictionerror.TheRMSEand˝ne-tunetrainingtimeareplottedinsolidblueanddashedredlines,respectively.Thetrainingincreaseslinearlywiththenumberofepochs,whiletheRMSErapidlydecreasesforthebeginningthenstartstoincreaseagainforthenumberofepochsthatislargerthan500.99PatientIDRMSE(cm)NormalizederrorDBNDBN+DOMix-e˙ectsDBNDBN+DOMix-e˙ectsP13.363.108.180.540.571.63P24.196.37.680.370.460.98P31.021.555.590.410.421.37P42.952.814.160.550.551.14P53.713.49.040.330.291.79P65.856.465.731.21.311.37Table5.4Comparisonamongdi˙erentpredictivemodelsintermsofRMSEandnormalizedunit.mixed-e˙ectsmodelMDCsareshowninsolidblack,dashedblue,anddotteddashedredlines,respectively.Asshowninthetable,ourproposedmethodoutperformsthemix-e˙ectsmethodwithaverage45%reductioninRMSE.NoticethatwehaveusedanadditionalGaussianprocessregressiontopredictthescaleoftheMDC.Thus,the˝nalpredictionsaresubjectedtoabiaserrorduetotheGaussianprocessregression.Forinstance,inthecaseofpatientP5,thewholedeeplearningpredictedcurvehasaconsistento˙setwithrespecttothetruecurve.NoticethatpatientsP2andP5haveasecondarylocalenlargementsandourproposedmethodsuccessfullyproducesaccordinglypredictionswhilethemixed-e˙ectmethodfailstodoso.100Figure5.6Thetrue,predictedbyourproposedmethod,andpredictedbythemixed-e˙ectsmodelareshowninsolidblackdashedblueanddotteddashedredlines,respectively.101Algorithm3TheProbabilisticCollocationMethod(PCM)modelPart1:Computecollocationpoints1:Initializeg1()=G1()=0andg0()=G0()=1.2:fori=1;;Ndo3:Gi()=gi1()hgi1();gi1()igi1()phGi1();Gi1()igi2()4:gi()=Gi()phGi();Gi()i5:endfor6:FindthezerosofgN+1()=0asNcollocationpoints.7:Repeatsteps1-5forotherrandomvariables.Wedenotethreerandomvariables2andthreesetsofbasisfunctionsasf[1]i;[2]i;[3]igandfg[1]j();g[2]j();g[3]j()g,respectively,wherei=1;;Nandj=0;;N18:Createapermutationofthreegroupsofcollocationpointsof3randomvariables,i.e.,([1]i;[2]j;[3]k)wherei=1;;N,j=1;;N,andk=1;;N.Part2:Runthecomputationalcodeatthecollocationpoints1:fori=1;;N,j=1;;N,k=1;;Ndo2:Run([1]i;[2]j;[3]k).3:endforPart3:Computethecoe˚cients1:Concatenatethecomputationaloutcomes:y=264([1]1;[2]1;[3]1)...([1]N;[2]N;[3]N)375.2:ArrangethematrixK=264g[1]N1([1]1)g[2]N1([2]1)g[3]N1([3]1)g[1]0([1]1)g[2]0([2]1)g[3]0([3]1).........g[1]N1([1]N)g[2]N1([2]N)g[3]N1([3]N)g[1]0([1]N)g[2]0([2]N)g[3]0([3]N)375.3:Computethecoe˚cients:=K1y:(5.11)Part4:Approximatethecomputationalcode1:Foranynewsetofrandomvariable([1];[2];[3]),theoutcomecanbeapproximatedby:([1];[2];[3])=N1Xi=0N1Xj=0N1Xk=0i;j;kg[1]i([1])g[2]j([2])g[3]k([3]):(5.12)102Chapter6ConclusionandfutureworksInthischapter,webrie˛ysummarizethemainfactorsofeachchaptersaswellthelimitationsofthediscussedmethods.Additionally,weshowhowtheworkscanbeimprovedinthefuturewithfurtherinvestigation.Inchapter2,wepresentanovelapproachtousevisiondatafortherobotlocalization.Thepredictivestatisticsofvisiondataislearnedinadvanceandusedinordertoestimatethepositionofavehicle,equippedjustwithanomnidirectionalcamerainbothindoorandoutdoorenvironments.ThemultivariateGPmodelisusedtomodelacollectionofselectedvisualfeatures.Thelocationsareestimatedbymaximizingthelikelihoodfunctionwithoutfusingcombiningvehicledynamicswithmeasuredfeaturesinordertoevaluatetheproposedschemealone.Hence,webelievethatthelocalizationperformancewillbefurtherimprovedwhenvehicledynamicsarefusedtogetherviaKalman˝lteringorparticle˝ltering.Oneofthelimitationofthemethodisthataftertheinitialtrainingphase,learningisdiscontinued.Iftheenvironmentchanges,itisdesirablethatthelocalizationroutinesadapttothechangesintheenvironment.Thus,afutureresearchdirectionistodevelopalocalizationschemethatisadaptivetochangesintheenvironment.Inchapter3,weintroduceanovelappearance-basedapproachbasedongroupLASSOregression.Wehaveshownthee˙ectivenessofourmethodinfeatureselectionandlocaliza-103tionenhancementbycombingthegroupLASSOregressionandtheEKF.Theexperimentstudyshowsthesigni˝cantreduction(75.5%)inthefeatureswhileimprovingthelocalizationperformance.Noticethatsincethedirectdynamicmodelthatmapsthevisualfeaturesintotherobot'spositions,i.e.,qi=h(x),arenotavailable,weusetheLASSO(aswellasthegroupLASSO)asalinearregressionapproximation.Thus,theobservationsfortheEKFandPFarenoisyestimatedlocalizationresultsfromtheLASSO.Thelinearityassumptioncouldrestrictthelocalizationperformance.Therefore,apotentialfuturestudyistoapplyanon-linearregres-siontechnique,suchasGaussianprocess,torelaxthelinearitycondition.Inchapter4,wehaveformulatedthemodelingandgrowthoftheAAAusingpatient-speci˝cpointcloudsdatainastatisticalframework.Afterutilizingthespatio-temporalGaussianprocessobservationmodeltoconstructtheimplicitsurface˝eld,wedevelopadynamicmodeltoinfertheevolutionofthe˝eldatafuturetime.Finally,weextractthesurfacefromthepredicted˝eldforvisualizationofananeurysm.Theresultsofthecasestudieshaveshownthee˚cacyofourproposedschemebycom-paringthepredictedAAAswiththegroundtruth.Tothebestofourknowledge,thisisthe˝rststudythatpredictsthegrowthofthe3DAAAshapebyusingpatient-speci˝cdatainastatisticalframeworkandprovidesuncertaintyofthepredictedAAAshape.Indoingso,thestudyyieldsinsightful˝ndingsaswellashighlightsthelimitationsofusingsuchmodelsforstudyingthenatureofthegrowthofanAAA.Possibleclinicalapplicationsandlimita-tionsofourapproacharealsodiscussedalongwithprospectiveresearchdirections.Withadvancesincomputingtechnologiesandnewsamplingmethods,theuseoftheBayesianapproachwillhaveagreatpotentialtorevolutionizeapplicationofcomputationalmodelinginthetreatmentofvasculardiseases.Inchapter5,weproposeanovelmethodthatutilizestheDeepBeliefNetworktopredictthegrowthofanAAAviathemaximumdiametercurves.Thedeepstructureispre-trainedonasetofsimulateddataand˝ne-tunedbythelongitudinaldataofthepatients.The104methodisimplementedonarealcasestudywith6patients.Theoutcomesvalidatethee˙ectivenessofourproposedmethod.Alimitationofthecurrentapproacharisesfromthefactthattheuseofaadditionalregressiontopredictthenormalizationfactorseparatelyincreasesthebiaserror.Thus,afutureresearchdirectionistoaccommodateuncertaintyinGaussianprocesstothe˝nalresults.Forinstance,drop-outcanbeusedasanapproximatedBayesiantechniquetorepresentrepresentmodeluncertaintyindeeplearning[33].105Appendices106AppendixAE-MalgorithmInthissection,weshowhowtousetheE-M(Expectation-Maximization)algorithmto˝tthelineardynamicmodeltothetrainingdataset.Firstofall,wede˝nealoglikelihoodfunctionconditionedontheavailabledatauptothetimeTas(fornotationalsimplicity,weomitthexargumentandputthetimetinthesubscriptforthefunctionf(:),i.e.,AandftimplyA(x)andf(x;t)):log(L):=12logj0j12(f00)T10(f00)12TXt=1logj2twj12TXt=1(ftft1tA)T2tw)1(ftft1tA)12TXt=1logjv(t)j12TXt=1(ytft)Tv(t)1(ytft):(1)TheE-stepLetA;w)=Er[log(L)jD1:T;A;w]betheexpectedvalueoftheloglikelihoodfunctionatiterationr,applytheexpectationtobothsidesof(1)andutilizethematrixidentityE[xTAx]=Tr(AE[xxT])(2)wecanderiveA;w)asfollows.Considereachlineof(1):Wecanseestraightforwardlythatthe˝rstlineof(1)becomesthe˝rstlineof(4)bysimplyapplying(2).Applytheidentity(2)tothesecondlineof(1),wehave:E"12TXt=1logj2twj12TXt=1(ftft1tA)T2tw)1(ftft1tA)#=12TXt=1logj2twj12TXt=1E(ftft1tA)T2tw)1(ftft1tA)107Apply(2)tothelatersumandexpandit,wehave:TXt=1Tr2tw)1E[(ftft1tA)(ftft1tA)T]=TXt=1Tr2tw)1CtBtBTt+Et+tHt+2tAAT;whereHt=(E[ft1]E[ft])AT+A(E[ft1]E[ft])T;Ct=Cov(ftjD1:T)+E[f(t)jD1:T]E[f(t)jD1:T]T;Bt=Cov(ft;ft1jD1:T)+E[ftjD1:T]E[ft1jD1:T]T;Et=Cov(ft1jD1:T)+E[ft1jD1:T]E[ft1jD1:T]T:(3)Applytheidentity(2)tothethirdlineof(1)wehave:T2logjvj12Tr(1vTXt=1E[(ytft)(ytft)T])=T2logjvj12Tr(1vTXt=1ytyTtytfTtyTtft+E[ftfTt])=T2logjvj12Tr(1vTXt=1(ytE[ft])(ytE[ft])T+Cov(ft;ft)):Finally,combingthethreelinestogether,wehave:A;w)=12logj0j12Tr10Cov(f(0)jD1:T)+E(f(0)jD1:T)E(f(0)jD1:T)T12TXt=1logj2twj12TXt=1Tr2tw)1CtBtBTt+Et+tHt+t)2AAT12TXt=1logjv(t)j12Tr(tv(t))1TXt=1(y(t)E[f(t)jD1:T])(y(t)E[f(t)jD1:T])T+Cov(f(t)jD1:T));(4)108whereHt,Ct,Bt,andEtarede˝nedin(3).ThisistheendoftheE-step.TheM-stepIntheM-stepofiterationr,we˝ndthevaluesofA(r+1)andw(r+1)thatmaximize(4).LetM=2twandtakethederivativeof(4)withrespecttoA:@@A=@Tr@A(TXt=1Mt(E[ft1]E[ft])AT+tA(E[ft1]TE[ft]T)+2tAAT)=TXt=1ˆ@Tr@AtM(E[ft1]E[ft])AT)+@Tr@AtMA(E[ft1]TE[ft]T))+@Tr@A2tMAAT)˙=TXt=1t(M+MT)(E[ft1]E[ft])+ TXt=12t(M+MT)!A=w+Tw)TXt=13t(E[ft1]E[ft])+w+Tw) TXt=14t!A=0:Thus,A(r+1)=A= TXt=14t!1 TXt=13t(E[ft]E[ft1])!Then,takethederivativeof(4)withrespecttowyields:@@w=@@w(TXt=1logj2twj+TXt=1Tr2tw)1(CtBtBTt+Et+tHt+2tAAT))=TXt=12tw)T2tTXt=12tw)T(CtBtBTt+Et+tHt+2tAAT)T2tw)T2t= TXt=11!TwTw NXt=12t(CtBtBTt+Et+tHt+2tAAT)T!Tw=0109Thus,w(r+1)=w=1T NXt=12t(CtBtBTt+Et+tHt+2tAAT)!;withAcomputedabove.Finally,wehavetheupdateforAandwasfollows.A(r+1)= TXt=14t!1 TXt=13t(E[f(t)jD1:T]E[f(t1)jD1:T])!;(5a)w(r+1)=1T NXt=12t(CtBtBTt+Et+tHt+2tAAT)!:(5b)NotethatwehavenotdiscussedhowtocomputeE[ftjD1:T],Cov(ftjD1:T),andCov(ft;ft1jD1:T),whichareneededtocomputeA(r+1),Bt,andCt.ThosestatisticalquantitiesareconditionedthewholedatasetD1:Tsotheyneedtobecomputedpriortothesumin(5).UsingtheKalman˝ltersmoothingframework[101],˝rstwepropagatethemeans(E[ft])andcovariances(Cov(ft))throughthedata.Foreachstepofpropagationthroughthedata,thequantitiesareconditionedontheavailabledatauptothatparticularstep.Then,oncewereachtheendofdataset,wecomputebackwardto˝ndthemean,covariance,andcorrelationconditionedonthewholedataset.Fortheinitialcondition,weassumethatf0jD0˘N(0;0),i.e.,E[f0jD0]=0andCov(f0jD0)=0where0and0areknown.TheoverallKalmanFilteralgorithmisshowninAlgorithm4.ComputedAandwarepluggedbackintotheloglikelihoodexpectation(4)thentheE-stepandM-steparerepeated.Thewholeprocesscontinuesuntiltheloglikelihoodexpectationconverges.OnceA(x)andw(x)arecalibratedtothetrainingdataset,thepredictivedistributionoftheIS˝eldforthetestdataset,whichisshownin(10)ofthemainmanuscript,isstraightforwardlyobtainedfromtheforwardcomputationoftheKalmanFilteralgorithm(the˝rsttwolinesoftheforloopoftheforwardcomputationsectionofAlgorithm.4).110Algorithm4KalmanFilterupdatingForwardcomputation:fort=1;;TdoE[f(t)jD1:t1]=E[f(t1)jD1:t1]+tACov(f(t)jD1:t1)=Cov(f(t1)jD1:t1)+w2tKt=Cov(f(t)jD1:t1)(Cov(f(t)jD1:t1)+v(t))1E[f(t)jD1:t]=E[f(t)jD1:t1]+Kt(y(t)E[f(t)jD1:t1])Cov(f(t)jD1:t)=(IKt)Cov(f(t)jD1:t1)endforBackwardcomputation:InordertocomputeE[f(t)jD1:T],Cov(f(t)jD1:T),andCov(f(t);f(t1)jD1:T),onecanusethebackwardcomputations:fort=T;;1doJt1=Cov(f(t1)jD1:t1)Cov(f(t)jD1:t1)1E[f(t1)jD1:T]=E[f(t1)jD1:t1]+Jt1(E[f(t)jD1:T]E[f(t1)jDt1])Cov(f(t1)jD1:T)=Cov(f(t1)jD1:t1)+Jt1(Cov(f(t)jD1:T)Cov(f(t)jD1:t1))JTt1ift6=1thenCov(f(t1);f(t2)jD1:T)=Cov(f(t1)jD1:t1)JTt2+Jt1(Cov(f(t);f(t1)jD1:T)Cov(f(t1)jD1:t1))JTt2endifendforFort=T:Cov(f(T);f(T1)jD1:T)=(IKT)Cov(f(T1)jD1:T1).111AppendixBSurfaceextractionThresholdDeterminationLetxbetheon-surfacepointsatthetimetsuchthatx=fxˆS:f(x;t)g;whereisthethresholdvalue.Todeterminethethresholdforthetrainingdata,werunanexhaustivesearchthroughtherangeofpossiblevaluesoftheIS˝eld.Foreachcandidatethreshold,wereconstructthetrainingpointcloud.Finally,wechoosethetrainingthresholdtobetheonethatyieldsthemostsimilarreconstructedpointclouds.Toevaluatethesimilaritybetweentwopointclouds,weusetheHausdor˙distanceH(:;:)[53]thatisde˝nedasfollows.H(P;O)=max(h(P;O);h(O;P));(6)whereh(P;O)=max^x2Pminx2Ok^xxk:Fortheexhaustivesearch,wegraduallyraisethethresholdfrom0to1,the˝nalthresholdischosensuchthatthecorrespondingpointcloudyieldsthelowestHausdor˙distance.Fig.A.1showsoneexampleofpatientP5.ThevaluesofIS˝eldareplottedwithrespecttoallpointsonthegrid.Thethresholdfortrainingandtest˝eldsareindicatedbytheblackanddashedsolidlines,respectively.BinaryencodingandmatchingNotethateachparticularvalueofthethresholdclassi˝esthewhole3Dlatticeintoauniquespatialpatternoftwocategories:on-surfaceando˙-surface.Therefore,weutilizethatfacttoassignthetestthreshold(thethresholdvalueforthepredictedIS˝eld)tothevaluethatyieldsthemostsimilarspatialpatterntothetrainingdata.Inparticular,we112FigureA.1ThresholddeterminationforpatientP5:the˝eldofpointsonthelatticeforthetrainingandtest˝eldsareplottedinblueplusesandreddots,respectively.Thethresholdsfortrainingandtestdataareshowninsolidanddashedlines,correspondingly.113encodetwospatialsiteS'sthatareclassi˝edbytrainingandtestthresholdsintotwobinaryvectors.Then,weselectthetestthresholdthatyieldstheclosestbinaryvectortothetrainingone.Thedetailoftheprocessisdiscussedasfollows.First,wemapon-surfacepointsontoabinaryvectorasfollows.g(x;)=8><>:1iff(x;L)0iff(x;L)>Letvt(x1:n;)=[g(x1;);;g(xn;)]Tbethebinaryvectorobtainedatthescantimetwiththreshold,wherex1:nisthecollectionofcoordinatesofthespatialsiteS.Then,weusetheJaccardindexes[92]toestimatethesimilaritybetweentwobinaryvectors.TheJaccardindexcanbecomputedasfollows.J(A;B)=jA\BjjA[Bj;whereA;Baretwosamplesets.ThethresholdthatyieldslowestJaccardindexischosentobethetestthreshold,i.e.,^L=argminLfJ(vL1(x1:n;L1);vL(x1:n;L))g:Pointcloudpost-processingInthissection,wediscussthe˝nalstepforsurfacere˝ning.Figs.A.2showthecrosssectionofthepredictedsurfaceattheheightsz=86:9(mm),whichisatthemiddleoftheAAA,andz=76:21(mm)wheretheAAAstartstobranchoutintotwosegments.Theselectedpointsbythetestthresholdareplottedinbluecircles.Notethatinnerpointsthatlocatenearthesurfacecanbefalselyclassi˝edtobeon-surface.Toeliminatethoseinnerpoints,weapplytwoconnectedcomponentoperatorsinthex-yplaneofeachcrosssection:k-meansclustering[42]andconvexhull[37].114(a)(b)FigureA.2CrossviewofpredictedsurfaceoftheAAAattwodi˙erentverticalheights:(a)z=86:9(mm)and(b)z=76:21(mm).Thepointcloudsandtwoclustersareplottedincirclesandred/bluestars,respectively.Thesurfacepointsthatareselectedbytheconvexhullareconnectedwiththeblackdashedline.Asshownin(b),withouttheclustering,twobrancheswillbefalselymergedintoone.First,weclusterthepointsintotwoclusters,thatareshowninredandbluestarsinFigs.A.2.Thenumberofclustersissetto2duetothespeci˝cstructureoftheAAA,whichhasatubularstructurewithtwobranchesattheend.Notethatclusteringthepointsintotwogrouppreventstheconvexhulloperatorfromfalselymergingtwobranchesintoonelargetube.Then,weapplytheconvexhulloperatorto˝lteroutonlythepointsthatlieontheboundary,whichareshownconnectedwithdashedlinesinFigs.A.2.Thosepointsthatdonotlieonthedashedblacklineareremoved.NoticethatwithoutclusteringinFig.A.2-(b),theconvexhulloperatorwillfalselymergetwosectionsintoone.Afterthispost-processingstep,we˝nallyobtainthepredictedpointcloudoftheAAA'ssurface.115AppendixCProofofCorollary2WeillustratetheproofforN=2,thentheproofforanarbitraryNisstraightforward.Letg0=1;g1=ax+b;g2=cx2+dx+e:So,applying(5.9),wehave:Zxˇ(x)g0(x)g3(x)dx=Zxˇ(x)g3(x)dx=0;Zxˇ(x)g1(x)g3(x)dx=Zxˇ(x)(ax+b)g3(x)dx=0;Zxˇ(x)g2(x)g3(x)dx=Zxˇ(x)(cx2+dx+e)g3(x)dx=0:Rearrangetheterms,wehaveanequivalentsystemofequations:(1+b+e)Zxˇ(x)g3(x)dx=0!Zxˇ(x)g3(x)dx=0;(a+d)Zxˇ(x)xg3(x)dx=0!Zxˇ(x)xg3(x)dx=0;(c)Zxˇ(x)x2gx(x)dx=0!Zxˇ(x)x2g3(x)dx=0:whichisthecondition(5.8).116BIBLIOGRAPHY117Bibliography[1]MSanjeevArulampalam,SimonMaskell,NeilGordon,andTimClapp.Atutorialonparticle˝ltersforonlinenonlinear/non-gaussianbayesiantracking.IEEETransactionsonSignalProcessing,2002.[2]YoshuaBengio.LearningdeeparchitecturesforAI.Foundationsandtrends®inMachineLearning,2009.[3]YoshuaBengio,PatriceSimard,andPaoloFrasconi.Learninglong-termdependencieswithgradientdescentisdi˚cult.IEEEtransactionsonneuralnetworks,1994.[4]ChristopherMBishopetal.Patternrecognitionandmachinelearning,volume1.springerNewYork,2006.[5]P.BlaerandP.Allen.Topologicalmobilerobotlocalizationusingfastvisiontech-niques.InProceedingsoftheIEEEInternationalConferenceonRoboticsandAutoma-tion,,volume1,pages2002.[6]D.Bluestein,K.Dumont,M.DeBeule,J.Ricotta,P.Impellizzeri,B.Verhegghe,andP.Verdonck.Intraluminalthrombusandriskofruptureinpatientspeci˝cabdominalaorticaneurysmFSImodelling.ComputerMethodsinBiomechanicsandBiomedicalEngineering,2009.[7]FranciscoBonin-Font,AlbertoOrtiz,andGabrielOliver.Visualnavigationformobilerobots:Asurvey.Journalofintelligentandroboticsystems,2008.[8]OlafBooij,ZoranZivkovic,andBenKrose.Sparseappearancebasedmodelingforrobotlocalization.In2006IEEE/RSJInternationalConferenceonIntelligentRobotsandSystems,pages151IEEE,2006.[9]T.Botterill,S.Mills,andR.Green.Speeded-upbag-of-wordsalgorithmforrobotlocalisationthroughscenerecognition.InImageandVisionComputingNewZealand,2008.IVCNZ2008.23rdInternationalConference,pagesNov2008.[10]GaryBradskiandAdrianKaehler.LearningOpenCV:ComputervisionwiththeOpenCVlibrary.O'ReillyMedia,Inc.,2008.[11]AnthonyRBrady,SimonGThompson,FGeraldRFowkes,RogerMGreenhalgh,JanetTPowell,etal.Abdominalaorticaneurysmexpansionriskfactorsandtimeintervalsforsurveillance.Circulation,2004.[12]A.Brooks,A.Makarenko,andB.Upcroft.Gaussianprocessmodelsforindoorandoutdoorsensor-centricrobotlocalization.IEEETransactionsonRobotics,1351,2008.118[13]YuCheng,FeiWang,PingZhang,andJianyingHu.RiskPredictionwithElectronicHealthRecords:ADeepLearningApproach.[14]SungjoonChoi,MahdiJadaliha,JongeunChoi,andSonghwaiOh.DistributedGaus-sianProcessRegressionUnderLocalizationUncertainty.JournalofDynamicSystems,Measurement,andControl,137(3):031002,2015.[15]RonanCollobertandJasonWeston.Auni˝edarchitecturefornaturallanguagepro-cessing:Deepneuralnetworkswithmultitasklearning.InProceedingsofthe25thinternationalconferenceonMachinelearning,pages167.ACM,2008.[16]JohnJCraig.Introductiontorobotics:mechanicsandcontrol,volume3.PearsonPrenticeHallUpperSaddleRiver,2005.[17]CédricDemonceaux,PascalVasseur,andYohanFougerolle.Centralcatadioptricimageprocessingwithgeodesicmetric.ImageandVisionComputing,2011.[18]SaandeepDepatla,LucasBuckland,andYasaminMosto˝.X-rayvisionwithonlywi˝powermeasurementsusingrytovwavemodels.IEEETransactionsonVehicularTechnology,2015.[19]GuilhermeN.DeSouzaandAvinashC.Kak.Visionformobilerobotnavigation:Asurvey.IEEETransactionsonPatternAnalysisandMachineIntelligence,,267,2002.[20]HuanNDo,JongeunChoi,ChaeYoungLim,andTapabrataMaiti.Appearance-basedlocalizationusingGroupLASSOregressionwithanindoorexperiment.In2015IEEEInternationalConferenceonAdvancedIntelligentMechatronics(AIM),pages.IEEE,2015.[21]HuanNDo,JongeunChoi,ChaeYoungLim,andTapabrataMaiti.Appearance-basedoutdoorlocalizationusingGroupLASSOregression.In2015DynamicSystemsandControl(DSCC).IEEE,inpress.[22]HuanNDo,MahdiJadaliha,MehmetTemel,andJongeunChoi.FullyBayesianFieldSlamUsingGaussianMarkovRandomFields.AsianJournalofControl,2015.[23]VladislavsDovgalecs,RémiMégret,andYannickBerthoumieu.Multiplefeaturefu-sionbasedonco-trainingapproachandtimeregularizationforplaceclassi˝cationinwearablevideo.AdvancesinMultimedia,1,2013.[24]StanimirDragiev,MarcToussaint,andMichaelGienger.Gaussianprocessimplicitsurfacesforshapeestimationandgrasping.In2011IEEEInternationalConferenceonRoboticsandAutomation(ICRA),pages2850.IEEE,2011.[25]Y.Dupuis,X.Savatier,andP.Vasseur.Featuresubsetselectionappliedtomodel-freegaitrecognition.ImageandVisionComputing,31(8):580591,2013.119[26]PEriksson,SJormsjö-Pettersson,ARBrady,HDeguchi,AHamsten,andJTPow-ell.Genotypyperelationshipsinaninvestigationoftheroleofproteasesinabdominalaorticaneurysmexpansion.BritishJournalofSurgery,92(112005.[27]MehdiFarsad,ShahrokhZeinali-Davarani,JongeunChoi,andSeungikBaek.Com-putationalgrowthandremodelingofabdominalaorticaneurysmsconstrainedbythespine.JournalofBiomechanicalEngineering,137(9):091008,2015.[28]MehdiFarsad,ShahrokhZeinali-Davarani,JongeunChoi,andSeungikBaek.Com-putationalgrowthandremodelingofabdominalaorticaneurysmsconstrainedbythespine.JournalofBiomechanicalEngineering,2015.[29]JoãoCarlosAparícioFernandesandJoséAlbertoBCamposNeves.Usingconicalandsphericalmirrorswithconventionalcamerasfor360panoramaviewsinasingleimage.InProceedingsoftheIEEEInternationalConferenceonMechatronics,pages2006.[30]B.Ferris,D.Fox,andN.Lawrence.WiFi-SLAMusingGaussianprocesslatentvari-ablemodels.InProceedingsofthe20thInternationalJointConferenceonArti˝cialIntelligence,pages24802007.[31]BernardFriedland.Controlsystemdesign:anintroductiontostate-spacemethods.CourierCorporation,2012.[32]KunihikoFukushima.Neocognitron:Aself-organizingneuralnetworkmodelforamechanismofpatternrecognitionuna˙ectedbyshiftinposition.Biologicalcybernetics,1980.[33]YarinGalandZoubinGhahramani.Dropoutasabayesianapproximation:Represent-ingmodeluncertaintyindeeplearning.arXivpreprintarXiv:1506.02142,2,2015.[34]HGharahi,B.A.Zambrano,CLim,JChoi,WLee,andSBaek.Ongrowthmea-surementsofabdominalaorticaneurysmsusingmaximallyinscribedspheres.MedicalEngineering&Physics,2015.[35]RBGopaluni.Aparticle˝lterapproachtoidenti˝cationofnonlinearprocessesundermissingobservations.TheCanadianJournalofChemicalEngineering,86(6):101092,2008.[36]AlexanderGraham.Kroneckerproductsandmatrixcalculus:withapplications,volume108.HorwoodEngland,UK,1981.[37]RonaldL.Graham.Ane˚cientalgorithfordeterminingtheconvexhullofa˝niteplanarset.InformationProcessingLetters,1972.[38]AndriiGrytsan,PaulNWatton,andGerhardAHolzapfel.AThick-WalledSolid-GrowthModelofAbdominalAorticAneurysmEvolution:ApplicationtoaPatient-Speci˝cGeometry.JournalofBiomechanicalEngineering,137(3):031008,2015.120[39]BaofengGuoandMarkSNixon.Gaitfeaturesubsetselectionbymutualinformation.IEEETransactionsonSystems,ManandCybernetics,PartA:SystemsandHumans,2009.[40]EfstathiosHadjidemetriou,MichaelD.Grossberg,andShreeK.Nayar.Multiresolutionhistogramsandtheiruseforrecognition.IEEETransactionsonPatternAnalysisandMachineIntelligence,2004.[41]JohnAHartiganandManchekAWong.Algorithmas136:Ak-meansclusteringalgorithm.Appliedstatistics,pages1001979.[42]JohnAHartiganandManchekAWong.Algorithmas136:Ak-meansclusteringalgorithm.AppliedStatistics,pages08,1979.[43]J.B.Hayet,F.Lerasle,andM.Devy.Visuallandmarksdetectionandrecognitionformobilerobotnavigation.InComputerVisionandPatternRecognition,2003.Pro-ceedings.2003IEEEComputerSocietyConferenceon,volume2,pagesIIvol.2,June2003.[44]XumingHe,RichardSZemel,andMiguelÁCarreira-Perpiñán.Multiscaleconditionalrandom˝eldsforimagelabeling.InComputervisionandpatternrecognition,2004.CVPR2004.Proceedingsofthe2004IEEEcomputersocietyconferenceon,volume2,pagesIIEEE,2004.[45]TinneTuytelaarsHerbertBay,AndreasEssandLucVanGool.Speeded-uprobustfeatures(surf).ComputerVisionandImageUnderstanding,2008.[46]DaveHigdon,JamesGattiker,BrianWilliams,andMariaRightley.Computermodelcalibrationusinghigh-dimensionaloutput.JournaloftheAmericanStatisticalAsso-ciation,2008.[47]Geo˙reyHinton.ApracticalguidetotrainingrestrictedBoltzmannmachines.Mo-mentum,9(1):926,2010.[48]Geo˙reyEHinton.Trainingproductsofexpertsbyminimizingcontrastivedivergence.Neuralcomputation,2002.[49]Geo˙reyEHinton.Learningmultiplelayersofrepresentation.Trendsincognitivesciences,2007.[50]Geo˙reyEHinton,SimonOsindero,andYee-WhyeTeh.Afastlearningalgorithmfordeepbeliefnets.Neuralcomputation,2006.[51]Geo˙reyEHinton,TerrenceJSejnowski,andDavidHAckley.Boltzmannmachines:Constraintsatisfactionnetworksthatlearn.Carnegie-MellonUniversity,DepartmentofComputerSciencePittsburgh,PA,1984.[52]WenhaoHuang,GuojieSong,HaikunHong,andKunqingXie.Deeparchitecturefortra˚c˛owprediction:deepbeliefnetworkswithmultitasklearning.IEEETransac-tionsonIntelligentTransportationSystems,2014.121[53]DanielP.Huttenlocher,GregoryA.Klanderman,andWilliamJRucklidge.Comparingimagesusingthehausdor˙distance.IEEETransactionsonPatternAnalysisandMachineIntelligence,1993.[54]AIjaz,JChoi,WLee,andSBaek.Predictionofabdominalaorticaneurysmsusingsparsegaussianprocessregression.InASME2013SummerBioengineeringConfer-ence,pagesV0AmericanSocietyofMechanicalEngineers,2013.[55]MahdiJadaliha,YunfeiXu,JongeunChoi,NicholasSJohnson,andWeimingLi.Gaussianprocessregressionforsensornetworksunderlocalizationuncertainty.IEEETransactionsonSignalProcessing,2013.[56]GijeongJang,SunghoLee,andInsoKweon.Colorlandmarkbasedself-localizationforindoormobilerobots.InProceedingsoftheIEEEInternationalConferenceonRoboticsandAutomation,volume1,pages2002.[57]WooYeonJeongandKyoungMuLee.Visualslamwithlineandcornerfeatures.InProceedingoftheIEEE/RSJInternationalConferenceonIntelligentRobotsandSystems,pages2575.IEEE,2006.[58]IanJolli˙e.Principalcomponentanalysis.WileyOnlineLibrary,2002.[59]ChristopherKananandGarrisonWCottrell.Color-to-grayscale:doesthemethodmatterinimagerecognition?PloSone,7(1):e29740,2012.[60]JamesMKeller,MichaelRGray,andJamesAGivens.Afuzzyk-nearestneighboralgorithm.Systems,ManandCybernetics,IEEETransactionson,1985.[61]Ji-HyunKim.Estimatingclassi˝cationerrorrate:Repeatedcross-validation,repeatedhold-outandbootstrap.ComputationalStatistics&DataAnalysis,2009.[62]AhmedKlink,FabienHya˝l,JamesRudd,PeterFaries,ValentinFuster,ZiadMallat,OlivierMeilhac,WillemJMMulder,Jean-BaptisteMichel,FrancescoRamirez,etal.Diagnosticandtherapeuticstrategiesforsmallabdominalaorticaneurysms.NatureReviewsCardiology,2011.[63]HWKniemeyer,TKessler,PUReber,HBRis,HHakki,andMKWidmer.Treatmentofrupturedabdominalaorticaneurysm,apermanentchallengeorawasteofresources?predictionofoutcomeusingamulti-organ-dysfunctionscore.EuropeanJournalofVascularandEndovascularSurgery,pages1902000.[64]SadanoriKonishiandGenshiroKitagawa.Bayesianinformationcriteria.InformationCriteriaandStatisticalModeling,pages37,2008.[65]NikolaosKontopodis,StellaLioudaki,DimitriosPantidis,GeorgePapadopoulos,Efs-tratiosGeorgakarakos,andChristosVIoannou.Advancesindeterminingabdominalaorticaneurysmsizeandgrowth.WorldJournalofRadiology,8(2):148,2016.122[66]NikolaosKontopodis,EleniMetaxa,MichalisGionis,YannisPapaharilaou,andChris-tosVIoannou.Discrepanciesindeterminationofabdominalaorticaneurysmsmax-imumdiameterandgrowthrate,usingaxialandorhtogonalcomputedtomographymeasurements.EuropeanJournalofRadiology,2013.[67]J.Kosecka,LiangZhou,P.Barber,andZ.Duric.Qualitativeimagebasedlocalizationinindoorsenvironments.InProceedingsoftheIEEEComputerSocietyConferenceonComputerVisionandPatternRecognition,volume2,pagesI2003.[68]HarrieKurvers,FrankJVeith,EvanCLipsitz,TakaoOhki,NicholasJGargiulo,NealSCayne,WilliamDSuggs,CarlosHTimaran,GraceYKwon,SooJRhee,etal.Discontinuous,staccatogrowthofabdominalaorticaneurysms.JournaloftheAmericanCollegeofSurgeons,2004.[69]MatthewLai.Deeplearningformedicalimagesegmentation.arXivpreprintarXiv:1505.02000,2015.[70]SvetlanaLazebnik,CordeliaSchmid,andJeanPonce.Beyondbagsoffeatures:Spatialpyramidmatchingforrecognizingnaturalscenecategories.InProceedingsoftheIEEEComputerSocietyConferenceonComputerVisionandPatternRecognition,volume2,pages2006.[71]YannLeCun,LéonBottou,YoshuaBengio,andPatrickHa˙ner.Gradient-basedlearn-ingappliedtodocumentrecognition.ProceedingsoftheIEEE,86(11):22781998.[72]YannLeCun,CorinnaCortes,andChristopherJCBurges.TheMNISTdatabaseofhandwrittendigits,1998.[73]AnneLong,LaurenceRouet,JesSLindholt,andEricAllaire.Measuringthemaximumdiameterofnativeabdominalaorticaneurysms:reviewandcriticalanalysis.EuropeanJournalofVascularandEndovascularSurgery,2012.[74]JonathanLong,EvanShelhamer,andTrevorDarrell.Fullyconvolutionalnetworksforsemanticsegmentation.InProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pages34312015.[75]DavidG.Lowe.Distinctiveimagefeaturesfromscale-invariantkeypoints.InternationalJournalofComputerVision,2004.[76]GMartu˝,MLindquistLiljeqvist,NSakalihasan,GPanuccio,RHultgren,JRoy,andTCGasser.LocalDiameter,WallStressandThrombusThicknessIn˛uencetheLocalGrowthofAbdominalAorticAneurysms.EuropeanJournalofVascularandEndovascularSurgery,48(3):349,2014.[77]RolandMemisevicandGeo˙reyHinton.Unsupervisedlearningofimagetransforma-tions.In2007IEEEConferenceonComputerVisionandPatternRecognition,pagesIEEE,2007.123[78]EmanueleMenegatti,TakeshiMaeda,andHiroshiIshiguro.Image-basedmemoryforrobotnavigationusingpropertiesofomnidirectionalimages.RoboticsandAutonomousSystems,2004.[79]EmanueleMenegatti,MauroZoccarato,EnricoPagello,andHiroshiIshiguro.Image-basedMonteCarlolocalisationwithomnidirectionalimages.RoboticsandAutonomousSystems,2004.[80]ChristopherZ.Mooney.Montecarlosimulation.Number116.Sage,1997.[81]MarkNixonandAlbertoSAguado.Featureextraction&imageprocessing.AcademicPress,2008.[82]GuillaumeObozinski,MartinJWainwright,MichaelIJordan,etal.Supportunionrecoveryinhigh-dimensionalmultivariateregression.TheAnnalsofStatistics,47,2011.[83]NaoyaOhnishiandAtsushiImiya.Appearance-basednavigationandhomingforau-tonomousmobilerobot.ImageandVisionComputing,31(6):511532,2013.[84]SugunaPappu,AlanDardik,HemantTagare,andRichardJ.Gusberg.Beyondfusiformandsaccular:anovelquantitativetortuosityindexmayhelpclassifyaneurysmshapeandpredictaneurysmrupturepotential.AnnalsofVascularSurgery,2008.[85]KosmasI.Paraskevas,DimitriP.Mikhailidis,VassiliosAndrikopoulos,NikolaosBessias,andSirPeterR.F.Bell.Shouldthesizethresholdforelectiveabdominalaorticaneurysmrepairbeloweredintheendovascularera?Yes.Angiology,61(7):61619,2010.[86]HanchuanPeng,FuhuiLong,andChrisDing.Featureselectionbasedonmutualinformationcriteriaofmax-dependency,max-relevance,andmin-redundancy.IEEETransactionsonPatternAnalysisandMachineIntelligence,2005.[87]PedroR.Peres-Neto,DonaldA.Jackson,andKeithM.Somers.Howmanyprincipalcomponents?stoppingrulesfordeterminingthenumberofnon-trivialaxesrevisited.ComputationalStatisticsandDataAnalysis,49(4):974997,2005.[88]LyPhan,KatherineCourchaine,AmirAzarbal,DavidAlanVorp,CindyGrimm,andSandraRugonyi.Ageodesics-basedsurfaceparameterizationtoassessaneurysmprogression.JournalofBiomechanicalEngineering,138(5):054503,2016.[89]CarolMPorth.Essentialsofpathophysiology:Conceptsofalteredhealthstates.Lip-pincottWilliams&Wilkins,2010.[90]CarlEdwardRasmussen.Gaussianprocessesformachinelearning.MITPress,2006.[91]SamarthSRaut,SantanuChandra,JudyShum,andEnderAFinol.Theroleofgeometricandbiomechanicalfactorsinabdominalaorticaneurysmruptureriskas-sessment.AnnalsofBiomedicalEngineering,pages9,2013.124[92]RaimundoRealandJuanMVargas.TheprobabilisticbasisofJaccard'sindexofsimilarity.SystematicBiology,pages385,1996.[93]RobertRoesser.Adiscretestate-spacemodelforlinearimageprocessing.IEEETrans-actionsonAutomaticControl,1975.[94]EricRoyer,MaximeLhuillier,MichelDhome,andJean-MarcLavest.Monocularvisionformobilerobotlocalizationandautonomousnavigation.InternationalJournalofComputerVision,2007.[95]DavidSalomon.Datacompression:thecompletereference.SpringerScience&BusinessMedia,2004.[96]TimoSchairer,BenjaminHuhle,PhilippVorst,AndreasSchilling,andWolfgangStraser.Visualmappingwithuncertaintyforcorrespondence-freelocalizationusingGaussianprocessregression.InProceedingsoftheIEEE/RSJInternationalConfer-enceonIntelligentRobotsandSystems(IROS),pages4235,2011.[97]DScharsteinandA.JBriggs.Real-timerecognitionofself-similarlandmarks.ImageandVisionComputing,19(11):763772,2001.[98]S.Se,D.G.Lowe,andJ.J.Little.Vision-basedgloballocalizationandmappingformobilerobots.Robotics,IEEETransactionson,June2005.[99]StephenSe,DavidLowe,andJimLittle.Mobilerobotlocalizationandmappingwithuncertaintyusingscale-invariantvisuallandmarks.TheinternationalJournalofroboticsResearch,2002.[100]ASheidaei,SCHunley,SZeinali-Davarani,LGRaguin,andSBaek.Simulationofabdominalaorticaneurysmgrowthwithupdatinghemodynamicloadsusingarealisticgeometry.Medicalengineering&physics,2011.[101]RobertHShumwayandDavidSSto˙er.AnapproachtotimeseriessmoothingandforecastingusingtheEMalgorithm.JournalofTimeSeriesAnalysis,1982.[102]RobertSimandGregoryDudek.Comparingimage-basedlocalizationmethods.InIJCAI,pages15602003.[103]EeroP.SimoncelliandWilliamT.Freeman.Thesteerablepyramid:a˛exiblearchi-tectureformulti-scalederivativecomputation.2ndIEEEInternationalConferenceonImageProcessing.,1995.[104]BrunoSinopoli,LucaSchenato,MassimoFranceschetti,KameshwarPoolla,MichaelIJordan,andShankarSSastry.Kalman˝lteringwithintermittentobservations.Auto-maticControl,IEEETransactionson,2004.[105]NitishSrivastava,Geo˙reyEHinton,AlexKrizhevsky,IlyaSutskever,andRuslanSalakhutdinov.Dropout:asimplewaytopreventneuralnetworksfromover˝tting.JournalofMachineLearningResearch,2014.125[106]ArthurHStroudandDonSecrest.Gaussianquadratureformulas.1966.[107]MJSweetingandSGThompson.Makingpredictionsfromcomplexlongitudinaldata,withapplicationtoplanningmonitoringintervalsinanationalscreeningprogramme.JournaloftheRoyalStatisticalSociety:SeriesA(StatisticsinSociety),586,2012.[108]SGThompson,HAAshton,LGao,MJBuxton,andRAPScott.Finalfollow-upoftheMulticentreAneurysmScreeningStudy(MASS)randomizedtrialofabdominalaorticaneurysmscreening.BritishJournalofSurgery,2012.[109]SebastianThrun.Bayesianlandmarklearningformobilerobotlocalization.MachineLearning,1998.[110]A.Torralba,K.P.Murphy,W.T.Freeman,andM.A.Rubin.Context-basedvisionsystemforplaceandobjectrecognition.InProceedingsoftheIEEEInternationalConferenceonComputerVision,pages2732003.[111]IwanUlrichandIllahNourbakhsh.Appearance-basedobstacledetectionwithmonoc-ularcolorvision.InProceedingsoftheNationalConferenceonArti˝cialIntelligence,pages2000.[112]C.ValgrenandA.Lilienthal.Incrementalspectralclusteringandseasons:Appearance-basedlocalizationinoutdoorenvironments.InIEEEInternationalConferenceonRoboticsandAutomation,2008.ICRA2008.,pages185May2008.[113]I.Vallivaara,J.Haverinen,A.Kemppainen,andJ.Roning.Simultaneouslocalizationandmappingusingambientmagnetic˝eld.InProceedingoftheIEEEInternationalConferenceonMultisensorFusionandIntegrationforIntelligentSystems,pages19,September2010.[114]JohnVilladsenandMichaelLMichelsen.Solutionofdi˙erentialequationmodelsbypolynomialapproximation,volume7.Prentice-HallEnglewoodCli˙s,NJ,1978.[115]N.Vlassis,R.Bunschoten,andB.Krose.Learningtask-relevantfeaturesfromrobotdata.InProceedingsoftheIEEEInternationalConferenceonRoboticsandAutoma-tion,volume1,pages2001.[116]GordonWellsandCarmeTorras.Assessingimagefeaturesforvision-basedrobotpositioning.JournalofIntelligentandRoboticSystems,2001.[117]GordonWells,ChristopheVenaille,andCarmeTorras.Vision-basedrobotpositioningusingneuralnetworks.ImageandVisionComputing,1996.[118]JSWilson,SBaek,andJDHumphrey.Importanceofinitialaorticpropertiesontheevolvingregionalanisotropy,sti˙nessandwallthicknessofhumanabdominalaorticaneurysms.JournalofTheRoyalSocietyInterface,2012.126[119]NiallWinters,JoséGaspar,GerardLacey,andJoséSantos-Victor.Omni-directionalvisionforrobotnavigation.InProceedings.IEEEWorkshoponOmnidirectionalVision,2000.,pages28.IEEE,2000.[120]CFJe˙Wu.OntheconvergencepropertiesoftheEMalgorithm.TheAnnalsofStatistics,pages103,1983.[121]JiachengWuandShawnCShadden.Coupledsimulationofhemodynamicsandvas-culargrowthandremodelinginasubject-speci˝cgeometry.AnnalsofBiomedicalEngineering,2015.[122]ZhirongWu,ShuranSong,AdityaKhosla,FisherYu,LinguangZhang,XiaoouTang,andJianxiongXiao.3dshapenets:Adeeprepresentationforvolumetricshapes.InProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pages2015.[123]Y.XuandJ.Choi.AdaptivesamplingforlearningGaussianprocessesusingmobilesensornetworks.Sensors,March2011.[124]YunfeiXuandJongeunChoi.AdaptivesamplingforlearningGaussianprocessesusingmobilesensornetworks.Sensors,2011.[125]ByronA.Zambrano,HamidrezaGharahi,ChaeYoungLim,FarhadA.Jaberi,JongeunChoi,WhalLee,andSeungikBaek.Associationofintraluminalthrombus,hemody-namicforces,andabdominalaorticaneurysmexpansionusinglongitudinalCTimages.AnnalsofBiomedicalEngineering,May2016.[126]ARZankl,HSchumacher,UKrumsdorf,HAKatus,LJahn,etal.Pathology,naturalhistoryandtreatmentofabdominalaorticaneurysms.ClinicalResearchinCardiology,2007.[127]SZeinali-DavaraniandSBaek.Medicalimage-basedsimulationofabdominalaorticaneurysmgrowth.MechanicsResearchCommunications,2012.[128]ShahrokhZeinali-Davarani,AzadehSheidaei,andSeungikBaek.A˝niteele-mentmodelofstress-mediatedvascularadaptation:applicationtoabdominalaor-ticaneurysms.ComputerMethodsinBiomechanicsandBiomedicalEngineering,2011.[129]YiZhou,YanWan,SandipRoy,ClarkTaylor,CraigWanke,DineshRamamurthy,andJunfeiXie.Multivariateprobabilisticcollocationmethodfore˙ectiveuncertaintyevaluationwithapplicationtoairtra˚c˛owmanagement.Systems,Man,andCyber-netics:Systems,IEEETransactionson,2014.[130]FengZhuge,Geo˙reyDRubin,ShaohuaSun,andSandyNapel.Anabdominalaor-ticaneurysmsegmentationmethod:levelsetwithregionandstatisticalinformation.MedicalPhysics,2006.127