FINGERPRINTRECOGNITION:CONTRIBUTIONSTOLATENTMATCHINGAND3DFINGERPRINTTARGETGENERATIONBySunpreetSinghAroraADISSERTATIONSubmittedtoMichiganStateUniversityinpartialoftherequirementsforthedegreeofComputerScienceŒDoctorofPhilosophy2016ABSTRACTFINGERPRINTRECOGNITION:CONTRIBUTIONSTOLATENTMATCHINGAND3DFINGERPRINTTARGETGENERATIONBySunpreetSinghAroraAutomaticcaptureandcomparisonmethodshaveledtotheubiquitoususeofpersonrecognitioninapplicationsrangingfromlawenforcementandbordercon-troltonationalandsmartphoneunlock.However,despitetremendousadvancementsinthestate-of-the-art,improvementsarestillneededincaseofsomechallengingapplications,e.g,torecognizepoorqualityanddistortedacquiredfromnon-cooperativeusers,improvereader,anddeterminecapabilityofdifferentreaders.Inthisthesis,weaddresstwosuchimpendingchallenges:(i)comparisonoflatentprintsfoundatcrimescenestolargecollectionsofreferenceprints(rolledtenprintsorslapinlawenforcementdatabases,and(ii)operationalevaluationofrecognitionsystemspriortolargescaledeployment.Wedevelopafeedbackparadigmthatusesreferenceprintfeaturestodynamicallyselectlatentfeaturesduringmatching.Theparadigmautomaticallydeterminesifdynamiclatentfeatureselec-tionwouldimproverecognitionperformanceusingastatisticalhypothesistestandqualitativelydecidestheregionsinlatentandreferenceprintsforapplyingfeedback.Theparadigmwhenusedinconjunctionwithastate-of-the-artlatentmatcherdemonstratesmarkedimprovement(0.5-3.5%)inlatentmatchingaccuracy.Further,wedevelopaframeworkforcrowdsourcinglatentprintfeaturemarkuptoapoolofexaminers.Theframeworkusesastatisticalcriteriontoautomaticallydeterminewhencrowdsourcingisrequired,andamethodtodynamicallydeterminethenumberofexaminersneededforlatentfeaturemarkup.recognitionperformanceimprovements(2.5-11.5%)areobtainedusingcrowdsourcedmarkupsinconjunctionwithastate-of-the-artlatentmatcher.Finally,wedesignandfabricateandwholehand3Dtargetsforoperationalevalu-ationofopticalandcapacitivereadersaswellasforend-to-endevaluationofrecognitionsystems.2Dcalibrationpatternswithknowncharacteristics(e.g.syntheticwithknownfeatures,sinegratingswithknownorientationandspacing)areprojectedontoelec-tronic3Dandhandsurfacestocreateelectronic3Dandwholehandtargets.Ahigh-resolution3Dprinterisusedtomanufacturephysical3Dandwholehandtar-getsfromelectronictargets.Othercontributionsinclude:(i)amethodtochemicallycleanthe3Dprintedtargetswithoutimpactingtheengravedtargetpatterns,(ii)aproceduretoapplyconductivecoatingofmetal/metaloxidesonthesurfaceof3DtargetsusingDCsputtering,(iii)mea-surementtechniquesusingopticalmicroscopytoassessthe3Dtargetgenerationprocess,and(iv)methodstoevaluatereadersusingthefabricated3Dtargets.Wedemonstratethatthe2Dcalibrationpatternfeaturesarereproducedwithhighbothontheelectronicandphys-ical3Dandwholehandtargetsandthattheintra-classvariationsbetweenimagesofthe3Dtargetsdonotdegradematchingaccuracy(at0.01%falseacceptrate).Weevaluateseveralcommerciallyavailablesingleandslapcontact-basedandcontactlessopticalreadersaswellascapacitivereadersusingthegenerated3Dtargets.CopyrightbySUNPREETSINGHARORA2016Toallmylovedones.vACKNOWLEDGMENTSAsIsitdowntowritethisdissertation,Iwouldliketotaketheopportunitytothankallmyteachers,includingmyparents,primary,middleschoolandhighschoolteachers,universitypro-fessorsandpeers,whohavetaughtmewonderfulthings,andfromwhomIhaveassimilatedamajorityofmylifelearnings.Iwillalwaysremainindebtedtoallofyoufornotonlysharingyourknowledgebutalsofuelingmycuriositytoexploreandlearn.Iamgratefultomyadvisor,Dr.AnilK.Jainforhisencouragementandsupportatbothprofessionalandpersonallevel.HebackedmetothehiltwhileIwastryingtostandonmyownfeetasaresearcher.OnceIdid,heletmetakemyownandexplore.IamalsothankfultoDr.XiaomingLiu,Dr.ArunRossandDr.SelinAviyenteforservingonmyPh.D.committee.ThePRIPlabprovidedmetherightkindofenvironmentforlearning.Interactionswithpeersinthelabduringthelastfouryearshavenotonlyadvancedmyknowledgebuthelpedmeappreciatethenuancesofresearch.,Iwouldliketomentionafewwhohaveleftalastingimpres-siononmeduetotheirsincerity,hardwork,andtenacity:Soweon,Serhat,Radha,Alessandra,Scott,Lacey,Tarang,Keyur,Charles,Inci,Josh,Debayan,Kai,Hu,andEryun.IwouldespeciallyliketothankKaiforallhishelpduringmyformativeyears.Afewspecialmentions:ScottforbeingtheoneIcouldexchangeideaswithforthecoupleofyears,CharlesandInciforbeingreallygoodfriendsandforlisteningtomyday-to-daygraduatelifeissuespatientlyoverlunch,andLaceyforbeingthefrequentcompaniontoattendvariousconferencestogetheraroundtheworld.Iwouldliketoacknowledgethefollowingorganizationsandindividualsforgenerouslysup-portingtheresearchprojectsIworkedon:NicholasG.PaulterJr.fromtheNationalInstituteofStandardsandTechnology(NIST),NISTMeasurementScienceprogramgrant60NANB11D155,theNationalScienceFoundation(NSF)CenterforTechnologyResearch(CITeR)grant1066197,KenWarmanfromtheBillandMelindaGatesFoundation,MarkThomasandShawnSarwarfromVaxTrac,andCaptainGregoryMichaudfromtheForensicScienceDivisionatviMichiganStatePolice.IamalsogratefultoBrianWright,MichiganStateUniversityandMatthewStaymates,NationalInstituteofStandardsandTechnology(NIST)fortheirassistancewith3Dprintingmodels,andAnnaSong,MichiganStateUniversityforherhelpincleaningthe3Dprintedmodels.IwouldalsoliketothankBrianWright,ChrisTraverseandLarsHaubold,MichiganStateUniversityfortheirhelpinsputtercoatingvariousmaterialson3Dtargets.IalsowanttothanktheadministrativestaffoftheComputerScienceDepartmentatMSU,includingNorma,Linda,Cathy,Debbie,KatyandCourtneyfortheirhelpwithvariousthingssuchascourseregistration,conferencetravelandreimbursementsduringthecourseofmystudy.IwouldliketoexpressmysinceregratitudetoDr.SalilPrabhakarforgivingmetheopportunitytointernatDeltaIDandextendingallpossiblehelptomakemystaycomfortableinthebayarea.ItwasawonderfulexperienceworkingatDeltaID,particularlywithAlexIvanisovandDr.YiChen.IowealottomyparentsandbrotherforbeingbymysidewhileIwasgoingthroughtheupsanddownsofmyjourneyasagraduatestudent.Withouttheirmoralsupport,writingthisdissertationwouldnothavebeenpossible.Lastbutnottheleast,IwouldliketomentionsomespecialfriendsImadeduringmytimehereinEastLansing.Theyprovidedmeimmensesupportindifferentwaystomakemystayworthwhile,TridipDas,GirishKasat,ShantanuKelkar,AbhishekSantra,SabyasachiHalder,SiddharthaDutta,Shailendra,RahulDeshpande,PrajaktaDeshpandeKulkarni,MayaPatel,TiasMaiti,RajibMandal,ShreyaNad,SoumenGhosh,ChetanTambe,OishiSanyal,SaptarshiDas,SaptarshiMukherjee,PortiaBanerjee,PikuMandal,ArkaprabhaKonar,PreetamGiri,andAritraChakroborty.Ialsothankmyfriendsbackhome,particularly,VidushiChaudharyandKajalJonejaforkeepingmemotivatedwhenthechipsweredown.viiTABLEOFCONTENTSLISTOFTABLES.......................................xiiLISTOFFIGURES......................................xviChapter1Introduction..................................11.1FingerprintFormation.................................21.2FundamentalTenets:UniquenessandPermanence..................41.3FingerprintMilestones................................51.3.1MajorStudies...........................61.3.2ApplicationsinLawEnforcement......................71.3.3OtherApplications..............................81.4ComparisonwithOtherTraits.............................101.5DesignofFingerprintRecognitionSystems.....................111.5.1FingerprintAcquisition............................121.5.1.1FingerprintSensingTechnologies.................151.5.2FeatureExtraction..............................181.5.3FingerprintMatching.............................201.5.3.1Exemplar-to-Exemplarmatching.................201.5.3.2Latent-to-Exemplarmatching...................221.6EvaluationofFingerprintRecognitionSystems...................231.6.1SensingTechnology.......................231.6.2FeatureExtractionandMatchingEvaluation.................241.7ChallengesinFingerprintRecognition........................261.7.1OpenResearchIssuesandChallenges....................261.7.1.1Automaticlatentmatching...............261.7.1.2Interoperabilityofreaders...............261.7.1.3Operationalevaluationofsystems...........271.7.1.4Fingerprintlivenessdetection...................271.7.1.5Fingerprinttemplatesecurity....................281.7.1.6Matchingnon-idealimages..............291.7.2AutomaticLatentFingerprintMatching...................291.7.3OperationalEvaluationofFingerprintSystems...............301.8DissertationContributions..............................34Chapter2LatentFingerprintMatching:PerformanceGainviaFeedbackfromEx-emplarPrints..................................372.1Introduction......................................372.1.1ManualLatentMatching...........................382.1.2Bottom-upLatentMatchingSystems.....................402.1.3ProposedTop-DownLatentMatchingFramework..............402.2FeedbackParadigmforLatentMatching.......................42viii2.3Re-sortingCandidateListbasedonFeedback....................432.3.1InitialMatchingandAlignment.......................452.3.2ExemplarFeatureExtraction.........................492.3.3LatentFeatureExtractionand..................492.3.4MatchScoreComputation..........................512.4TheAdequacyofFeedback..............................522.4.1GlobalCriterion................................522.4.1.1ModellingtheMatchScoreDistribution..............522.4.1.2Testforthepresenceofanupperoutlier..............532.4.2LocalCriterion................................562.5ExperimentalEvaluation...............................602.5.1Databases...................................602.5.1.1NISTSD27............................612.5.1.2WVU................................612.5.2SizeoftheCandidateList(K)........................622.5.3EffectivenessoftheGlobalCriterionforFeedback.............622.5.4PerformanceonNISTSD27Database....................652.5.5PerformanceonWVUDatabase.......................662.5.6ComputationalComplexity..........................682.6Conclusions......................................68Chapter3CrowdPoweredLatentFingerprintMatching:FusingAFISwithExam-inerMarkups..................................703.1Introduction......................................703.1.1Semi-automaticLatentMatching:AdvantagesandDisadvantages.....713.1.2ProposedCrowdPoweredLatentMatchingFramework...........723.2CollectiveWisdomofMultipleExaminers......................743.2.1Expertcrowdsourcingframework......................743.2.2Whentocrowdsource?............................763.2.3Howmanyexpertsareenough?........................773.3ExperimentalDetails.................................783.3.1Databases...................................793.3.2LatentMarkup................................803.3.3Experiments..................................813.3.3.1Lights-outMatching........................813.3.3.2MatchingIndividualExaminerMarkups..............813.3.3.3FusingMultipleExaminerMarkups................843.3.3.4Fusinglights-outAFISwithMultipleMarkups..........843.3.3.5Determiningtheneedforcrowdsourcing.............853.3.3.6Greedycrowdsourcing.......................863.4Conclusions......................................87Chapter4DesignandFabricationof3DSingle-FingerTargets............894.1Introduction......................................894.1.1StructuralEvaluationofFingerprintReaders.................91ix4.1.2BehavioralEvaluationofFingerprintReaders................924.1.33DTargetsforBehavioralEvaluation....................934.2Generating3DTargets................................974.2.1Preprocessing3Dsurface.......................994.2.1.1Alignment.............................994.2.1.2Remeshing.............................994.2.1.3Subdivision.............................1014.2.1.4Creatingoutersurface.......................1014.2.1.5Separatingfrontandrearportions.................1024.2.2Preprocessing2Dcalibrationpattern.....................1024.2.3Mapping2Dcalibrationpatternto3Dsurface................1034.2.4Engraving2Dcalibrationpatternon3Dsurface...............1064.2.5Postprocessing3Dsurface.......................1074.2.63Dprinting..................................1074.2.7Chemicalcleaning..............................1084.3Fidelityof3DTargetGeneration...........................1094.3.12Dto3DProjectionError..........................1094.3.23DprintingFabricationError.........................1104.3.3Fidelityof2Dpatternfeaturesduring3Dtargetcreation..........1114.3.3.1Fidelityof2Dpatternfeaturesafterprojectionto3Dsurface...1124.3.3.2Fidelityoftheengravedfeaturesonthe3Dsurfaceafter3Dprinting1134.3.3.3End-to-endof2Dcalibrationpatternfeaturesafter3Dprinting...............................1174.3.3.4Intra-classvariabilitybetween3Dtargetimpressions.......1174.4BehavioralEvaluationofFingerprintReadersusing3DTargets...........1184.4.1ExperimentI:SyntheticSineGratingTargets................1184.4.2ExperimentII:FingerprintTargets......................1204.5Conclusions......................................123Chapter53DWholeHandTargets:EvaluatingSlapandContactlessReaders....1255.1Introduction......................................1255.2GeneratingWholeHandTarget............................1315.3Fidelityof3DWholeHandTargetGeneration....................1355.3.1Replicationof2Dcalibrationpatternfeaturesonelectronic3Dhandtarget.1365.3.2Replicationofelectronic3Dhandtargetfeaturesonphysical3Dhandtarget1375.3.3Replicationof2Dcalibrationpatternfeaturesonphysical3Dhandtarget..1385.3.4Consistencybetweendifferentimpressionsofthephysical3Dhandtarget.1395.4EvaluatingContact-basedSlapFingerprintReaders.................1405.5EvaluatingContactlessSlapFingerprintReader...................1425.6Conclusions......................................145Chapter6Generating3DConductiveFingerprintTargets...............1466.1Introduction......................................1466.2SputterCoating3DTargets..............................1506.2.1DCSputteringProcess............................152x6.2.2ChoiceofSputteringMaterials........................1526.2.3SputteringTi+Au...............................1546.3ImpactofSputterCoatingon3DTargetFeatures..................1566.3.1Fidelityofphysical3Dtargetfeatureson............1586.3.2Fidelityof2Dcalibrationpatternfeatureson..........1596.3.3Intra-classvariabilitybetweenimpressionsof..........1606.4EvaluationofCapacitiveReaders...........................1616.4.1Largeareareader...............................1616.4.2Embeddedsmallareareaders.........................1626.5PresentationAttacksonCapacitiveReaders.....................1636.6Conclusions......................................165Chapter7Summary....................................1667.1FutureWork......................................168BIBLIOGRAPHY.......................................170xiLISTOFTABLESTable1.1Qualitativecomparisonofwithfaceandiris............10Table2.1Thetotalnumberoflatentswhere(a)feedbackisapplied,(b)feedbackisappliedwhenitisnotneeded(matedexamplarretrievedatrank-1bythebaselinematcher),and(c)feedbackisnotappliedwhenitcouldhavebeenuseful(matedexemplarreturnedamongstthetop200candidatesbutnotatrank-1bythebaselinematcher)basedontheglobalcriterionforfeedback(atlevel=0.05)..62Table3.1Summaryofthelatentdatabasesused......................80Table3.2Numberoflatentsmarkupsprovidedbyeachofthesixexaminers(outof258)fortheNISTSD27latents............................80Table3.3accuracy(%)oftheAFIS,onaverage,ontheNISTSD27against250Kreferenceprintswhenfedwithmarkupsfromdifferentsubsetsoflatentexaminers....................................84Table3.4Numberoflatentswheremarkupisrequired,markupisnotrequiredwhenmatedreferenceprintisnotatrank-1,andmarkupisrequireddespitethematedreferenceprintbeingretrievedatrank-1forNISTSD27(NIST27),ELFT-EFS(ELFT),andRS&A(RSA)databases.Thenumberoflatentsinthesethreedatabasesis258,255,and200,respectively......................86Table4.1Comparisonofprevailing2Dsyntheticbasedevaluationmethodswiththeproposed3Dtargetgenerationmethod....................94Table4.2Comparisonofmechanicalpropertiesofthetwo3Dprintermaterialsusedfor3Dtargetfabricationwiththehumanskin..................107Table4.3ObservedaveragegratingspacingonthreedifferenttargetswhenviewedundertheKeyenceVHX-600DigitalMicroscopeattwodifferent(50Xand100X).Expectedaveragespacingforeachtargetis0.478mm.......110Table4.4Similarityscoresbetweentheimages(2D)oftheelectronic3DtargetsinMeshlabandthe2DimagesfromNISTSD4usedfortargetgeneration.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33..............................112xiiTable4.5Similarityscoresbetweentheimages(2D)oftheelectronic3Dtargetsandtheimagescapturedbythethreeopticalreadersofthephysical3Dtargetsfabricatedwithtwodifferentmaterials(TangoBlackPlusFLX980andFLX9840-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.......................114Table4.6Similarityscoresbetweentheimagescapturedbythethreeop-ticalreadersofthe3Dtargetsfabricatedwithtwodifferentmaterials(TangoBlack-PlusFLX980andFLX9840-DM)andthefromNISTSD4usedintheirgeneration.Vnger6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.....................115Table4.7Rangeofsimilarityscoresforpairwisecomparisonsbetweenvedifferentimagescapturedbythethreeopticalreadersofthesame3Dtargetfabricatedwithtwodifferentmaterials(TangoBlackPlusFLX980andFLX9840-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33............................116Table4.8Mean()andstd.deviation(˙)ofcenter-to-centerspacingintheimagesofthethreedirectionaltesttargetscapturedusingthethreeopticalreaders(OR).(Expectedgratingspacing=8.278pixels.)..............119Table4.9Mean()andstd.deviation(˙)ofcenter-to-centerridgespacinginthegerprinttargetimagescapturedusingthethreeopticalreaders(OR).Theexpectedaverageridgespacing(inpixels)inthetargetimagesisindicatedinbrackets.........................................122Table5.1Comparisonofthemechanicalpropertiesofthethreeprintingmaterialsusedfor3Dwholehandtargetfabricationwiththehumanskin.TangoPlusFLX930andFLX9740-DMarerubber-likematerialssimilarinmechanicalpropertiestothehu-manskinandaresuitableforusewithcontact-basedslapreaders.RGD8520-DMisarigidopaquematerialthatprovidesoptimumridge-valleycontrastforusewiththecontactlessslapreader.........................135Table5.2Similarityscoresbetweenfrontalimages(2D)ofindividualen-gravedontheelectronic3DhandtargetcapturedinMeshlabandthecorresponding2DimagesfromNISTSD4usedfortargetgeneration.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.......................................137xiiiTable5.3Similarityscoresbetweenthefrontalimages(2D)oftheindividual-printsengravedontheelectronic3Dhandtargetandthecorrespondingplainprintsextractedfromaslapimageofthephysical3Dhandtargetscapturedbyeachofthethreecontact-basedslapreaders(SR1,SR2andSR3).Physicaltargetswerefabricatedwithtwodifferentmaterials(TangoPlusFLX930andFLX9740-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33..............................138Table5.4Similarityscoresbetweentheplainprintsextractedfromslapimpressionscapturedbythethreecontact-basedreaders(SR1,SR2andSR3)ofthephysical3DhandtargetsandthecorrespondingfromNISTSD4usedintheirgeneration.Physicaltargetswerefabricatedwithtwodifferentmaterials(Tango-PlusFLX930andFLX9740-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33...........139Table5.5Rangeofsimilarityscoresforpairwisecomparisonsbetweenplainprintsofthesameextractedfromvedifferentslapprintscapturedbythethreecontact-basedslapreaders(SR1,SR2andSR3)ofthesame3Dwholehandtarget.Resultsareshownfortwophysicalhandtargetsfabricatedwiththetwoprintingmaterials(TangoPlusFLX930andFLX9740-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33...140Table5.6Mean()andstd.deviation(˙)ofcenter-to-centerridgespacings(inpixels)intheplainprintsextractedfromvedifferentslapimagesofthe3Dwholehandtargetscapturedusingthethreecontact-basedslapreaders(SR1,SR2andSR3).Expectedaverageridgespacing(inpixels)foreach2DfromNISTSD4isshowninbrackets.Thespacingmeasurementstakeintoconsiderationthereduc-tioninspacingdueto2Dto3Dprojectionand3Dprintingfabricationerrors....141Table5.7Mean()andstd.deviation(˙)ofcenter-to-centerridgespacings(inpixels)intheplainprintsextractedfromvedifferentslapimagesofthecirculargratingwholehandtargetcapturedusingthecontactlessslapreader(CR).Ex-pectedaverageridgespacing(inpixels)ofthecirculargratingengravedonthehandtargetis8.28.Thespacingmeasurementstakeintoconsiderationthereductioninspacingdueto2Dto3Dprojectionand3Dprintingfabricationerrors........144Table6.1Mechanicalandelectricalpropertiesofhumanskin...............148Table6.2Qualitativecomparisonofdifferentthincoatingsappliedon3DtargetsusingDCsputtering..................................154Table6.3ParametersettingsforTi+AuDCSputtering..................156xivTable6.4Similarityscoresbetween500ppiplainimpressionoffabricatedphysical3Dtargetscapturedbytheopticalreaderto500ppiplainimpressionofthecor-respondingsputtercoatedcapturedbythecapacitivereader.Physical3DtargetsS0005andS0010werefabricatedwithTangoBlackPlusFLX980,andS0083andS0096werefabricatedwithFLX9840-DM.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.158Table6.5Similarityscoresbetweenplainimpressionsofthesputtercoatedgolcapturedusinga500ppicapacitivereadertothecorresponding2DfromNISTSD4usedintheirgeneration.Physical3DtargetsS0005andS0010werefabricatedwithTangoBlackPlusFLX980,andS0083andS0096werefabri-catedwithFLX9840-DM.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.................159Table6.6Rangeofsimilarityscoresbetweenvedifferent500ppiplainimpressionsofeachsputtercoatedgol.Physical3DtargetsS0005andS0010werefab-ricatedwithTangoBlackPlusFLX980,andS0083andS0096werefabricatedwithFLX9840-DM.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.......................160Table6.7Mean()andstd.deviation(˙)ofcenter-to-centerspacing(inpixels)intheimagesoftheerscapturedusingthe500ppicapacitivereader(CR).Expectedgratingspacing(inpixels)isshowninbrackets...........161xvLISTOFFIGURESFigure1.1Illustratingthefrictionridgepatternspresenton(a)thepalmsandofourhandand(b)ourfeet.Imagesreproducedfrom[110].............2Figure1.2Simulationoftheformationprocess.(a)-(b)Localizedridgeunitsappear,and(c)-(f)ridgeunitsmergetoformridgeswithuniquecharacteris-tics.Imageadaptedfrom[110].............................3Figure1.3FingerprintsofWilliamHerschel'ssonatages(a)7,(b)17,and(c)40years.Imagesreproducedfrom[102].............................4Figure1.4ClaysealswithimpressionsfromancientChina.Imagerepro-ducedfrom[141]....................................5Figure1.5Majormilestonesinrecognition.Imagereproducedfrom[122]..6Figure1.6TenprintcardusedbytheFBIforcollectingalltenImagereproducedfrom[189].Thetoptworowsshowtherolledandthebottomrowshowsthefourslapimpressionsandtheplainimpressionsofthetwothumbs.....................................8Figure1.7Exampleapplicationsofrecognition.(a)India'sAadharPro-gram[38],(b)ApplePay[1],(c)U.S.Visit(OBIM)[20],and(d)AccessControl..9Figure1.8Potentialapplicationsrequiringbiometricrecognitionofchildren(agerange:0-5years).(a)OperationASHA'smobilehealthcaree-compliancebio-metricsystem[22],and(b)AadhaarcivilregistryprojectinIndia[38]........10Figure1.9Designofarecognitionsystem.Themajorstepsinvolvedare:(a)Fingerprintacquisition,(b)preprocessing,(c)featureextraction,(d)compari-sonofthegeneratedtemplateagainstthereferencedatabase,and(e)dependingontherecognitionscenario,voftheclaimedidentity(1:1comparison)orestablishmentoftheidentity(1:Ncomparisons).Imageadaptedfrom[111].....11Figure1.10Off-linev.live-scanacquisitionmethods.(a)Traditionalink-on-paperbasedacquisition(off-line)[79],and(b)captureusingreader(live-scan)[142].Foracquiringrolledprints,operatorholdstheofthesubjectandguidesittoﬁrollﬂtheonpaperorreaderplaten....13Figure1.11Differenttypesofimpressions:(a)rolled,(b)plain,(c)slap,and(d)latent........................................14xviFigure1.12Acquisitionprinciplesofopticalandsolidstatesensingtechnologies.Im-agesreproducedfrom[50]...............................15Figure1.13Examplesofcommercialtouchlessreaders.(a)TBScontactless3Dreader[32],FlashScan3D's3Dtouchlessreader[10],(b)IDair'sOnePrint[21],and(c)Morpho'sFingerontheFly[17]................16Figure1.14Thethreedifferentlevelsoffeatures.Imagereproducedfrom[115]..........................................19Figure1.15Asimplemethodforminutiaematching.(a)and(b)Theenrolledandprobetemplatesmarkedwithminutiaesets,(c)alignmentofthetwotem-platesbasedonaninitialcorrespondingminutiaepairmarkedingreen,and(d)correspondingminutiaepointsgeneratedbasedonthealignmentin(c).Imagereproducedfrom[119].................................21Figure1.16Exampletargetsusedforcalibratingimagingsystems.Imagesadaptedfrom[107][137][163]....................................23Figure1.17Exampleproceduretocreateandirectlyfromalive.Plasticisusedtocreatethemoldandgelatinisusedasthecastingmaterial.Imagereproducedfrom[143].............................27Figure1.18Examplesofnon-idealimages.(a)Awithworn-outridgedetails,and(b)awithalteredpatterns(imagereproducedfrom[188])..........................................28Figure1.192Dsyntheticgenerationprocessusingthemethodin[192].(a)Fingerprinttypeis(b)ridgewmapisgeneratedfromalearnedstatis-ticalmodel,(c)minutiaearegeneratedbasedontheridgewin(b)andalearnedstatisticalminutiaemodel,an(d)2Dsyntheticissynthesizedusing(b)and(c).........................................31Figure1.20Exampleofagenerated3Dtarget.Shownontheleftisthe2Dimageandontherightisthe3Dsurfaceusedtocreatethe3Dtarget....................................32Figure1.21Sample3Dprintedwholehandtarget(a),and(b)evaluatingaslap-printreaderusingthe3Dwholehandtargetshownin(a)...............33Figure1.22Sampleconductive3Dtargetcreatedbydepositingthinlayersoftitaniumandgoldon3Dprintedtargetsshownin(a),and(b)animpressionofthein(a)capturedusingacapacitivereader.........34xviiFigure2.1SampleimagesfromtheNISTSD27databaseshownheretoelucidatesomeofthechallengesinlatentmatching:(a)poorridgeclarity,(b)insufcientamountofusableridgevalleypatternsand(c)presenceofcomplexback-groundnoise.Theredcurvesaremanuallymarkedforegroundareaintheimage..38Figure2.2Illustratingthetypicalbottom-updatawusedinlatenttoexemplarmatchingsystems.Thedottedlineshowsthefeedbackpath(top-downdataw)intheproposedmatchingparadigm..........................41Figure2.3Re-sortingthecandidatelistusingfeedback.Notetheoflatentfeaturesduetofeedback................................44Figure2.4Majorstepsinvolvedinlatentmatchingusingfeedbackfromexemplar.Showninyellowisapairofcorrespondinglatentandexemplarblocks..46Figure2.5Majorstepsinvolvedinlatentmatchingusingfeedbackfromexemplar.Thelatentfeaturesillustratedin(b)areusedtorematchthelatenttotheexemplarandre-sortthecandidatelist.Showninyellowisapairofcorrespondinglatentandexemplarblocks.......................48Figure2.6Exemplifyingtheglobalcriterionforfeedback:(a)matchscoredistributionforaparticularlatentquerywithoutanupperoutlier,and(b)withanupperoutlierpresent(markedinred).Feedbackisneededincase(a),butnotneededin(b)....53Figure2.7Exemplifyingthelocalcriterionforfeedback:(a)alatentimage,(b)itsridgeclaritymapand(c)regionswhichneedfeedback(showningrey);(d)anexemplarimage,(e)itsridgeclaritymapand(f)regionswhicharereliableforprovidingfeedback(showninwhite)..........................56Figure2.8Samplelatentimagesfrom(a)NISTSD27and(c)WVUlatentdatabases.Theirmatedexemplarsareshownin(b)and(d),respectively.............60Figure2.9Performanceofthebaselinelatentmatcheronthetwolatentdatabasesagainstareferencedatabaseof100,000exemplars..................61Figure2.10Performanceofthebaselinematcherwithandwithoutridgeorientationandfrequencyfeedbackon(a)NISTSD27and(b)WVUlatentdatabase(againstareferencedatabaseof100,000exemplars).......................63Figure2.11Genuineandimpostorsimilarityscoredistributions(scaledtothesamesim-ilarityscorerange)fortheNISTSD27database(a)beforeand(b)afterapplyingfeedbackusingthetop200candidatesretrievedbythebaselinematcher(againstareferencedatabaseof100,000exemplars).TheoverlapbetweenthegenuineandtheimpostorscoredistributionsreducesbyŸ25%afterapplyingfeedback......64xviiiFigure2.12SuccessfullatentfeatureviafeedbackforalatentintheNISTSD27database.Showninredistheexemplarorientationandinblueistheinitialandlatentorientationin(c)and(d),respectively.NotethatthelatentorientationisclosertotheexemplarorientationcomparedtotheinitiallatentorientationTherankofthematedexemplarofthelatentin(a)improvedfrom49to16amongstthe200candidateexemplarsreturnedbythebaselinematcherafterfeedback..........................65Figure2.13SuccessfullatentfeatureviafeedbackforalatentintheNISTSD27database.Showninredistheexemplarorientationandinblueistheinitialandorientationin(c)and(d),respectively.NotethatthelatentorientationisclosertotheexemplarorientationcomparedtotheinitiallatentorientationTherankofthematedexemplarofthelatent(a)im-provedfrom20to8amongstthe200candidateexemplarsreturnedbythebaselinematcherafterfeedback.................................66Figure2.14FailureoffeedbackforalatentintheWVUdatabase.Showninredistheexemplarorientationandinblueistheinitialandorientationin(c)and(d),respectively.ThelatentorientationisclosertotheexemplarorientationcomparedtotheinitiallatentorientationHowever,theretrievalrankofthematedexemplardegradedfrom16to49amongstthe200candidateexemplarsreturnedbythebaselinematcherafterfeedback.........67Figure3.1Twomarkups(bytwodifferentexaminers)foralatentimagefrom1000ppiELFT-EFSdatabase.Astate-of-the-artAFISwasunabletomakeahitforthelatentimageinlights-outmode(scoreof0withthetruemateinthereferencedatabase).However,feedingtheAFISwiththemarkupsshownin(a)and(b)resultedinthematedprintbeingretrievedatrank-1andrank-129,respectively...........71Figure3.2Theproposedcrowdpoweredlatentmatchingframework.(a)LatentisfedtoanAFIS,(b)itisdeterminedwhethermanualmarkupisneeded,(c)markupsareobtainedviaexpertcrowdsourcing,(d)multiplemarkupsarefedtotheAFIS,and(e)AFISscoresin(b)arefusedwiththemultiplemarkupscoresin(d)..........................................75Figure3.3Markupsbysixdifferentlatentexaminersforalatentimageinthe500ppiNISTSD27.......................................78Figure3.4Markupsbytwoexaminersforalatentinthe1000ppiELFT-EFSpublicchallengedatabase...................................79Figure3.5Markupforalatentimage(a)inthe1000ppiRS&Adatabase.Thematedreferenceprintofthelatentisshownin(b)......................79xixFigure3.6performance(CMCcurves)oftheAFISonNISTSD27when(i)operatinginlights-outmode(Imageonly),(ii)fedwithmarkupfromasingleexaminer(Image+Markup),and(iii)fusionoflights-outand500ppimarkupsfromallsixexaminers(Fusion)for(a)all258latents,(b)88goodqualitylatents,(c)85badqualitylatents,and(d)85uglyqualitylatents.Thesizeoftherefer-encedatabaseis250Krolledprints,includingthetruematesoflatentsfromNISTSD27.Theperformancebandofthelatentexaminersindicatesthemaximumandminimumaccuracyobtainedusinganindividualexaminermarkupatdifferentranks.82Figure3.7performance(CMCcurves)oftheAFISwhen(i)operatinginlights-outmode(Imageonly),(ii)fedwithanindividual1000ppimarkup(Image+Markup),and(iii)fusionoflights-outAFISscoreswiththescoresobtainedusingthetwo1000ppimarkups(Fusion)forall255latentsintheELFT-EFSdatabaseagainstareferencedatabaseof250Krolledprints...................83Figure3.8Performance(CMCcurves)oftheAFISwhen(i)operatinginlights-outmode(Imageonly),(ii)fedwiththesingleavailablemarkup(Image+Markup),and(iii)fusionoflights-outwithexaminermarkup(Fusion)forthe200latentsintheRS&Adatabaseagainstareferencedatabaseof250Krolledprints...83Figure3.9AnexamplelatentforwhichthematedreferenceprintisretrievedatahigherrankafterfusingthesixcrowdsourcedmarkupswiththeAFIS.Inthelights-outmode,theAFIScouldnotmatchthelatenttothematedprintshownin(g)(score=0).Therankofthematedprintusingtheindividualmarkupsbythesixexaminersshownin(a)-(f)is80,-(score=0),45,7,57and12971,respectively.Thematedprintisretrievedatrank-2usingthecombinationoftheAFISwiththesixmarkups.......................................85Figure3.10AnexamplelatentforwhichthematedreferenceprintisretrievedatalowerrankafterfusingthecrowdsourcedmarkupswiththeAFIS.Inthelights-outmode,theAFISretrievedthematedprintshownin(g)atrank-1.Therankofthematedprintin(g)usingtheindividualmarkupsbythesixexaminersshownin(a)-(f)is54,1171,3426,595,22and8450,respectively.Thematedprintisretrievedatrank-26usingthecombinationoftheAFISwiththesixmarkups...........87Figure3.11accuracyoftheAFISusinggreedycrowdsourcingforthe258NISTSD27latents.Startingwiththebestexaminer,alevelof0.05isusedtodecideifmarkupfromthenextbestexaminerisneeded.Numbersoflatentsgiventothenextbestexaminerareindicatedinred.DuetothepreponderanceoflowqualityprintsinNISTSD27,therank-1accuracytapersoffafterthreeexaminermarkups................................88xxFigure4.1Structural(White-Box)v.Behavorial(Black-Box)evaluationofgerprintreaders.Instructuralevaluation,detailsoftheinternalsetupofthereaderareknownandreadercomponentassemblyoperationistested.Ontheotherhand,inbehavorialevaluation,theinternaldetailsofthereaderarenotknownandonlyfunctionalityofthereaderistestedbasedonitsinputandoutput...........90Figure4.2Examplesofimagingphantomsusedinmedicalimaging:(a)Phannie,aphantomtocalibrateMRImachinesdevelopedatNIST[14],(b)aphantomhandusedforevaluatingX-raymachines[41],and(c)atorsophantomusedtocalibrateCT-Scanmachines[34].................................91Figure4.32Dimagesofstandardtargetsusedforcalibratingreaders,(a)ronchi(verticalbar)targetforcalibratingthegeometricaccuracy,(b)sinewavetargetformeasuringtheresolution,and(c)multiplebartargetforestimatingthespatialfrequencyresponseofareader(imagestakenfrom[155]).....92Figure4.4Evaluatingaopticalreaderusingthe3Dtargetsdesignedandfabricatedbytheauthors.(a)The3Dtargetiswornona,(b)theisplacedonthereaderplaten,and(c)-(f)multiple2Dimpressions(fourshownhere)ofthe3Dtargetarecapturedtoevaluatethereader..93Figure4.5Generatinga3DtargetA,givena2DcalibrationpatternIanda3DsurfaceS..................................95Figure4.6Preprocessing3Dsurface.(a)OriginalsurfaceS,(b)aligningSsuchthatthelengthisalongtheyaxis,(c)alignedS(triangularmesh),(d)remeshingS(triangularmesh),(e)subdividingS(triangularmesh),(f)subdividedSview),(g)creatingoutersurfaceSOfrom(f),and(h)separatingfrontandrearportions,SOFandSOR,ofSO......................100Figure4.7Preprocessinga2Dpatternbeforeprojectingitonto3Dsurface.(a)OriginalimageI,(b)extractedskeletonISofthein(a),(c)skeletonISin(b)afterapplyingthemorphologicaloperationofdilation,and(d)dilatedskeletonin(c)smoothedusingagaussian............103Figure4.8Mappingandengraving2Dcalibrationpatternontothefrontportionoftheouter3DsurfaceSOF.(a)3DfrontaloutersurfaceSOF,(b)frontalsurfaceSOFin(a)isprojectedinto2D,(c)the2DprojectedfrontalsurfaceSOFPissubdivided,(d)correspondencesaredeterminedbetweenthe2DprojectedfrontalsurfaceSOFPand2DcalibrationpatternI,(e)3DfrontaloutersurfaceSOFin(a)isdisplacedalongthesurfacenormalstoengravethepattern.......104xxiFigure4.9Postprocessing3Dsurface.(a)Separatedfrontandrearportionsofouter3Dsurface,(b)frontandrearportionsshownin(a)arecombinedtocreatetheouter3Dsurface,(c)outer3Dsurface(bottomview),(d)theretainedoriginal3Dsurface(bottomview),(e)electronic3Dtargetcreatedbystitchingtheouterandoriginalsurfacein(c)and(d)................106Figure4.10The2Dimagesofamanuallycleaned3Dtarget(shownontheleft)andthesametargetafterchemicalcleaning(shownontheright)capturedusingaopticalreader.Chemicalcleaningofthe3Dtargetwith2MNaOHsolutionandwaterremovesthe3Dprintersupportmaterialresidueandprovidesabetterqualityimage.............................108Figure4.11Thetwosourcesoferrorin3Dtargetgeneration(showninred)givena2Dcalibrationpatternanda3Dsurface:(i)2Dto3Dmapping,(ii)3Dprintingfabrication.......................................109Figure4.12Estimating3Dprintingfabricationerrorbymeasuringpoint-to-pointdis-tancesbetweenhorizontalgratingsona3Dtargetat(a)50Xand(b)100Xmagni-usingtheKeyenceDigitalMicroscopeVHX-600...............110Figure4.13Minutiaecorrespondencebetween(a)rolledimage(S0083fromtheNISTSD4),and(b)2Drenderingoftheelectronic3Dtargetgeneratedusing(a).Similarityscoreof116isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.............................113Figure4.14Minutiaecorrespondencebetween(a)imageoftheelectronic3Dtarget(ofS0083inNISTSD4),and(b)theimagecapturedbyopticalreader2(1000ppi)ofthephysical3DtargetfabricatedwithFLX9840-DM.Similarityscoreof473isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.......................................114Figure4.15Minutiaecorrespondencebetween(a)rolledimage(S0083fromtheNISTSD4),and(b)theimagecapturedbyopticalreader2(1000ppi)ofthe3Dtargetgeneratedusing(a)andfabricatedwithFLX9840-DM.Similarityscoreof374isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR...................................115Figure4.16Minutiaecorrespondencebetweentwoimages(a)and(b)capturedbyopti-calreader2(1000ppi)ofthe3DtargetgeneratedfromS0083inNISTSD4andfabricatedwithFLX9840-DM.Similarityscoreof1494isobtainedbe-tween(a)and(b)whichisabovethethresholdof33at0.01%FAR.........116xxiiFigure4.17Evaluatingopticalreaderswitha3Dtargetgeneratedusingahorizontalsinegrating.(a)Horizontalsinegrating(10pixelseparationbetweenthegratings);(b)electronic3Dtargetgeneratedusing(a);(c),(d)and(e)aresam-pleimagesofthefabricatedtargetcapturedusingopticalreaders1,2and3,re-spectively.Thereisaslightdistortionapparentin(b)thatisduetothe2Dto3Dprojectionerror.....................................119Figure4.18Evaluatingopticalreaderswitha3Dtargetgeneratedusingaverticalsinegrating.(a)Verticalsinegrating(10pixelseparationbetweenthegrat-ings);(b)electronic3Dtargetgeneratedusing(a);(c),(d)and(e)aresampleim-agesofthefabricatedtargetcapturedusingopticalreaders1,2and3,respectively.Thereisaslightdistortionapparentin(b)thatisduetothe2Dto3Dprojectionerror.120Figure4.19Evaluatingreaderswitha3Dtargetgeneratedusingacircularsinegrating.(a)Circularsinegrating(10pixelseparationbetweenthegratings);(b)electronic3Dtargetgeneratedusing(a);(c),(d)and(e)aresampleimagesofthefabricatedtargetcapturedusingopticalreaders1,2and3,respectively.Thereisaslightdistortionapparentin(b)thatisduetothe2Dto3Dprojectionerror...121Figure5.1Tenprintcapture(fourcaptureofeachofthetwohands(shownin(a)and(b))followedbysimultaneouscaptureofthetwothumbs)byaUnitedStates(US)CustomsandBorderProtection(CBP)ofataportofentryintheUS.Imagereproducedfrom[158].............................126Figure5.23Dwholehandtargetforevaluatingslapandcontactlessreaders.(a)Electronic3Dhandtargetcompletewiththefourthumbandglove;theindexandmiddleengravedonthetargetareshownatfullscaleinredandblueboxes,respectively.(b)Fabricatedhandtargetwithtranslu-centrubber-likematerialTangoPlusFLX930[31].(c)slapcapturebyacontact-basedreaderusingthefabricatedhandtargetin(b)..............127Figure5.3Imagesofa3Dtargetfabricatedwithtranslucentrubber-likematerialTangoPlusFLX930[31](shownin(a))capturedbythreedifferentPIV[155]opticalreadersusingdifferentwavelengthsoflightforcapture:(b)bluewavelength,(c)combinationofblueandredwave-lengths,and(d)redwavelength.Targetsprintedwithblackcoloredrubber-likematerials(TangoBlackPlusFLX980[31]andFLX9840-DM[30])couldnotbeimagedusingthesethreereaders............................129Figure5.4Generatinga3Dwholehandtargetfromageneric3Dhandsurfaceandasetof2Dcalibrationpatterns..............................131Figure5.5Cleaningandassemblingthe3Dprintedandglovestocreateawholehand3Dtarget.....................................132xxiiiFigure5.6Sample3Dtargetsfabricatedfor(a)contact-basedreaders(us-ingtranslucentrubber-likematerialFLX9740-DM[30])and(b)contactlessreaders(usingrigidopaquematerialRGD8520-DM[30])...................134Figure5.7Sampleslapimpressionofthe3Dwholehandtargetcapturedusingacontact-basedslapreader................................137Figure5.8Circularsinegrating(ridgespacing=10pixels)usedtogeneratethe3Dwholehandtarget(shownin(a))andtheslapimpressionofthecorrespondinghandtargetcapturedusingthecontactlessslapreader(shownin(b)).Thecircularsinegratingappearstoexhibitthemoireeffect[16]..................143Figure6.13Dtargetsforevaluatingcapacitivereaders.(a)Sample(er)target,and(b)animpressionofthein(a)capturedusingaPIV500ppicapacitivereader...........................147Figure6.2Mainstepsinvolvedincreatingagivena2Dcalibrationpatternanda3Dsurface.................................150Figure6.3DCsputterdepositiontocoatthe3Dtargetswiththinlayersofconductivematerials.(a)SimplrepresentationoftheDCsputteringprocess(imagere-producedfrom[175]),and(b)theDentonVacuumDCsputteringsystem[4]usedforDCsputtering.Titanium(Ti)andGold(Au)ionsfromthecathodetargetaredepositedontheanodesubstrateusingArgon(Ar)astheprocessgas........151Figure6.43Dtargetscoatedwith(a)and(b)athinlayer(300nm)ofsilver(Ag)andcopper(Cu),respectivelyoverathinlayer(30nm)oftitanium(Ti),and(c)100nmoftin(Sn)dopedindiumoxide(ITO).Thetargetsin(a)and(b)wereprintedwithTangoBlackPlusFLX980[31]andthetargetin(c)wasprintedwithTangoPlusFLX930[31].Targetscoatedwithotherconductivetransparentoxidesarenotshownherebecausetheyarevisuallysimilarto(c)..............153Figure6.53Dmountfabricatedtoholda3Dtargetforstableplacementonthesput-teringsystem'srotaryplatform.(a)Electronic3Dmodel,(b)3Dprintedphysicalmodeland(c)3Dtargetonthemountshownin(b)aftergoldcoating........155Figure6.6Sampleimpressionsshownin(b)ofacapturedusingtheembed-dedcapacitivereaderdesignedforsmartphonesin(a).................157Figure6.7Minutiaecorrespondencebetween(a)plainimpressionofthe3DtargetgeneratedusingimageS0083fromNISTSD4capturedbytheopti-calreader,and(b)plainimpressionofthesametargetcapturedbythecapacitivereader(a).Similarityscoreof680isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR...........................158xxivFigure6.8Minutiaecorrespondencebetween(a)rolledimageS0083fromNISTSD4,and(b)plainimpressionofthe3Dtargetgeneratedusing(a)capturedbythecapacitivereader.Similarityscoreof183isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR...................159Figure6.9Minutiaecorrespondencebetweentwodifferentplainimpressions(a)and(b)ofthesame3DtargetgeneratedusingimageS0083fromNISTSD4capturedbythecapacitivereader.Similarityscoreof1164isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR...................160Figure6.10Evaluationofcapacitivereadersembeddedinsmartphonesusinggers.(a)EnrolmentofaonanAppleiPhone6s,and(b)unlockingoftheiPhone6susingthesame........................162Figure6.11Sample3Dspoof.(a)Electronic3Dspoof,and(b)physical3Dspoofafterconductivecarboncoating............................163xxvChapter1IntroductionﬂPerhapsthemostbeautifulandcharacteristicofallmarksarethesmallfurrowswiththeinterveningridgesandtheirporesthataredisposedinasingularlycomplexyetevenorderontheundersurfacesofthehandsandthefeet.ﬂ-FrancisGalton,1889[97]Theepidermalridgepatternsfoundonthepalmsandofourhandsandthesolesofourfeet(Figure1.1)havelongcaptivatedtheimaginationofthelaymanandintriguedscientistsandexpertsalike.Itistheseridgepatternspresentonourthatarecommonlycallederprints.Fingerprintsdifferfrompersontoperson(evenidenticaltwinshavedifferentprints[141])anddonotchangeovertime.Hence,theyareareliablesourceforuniquelyidenti-fyingindividuals.FrombeingusedinancientBabylonandChinaasaproofofinbusinessandlegaltransactionstobeingdeployedinthe21stcenturyforpersonalinlarge-scalecriminal,civilianandgovernmentalapplications[141],theutilityofasapersonalhasmanifestedubiquitously.Advancesinboththescienceandtechnologyofoverthelastfewdecadeshaveresultedinwidespreadapplicationsofbasedpersonrecognition,includingdeviceunlockmechanismsinmoderndaysmartphones[36]andonlinetransactions[1][26](seeFigure1.7).1(a)(b)Figure1.1Illustratingthefrictionridgepatternspresenton(a)thepalmsandofourhandand(b)ourfeet.Imagesreproducedfrom[110].Inthischapter,wedescribetheformationprocessandhighlightthetwofunda-mentalpropertieswhichpurportedlymakeusefulforpersonrecognition:uniquenessandpermanence.Wethenenumeratethekeymilestonesintheprogressionoftheuseof-prints.Subsequently,wedescribethedesignofmoderndayautomatedsystems(AFIS),andtheexistingevaluationmethodsforthesesystems.Althoughtheresearchcommunityhasmadeadvancesoverthelastfewdecades,therearestillcertainchallengingavenuesinrecognitionwherefurtheradvancesarerequired.Weidentifyanddiscusssomeoftheseproblemsinrecognition.Finally,weconcludethechapterbydetailingthecontributionsofthisdissertationinsolvingtwooftheaforementionedproblems.1.1FingerprintFormationItistypicallypresumedthattheoutermorphologyofthefrictionridgeskinpresentonourisadirectofitsfunction:toprovideappropriatefrictionforassistanceingraspingorholdingobjectsandhelpinsensingtexture[139].Thegenerallybelievednotionisthatfric-tionridgeskiniscreatedfrommanysmalllocalizedridgeunits[57].Theseridgeunitsappearat1or2focalpointsontheAtapproximately10.5weeksofgestationalage,theridge2Figure1.2Simulationoftheformationprocess.(a)-(b)Localizedridgeunitsappear,and(c)-(f)ridgeunitsmergetoformridgeswithuniquecharacteristics.Imageadaptedfrom[110].unitsmergetogetherunderrandomforcestoformveridgecharacteristics,suchasridgebifurcationsandendings[57](seeFigure1.2).Duetotherandomnatureofforcesactingontheridgeunits,thesecharacteristicsarebelievedtobeunique.Theformationprocesspre-sumablystartsdeepbeneaththeskininthesecondarydermallayers,whereskincellsareproducedandmoveupwardstotheepidermis[71].IntheirstudyonmicrocirculationofhumanSangiorgietal.[170]notedthattheﬁregulardispositionofcapillariesbeneaththedermissharplyfollowedtheerprintpattern,reproducinganidenticalvascularerprintwiththesamein-dividualarchitectureﬂ.Theseobservationssuggestthepermanenceofminorcutsandbruisesonthedonotchangepatternsbecausenewskincellsaregeneratedbeneaththeepidermisandfacilitatethereformulationofpatternsontheepidermis.3(a)(b)(c)Figure1.3FingerprintsofWilliamHerschel'ssonatages(a)7,(b)17,and(c)40years.Imagesreproducedfrom[102].1.2FundamentalTenets:UniquenessandPermanenceTherearetwofundamentaltenetsofthatunderlietheiruseforrecognizingindividuals.1.Uniqueness:Fingerprintsareabletouniquelyidentifyeachindividual,i.e.,notwoevenofthesameindividual,haveidenticalridgestructure.2.Permanence:Fingerprintsdonotchangeoverthelifetimeofanindividual.Tothisdate,onlyafewstudieshaveattemptedtovalidatethesetwotenets.Severalofthemhaveattemptedtoshowthateveryindividualisunique[164][193][174][173][174].ItisalsobelievedthatindividualssharingthesameDNAhavedifferentForexample,Jainetal.[123]analyzedcollectedfrom94pairsofidenticaltwinsanddemonstratedthatidenticaltwinscanbedistinguishedusingHowever,alloftheaforementionedstudiesareeitherbasedonrelativelysimplestatisticalmodelsoffeaturesorbasedonempiricalstudiesinvolvingonlyasmallnumberofsubjects.WilliamHerschelwasthetodemonstratethepermanenceofHecapturedofhissonatthreedifferentages7,17and40yearsoldandconcludedthattheridgedetailspresentdonotchangeovertime[102].However,Herschel'sconclusionswerebasedoncollectedfromjustonesubject.Recently,YoonandJain[189]conductedaformalstudyinvolvinglongitudinalrecordsof15,597subjects.Theyusedmultilevelstatistical4Figure1.4ClaysealswithimpressionsfromancientChina.Imagereproducedfrom[141].modelsandastate-of-the-artAFIStoshowthatrecognitionaccuracyofanAFISdoesnotdegradewithtime.Insummary,whileanecdotallywehavebeenleadtobelievethatexhibitthetwoessentialtenetsforpersonrecognition,uniquenessandpermanence,itisnotyetsupportedbysoundstudies.ThiswasoneofthecriticalissuespointedoutbytheNationalResearchCouncil(NRC)initsreportonstrengtheningforensicscienceintheUnitedStates[78].Itisalsoextensivelydiscussedintherecentlyreleasedreportonensuringvalidityoffeature-comparisonmethodsbythePresidentsCouncilofAdvisorsonScienceandTechnology(PCAST)[45].1.3FingerprintMilestonesTheearliestrecordoftheuseoffrictionridgeimpressionsdatesbackto1955-1913BCwhenclaytabletswithwereusedforconductingbusinesstransactionsinancientBabylon[141].ClaysealswithimpressionsthatwerebeingusedforlegaltransactionsinancientChinabetween600-700ADhavealsobeendiscovered[141](seeFigure1.4).AprehistoricpictureofahandwithfrictionridgepatternswasfoundinNovaScotia[149].Historicalevidence,clearly,seemstosuggestthathumanwereusedinancienttimesasameansforpersoniden-5Figure1.5Majormilestonesinrecognition.Imagereproducedfrom[122].However,thereisnoevidencethatanysystematicmethodswerebeingusedforpersonusing1.3.1MajorStudiesDespitetheevidentuseofasaﬁsealﬂforthepurposeofpersonsincetheancientera,recordsofworkonarecomparativelyrecentandonlybegantoemergeinthelate19thcentury.Intheyear1858,SirWilliamHerschelinhiscapacityasaBritishadministratorinthestateofWestBengalinIndia,madeitmandatorytouseoncivilcontractsforpayrollpurposes[98].In1880,HenryFauldsusedprinterinktocapture[90].In1892,FrancisGaltonwrotethelandmarkbooktitledFingerPrints[96],wherehefeatureswhichpurportedlymakeeachunique,suchasridgeendingsandbifurcations,andproposedthatcouldbeusedforpersonIn1899,EdwardHenryintroducedantsystem,whichlaterbecamepopularasthe6ﬁHenrySystemof[98].Over60yearshence,thepaperonautomaticcomparisonofbyMitchellTrauringappearedinthejournalNature[179].1.3.2ApplicationsinLawEnforcementInthelate19thcentury,begantobeusedbylawenforcementagenciesforestablishingtheidentityofcrimesuspects.Intheyear1893,weresupposedlyusedforthetimeasanofevidencetoconvictamotherwhohadmurderedhertwochildreninArgentina[101].TheScotlandYardstartedrecordingofcriminalsaround1900[76].Postthesedevel-opments,theuseofforidentifyingandapprehendingcriminalsbecamewidespread.TheUnitedStatesCongressmadeitmandatorytocollectofcriminalsin1924[98].Consequently,theFederalBureauofInvestigation(FBI)establisheditsdivisionandbegancollectingofcriminals[159].Manualcomparisonandmaintenanceofalargenumberofprintsbecameincreasinglydifcult.Asaresult,therewasacompellingneedtoautomatethemanualprocesses.Followingthis,researchanddevelopmentofAutomatedFingerprintSystems(AFIS)wasinitiatedbytheFBIinthe1970s[130].Lawenforcementagenciesatthestateandlocallevelalsobeganinstallingsuchsystems.In1999,FBI'sIntegratedAFIS(IAFIS)startedallowingelectronicrecordsubmissionsfromstateandlocallawenforcementagenciestothenationalrepositoryaswellasin-troducedcapabilitiesfortheseagenciestodirectlysearchrecordsinthenationalrepository[130].TheFBI'srepositoryhasover70millioncriminaland34millionciviliansetsoftenprints(Figure1.6)currentlyon[91].In2011,theFBIintroducedtheNextGeneration(NGI)systemwithenhanced(aswellasface,palmprint,andiris)recognitioncapabilitieswithreportedmatchingaccuracyashighas99.6%[160].Accordingtotheirestimates,theintroductionofthissystemhasreducedtheneedformanualreviewsbyexaminersbyasmuchas90%[160].Atpresent,arebeingusedfortwomainpurposesbylawenforce-mentagencies:(i)identifyingrepeatoffenders(tenprint-to-tenprintmatching),and(ii)determiningwholeftlatentoratacrimescene[124].7Figure1.6TenprintcardusedbytheFBIforcollectingalltenImagereproducedfrom[189].Thetoptworowsshowtherolledandthebottomrowshowsthefour-slapimpressionsandtheplainimpressionsofthetwothumbs.1.3.3OtherApplicationsThelasttwodecadeshaveseengrowinguseofngerprintsinbordercontrol,accesscontrol,civilregistryandahostofotherapplications(seeFigure1.7).Examplesincludethefollowing:(i)India'sAadhaarprograminitiatedbytheUniqueAuthorityofIndia(UIDAI)thataimstoassignaunique12-digitnumbertoeveryresidentofIndia,andhasalreadyenrolledoveronebillionIndianresidents[38].(ii)thesystemtopreventcriminalsandimmigrationviolatorsfromcrossingtheUnitedStatesborderbytheOfofBiometricIdentityManagementServices(formerlytheUS-VISITprogram)[20],(iii)thescansystemdeployedatWaltDisneyWorldThemeParkssince2005tohelppreventtheuseofstolenorfraudulentticketsforenteringtheirpremises[40],and(iv)inmobiledevicesforauthenticatingusers(e.g.theTouchIDsystemintroducedin2013intheAppleiPhones[36]andthesystemin8Figure1.7Exampleapplicationsofrecognition.(a)India'sAadharProgram[38],(b)ApplePay[1],(c)U.S.Visit(OBIM)[20],and(d)AccessControl.Samsungphones[25])andconductingonlinetransactions(e.g.ApplePay[1]introducedin2014andSamsungPay[26]in2015).Figure1.5summarizesthemajormilestonesinrecognition.Emergenceofimportantsocietalapplicationsinthelastfewyears,e.g.,vaccinationtrackingofchildren,preventingnewbornswappinginhospitalsandidentifyingmissingchildren(seeFigure1.8),hasignitedtheinterestofnationalandinternationalhealthorganizations,aswellasnon-governmentalorganizations,inexploringmethodstorecognizechildren(agerange:0-5years)usingtheirphysicaltraits.Comparedtootherphysicaltraits,recognitionofchildrenappearspromising[114]because(i)canbecapturedwithrelativeease,incontrasttoiris,forexample,whichrequiresthechildtobesteadyandstaredirectlyintotheiriscapturedevice,and(ii)areknowntobepersistentcomparedtofacialcharacteristics,forinstance,whichcanchangedrasticallyasthechildgrows.Researcheffortsarebeingmadetoactivelyexploretheuseofforinfantandtoddlerrecognition[114][112][113].9(a)(b)Figure1.8Potentialapplicationsrequiringbiometricrecognitionofchildren(agerange:0-5years).(a)OperationASHA'smobilehealthcaree-compliancebiometricsystem[22],and(b)AadhaarcivilregistryprojectinIndia[38].Table1.1Qualitativecomparisonofwithfaceandiris.TraitUniquenessPermanenceEaseofCaptureRecognitionPerformanceLegacydatabasesFingerprintHighHighMedium(obtrusive)HighYesFaceMedium(identicaltwins)Medium(facialaging)High(unobtrusive)MediumYesIrisHighHighLow(mostobtrusive)HighNo1.4ComparisonwithOtherTraitsFingerprintsarearguablythemostcommonlyusedphysicaltraitforpersonrecognition.How-ever,besidesthereareotherphysicalandbehavioraltraits,e.g,face,iris,voiceandgait,thatareusefulforpersonrecognition[118].Faceandiris,inparticular,havebeenusedforpersonrecognitioninavarietyofapplications.Lawenforcementagencies,suchastheFBI,usefacerecognitiontoidentifysuspectsfromstillphotosandvideosrecordedatcrimescenes,andtheDepartmentofMotorVehicles(DMV)intheUnitedStatesusesfacerecognitiontechnologytopreventdriver'slicensefraud[120].India'sAadhaarprogram[38]usesirisinadditionto-10Figure1.9Designofarecognitionsystem.Themajorstepsinvolvedare:(a)Finger-printacquisition,(b)preprocessing,(c)featureextraction,(d)comparisonofthegeneratedtem-plateagainstthereferencedatabase,and(e)dependingontherecognitionscenario,voftheclaimedidentity(1:1comparison)orestablishmentoftheidentity(1:Ncomparisons).Imageadaptedfrom[111].printsforassigningauniqueidentitytoeveryIndianresident.Irisrecognitionisalsobeingusedforsmartphoneunlockandpayments[81].Table1.1showsaqualitativecomparisonofwithfaceandiris.Fingerprintsaremoredistinctiveandpermanentandtypicallyprovideahigherrecognitionperformancerelativetoface.Theyareeasiertocaptureandhavelegacylawenforcementdatabasesincontrasttoiris.Duetothesereasons,areoftenpreferredoverfaceandirisinlargescalepersonrecognitionapplications.1.5DesignofFingerprintRecognitionSystemsWithrecognitionbeingusedinawidevarietyofapplications,recognitionmethodshaveevolvedrapidlyovertheyears.Advancementsinbothsensingtechnology11andrecognitionalgorithmshaveresultedinextremelyefandaccurateAutomatedFingerprintSystems(AFIS).Automatedrecognitionprocessconsistsofthefollowingtwostages:1.Enrolment:Duringenrolment,ofauserisacquiredandsalientfeaturesareex-tractedfromtheresultingimagetogenerateaerprinttemplate.ThetemplateisthenstoredwiththeuserIDinadatabasewhichisgenerallycalledthereference,background,orenrolmentdatabase.2.Recognition:Intherecognitionphase,thegoalistoeitherverifytheclaimedidentityofaperson()orestablishtheidentityofaperson().Inboththesescenarios,isacquiredandfeaturesareextractedtogenerateatemplatewhichisoftencalledtheprobeorthequerytemplate.Forvthequerytemplateiscomparedtotheenrolledtemplatesoftheclaimedidentity(1:1comparison)inthereferencedatabase.Ontheotherhand,forwherenoexplicitidentityclaimismade,thequerytemplateismatchedagainsteachenrolledtemplateinthereferencedatabase(1:Nsearch)toestablishtheidentity.Below,weexplainindetail,themajorstepsinvolvedinatypicalrecognitionsystem:acquisition,featureextractionandmatching1(seeFigure1.9):1.5.1FingerprintAcquisitionBroadlycategorizing,therearetwomainmethodsforcontrolledcaptureofimpressions:(i)off-linemethodsusing,e.g.,ink-on-paper,whichacquireonaphysicalmedia,and(ii)live-scanmethods,usingreaderswhichsenseelectronically(seeFigure1.10).Off-lineacquisitionmethodswereprimarilyusedbylawenforcementagenciestorecordofcrimeperpetrators.However,inthelastfewyears,live-scanmethodshavelargely1Thetermmatchingherereferstothecomparisonoftwofeaturesetstoascertainwhethertheybelongtothesameordifferentsource.12(a)(b)Figure1.10Off-linev.live-scanacquisitionmethods.(a)Traditionalink-on-paperbasedacquisition(off-line)[79],and(b)captureusingreader(live-scan)[142].Foracquiringrolledprints,operatorholdstheofthesubjectandguidesittoﬁrollﬂtheonpaperorreaderplaten.replacedoff-linemethods.Live-scanmethodsarealsousedinmostmoderndayapplications,e.g.,civilian,governmental,andaccesscontrol.Dependingontheapplicationscenario,oneormoreofthefollowingthreemajortypesareusuallyacquired,(i)rolledimpressions,(ii)plain/slapimpressions,(iii)latents(seeFigure1.11).Whereasrolledorslapimpressionsareacquiredinacontrolledmanner,latentimpressionsareliftedfromsurfacesofobjects.Rolledandslapprints,storedinthereferencedatabase,areoftenreferredtoasexemplarorreference1.Rolled:Rolledimpressionsareacquiredbyrollingafromﬁnail-to-nailﬂonthesensingsurface.Expertassistanceisgenerallyrequiredforrollingtheinthecorrectmanner.Rolledimpressionscapturethecompleteridgedetailpresentonafromthetipofthetothejoint.Therefore,theyprovidehigherrecognitionaccuraciescomparedtoplainimpressions.Onedisadvantage,however,isthepresenceofgreaterdistortioninrolledimpressionsthanplainimpressionsduetotheacquisitiondynamics(pressure,shear,slippage).2.Plain/Slap:Plain/slapimpressionsarecapturedbypressingoneormoreagainstasurfacewhichcouldeitherbeapaperincaseofink-basedacquisitionortheplatenofalive-scanreader.Asinglecaptureistermedaplainimpression(typicalincivilian13(a)(b)(c)(d)Figure1.11Differenttypesofimpressions:(a)rolled,(b)plain,(c)slap,and(d)latent.andaccesscontrolapplications),whereasafoursimultaneouscapture(index,middle,ringandlittlealtogether)iscalledaslapimpression(mostlyusedinlawenforcementapplications).Individualplainimpressionsofthefouraresegmentedoutfromaslapcapturebeforematching.Apopularwayofacquiringallten(tenprints)isthe4-4-2capturewheretwoslapimpressionsofthefouroftheleftandrighthandarecaptured,followedbysimultaneouscaptureofthetwothumbprints.3.Latent:Latentprints,alsoknownasintheforensicscommunity,arethe-printimpressionsinadvertentlyleftbehindonthesurfacesofobjectswhentheyaretouchedorhandled[116].Latentsarepoorqualitypartialimpressionswithincompleteridgedetailsimpressedagainstacomplexsurfacebackground,andcanbedistortedduetotheuncontrolledmannerofdepositiononthesurface.Properimagingofsuchimpressionsis14(a)(b)Figure1.12Acquisitionprinciplesofopticalandsolidstatesensingtechnologies.Imagesrepro-ducedfrom[50].veryimportantforlawenforcementagenciesbecausetheycanbeavitalevidencetoidentifycrimesuspects.Dependingonthecharacteristicsofthesurfacefromwhichlatentshavetobeacquired,forensicexpertsusephysical(e.g.dustwithpowder),chemical(e.g.ninhydrintreatment),and/orphotographical(e.g.ultravioletimaging)methodsforlatentacquisition.1.5.1.1FingerprintSensingTechnologiesInk-on-paperbasedacquisitionmethodshavebeentraditionallyusedbylawenforcementagenciestocaptureplain,slapand/orrolledofcriminals.However,thespreadofrecognitiontomanyconsumerandgovernmentapplications,hasresultedinthedevelopmentofcompact,highresolutionandlow-costsensingtechnologies.Someofthepopularsens-ingtechnologiesinusetodayaredescribedbelow.Optical:Fingerprintreadersbasedonopticalimagingtechnologyarethemostprevalentinthecommercialsector.TheacquisitionprincipleofmostopticalreadersisbasedonFrustratedTotalInternal(FTIR)(seeFigure1.12(a)).ThemajorcomponentsinthereaderassemblytypicallyareacombinationofvisiblespectrumorinfraredLEDs,aglassprismandaCCDoraCMOSsensorarray.Fingerprintacquisitioninvolvesthefollowingsteps:(i)placementoftheononefaceoftheglassprism(calledtheglass15(a)(b)(c)(d)Figure1.13Examplesofcommercialtouchlessreaders.(a)TBScontactless3Dgerprintreader[32],FlashScan3D's3Dtouchlessreader[10],(b)IDair'sOnePrint[21],and(c)Morpho'sFingerontheFly[17].platen),(ii)illuminationoftheusingLEDs,(iii)absorptionofthelightincidentattheridgesandofthelightfromthevalleys,and(iv)ofthelightontotheCCDorCMOSarraybytheglassprismforimagingtheOpticalreaders,ingeneral,providegoodimagequality,andtherefore,arethepreferredchoiceoverotherkindsofreadersinmostapplications.Onelimitationofopticalreaders,however,istheirbiggerformfactorcomparedto,e.g.,solidstatereaders.Asaresult,sofarithasnotbeeneasytoembedthemintosmallelectronicdevicessuchasmobilephones.Ontheotherhand,almostalltheslapreadersareoptical.16SolidState:Solidstatereaderstypicallyconsistofasiliconplate,whereeachelementoftheplateisamini-sensorinitself(seeFigure1.12(b)).Dependingonthetypeofsolidstatesensor,acquisitionisbasedononeofthefollowingphysicalcharacteristicsofthe,(i)capacitancedifferencebetweenridgesandvalleys,(ii)thermalbehaviorofthefrictionridgeskinuponcontactwiththesiliconplate,or(iii)pressurevariationsduetointeractionofthewiththesensingelements.Outofthesethree,capacitivesolidstatereadersarethemostcommonlyused.Solidstatereadersgenerallyhaveasmallsensingarray,typically5-8cm2,tokeepthereadercostlow.Becauseoftheirlowcostandsmallformfactor,theyareeasytoembedinlaptops,PDAsandmobilephones.Fingerprintreadersembeddedinpersonaldevices,e.g,laptopsandsmartphones,haveasmallsensingareawhichonlycapturesapartofthe.Tocapturetheentiresurfacearea,thesereadersusetwodifferentmethods:(i)aswipeofthefromtoptobottom,or(ii)multiplepartialcapturesbyaligningtheindifferentorientationswithrespecttothesensingarea.Ultrasound:Theultrasoundsensingtechnologyusesﬂactiveﬂsensingbytransmittingacous-ticsignalsofawavelength(e.g.,100m)forimaging[138].Ultrasoundsignalssenttothesurfacearebackandcapturedtoformtheim-age.Thistechnologyisbelievedtoberobusttodirt,oilandotherfactorswhichcanpoten-tiallydegradeimagequality.Yetthecommercialapplicationofthistechnologywas,untilrecently,limited.Thiswasduetoultrasoundsensorsbeingbulkyandexpensive,andcapturerequiringatleastafewseconds.However,QualcommintroducedamajorbreakthroughwiththereleaseoftheSnapdragonSense[46]whichisarealtimeauthenticationtechnologyformobiledevicesbasedonultrasoundsensing.Multi-Spectral:Themulti-spectralscanningtechnologywasdevelopedin2005byLumidigm[169],andcanbeconsideredasanextensionoftheopticalimagingmethoddescribedearlier.Themainideainmulti-spectralimagingistoilluminatethewithLEDsofdifferentwavelengths(visibleandnearinfrared).Someofthewavelengthsget17fromtheepidermallayer,whereasotherwavelengthsarebytheunderlyingsecondarydermallayers.Theresponseobtainedfromdifferentwavelengthsiscombinedtoproducetheimage.Becausesecondaryskinlayerscanbeimagedusingthismethod,anadvantageofthemethodisthattheresultingimageisrobusttonoiseduetodirt,sweat,andoiloftenpresentontheouterskinlayer.Touchless:Onemajorissuewithtraditionaltouch-basedlivescanmethodsistheinherentdistortioninducedinthecapturedimagewhentheispressedagainstthereaderplaten.Toalleviatethisissue,touchlesslive-scantechnologywasproposed[165].Oneofthefollowingtwoimagingtechniquesisusedintouchlessreaders(seeFigure1.13):structuredlightingwhereaedlightpatternisusedtoilluminatethetoestimatethedepthandgeneratea3Drepresentationofthe,orthemulti-viewimagingtechniquewheremultiplecamerasareusedtoimagethefromdifferentviewpointstoconstructa3Drepresentation.Tosumup,thelastfewdecadeshasseenthedevelopmentandadoptionofavarietyoflive-scantechnologiesforsensing.Easytousereal-timeacquisitionmethodshaveenabledthespreadofrecognitionsystemstodifferentapplications.1.5.2FeatureExtractionFingerprintfeaturesareusuallycategorizedintothreedifferentlevelsbasedontheirgranularity(seeFigure1.14).Level-1:Prominentfeaturessuchastypeofthepattern(loop,whorl,arch),di-rectionoftheridgew(ridgeorientation),discontinuities(singularities)intheridgew(cores,deltas),andmeasurementofthespacingbetweenridges(ridgefrequency)arecatego-rizedaslevel-1features.Note,however,thatthesefeaturesarenotuniquetoeachLevel-2:Salientpointswherearidgeexhibitsspecialcharacteristics,e.g,endingsandbifur-cations(alsocalledminutiae),areaslevel-2features.Therecommendedscanning18Figure1.14Thethreedifferentlevelsoffeatures.Imagereproducedfrom[115].resolutiontoclearlycapturelevel-2featuresis500ppi.Thesefeaturesareconsideredtobeuniquetoeachandhence,aremostcommonlyusedinmatching.Level-3:Featuresatalevelofgranularitysuchassweatporespresentinsidetheridges,dotsbetweentheridges,incipientridges,andpeculiarfeaturessuchascreasesandwartsaretermedlevel-3features.Level-3featurescanprovideadditionaldistinctiveness,butareonlyvisibleinacquiredatascanningresolutionof1000ppiormore.Further,theavailablealgorithmsforlevel-3featureextractionarenotveryaccurateandrobust.However,theyappeartobeimplicitlyusedbyforensicexaminersforﬁexclusionﬂ2.Lawenforcementagenciesgenerallyacquireatascanningresolutionof500ppi,although,the1000ppiresolutionisbeingconsideredforadoption.State-of-the-artfeatureex-tractionalgorithmstypicallyextractonlylevel-1andlevel-2features(e.g.,ridgeorientation,ridge2Exclusionreferstoexcludingthepossibilityofmatchwhenmanuallycomparingtwo19frequency,andminutiae)andtheirderivatives.Thederivedmeta-featuresfromlevel-1orlevel-2featuresarealsocalleddescriptors.Mostpopularonesare(i)ridgeorientationdescriptorsbasedonorientationvaluesintheminutiaeneighborhood(e.g.[178])and(ii)neighborhoodminutiaebaseddescriptors(e.g.[68]).Allimagesundergoapreprocessingstep(foregroundextractionandenhancement)priortoextraction.Thisstepisparticularlycrucial,incaseoflatentwhereimagequalityisamajorissue.Techniquesdevelopedinmachinelearningandcomputervisionsuchasdictionarylearning[66]andconvolutionalneuralnetworks[64]havebeenproposedforthispurpose.State-of-the-artcommercialfeatureextractionalgorithmsdesignedforlatentsarebelievedtoextractmultiplefeaturerepresentations(e.g.atdifferentscales)fromalatentwiththegoalofimprovingtheoverallmatchingaccuracy.1.5.3FingerprintMatchingTherearetwodifferentmatchingscenariostypicallyencounteredinmostrecognitionapplications:(i)matchingrolled/plain(exemplar)printstoexemplars,and(ii)matchinglatenttoexemplarprints.1.5.3.1Exemplar-to-ExemplarmatchingThisisthemostcommonlyencounteredscenarioinapplicationsrangingfrommobilephoneunlockandbordercontroltocriminalbackgroundcheckandnationalregistry.Forexample,innationalsystemssuchasAadhaar[38],exemplar-to-exemplar(tenprint)matchingisusedforﬁde-duplicatingidentitiesﬂi.e.topreventenrolmentofduplicateidentities.Exemplar(rolled/plain)are,ingeneral,goodqualityprintswithclearridgedetail.Thisallowsaccurateandrobustfeatureextractionfromexemplarprints.Minutiaearethemostcommonlyusedfeaturesinexemplarmatching.Matchingminutiaesetsextractedfromtwodifferentisaclassicapplicationofthepointpatternmatchingproblem[140].Asanexample,onesimpleapproachformatchingtwodifferentminutiaesetsistogenerateaninitialsetofcorrespondences20(a)(b)(c)(d)Figure1.15Asimplemethodforminutiaematching.(a)and(b)Theenrolledandprobetemplatesmarkedwithminutiaesets,(c)alignmentofthetwotemplatesbasedonaninitialcor-respondingminutiaepairmarkedingreen,and(d)correspondingminutiaepointsgeneratedbasedonthealignmentin(c).Imagereproducedfrom[119].betweenthetwominutiaesets,andtheniteratively(i)generatealignmenthypothesisbasedonthecurrentsetofcorrespondingminutiaepairs(measureofsimilaritybetweentheminutiaesets),and(ii)updateminutiaecorrespondencesbasedonthecurrentalignmenthypothesis[125](Fig-ure1.15).Descriptor-basedmatchingmethods[178][68],ontheotherhand,establishminutiaecorrespondenceeitherina(i)top-downmannerbygeneratingminutiaecorrespondencesfromhighsimilaritydescriptorpairsandtheneliminatingfalsecorrespondencesusinglocalstruc-turalconstraints,or(ii)bottom-upmannerwheretypically,thetopn(e.g.,n=5)highestsimilaritydescriptorpairsareused;eachofthendescriptorpairsisusedtoestablishminutiaecorrespon-dencesbyaligningthepairorgrowingthecorrespondingregion,andthepairingthatresultsinthemaximumnumberofminutiaecorrespondencesisselectedastheresult.21Thegoalofmatchingistocomputethesimilaritybetweenthetwoim-pressions.Onceminutiaecorrespondencesbetweenthetwoaregenerated,similarityiscomputedbasedonthenumberaswellasthestrengthofthesecorrespondences.Proprietarygerprintmatchingalgorithmscommonlyuseadditionalfeatures,e.g.,ridgewandridgespacing,besidesminutiae.1.5.3.2Latent-to-ExemplarmatchingForlawenforcementagenciesandforensiccrimelabs,matchinglatentprintsliftedfromcrimelocationstoexemplarprintsinlegacylawenforcementdatabasesisimportanttoestablishpos-siblecrimesuspects.Latentimagestypicallyhaveincompleteridgedetailandpossiblyseverebackgroundnoiseleadingtodifinautomaticextractionofreliablefeaturesformatching.Intheabsenceofrobustautomaticmatchingmethods,asemi-automaticmatchingmethodbasedontheAnalysis,Comparison,EvaluationandV(ACE-V)protocol[57]ispractisedinmostcrimelabs.Underthepurviewofthisprotocol,aexaminerdeterminesthequalityofthelatent,andifthelatenthassufquality,marksfeatures,suchasregionofinterestandminutiaeonthelatentimage.Thelatentwithmarkedfeaturesisthensubmittedtoalatentmatcherforcomparingthelatenttoexemplarsinthereferencedatabase.Acandidatelistoftop-Kmatch-ingexemplarsfromthereferencedatabaseisreturnedbythelatentmatcher3.Theexaminerthencomparesthelatenttoeachexemplarimageinthecandidatelisttodeterminethecorrespondingfeatures.Basedonthestrengthofthecorrespondencesobtainedfordifferentlatent-exemplarpairs,theevaluationofwhetheracandidateexemplarmateswiththelatent(hitisfound)ismade.Fol-lowingthis,asecondexaminerthenindependentlyinspectsthelatent-exemplarpairsandvtheauthenticityofthedecisionmadebytheexaminer.3State-of-the-artlatentmatchersmatchmultiplefeaturerepresentationsandfusetheresultstoimprovethelikeli-hoodofobtainingahitinthecandidatelist.22Figure1.16Exampletargetsusedforcalibratingimagingsystems.Imagesadaptedfrom[107][137][163].1.6EvaluationofFingerprintRecognitionSystemsPerformanceevaluationisacriticalstepduringthedesignanddevelopmentofarecog-nitionsystembeforeitsactualdeployment.Forsystemevaluation,atwo-stepevaluationprocedureisusuallyfollowed:internaltestingtoensuredesiredaccuracy,followedbytestingtovalidatethelaboratorytestingresultsinoperations.Beforedeployment,eachmoduleofthe-printsystem(sensing,featureextractionandmatching)needstobethoroughlyevaluated.Inthefollowingsections,wedescribethestandardtestingproceduresforof-printreaders,aswellastheevaluationstudiesconductedtobenchmarkexistingfeatureextractionandmatchingalgorithms.1.6.1SensingTechnologyIntheUnitedStates,readersarebytheTechnologyEvaluationStandardsTestUnit,partoftheFBIsBiometricCenterofExcellence(BCOE)ledbytheCriminalJusticeInforma-tion(CJI)ServicesDivision[9].TwodifferentstandardshavebeenestablishedbytheFBIfortheofreaders.ThePIVstandard[155]caterstoreadersdesignedforthevescenario.Fingerprintreadersbuiltforuseinapplicationsinvolvinglarge-scaleareundertheAppendixFstandard[156]whichenforcesstricterqualityrequirementscomparedtothePIVstandard.Thesestandardscontainthedesiredreader23fordifferentaspectsofthereader,suchasgeometricaccuracy,resolutionandspa-tialfrequencyresponse.Thereaderprocessrequiresvendorstoshowthatimagescapturedbytheirreadersexceedtheminimumprescribedintherelevantstandard[8].Asastep,vendorsinternallytesttheirreadersusingcalibra-tiontargets(see,e.g.,Figure1.16)toensurethatthetargetimagescapturedusingthereaderareofsufqualitytomeettheprescribedinthestandard.Oncetheyarewiththecapturedimagequality,theysubmittestimagestotheagency.Theagencyindependentlyvthatthetestimagesmeetthestandardandthereaderundertherelevantstandard.Thetestingandofbiometricdevicesirisandface)forusebytheUniqueAuthorityofIndia(UIDAI)intheAadhaarprojectisperformedbytheStandardizationTestingandQuality(STQC)Directorate,GovernmentofIndia[37].UIDAIisoneofthelargestconsumersofbiometricreadersintheworldwith36,000enrolmentstationsdeploying11differentbiometricreaders(5slapsensors,4irissensors,and2facecameras)[80].ImageacquisitionrequirementsequivalenttotheAppendixFstandardaremandatedforreadersusedforenrolmentinAadhaar[42].Forgettingtheirread-ersvendorssubmitaagreementtotheagency,theSTQCDirectorate.Theagencyevaluatestheevidenceofconformityofthesubmittedagreementtotheprocedureguidelines.Thereafter,providedthatthetestingprocedureresultsaresatisfactory,thereaderisbytheagencyforuseinAadhaar[43].1.6.2FeatureExtractionandMatchingEvaluationFingerprintfeatureextractionandmatchingalgorithmsaretypicallyevaluatedtogetherasasinglecomponentwhereimagesarefedasinputandsimilarityscoresaregeneratedasoutput.Differentmeasuresareusedtoevaluatetheirperformanceinthetwomatchingscenarioscommonlyencountered,v(1:1comparison)and(1:Ncomparison).24Toevaluatevperformance,twodifferentmetricsarefrequentlyused,(i)trueacceptrate(TAR),i.e.proportionofsubjects,amongstthosepreviouslyenrolled,thatcanbesuccessfullyvand(ii)falseacceptrate(FAR),i.e.proportionofsubjects,amongstthosenotpreviouslyenrolled,thatareincorrectlydeterminedtohavebeenpreviouslyenrolled[141].Thesetwoquantities,TARandFAR,arenotindependentofeachother,sothereisatrade-offbetweenTARandFAR.ReceiverOperatingCharacteristic(ROC)curve,aplotofTARv.FARatdifferentoperatingthresholds,isoftenusedtoindicatethevperformance.SomestudiesprefertousethefalseRejectRate(FRR)insteadofTAR.FRRindicatestheproportionofsubjects,amongstthosepreviouslyenrolled,thatcannotbesuccessfullyvd.Inthiscase,aDetectionErrorTrade-off(DET)curvethatplotsFRRv.FARisusedtoreportvperformance.Therearetwodistincttypesofonscenarios:(i)closedsetwheretheprobeorthequeryisknowntohaveamateinthereferencedatabase,and(ii)opensetwheretheprobemayormaynothaveamateinthereferencedatabase.Forclosedsettypically,acandidatelistofthetop-Kmatchesisretrieved,andtheretrievalrankofthetruemateinthecandidatelistisusedasanevaluationmetric[141].CumulativeMatchCharacteristics(CMC)curve,whereeachpointonthecurvedenoteswhetherthetruematewasretrievedatrankiinthecandidatelist,isplottedtoindicatetheclosedsetperformance.Incaseofopensetthetwomostcommonlyusedperformanceevaluationmetricsare:falsepositiverate(FPIR)whichmeasurestheproportionofquerieswhichdonothaveamateinthereferencedatabasebutwerefalselyedtohaveamate,andfalsenegativerate(FNIR)whichmeasurestheproportionofquerieswhichhaveamateinthereferencedatabase,butcouldnotbesuccessfullytohaveamate.Sincetheearly2000s,NISThasconductedseveralevaluationsoffeatureextractionandmatchingalgorithms.TheFpVTE2003evaluation[185],performedonadatabaseof10,000plainfoundthatthebestperformingalgorithmhadaTARof99.4%atFARof0.01%.InthemostrecentevaluationFpVTE2012[184],plainof30,000subjects(10,000matesand20,000non-mates)weresearchedagainstplainof100,000subjectsandthe25FNIRofthebestperformingalgorithm,atFPIRof0.1%,wasreportedtobe1.9%forsingleindexcapturesand0.27%fortwoindexToevaluatethestate-of-the-artlatentmatchingalgorithms,NISTperformedELFT-EFSevalu-ationintwophases[109][108].InPhaseI,1,114latentswerematchedagainstexemplars(rolled+plain)obtainedfrom100,000subjects.Therank-1cationaccuracyofthebestalgorithmwasreportedtobe62.2%.InPhaseII,1,066latentswerecomparedagainstreferencedatabaseofexemplarsof100,000subjects,andtherank-1accuracyofthebestperformingmethodwas67.2%.1.7ChallengesinFingerprintRecognitionAlthoughthedesignofautomaticrecognitionsystemsandthemethodstoevaluatethesesystemshaveevolvedoverthepast50years,thereremainanumberofopenresearchissuesandchallenges.Weprovidethislistofproblemsfromourperspectiveandthenaddresstwoofthemthatconstitutethecontributionsofthisdissertation.1.7.1OpenResearchIssuesandChallenges1.7.1.1AutomaticlatentprintmatchingLatentareimportantforlawenforcementagenciesandforensiccrimelabstoidentifyfugitivesandtoassessiftheyareguiltyorinnocent.However,despiterecentdevelopments,fullyautomaticandaccuratematchingoflatentstoreferenceremainsanopenchallengeforresearchers.1.7.1.2InteroperabilityofprintreadersLarge-scalesystemdeploymentse.g.,Aadhaar[38]havemultipleenrolmentstationsequippedwithdifferentreaders.Compatibilityofacquiredbythedifferent26Figure1.17Exampleproceduretocreateandirectlyfromaliveger.Plasticisusedtocreatethemoldandgelatinisusedasthecastingmaterial.Imagereproducedfrom[143].readersisessentialforsuccessfuloperationofsuchalarge-scalesystem.Furthermore,withthead-ventofnewsensingtechnologies,suchascontact-less3Dsensing[165],itisimportanttodevelopmethodstomatchacquiredbythesereaderswithreferenceprintsinlegacydatabases.1.7.1.3OperationalevaluationofprintsystemsEvaluationstandardshavebeendevelopedforofreaders.However,testresultsobtainedinacontrolledenvironmentdonotgeneralizetotheoperationalsettings.Thereisaneedtodevelopmethodsforevaluatingerprintreadersinthefunctionalenvironment.Fur-thermore,thereisalackofstandardproceduresforend-to-endevaluationofntsystems,fromacquisitiontofeatureextractionandmatching.1.7.1.4FingerprintlivenessdetectionFingerprintreadersembeddedinconsumerdevices,e.g.,mobilephonesandtablets,andthatarebeingusedforconductingtransactions,havebeenshowntobevulnerabletospoofattacks[144][145][65][143].Figure1.17illustratesasimpleproceduretocreatean27(a)(b)Figure1.18Examplesofnon-idealimages.(a)Awithworn-outridgede-tails,and(b)awithalteredpatterns(imagereproducedfrom[188]).directlyfromalivethatcanbeusedforreaders.Severalalgorithmshavebeendevelopedbyacademicresearchers[47][95][99].However,thereisaneedtodevelopcommercial-gradelivenessdetectionmethodstopreventimpostorsfrommisusingrecognitiontechnology.1.7.1.5FingerprinttemplatesecurityInmostoperationalsystems,templatesaretypicallysecuredbyusingstan-dardencryptiontechniques,e.g.,AES.Thesecurityofthetemplate,therefore,dependsonthelackofadversary'sknowledgeaboutthedecryptionkey.Further,templatematchingisusuallynotperformedintheencrypteddomain.Asaresult,templatesaredecryptedatthetimeofauthentica-tion,andthisleavesthemvulnerabletopossibleattacksduringauthentication[154].Toovercomethislimitation,onecommonapproachistostoretheencryptedtemplatesanddecryptionkeysinasecuremodule(e.g.,A10chiponAppleiPhone74)andperformtemplatematchinginatrust-worthyenvironment.However,thisrequirestheusertocarryanadditionaldevicethatstorestheencryptedtemplates.Althoughnumeroustemplateprotectiontechniquesthataimto4http://support.apple.com/en-sg/HT594928ensurenon-invertibility,revocabilityandnon-linkabilityoftemplateswhilemaintainingtherecog-nitionperformancehavebeenproposedovertheyears(e.g.[182],[153],[152],[126]),thereisstillawidegapbetweenthetheoreticalclaimsandthepracticalapplicabilityofthesemethods[154].1.7.1.6Matchingnon-idealprintimagesIthasbeenobservedthatofolderpeopleandthoseincertainprofessions,e.g.,farmingandwelding,areofpoorquality[172][183].Thisisbecausewithrepeateduseoforbecauseofthecomingincontactwithcertainchemicals,ridgespresentontheirwearoutovertime(seeFigure1.18(a)).SomepeoplehavegeneticallypoorqualityBesides,therehavebeencaseswherecriminalsandthoseguiltyofotherfelonieshaveintentionallyobliteratedoralteredtheirtoevadebytheauthorities[188](seeFigure1.18(b)).Matchingthesenon-idealprintsisachallengingtask.1.7.2AutomaticLatentFingerprintMatchingInits2009reportonstrengtheningforensicscienceintheUnitedStates[78],theNationalResearchCouncilemphasizedtheneedtoaddressthefollowingtwomajorissuesfacingforensicscience:(i)ﬁlackofmandatoryandenforceablestandardsﬂforreadyreferencebycrimelabsaroundtheworldand(ii)ﬁunacceptablecasebacklogsinstateandlocalcrimelabswhichlikelymakeitdifforlaboratoriestoprovidestrongevidenceforprosecutionsandavoiderrorsthatcouldleadtoimperfectjusticeﬂ.Followingthis,effortsweremadetounderstandindepththedifferentfactorswhichimpactthelatentexaminationwwandtostandardizetheprocesses[44].Asanexample,toimprovetheoddsofobtainingamatch,acommonpracticeusedbyseverallawenforcementandforensicagenciesistoinvolveaexaminerto(i)markfeaturesonalatentimagebeforesubmittingittoanAFIS,and(ii)inspectthelistoftop-KcandidatesreturnedbytheAFIStoverifythatahithasbeenmade.Althoughthismanualinterventionprocessissupposedtobeforincreasingtheoverallmatchingaccuracy,studiesonhumanfactorshaveshownthatitinducesbiasandsubjectivityinthelatentmatchingprocess[85][86].Further,ithasalso29beendemonstratedthatexaminersoftenhavealowdegreeofagreementwiththeirowndecisions,aswellasthedecisionsmadebyotherexaminers,reducingtherepeatabilityoftheoutcome[181].Anotherimpendingissueistheobjectivedeterminationoftheevidentialvalueofalatentprint[75].The2016reportbythePresidentsCouncilofAdvisorsonScienceandTechnology(PCAST)[45]statesthatﬁlatentanalysisisafoundationallyvalidsubjectivemethodologyalbeitwithafalsepositiveratethatissubstantialandislikelytobehigherthanexpectedbymanyjurorsbasedonlongstandingclaimsabouttheinfallibilityofanalysisﬂandthatﬁinreportingresultsoflatentexamination,itisimportanttostatethefalsepositiveratesbasedonproperlydesignedvalidationstudiesﬂ.Thisnecessitatestheneedforfullyautomaticlatentmatch-ingtoeliminatehumanbiasandsubjectivity.Theevaluationoflatentmatchingtechnologies(ELFT-EFSII[108])conductedbytheNationalInstituteofStandardsandTechnology(NIST)in2012,reportedtheaccuracyofthebestautomaticlatentmatchingalgorithmtobemere67.2%.Incontrast,theresultsofthegerprintvendortechnologyevaluationperformedbyNISTin2012[184]reportedaccuracynumbersashighas99%forautomaticmatchingofrolled/plainimpressions.Thisindi-catesthatinspiteofdevelopmentsinrecognitiontechnologyduringthepast40years,improvementstotheautomaticlatentprintmatchingprocedureareurgentlyneeded.Forthisreason,wepursuedthisprobleminthisdissertation.Wedevelopedatop-downmatch-ingparadigmthattakesfeedbackfromreferenceprintsandre-sortsthecandidatelistgeneratedbyabottom-uplatentmatchertoimproveitsaccuracy.Wealsodevelopedalatentmarkupcrowd-sourcingframeworkwhereexaminersandthelatentmatcherworkinconjunctionwitheachothertoboostthelatentmatchingaccuracy.1.7.3OperationalEvaluationofFingerprintSystemsIndeployingalarge-scalerecognitionsystem,oneofthecriticalfactorsistohaveareasonableestimateofthematchingperformanceofthesystemintheoperationalsettings.For30Figure1.192Dsyntheticgenerationprocessusingthemethodin[192].(a)Fingerprinttypeis(b)ridgewmapisgeneratedfromalearnedstatisticalmodel,(c)minutiaearegeneratedbasedontheridgewin(b)andalearnedstatisticalminutiaemodel,an(d)2Dsyntheticissynthesizedusing(b)and(c).thispurpose,typically,pilotstudiesareconductedonalargenumberofofmanysubjectstoascertaintheoperationalthresholdsoncomparisonscorestoachievethedesiredfalseacceptrate(FAR).Thisisatediousprocessbothintermsoftimeandresourcecommitment.Besides,theresultingperformanceestimateislimitedintheofitsaccuracybytheamountandnatureofdatawhichisavailable.Onepossiblesolutiontoalleviatethisshortcomingofsmallsamplesizeistosyntheticallygenerateverylargeamountsofrealisticlookingimageswhichcanthenbeusedforsystemperformanceevaluation.Thiswouldentailgenerating,say,millionsofsyntheticrprintsforevaluatinglarge-scalerecognitionsystems[69][192].State-of-the-artgenerationmethods[67][192]output2Dsyntheticus-ingmathematicalorstatisticalmodelsoffeatures(e.g.type,orientationandminutiae).The2Dsyntheticgeneratorproposedin[67]generatesridgewmapusingamathematicalmodelandridgedensitymapbasedonheuristicslearnedfromseveralimages.Directionaltunedtolocalridgeorientationandfrequencyvaluesaretheniterativelyappliedstartingfromafewseedlocationstogenerateridgepatterns.Note,however,thatminutiaeplacementcannotbecontrolledduringthe2Dsynthetic31Figure1.20Exampleofagenerated3Dtarget.Shownontheleftisthe2Dimageandontherightisthe3Dsurfaceusedtocreatethe3Dtarget.generationprocess.Ontheotherhand,the2Dsyntheticgenerationmethodin[192]outputs2Dsyntheticusingstatisticalmodelsoffeaturestype,orientationandminutiae).Thefeaturesaresampledfromtheirrespectivestatisticaldis-tributions,followedbyareconstructionmethod(describedin[92])togeneratevisuallyrealisticsynthetic(seeFigure1.19).Theaforementionedmethodscangeneratesynthetictoevaluatefeatureextractionandmatching.However,thereisalackofanapproachtoevaluatereadersinoperationalsettings(e.g.placementofhumanonthereaderplaten),andconsequentlyanﬁend-to-endﬂbiometricsystem,fromsensingaphysicalandacquiringitsimpression(image)toextractingthetemplateandestablishingorverifyinganidentity.Operationalevaluationofsystems,therefore,stillremainsachallenge.Toaddresstheaforementionedlimitations,wegenerated3Dtargets(seeFigure1.20).Weprojected2Dcalibrationpatternswithknownfeatures(e.g.sinegratingsofknownorientationandfrequency,2Dwithknownsingularitiesandminutiae)onto32(a)(b)Figure1.21Sample3Dprintedwholehandtarget(a),and(b)evaluatingaslapreaderusingthe3Dwholehandtargetshownin(a).a3Dsurfacetocreateelectronic3Dtargets.Wethenusedastate-of-the-art3Dprintertofabricatethesetargetswithmaterialshavingsimilarhardnessandelasticitytothehumanskin.Weshowedtheutilityof3Dtargetsinevaluatingthreedifferent500/1000ppiopticalreaders.Forevaluatingcontact-basedandcontactlessslapreaders,wecreated3Dwholehandtargetscompletewithallfourandthethumbprint(seeFigure1.21).Givenanelectronic3Dhandmodel,3Dsurfacescorrespondingtoeachoftheandthemiddleportionofthehandweresegmented.2Dcalibrationpatternswerethenprojectedandetchedonthesegmented3Dsurfaces.Followingthis,wearableelectronic3Dtargetsforallandthethumbaswellasglovewerefabricatedwithastate-of-the-art3Dprinter.Thegenerated3Dwholehandtargetswereusedforevaluatingthreedifferent500/1000ppicontact-basedslapreadersanda500ppicontactlessslapreader.The3Dtargetsfabricatedwiththe3Dprinter,althoughsimilarinhardnessandelasticitytothehumanskin,werenon-conductive.Consequently,theycouldnotbeusedforevaluatingcapacitivereaderssuchasthoseembeddedinmoderndaysmartphones.Toimpartconductivityto3Dprintedtargets,wecoatedtheirsurfacewiththinlayersofconductivematerials(titanium+gold)viaDCsputtering(see,e.g,Figure1.22).Thegeneratedconductivetargetswereusedforevaluatinga500ppicapacitivereader.Itisimportanttonotethatbesides33(a)(b)Figure1.22Sampleconductive3Dtargetcreatedbydepositingthinlayersoftitaniumandgoldon3Dprintedngertargetsshownin(a),and(b)animpressionofthein(a)capturedusingacapacitivereader.readerevaluation,ourtargetscanalsobeusedforend-to-endevaluationofrecognitionsystems.1.8DissertationContributionsThecontributions(includingtheorganization)ofthisdissertationareasfollows:Designofaframeworktoimprovelatentmatchingaccuracybyincorporatingtop-downin-formationorfeedbackfromanexemplarprinttothefeaturesextractedfromalatent(Chapter2).Thesetoflatentfeatures(e.g.ridgeorientationandfrequency),afterfeedback,arecomparedagaintothetop-Kcandidateexemplarsreturnedbythebaselinematcherandtogenerateanewrankedcandidatelist.Ourcontributionsare:we(i)devisesystemicwaystouseinformationinexemplarsforlatentfeature(ii)developafeedbackparadigmwhichcanbewrappedaroundanylatentmatcherforimprovingitsmatchingperformance,and(iii)determinewhenfeedbackisactuallynecessarytoimprovelatentmatchingaccuracy.34Designofacrowdpoweredlatentmatchingframeworkwheremultiplelatentexaminersandanautomaticlatentmatcherworkinasynergisticmannertoboosttheoverallaccuracy(Chapter3).Givenalatent,thecandidatelistoutputbythelatentmatcherisusedtodeterminethelikelihoodofahitatrank-1.Alatentforwhichthislikelihoodislowiscrowdsourcedtoapooloflatentexaminersforadditionalfeaturemarkup.Themanualmarkupsaretheninputtotheautomaticlatentmatchertoincreasethelikelihoodofahitinthereferencedatabase.Furthermore,agreedyparadigmwheremarkupsareobtainedfromtheexaminersinanincrementalmannerwhenrequired,isalsoproposed.Thisisshowntoreducetheexaminerworkloadbyonlyrequiringamaximumofthreeexaminerstoprovidemarkupforalatent.Designandfabricationof3Dtargetsforrepeatablebehavioralevaluationofopticalreaders(Chapter4).2Dcalibrationpatternswithknowncharacteris-tics(e.g.sinusoidalgratingsofpre-sorientationandfrequency,withknownsingularpointsandminutiae)areprojectedontoageneric3Dsurfacetocreateelectronic3Dtargets.Astate-of-the-art3Dprinterisusedtofabricatewearable3Dtargetswithmaterialsimilarinhardnessandelasticitytothehumanskin.Thegenerated3Dtargetsaresuitableforbehavioralevaluationofthreedifferent(500/1000ppi)PIV/AppendixFopticalreaders.Generationof3Dwholehandtargetscompletewithfourthethumbprintandthemiddleportionofthehandforrepeatableevaluationofslapandcontactlessreaders(Chapter5).2Dcalibrationpatternswithknowncharacteristicsareprojectedonto3Dsurfacescorrespondingtoeachofthefourandthethumbtocreateelectronicwholehand3Dtarget.Physical3Dwholehandtargetsaresubsequentlyfabricatedusingastate-of-the-art3Dprinterwithmaterialsthataresimilarinhardnessandelasticitytothehumanskinaswellasopticallycompatiblewithavarietyofopticalreaders.35Generatedwholehand3DtargetsareusedforevaluatingthreeAppendixFcontact-basedslapreadersandaPIVcontactlessslapreader.Fabricationofconductive3Dtargetsforevaluationofcapacitivereaders(Chapter6).3Dprintedtargetsarecoatedwiththinlayersofconductivematerials(titanium+gold)viaDCsputteringtoimpartconductivitytotheirsurface.Weshowthatthecoatingproce-duredoesnotimpacttheofthecalibrationpatternsetchedonthe3Dtargets.Theconductive3DtargetsareusedforevaluatingaPIVcapacitivereader.Furthermore,asimpleproceduretocreate3Dspoofsforperformingpresentationattacksoncapacitivereadersisdescribed.Thegenerated3Dspoofsaresuccessfullyusedforthecapacitivereaderandanembeddedreaderinanaccesscontrolterminal.Abriefsummaryofthecontributionsofthisdissertationandpossiblefutureresearchdirec-tionsarediscussedinChapter7.36Chapter2LatentFingerprintMatching:PerformanceGainviaFeedbackfromExemplarPrints2.1IntroductionLatent1arepartialimpressionsofthewithrelativelysmallerareacontainingfrictionridgepatterns.Automaticmatchingoflatenttoexemplarsischallengingbecauselatents(i)generallyexhibitpoorqualityintermsofridgeclarity,(ii)havecomplexbackgroundnoise(Figure2.1),and(iii)havelargenon-lineardistortionsduetovariationsinpressurewhenanobjectistouched,resultinginthedepositionoflatentprintonitssurface.IntheEvaluationofLatentFingerprintTechnologies(ELFT)[109]conductedbyNIST,thePhase-Iresultsshowedthatthebestrank-1latentmatchingaccuracywas80%inidentifying100latentimagesfromamongstasetof10,000rolledprints[157].Morerecently,intheNISTEvaluationofLatentFingerprintTechnologies:ExtendedFeatureSets(ELFTEFS)PhaseII[108],therank-1accuracyofthebestperforminglatentmatcherwasonly67.2%intheﬁlights-outﬂ(fullyautomatic)mode2.So,whileAutomatedFingerprintSystems1Thetermermarkisalsousedintheforensicsciencecommunitytorefertotheimpressionsaccidentallyleftbehindonthesurfaceofobjects.Weusethetermlatentbecauseitismorepopularinthebiometricscommunity.2ThelatentmatchingaccuracyishigherintheELFTPhase-IascomparedtoPhase-IIbecausethequalityoflatentsusedinPhase-Ievaluationwascomparativelybetter.37(a)(b)(c)Figure2.1SampleimagesfromtheNISTSD27databaseshownheretoelucidatesomeofthechallengesinlatentmatching:(a)poorridgeclarity,(b)insufamountofusableridgevalleypatternsand(c)presenceofcomplexbackgroundnoise.Theredcurvesaremanuallymarkedforegroundareaintheimage.(AFIS)workextremelywellinmatchingexemplartoeachother,thereisaconsiderableperformancedropwhenmatchinglatentimagestoexemplarimages.Itisgenerallyagreedthatlatentmatchingisachallengingproblemwhoseperformanceneedstobeimprovedtoreducethebacklogofoperationalcasesinlawenforcementagencies.TheFBI'sNextGeneration(NGI)program[160]listsﬁlights-outﬂcapabilityforlatentmatchingasoneofitsmajorobjectives.2.1.1ManualLatentMatchingInmanualmatchingoflatentprints,latentexaminersusuallyfollowtheAnalysis,Com-parison,EvaluationandV(ACE-V)methodology[57].Thisbasically,isafourstepprocess:1.Analysis:Thepreliminarystepinvolvesanalyzingthelatentimagetoascertainifthelatentisofsufvalueforprocessingandmanuallymarkingfeaturessuchasminutiae,ori-entationandridgefrequency.Thisisusuallydonebyobservingthelatentimageinisolation.382.Comparison:Thisconsistsofcomparingthelatentimagetotheexemplarimageintermsoftheirfeatures,andassessingthedegreeofsimilarity/dissimilaritybetweenlatentandexem-plar.3.Evaluation:Thelatentexaminerdeterminesthestrengthoftheevidencebetweenthelatentandexemplarbasedontheassesseddegreeofsimilarity/dissimilaritybetweenthelatentandexemplarinthecomparisonstep.4.V:Asecondlatentexaminerindependentlyevaluatesthelatent-exemplarpairtovalidatetheresultsofthelatentexaminer.TheACE-Vprocedureisatediousandtimeconsumingprocessforthelatentexaminerasitmayinvolvealargenumberofcomparisonsbetweendifferentexemplarpairs.Forthisreason,AFISisusedinthecomparisonstep.Typically,alistoftopKmatchingcandidates(withKgenerallybeing50)isretrievedfromtheexemplardatabaseusingalatentmatcher,whicharethenvisuallyinspectedbythelatentexaminertoascertainthebestmatch.Thisresultsinoneofthefollowingveoutcomes:1.Thelatentexaminercorrectlymatchesthelatenttoitstruematedexemplarfromthecandidatelist.2.Theexaminererroneouslymatchesthelatenttoanexemplarfromthecandidatelist(whichisnotthetruemate).3.Theexaminercorrectlyexcludesanexemplarfromthecandidatelist(whichisnotthetruemate)tobethepossiblemateofthelatent4.Theexaminererroneouslyexcludesthetruematedexemplarofthelatent-printfromthecandidatelisttobethepossiblemateofthelatent.5.Theexaminerdeemsthematchingresulttobeinconclusivebecauseheisunabletoanycandidateexemplarthatissufsimilartothelatentprint.39Notethatwhileoutcome2)isanerroneousmatchandoutcome4)anerroneousexclusion,outcome5)isarejectinthesensethatthetruematedoesnotexistinthereferencedatabase.Theproposedfeedbackbasedmethodologyisdesignedtominimizetheoccurrenceofoutcomes2),4)and5)byinitiallyretrievingamuchlargercandidatelist(e.g.N=200)usingtheAFIS.Eachofthesecandidatesisthenviewedastheoutputofacoarselevelmatchwhichcanbeusedtothefeaturesextractedfromthelatentimages.ThesimilaritiesoftheseNcandidatestothequerylatentarethenrecomputedbasedonthelatentfeaturestore-rankthecandidatelist.ThelatentexaminercanthenexaminethetopKcandidates(K<>:jj;ifjj<90;180jj;otherwise:(2.12)Here,'(;)isthefunctiontodeterminethedifferencebetweenridgeorientationsandandistherotationangleusedtoalignthetworidgeorientationsand.7.Theridgeorientationandridgefrequencyvaluescorrespondingtotheselectedpeakpoint(ul;vl)arethenchosenastheridgeorientationandridgefrequencyfeaturesfortheblockILBinlatentimage.Notethattheridgeorientationandridgefrequencyfeaturesareselectedbasedontheex-emplarfeatures,andthisessentiallyconstitutesthetop-downinformationworfeedbackfromtheexemplar.502.3.4MatchScoreComputationThefunctionstocomputethesimilaritybetweentheexemplarfeaturesandthelatentfea-turesafterfeedbackshouldresultinimprovedsimilaritybetweenmatedlatent-exemplarpairs.Thus,thesimilarityfunctionshouldbebasedontheunderlyingdistributionoffeaturedifferencesobtainedfromgenuinelatent-exemplarmatches.Assumethattheorientationandfrequencydiffer-encesbetweenthelatentfeaturesandexemplarfeatureswithineachblockareindependentandidenticallydistributed.Tolearnthecharacteristicsofthegenuinedistributionmodel,50matedlatent-exemplarpairsfromtheNISTSD27[148]andWVUdatabase[150]arerandomlysampledtoestimatethedistributions.Weobservethatthegenuinedistributionoforientationdifferencesapproximatelyfollowsacosinecurvewhereasthatofridgefrequencydifferencesapproximatesanexponentialcurve;cosineandexponentialfunctionsarehenceusedforcomputingfeedbackorientationandfrequencysimilarities,respectively.Forcomputingthefeedbackridgeorientationandfrequencysimilarities,theoverlappingregionbetweenthelatentandexemplarisdeterminedusingthetransformationfunctionT.Withintheoverlappingregion,theridgeorientationandridgefrequencysimilaritiesSimandSimfarethencomputedas:Sim=1NumNumXi=1cos'(Li+;Ri);(2.13)Simf=1NumNumXi=1exp0@1fLi1fRif1A;(2.14)NumNummin;whereNumisthenumberofoverlappingblocks;Numminisathresholdontheminimumnumberofblocksneededintheoverlappingregionandissetto10inourexperiments;LiandRiaretheridgeorientationsandfLiandfRiaretheridgefrequenciesoftheithoverlappingblockfromthelatentandexemplar,respectively;andfaretwonormalizationparameterswhichareempiri-callysetto12and8,respectively.TheinitialmatchscoreSimIisnormalizedusingmin-max51scorenormalization[121],andtheorientationandfrequencysimilaritiesSimandSimfarethencombinedwiththenormalizedinitialmatchscoreSimNIbasedonproductfusiontoobtaintheupdatedmatchscoreSimUasfollows:SimU=SimNISimSimf:(2.15)2.4TheAdequacyofFeedbackAlthoughfeedbackfromexemplarscanbeusedtolatentfeatures,thefeedbackmaynotbenecessarywhenthelatentisofsufgoodqualitysuchthatitsfeaturescanbereliablyextracted.Bottom-uplatenttoexemplarmatchingmaysufforsuchcasesandfeedbackmaynotaddanyvaluetothelatentmatchingprocess.Clearly,itwouldbeusefultohaveanobjectivecriteriontoascertainiffeedbackcanpotentiallyimprovethematchingaccuracyforeachlatentquery.Besides,sincefeedbackisappliedwithineachblockinthelatent,decisiontoapplyfeedbackcanalsobemadelocallyattheblocklevel.Todeterminetheneedforfeedback,wedesignaglobalcriterionbasedonthematchscoredistribution(ofthetopKmatchscoresreturnedbythebaselinelatentmatcher),andalocalcriterionbasedonthelocalqualityofthelatent-exemplarpairbeingmatched.2.4.1GlobalCriterionWedesignasimplecriteriontodecidewhetherfeedbackisneededforaparticularlatentquerybasedontheprobabilitydistributionofthetopKmatchscoresreturnedbythebaselinematcher.2.4.1.1ModellingtheMatchScoreDistributionThisdistributionisbasedonthesimilarityfunctionusedinthebaselinematcher.Thelatentmatcherusedinourexperiments[166]usesanexponentialsimilarityfunction,soweusetheex-ponentialdistributiontomodeltheprobabilitydistributionofmatchscores.Alternately,wecould52(a)(b)Figure2.6Exemplifyingtheglobalcriterionforfeedback:(a)matchscoredistributionforapar-ticularlatentquerywithoutanupperoutlier,and(b)withanupperoutlierpresent(markedinred).Feedbackisneededincase(a),butnotneededin(b).estimatetheprobabilitydensitiesusingthematchscorehistogram,andthenaparametricdistri-butiontothehistogram.Tomeasurethegoodnessofoftheexponentialdistributionmodelinourcase,weusedthechi-squaregoodnessoftest[133].Forthis,werandomlysampled40latentimagesfromtheNISTSD27database[148],andthentestedthegoodnessofoftheexponentialdistributiononthesetoftopKmatchscoresgeneratedbythematcherforeachlatent.2.4.1.2TestforthepresenceofanupperoutlierWeobservethatifthetruematedexemplarprintisindeedretrievedatrank-1bythebaselinelatentmatcheroperatinginbottom-upmode,thenthereisasizeabledifferencebetweentherank-1andothermatchscores.Inotherwords,therank-1matchscoreisanupperoutlierintheprobabilitydistributionofthetopKmatchscores.Thus,theproblemofdeterminingwhetherfeedbackisneededornotbecomesequivalenttotheproblemofdetectingwhetheranupperoutlierexistsinthematchscoredistribution(seeFigure2.6).Wenowdescribeahypothesistestfordetectingthepresenceofanupperoutlierforexponentialdensityusedhere[132]anditsusageindeterminingtheneedforfeedback.53Thepdfoftheexponentialdistributionwithscaleparameterisgivenbyf(x)=1ex;x>0;>0:(2.16)LetX=fX1;X2;:::;Xngbeanindependentandidenticallydistributed(i.i.d)randomsampleofsizengeneratedfromanexponentialdistributiongivenbyEqn.(2.16),andX(1);X(2);:::;X(n)bethecorrespondingorderstatistics.OrderstatisticsarethesamplevaluesintheorderoftheirmagnitudewithX(1)1;z1>z2.Now,thedensityofZcanbecomputedusingabivariatetransformationonthejointdensityofZ1andZ2[132](Eqn.(2.19)):m(z)=n(n1)2nn2ˆ(n2)n22n21(n3)n23++(1)n3n2n31n1˙(1z)n2:(2.20)Here0z()jH0]=1Zz()m(z)=(1z())n1=:(2.21)Thus,thecriticalvaluez()is:z()=11n1:(2.22)ForX(n)tobetheoutlier,therealizedvalueoftheteststatisticZ=zshouldbegreaterthanthecriticalvaluez().Fortheglobalcriterionforfeedback,weanindicatorrandomvariableIFwhichtakesthevalue1whenfeedbackisneededand0,ifitisnotneeded:55IF=8>><>>:0;z>z();1;otherwise(2.23)(a)(b)(c)(d)(e)(f)Figure2.7Exemplifyingthelocalcriterionforfeedback:(a)alatentimage,(b)itsridgeclaritymapand(c)regionswhichneedfeedback(showningrey);(d)anexemplarimage,(e)itsridgeclaritymapand(f)regionswhicharereliableforprovidingfeedback(showninwhite).2.4.2LocalCriterionEventhoughfeedbackmaybepotentiallyusefulforaparticularlatentquery,theremaybesomegoodqualityregionswithinthelatentimagewhichdonotrequirefeedback.Besides,theexemplarprintregionfromwherefeedbackisbeingtakenmaybeofpoorqualitywhichmaynotbereliable56forfeedback.Fordecidingwhetherfeedbackisneededlocally,weusethelocalqualitymetricproposedin[190]calledtheRidgeClarity.Whilethismetricwasproposedforlatentim-ages,wethatitisappropriateforestimatingthelocalqualityofexemplar(Figure2.7).ThecomputationofridgeclarityforanimageIinvolvesthefollowingfoursteps:1.ContrastEnhancement:Obtainthecontrast-enhancedimageIC[94]:IC=sign(IIS)log(1+jIISj)(2.24)Here,ISistheimageobtainedusinga15x15averagingonI,andsign(x)isthesignumfunctionwhichoutputs1ifx>0and0otherwise.2.FrequencyDomainAnalysis:Thecontrast-enhancedimageICisdividedintoblocksofsize16x16,anda32x32subimageIC(x;y)isobtainedaroundthecenter(x;y)ofeachblock.IC(x;y)isthenpaddedwithzerostoobtaina64x64subimageIC(x;y).Thissubim-ageIC(x;y)istransformedintothefrequencydomaintoobtainFC(s;t).Twopeakpoints(s1;t1)and(s2;t2)correspondingtothetwolocalamplitudemaximawithinfrequencyrange[0:0625;0:2]inFC(s;t)arethenselected[116].The2-Dsinewavewi(p;q)attheithpeakpointinFC(s;t);i=f1;2gwithamplitudeai,frequencyfi,angleiandphase˚iisgivenby:wi(p;q)=aisin(2ˇfi(cos(i)p+sin(i)q)+˚i);(2.25)whereai=jFC(si;ti)j;fi=ps2i+t2i64;57i=arctansiti;˚i=arctanIm(FC(si;ti))Re(FC(si;ti)):3.RidgeContinuityMapComputation:Twoneighbouringblocksb1andb2aresaidtobecon-tinuousifthefollowingconditionsholdfortheircorrespondingsinewavesbw1andbw2:minfj1;2j;ˇj1;2jgT;1bf11bf2Tbf;116Xfp;q2 gbw1(p;q)ba1bw2(p;q)ba2Tbp:(2.26)Here,Tb;Tbf;Tbpareconstantssettoˇ=10,3and0.6,respectively,and referstothesetof16pixelswhichlieontheborderoftwoneighbouringblocks.anindicatorfunctionIfcforridgecontinuityas:Ifc=8>><>>:1;sw1andsw2arecontinuous,0;otherwise.(2.27)TheridgecontinuitymapRcontisthencomputedas:Rcont[p;q]=X[p;qN]maxfIfc(w1(p;q);w1(p;q));Ifc(w2(p;q);w2(p;q))g(2.28)584.RidgeClarityMapComputation:Finally,theridgeclarityforeachblockcenteredat[p;q]canbecomputedbytakingtheproductoftheamplitudewiththeridgecontinuitymapasfollows:Rclar[p;q]=a1(p;q)Rcont[p;q]:(2.29)TodeterminetheregionswithineachlatentimageILwhichneedfeedback,weapplyathresholdth1onthelocalridgeclarityvalue.LetusanindicatorrandomvariableILFforeachblockcenteredat[p;q]whichequals1forlatentregionswhichneedfeedback(Figure2.7c):ILF=8>><>>:1;IL(Rclar[p;q])>th1;0;otherwise:(2.30)Similarly,todecidetheregionswithineachexemplarIRwhichcanprovidefeedbackweuseathresholdth2onthelocalridgeclarityvalue.LetusanindicatorfunctionIRFforeachblockcenteredat[p;q]whichtakesthevalue1inexemplarregionswhichcanprovidefeedback(Figure2.7f):IRF=8>><>>:1;IR(Rclar[p;q])>th2;0;otherwise:(2.31)Differentvaluesofthethresholdsth1andth2weretested,andtheyareempiricallysetto0:1and0:9,respectively.59(a)(b)(c)(d)Figure2.8Samplelatentimagesfrom(a)NISTSD27and(c)WVUlatentdatabases.Theirmatedexemplarsareshownin(b)and(d),respectively.2.5ExperimentalEvaluation2.5.1DatabasesTheproposedfeedbackparadigmwasevaluatedontwodifferentlatentdatabases,NISTSD27[148]andWVU[150].Toincreasethesizeofthereferencedatabase,weincluded27,000rolledimagesfromNISTSD14[147]databaseand68,002rolledimagesprovidedbytheMichiganStatePolice.So,thereferencedatabaseconsistedof100,000rolled60Figure2.9Performanceofthebaselinelatentmatcheronthetwolatentdatabasesagainstarefer-encedatabaseof100,000exemplars.2.5.1.1NISTSD27NISTSD27databasecontains258latentimagesaswellastheircorrespondingexemplarimagesfromoperationalcases.ThelatentimagesinNISTSD27havegoodcontrastbutcontaincomplexbackgroundnoise(Figure2.8(a)).Theresolutionofeachimageis500ppi.2.5.1.2WVUTheWVUdatabasewascollectedinalaboratoryenvironmentatWestVirginiaUniversity.Itincludes449latentimagesand4,740exemplarimagesoutofwhich449exemplarsarethetruematesofthelatents.TheoriginalresolutionofeachimageintheWVUdatabaseis1000ppibutitwasdownsampledto500ppiforourexperiments.Thelatentimagesinthisdatabasehaverelativelycleanbackground,butpoorimagecontrastascomparedtolatentsinNISTSD27(Figure2.8(c)).61Table2.1Thetotalnumberoflatentswhere(a)feedbackisapplied,(b)feedbackisappliedwhenitisnotneeded(matedexamplarretrievedatrank-1bythebaselinematcher),and(c)feedbackisnotappliedwhenitcouldhavebeenuseful(matedexemplarreturnedamongstthetop200candidatesbutnotatrank-1bythebaselinematcher)basedontheglobalcriterionforfeedback(atlevel=0.05).Database#Latentsforwhichfeedbackapplied#Latentswherefeedbackappliedbutnotneeded#LatentswherefeedbacknotappliedbutneededNISTSD27(258latents)17211WVU(449latents)2541202.5.2SizeoftheCandidateList(K)OneofthecriticalparameterswhileapplyingtheparadigmisthelengthofthecandidatelistK.WhilechoosingalargevalueofKwouldimprovetheoddsofthematedexemplarbeingretrievedinthecandidatelist,itwouldalsotakemoretimetore-sortthecandidatelist.TotheoptimalvalueofK,weplottheCumulativeMatchCharacteristics(CMC)curvesofthebaselinematchersusedinourexperiments(Figure2.9).Wecanseethattheperformancegainstabilizesbyrank200.So,tooptimizebothaccuracyandspeed,thevalueofKissetto200.2.5.3EffectivenessoftheGlobalCriterionforFeedbackApplyingfeedbacktoalatentwhenitisnotneededaddscomputationalcomplexitywithoutim-provingtheaccuracy.Theglobalcriterionforfeedbackobviatestheneedforfeedbackinabout86outof258latentqueriesfortheNISTSD27databaseandinabout195outof449casesfortheWVUdatabaseatlevelof0.05.Table2.1liststhenumberoflatentqueriesforwhich(i)feedbackisappliedevenwhenthematedexemplarisretrievedatrank-1bythebaselinematcher[166],and(ii)feedbackisnotappliedwhenthematedexemplarisnotretrievedatrank-1(butisamongstthetop200candidatesreturnedbythebaselinematcher[166]).Thelownumberofsuchcasesdemonstratetheefyoftheproposedcriterionindeterminingtheneedforfeedback.62(a)(b)Figure2.10Performanceofthebaselinematcherwithandwithoutridgeorientationandfrequencyfeedbackon(a)NISTSD27and(b)WVUlatentdatabase(againstareferencedatabaseof100,000exemplars).63(a)(b)Figure2.11Genuineandimpostorsimilarityscoredistributions(scaledtothesamesimilarityscorerange)fortheNISTSD27database(a)beforeand(b)afterapplyingfeedbackusingthetop200candidatesretrievedbythebaselinematcher(againstareferencedatabaseof100,000exemplars).TheoverlapbetweenthegenuineandtheimpostorscoredistributionsreducesbyŸ25%afterapplyingfeedback.64(a)Latent1fromNISTSD27(b)MatedExemplarof(a)(c)InitialOrientationField(d)OrientationFieldFigure2.12SuccessfullatentfeatureviafeedbackforalatentintheNISTSD27database.Showninredistheexemplarorientationandinblueistheinitialandlatentorientationin(c)and(d),respectively.NotethatthelatentorientationisclosertotheexemplarorientationcomparedtotheinitiallatentorientationTherankofthematedexemplarofthelatentin(a)improvedfrom49to16amongstthe200candidateexemplarsreturnedbythebaselinematcherafterfeedback.2.5.4PerformanceonNISTSD27DatabaseTheCumulativeMatchCharacteristics(CMC)curvesshowninFigure2.10aillustratetheperfor-manceofthebaselinematcher[166]withandwithoutfeedbackontheNISTSD27database.Us-ingtheproposedridgeorientationandfrequencyfeedbacktothelatentfeaturesimprovestherank-1accuracyimprovesbyaround3.5%.Consistentaccuracyimprovementforallranksisalsoobserved.Figure2.11showsthegenuineandimpostorsimilarityscoredistributionsbeforeandafterapplyingfeedback.Theoverlapbetweenthegenuineandimpostordistributionsdecreasesbyapprox.25%afterapplyingfeedback.Figures2.12and2.13showstwolatentsfor65(a)Latent2fromNISTSD27(b)MatedExemplarof(a)(c)InitialOrientationField(d)OrientationFieldFigure2.13SuccessfullatentfeatureviafeedbackforalatentintheNISTSD27database.Showninredistheexemplarorientationandinblueistheinitialandorientationin(c)and(d),respectively.NotethatthelatentorientationisclosertotheexemplarorientationcomparedtotheinitiallatentorientationTherankofthematedexemplarofthelatent(a)improvedfrom20to8amongstthe200candidateexemplarsreturnedbythebaselinematcherafterfeedback.whichtheretrievalrankofthematedprintisimprovedbyapplyingridgeorientationandfrequencyfeedbackforthelatentmatcherin[166].2.5.5PerformanceonWVUDatabaseTheCumulativeMatchCharacteristicscurves(Figure2.10b)fortheWVUdatabasealsodemon-stratetheadvantageofusingtheproposedfeedbackframeworkwiththebaselinematcher.Al-thoughthereisamarginaldecreaseintherank-1accuracy,itisoffsetbytheperfor-manceimprovementofabout1-1.5%forthehigherranks.Notethattheimprovementissmaller66(a)Latent1fromWVU(b)ImpostorExemplar(c)InitialOrientationField(d)OrientationFieldFigure2.14FailureoffeedbackforalatentintheWVUdatabase.Showninredistheexemplarorientationandinblueistheinitialandorientationin(c)and(d),respectively.ThelatentorientationisclosertotheexemplarorientationcomparedtotheinitiallatentorientationHowever,theretrievalrankofthematedexemplardegradedfrom16to49amongstthe200candidateexemplarsreturnedbythebaselinematcherafterfeedback.67ascomparedtoNISTSD27becausethecontrastoflatentsinWVUis,ingeneral,poormakingitdiftoextractlevelonefeaturesinthefrequencydomain.Figure2.14showsanexamplewheretheretrievalrankofthematedprintdegradesafterfeedbackforalatent.Thematchingperformancegenerallydegradeswhen(i)theridgestructureoftheimpostorissimilartolatentand(ii)theimpostorexemplarisofbetterqualityascomparedtothetruemateresultinginbetterqualityfeaturesbeingextractedfromtheimpostor.2.5.6ComputationalComplexityThecurrentimplementationofthefeedbackparadigmuseslocalridgeorientationandridgefre-quencyfeaturesextractedatmultiplepeakpointsinthefrequencyrepresentationofthelatentimage.Toreducethecomputationalcomplexity,thesefeaturesarecomputedonlyonceforeachquery,andthenusedinmatchingagainstallexemplarcandidates.Sincethefeedbackmechanismdoesnotinvolvetheentireexemplardatabasebutisusedonlytore-rankthetopKcandidatesreturnedbythebaselinematcher,thealgorithmiccomplexityofthealgorithmisO(K).ThealgorithmhasbeenimplementedinMATLABandrunsonadesktopsystemwithIntelRCoreTM2DuoCPUof2.93GHzand4.00GBofRAMwithWindows7Operatingsys-tem.FortheNISTSD27database,theaveragetimetoextractlocalorientationandfrequencyfeaturesforalatentisabout0.74secandtheaveragetimetomatchalatentagainstthetop200candidatesisabout4sec.Theextracomputationalcostincurredinmatchingthelatentisworththeimprovementinperformance,especiallyinforensicapplicationswhichdemandhighlatentmatchingaccuracy.2.6ConclusionsGiventherelativelypoorqualityofoperationallatentimages,featureextractionisoneofthemajorchallengesforalatentmatchingsystem.Todealwithcomplexbackgroundnoiseinthelatent,weproposeincorporatingfeedbackfromexemplar(rolledorplainto68featureextractioninlatentwiththeeventualgoalofimprovingthelatentmatchingaccuracy.Wedeviseamethodtouseexemplarfeatures(ridgeorientationandfrequency)forthelatentfeaturesandthendevelopafeedbackparadigmtousethelatentfeaturestore-sortthecandidatelistreturnedbyalatentmatcher.Experimentalresultsshow0.5-3.5%improvementinthelatentmatchingaccuracyusingthefeedbackmechanism.Wealsoproposeaglobalcriteriontodecideiffeedbackisneededforalatentquery.Alocalqualitybasedcriterionisusedtodeterminetheregionsinlatentwhereitshouldbeappliedifneededandtoidentifyreliableregionsinexemplarforprovidingfeedback.69Chapter3CrowdPoweredLatentFingerprintMatching:FusingAFISwithExaminerMarkups3.1IntroductionInthepreviouschapter,wepresentedaframeworktoimprovetheperformanceofautomaticlatentmatching.Still,latentmatchingisanextremelydifproblem,particularlywhenthequal-ityorinformationcontentoflatentsisinadequate.Mostforensicagencies,therefore,followasemi-automaticlatentmatchingprocess,whereaexaminermarksfeaturesonalatent,submitsaquery(imageplusmarkup)toanAFIS,andsubsequentlyreviewsthetop-K(usuallyK=20to50)retrievalsfromthedatabasetodetermineifthelatenthitagainstareferenceprint.Al-thoughthepracticeofobtainingmarkupsfromexaminersincreasestheoverallchancesofobtainingahitfromthedatabase[108],inaccuratefeaturemarkupscanleadtomatedreferenceprintsbeingretrievedatalowerrankinthecandidatelist.Thiscan,inturn,adverselyimpacttheexaminerdecisionprocess[86][181].Thegoalofthisresearchistoharnessthecombinedexper-70(a)(b)Figure3.1Twomarkups(bytwodifferentexaminers)foralatentimagefrom1000ppiELFT-EFSdatabase.Astate-of-the-artAFISwasunabletomakeahitforthelatentimageinlights-outmode(scoreof0withthetruemateinthereferencedatabase).However,feedingtheAFISwiththemarkupsshownin(a)and(b)resultedinthematedprintbeingretrievedatrank-1andrank-129,respectively.tiseofmultipleexaminersandtheAFIStoincreasethelikelihoodofobtainingahitatahigherrankfromthereferencedatabase.3.1.1Semi-automaticLatentMatching:AdvantagesandDisadvantagesTheNISTEvaluationofLatentFingerprintTechnologies,ExtendedFeatureSets(ELFT-EFS)2[108]reportedthatthelikelihoodofahitinthereferencedatabaseimproveswhenanAFISisprovidedwithamarkup1(seeFigure3.1).TheaccuracyofthebestAFISoperatinginthelights-outmode2isreportedtobe67.2%inidentifying1,066latentprintsagainstreferenceprintsfrom100,000subjects.However,theaboveaccuracyimprovesto70.2%whentheAFISisfedwithboththelatentimageandtheextendedfeatureset(EFS)markupprovidedbyNIST.Theaboveperformancegain,however,dependsontheprecisionofthemarkupbeingfedtotheAFIS[109].Imprecisemarkupscanresultinthematedreferenceprintbeingreturnedatalowerrankamongsttheretrievedcandidates[86][181]comparedtotheimagealonebeingfedto1Markup,inthischapter,referstothelatentimagewithfeaturesmarkedbyaexaminer.2Inthelights-outmode,AFISautomaticallyextractsfeaturesandcomparesthelatenttothereferenceprints,withoutanyhumanintervention.71theAFIS.Furthermore,markupsforthesamelatentbydifferentexaminerscandifferand,consequently,differentmarkupsmayleadtodifferenceinperformanceoftheAFIS(seeFigure3.1).3.1.2ProposedCrowdPoweredLatentMatchingFrameworkToovercometheaforementionedlimitations,weproposealatentmatchingframeworkwheretheAFISandlatentexaminers3operateinsynergytoimprovethelatentmatchingaccuracy4.Inthisframework,alatentissubmittedtotheAFIStobematchedinthelights-outmode.BasedontheoutputoftheAFIS,thelikelihoodthattheAFIShitagainstareferenceprintatrank-1isdeterminedusingavariantofthecriteriondescribedin[56].IfthelikelihoodoftheAFISmakingahitatrank-1islow,thelatentiscrowdsourcedtoapooloflatentexaminersformarkingfeatures.Inthismanner,thecollectiveﬁwisdomﬂofseverallatentexaminersisutilizedtoobtainmultiplemarkupsforalatentonlywhenrequired.ThemanualmarkupsarethenusedinconjunctionwiththeAFISforimprovingthelatentmatchingaccuracy.TheproposedframeworkisbasedontheconjecturethatcombiningmarkupsobtainedfromdifferentexaminerswiththeautomatedencodingoftheAFIScantheperfor-manceoftheAFIS.Theconjecturestemsfromtheclassicpatternrecognitiontheorythatagroupofexpertswithdiverseandcomplementaryskillscancollectivelysolveadifproblem,onaverage,betterthaneachindividualexpert[83][100].Eachlatentexaminer,aswellastheAFIS,canbeviewedasanexpertforlatentmarkup.Becausemanualmarkupsobtainedfromdiffer-entlatentexaminersleadtodifferentcandidatelistsbeingretrievedbytheAFIS,theirexpertiseisratherdiverse.Thus,acombinationofAFISwithexaminermarkupsshouldboosttheperformanceoftheAFIS.State-of-the-artAFIStypicallygeneratemultipletemplates(encodings)fromaninputlatent.Thesetemplatesarethenindividuallycomparedwiththereferenceprintsandtheresultingcompar-3Thetermlatentexaminerisusedtorefertoaexaminerwhoistrainedtoanalyzeandcomparelatentprints.4ThisworkwaspublishedintheproceedingsoftheInternationalConferenceonBiometrics(ICB),2015[52].72isonscoresarecombinedtogenerateasinglecandidatelist.Ourmethodcanbeviewedanalogoustogeneratingmultipletemplates,albeitbasedonfeaturemarkupbymultiplelatentexaminers,andfusingthemwiththemultipletemplatesinternallygeneratedbyanAFIS.Toevaluatetheproposedframework,wecrowdsourcedmarkupsfortheNISTSpecialDatabase27(NISTSD27)latents[18]tosixlatentprintexaminersaftoMichiganStatePolice.WealsoconductexperimentsusingtwoindividualmarkupsprovidedintheELFT-EFSpublicchallengedatabase[6]andoneindividualmarkupprovidedintheRS&Adatabase[24].WecomputetheefyoftheproposedcriteriontocomputethelikelihoodthattheAFISmakesahitforthelatentatrank-1.Theproposedcriterionisabletoreducethenumberoflatentsthatneedtobecrowdsourcedformanualmarkupfrom258to151forNISTSD27,from255to151forELFT-EFSdatabase,andfrom200to35forRS&Adatabasewithoutimpactingtheoverallhitrate.Ourexperimentalresultsonareferencedatabaseof250,000rolledprintsshowthatbyfusingthescoresfromlights-outcomparisonwiththescoresobtainedusingtheexaminermarkups,therank-1accuracyoftheAFISimprovesby7.75%on500ppiNISTSD27(usingsixmarkups),by11.37%on1000ppiELFT-EFSdatabase(usingtwomarkups),andby2.5%on1000ppiRS&Adatabase.Ourexperimentalresultsindicatethatmarkupsobtainedfromdifferentlatentexaminerscontaincomplementaryinformationwhich,inturn,helpstoboosttheperformanceofanAFIS.Thecontributionsofthischapterare:AsystemicwaytocombinetheAFISwithexaminermarkupstoboostthelatenthitrate,Acrowdpoweredlatentmatchingframeworkwherealatentiscrowdsourcedtoapoolofexaminersforobtainingmultiplemarkups,Acriteriontoautomaticallydeterminewhencrowdsourcingisrequired,andAmethodtodynamicallydeterminehowmanycrowdexpertsareneeded.733.2CollectiveWisdomofMultipleExaminersHarnessingtheﬁcollectivewisdomﬂofthecrowdisacommonlyusedmethodologyforperform-ingrelativelysimpletasks(e.g.,imagelabeling,productrecommendations).Forinstance,recom-mendationsystemsin5andAmazon6usethecollectivepreferencesofalargenumberofcustomerswhenrecommendingmoviesorproductstoacustomer.Expertcrowdsourcingisanotherconceptwhichhasrecentlygainedprominence[168][191].Thisinvolvesdynamicallyassemblingateamofexpertcrowdworkersforaccomplishingspecializedtasks.Weextendtheseconceptstolatentmatchinginthefollowingmanner.Givenalatent,anAFISoperatinginlights-outmodeisusedtocompareittoreferenceprintsinthebackgrounddatabase.Basedonthescoredistributionofthetop-KcandidatematchesoutputbytheAFIS,weascertainwhethermanualmarkupisneededtoboosttheperformance.Ifitisdeterminedthatmanualmarkupisneeded,thelatentmarkupiscrowdsourcedtoapooloflatentexaminers.TheobtainedmarkupsareinputtoAFISindividuallytogeneratemultiplescoresforeachreferenceprintinthedatabase.Theseindividualmarkupandlights-outscoresarethenfusedtoboosttheaccuracyoftheAFIS7(seeFigure3.2).3.2.1ExpertcrowdsourcingframeworkLetaquerylatentimagebedenotedbyIL,andthesetofreferenceprintimagesinthedatabasebedenotedbyIR.LetthetotalnumberofreferenceprintsinthedatabasebeN,andtheithreferenceprintbedenotedbyIR(i).ThelatentILiscomparedagainstthesetofreferenceprintsIRusinganAFISoperatinginlights-outmodetogenerateasetofsimilarityscoresSLO:SLO(i)=S(IL;IR(i));8i=f1;2;:::Ng:(3.1)5http://www6http://www.amazon.com/7Rank-levelfusionwasalsoinvestigatedforfusingthelights-outresultswiththeresultsobtainedfordifferentmarkups.However,score-levelfusionoutperformedrank-levelfusioninourexperiments.74Figure3.2Theproposedcrowdpoweredlatentmatchingframework.(a)LatentisfedtoanAFIS,(b)itisdeterminedwhethermanualmarkupisneeded,(c)markupsareobtainedviaexpertcrowdsourcing,(d)multiplemarkupsarefedtotheAFIS,and(e)AFISscoresin(b)arefusedwiththemultiplemarkupscoresin(d).whereS(IL;IR(i))isthesimilaritybetweenILandIR(i)outputbytheAFIS.Aparametricprobabilitydistributionmodelistothedistributionofthetop-KscoresfromthesetSLO,andavariantofthemethoddescribedin[56]isusedtoascertainwhethermanualmarkupisrequired(seeSection3.2.2).Ifitisdeterminedthatmanualmarkupisnotrequired,thesetoftop-Kcandidatesandtheircorrespondingscoresaredirectlyoutputforvalidationbyalatentexaminer.Otherwise,thelatentimageILiscrowdsourcedtoapooloflatentexaminersforprovidingmanualmarkups.LetPbethenumberofexaminersthatprovidemanualmarkups.DenotethesemarkupsasIM,wherethejthmarkupisIM(j).EachofthePmarkupsareindividuallyinputtotheAFIStoobtainsimilarityscoresagainstthesetofreferenceprintsIR,75SMj(i)=S(IM(j);IR(i));(3.2)8j=f1;2;:::;Pg;8i=f1;2;:::;Ng:HereSMj(i)denotesthesimilarityscorebycomparingthejthlatentmarkuptotheithreferenceprint.Finally,wefusethelights-outscoresSLOwiththesimilarityscoreofeachmarkupSMjforeveryreferenceprinttoobtainacombinedscoreSF,SM(i)=SMj(i)g;(3.3)8j=f1;2;:::;Pg;8i=f1;2;:::;Ng;SF(i)=SLO(i)SM(i);8i=f1;2;:::;Ng:(3.4)Here,isthescorefusionoperator,andSMdenotesthescoreobtainedbyfusingthescoresofthePdifferentmarkups.Thetop-KfusedscoresfromthesetSF,andthecorrespondingcandidatesbasedonthefusedscoresarethenoutputtothelatentexaminerforevaluation.3.2.2Whentocrowdsource?Althoughexpertcrowdsourcinghasitsadvantages,crowdsourcingeverylatenttoapooloflatentexaminersiscostlyintermsoftimeandeffort.IfitcanbeestablishedthatthelikelihoodofAFISmakingahitatrank-1,foragivenlatent,isfairlyhigh,thenexpertcrowdsourcingisnotutilizedforthatlatent.Whilelatentqualitycanbeanindicatorofthislikelihood,toourknowledge,thereisnoexistingsatisfactoryindicatoroflatentquality[186].Therefore,webaseourdecisionontheorderstatisticofthetop-KcandidatescoresreturnedbytheAFIS[56].Letthesetoftop-KscoresreturnedbythetheAFISbedenotedasX=fX(1);X(2);:::X(K)gwhereX(i)denotestherank-iscore.AnexponentialdistributionmodelistothesetX.Ahy-76pothesistestisconductedtodeterminewhetherthereisanupperoutlierpresentinthedistributionofthetop-Kscores.Thenullhypothesis(H0)andthealternativehypothesis(H1)areasfollows:H0:AllscoresinthesetXarei.i.dfromanexponentialdistribution.H1:Rank-1scoreX(1)isanupperoutlierofthescoredistribution.TheteststatisticZfortestingH0againstH1isas:Z=X(1)X(2)SK;SK=KXi=1X(i):(3.5)Thecriticalvalueofthetestz()atlevelis:z()=11K1:(3.6)ThevalueoftheteststatisticZ=zshouldbegreaterthanthecriticalvaluez()whenrank-1scoreX(1)isanoutlier.Thus,weanindicatorrandomvariableICwhichtakesthevalue1whenexpertcrowdsourcingisneeded,and0whenitisnotneeded:IC=8>><>>:0;z>z();1;otherwise(3.7)Inotherwords,iftherank-1scoreisindeedanupperoutlier,wearesufthatlights-outretrievedthematedreferenceprintatrank-1.Therefore,thequerylatentdoesnotneedmarkupsfromlatentexaminers.3.2.3Howmanyexpertsareenough?Aprioriinformationaboutlatentexaminers(e.g.,yearsofexperience,thenumberofcasessolved)isoftenknownandcanbeutilizedwhilecrowdsourcinglatentmarkup.Assumethatthelatentexaminerscanberatedbasedonsuchpriorinformation.Whenadditionalmarkupisrequiredforalatent,insteadofcrowdsourcingthelatenttoeveryexaminer,itcanbesenttothebest77(a)(b)(c)(d)(e)(f)Figure3.3Markupsbysixdifferentlatentexaminersforalatentimageinthe500ppiNISTSD27.examinertoobtainamarkup.Thebestexaminer'smarkupcanthenbefusedwiththelights-outAFIS,andthedecisionwhetheradditionalmarkupisneededmade.Subsequently,thelatentcanbesenttothenextbestexaminer,ifrequired.Suchagreedy(sequential)strategycandynamicallydeterminethenumberofexaminersneededforprovidingmarkups,inturn,reducingtherequiredcostandeffort[125].3.3ExperimentalDetailsAstate-of-the-artAFIS,whichwasoneofthetopperformingAFISintheNISTELFT-EFS2evaluation[108],isusedforconductingallexperiments.78(a)(b)Figure3.4Markupsbytwoexaminersforalatentinthe1000ppiELFT-EFSpublicchallengedatabase.(a)(b)Figure3.5Markupforalatentimage(a)inthe1000ppiRS&Adatabase.Thematedreferenceprintofthelatentisshownin(b).3.3.1DatabasesTheproposedlatentmarkupcrowdsourcingframeworkisevaluatedonthreedifferentlatentdatabases(summarizedinTable3.1),theNISTSD27[18],ELFT-EFS[6]andtheRS&A[24].Inadditiontothematedreferenceprintsofthelatentsavailablefromthesedatabases,weuserolledprints,providedbytheMichiganStatePolice(MSP),toenlargeourreferencedatabaseto250,000rolledprintsforalltheexperimentsreportedhere.TherolledprintsprovidedbyMSPhavesimilarcharacteristicstothematedreferenceprintsprovidedwiththethreelatentdatabases.79Table3.1Summaryofthelatentdatabasesused.Database#LatentsResolutionLatentType#MarkupsNISTSD27258500ppioperational6*ELFT-EFS**2551000ppioperational2*RS&A2001000ppicollectedinlab1*Thescopeofthisresearchistoinvestigatehowbesttocombineindependentmarkups.Therefore,juriedmarkups,althoughavailable,arenotusedbecausetheyinvolvetheex-pertiseofmultipleexaminers.**ELFT-EFSdatabasecontains255latentsfromNISTSD27rescannedat1000ppi.Table3.2Numberoflatentsmarkupsprovidedbyeachofthesixexaminers(outof258)fortheNISTSD27latents.Examiner123456No.ofmarkups2532552552552532573.3.2LatentMarkupIndependentfeaturemarkupsforNISTSD27latentswereobtainedfromsixcertlatentprintexaminersaftoMichiganStatePolice.Theaveragefeaturemarkuptimeisabout5min.perlatent(around20hoursforall258latents).Examinerswereaskedtomarkminutiae,ridgecountsbetweenminutiaeand/orregionofinterest(ROI)onthelatents.However,notallexaminersmarkedall258latents(seeTable3.2).Figure3.3showssamplemarkupsobtainedfromthesixexaminersforalatentinNISTSD27.SomeexaminersmarkedROIwhileothersdidnot.ForeachlatentintheELFT-EFSdatabase,atleasttwoindependentfeaturemarkupsareavailablewiththedatabase.StandardEFTS-LFFSfeaturemarkups(minutiae,ridgecountsbetweenminu-tiae,singularpointsandROI)areusedinourexperiments.Notethatlatentexaminers,ingeneral,donotmarkextendedfeaturesonalatentbecauseitisachallenging(ambiguous)andtimecon-sumingprocess.Hence,ourexperimentsareinaccordancewiththegeneralmarkupprotocolbeingfollowedbyexaminersinlawenforcementagencies.Figure3.4showssamplemarkupsforalatentfromtheELFT-EFSdatabase.OnlyasinglemarkupisavailableintheRS&Adatabase[24]whichisutilizedinourexperiments(Figure3.5).803.3.3ExperimentsToevaluatetheefyoftheproposedexpertcrowdsourcingframework,weperformthefollow-ingsetofexperiments8.3.3.3.1Lights-outMatchingTheCumulativeMatchCharacteristic(CMC)curvesoftheAFISinthelights-outmodeonNISTSD27aremarkedasImageonlyinFigure3.6(a).Therank-1accuracyis64.34%.NoticethereductioninperformanceoftheAFISonbadanduglyqualitylatentsascomparedtothegoodqualitylatentsintheNISTSD27(Figures3.6(b)-(d)).Figure3.7(Imageonly)showstheCMCcurvesforlights-outonELFT-EFSdatabase.Therank-1rateis65.10%.Figure3.8(Imageonly)showstheCMCcurveforlights-outontheRS&Adatabase.Therank-1accuracyobtainedontheRS&Adatabaseis87.50%.ThisismuchhigherthantheaccuracyobtainedontheNISTSD27andELFT-EFSdatabasesbecausethelatentsintheRS&Adatabasewerecollectedinalaboratoryandarecomparativelyofbetterquality.3.3.3.2MatchingIndividualExaminerMarkupsTheImageplusMarkupperformancebandinFigure3.6(a)indicatestheaccuracyoftheAFISontheNISTSD27whenfedwithindividual500ppimarkups.Thebestrank-1accuracyobtainedusinganindividualmarkupis66.67%.Notethatthelights-outperformanceiswithintheperformancebandoftheexaminers.Asexpected,theaccuracyishigherforgoodqualitylatents,comparedtothebadanduglyqualitylatents(Figures3.6(b)-(d)).Figure3.7showstheperformanceoftheAFISwhenfedwith1000ppimarkupsavailablefortheELFT-EFSdatabase.Thebestindividualrank-1accuracyobtainedis72.16%.8Opensetexperimentsareplannedforsubsequentstudies81(a)Alllatents(b)Goodqualitylatents(c)Badqualitylatents(d)UglyqualitylatentsFigure3.6performance(CMCcurves)oftheAFISonNISTSD27when(i)operatinginlights-outmode(Imageonly),(ii)fedwithmarkupfromasingleexaminer(Image+Markup),and(iii)fusionoflights-outand500ppimarkupsfromallsixexaminers(Fusion)for(a)all258latents,(b)88goodqualitylatents,(c)85badqualitylatents,and(d)85uglyqualitylatents.Thesizeofthereferencedatabaseis250Krolledprints,includingthetruematesoflatentsfromNISTSD27.Theperformancebandofthelatentexaminersindicatesthemaximumandminimumaccuracyobtainedusinganindividualexaminermarkupatdifferentranks.82Figure3.7performance(CMCcurves)oftheAFISwhen(i)operatinginlights-outmode(Imageonly),(ii)fedwithanindividual1000ppimarkup(Image+Markup),and(iii)fusionoflights-outAFISscoreswiththescoresobtainedusingthetwo1000ppimarkups(Fusion)forall255latentsintheELFT-EFSdatabaseagainstareferencedatabaseof250Krolledprints.Figure3.8Performance(CMCcurves)oftheAFISwhen(i)operatinginlights-outmode(Imageonly),(ii)fedwiththesingleavailablemarkup(Image+Markup),and(iii)fusionoflights-outwithexaminermarkup(Fusion)forthe200latentsintheRS&Adatabaseagainstareferencedatabaseof250Krolledprints.83Table3.3accuracy(%)oftheAFIS,onaverage,ontheNISTSD27against250Kreferenceprintswhenfedwithmarkupsfromdifferentsubsetsoflatentexaminers.CombinationRank-1Rank-50Rank-100Oneexaminer63.1177.1378.23Twoexaminers68.0480.8881.96Threeexaminers69.4282.1583.29Fourexaminers70.0082.7183.98Fiveexaminers70.8083.1484.56Allsixexaminers70.9382.9584.88OntheRS&Adatabase,ontheotherhand,therank-1accuracyobtainedusingthesingleavailablemarkupis90%(Figure3.8).3.3.3.3FusingMultipleExaminerMarkupsSincewehavesixdifferentmarkupsavailablefortheNISTSD27latents,wefusethescoresob-tainedusingdifferentmarkupcombinations,andthencomputetheaverageaccuracyoftheAFISwhenfedwithdifferentsubsetsofexaminermarkups.Severaldifferentscorelevelfusionstrategieswereinvestigated.Simplesumfusionruleprovidedthebestperformance.NoscorenormalizationisnecessaryheresinceallthescoresarebeinggeneratedbythesameAFIS.Table3.3showsthatwhileidentperformanceoftheAFISimproveswithadditionalmarkups,thereisasatura-tionafter3or4markupsperlatent.FortheNISTSD27with258latents,each1%improvementinperformance,sayatrank-1,correspondstoroughlytwoorthreelatentsbeingpromotedtorank-1.3.3.3.4Fusinglights-outAFISwithMultipleMarkupsTheCMCcurvesplottedinFigure3.6showthattherank-1ideaccuracyoftheAFISincreasesby7.75%ontheNISTSD27byfusingthescoresobtainedusingthesixmarkupswiththescoresobtainedfromlights-outOntheotherhand,aperformanceimprovementof11.37%isobservedwhenfusingthescoresobtainedfromthetwoindividualmarkupsforthe84(a)(b)(c)(d)(e)(f)(g)Figure3.9AnexamplelatentforwhichthematedreferenceprintisretrievedatahigherrankafterfusingthesixcrowdsourcedmarkupswiththeAFIS.Inthelights-outmode,theAFIScouldnotmatchthelatenttothematedprintshownin(g)(score=0).Therankofthematedprintusingtheindividualmarkupsbythesixexaminersshownin(a)-(f)is80,-(score=0),45,7,57and12971,respectively.Thematedprintisretrievedatrank-2usingthecombinationoftheAFISwiththesixmarkups.ELFT-EFSdatabasewiththelights-outscores.Figures3.9and3.10,respectively,showanexampleofasuccessfulandfailurecaseusingfusionoftheAFISwiththeexaminermarkups.FortheRS&Adatabase,althoughfusionoflights-outmatchscoreswithmarkupscoresdoesnotseemtointermsoftherank-1accuracyincomparisontoonlyusingthemanualmarkup,performanceimprovementisobservedforhigherranks(seeFigure3.8).3.3.3.5DeterminingtheneedforcrowdsourcingTomeasuretheefyofthetestbasedonorderstatisticfordeterminingtheneedforcrowd-sourcingmanualmarkup,wecomputethe(i)numberoflatentswheremarkupisnotneededand85Table3.4Numberoflatentswheremarkupisrequired,markupisnotrequiredwhenmatedrefer-enceprintisnotatrank-1,andmarkupisrequireddespitethematedreferenceprintbeingretrievedatrank-1forNISTSD27(NIST27),ELFT-EFS(ELFT),andRS&A(RSA)databases.Thenumberoflatentsinthesethreedatabasesis258,255,and200,respectively.level()#Latentsrequiringmarkup#Latentsnotrequiringmarkupwhenmatedreferenceprintisnotatrank-1#Latentsrequiringmarkupwhenmatedreferenceprintisatrank-1NIS27ELFTRSANIST27ELFTRSANIST27ELFTRSA0.0116616646002*7474220.0515115135002*5959110.113713733002*45459*Thematedreferenceprintsareincorrectlylabelledfortheselatents;doesnotimpacttheaccuracyoftheAFIS.thematedprintwasnotretrievedatrank-1,and(ii)numberoflatentswheremarkupisascertainedbutthematedprintwasretrievedatrank-1(Table3.4).ThevalueofKusedhereis200.Forcase(i)wefoundthattherankofthematedprintdidnotdecreaseafterfusionoflights-outwithmarkupscores.Thisdemonstratestheefyoftheorderstatisticbasedtest.3.3.3.6GreedycrowdsourcingTotesttheofusingthegreedysequentialstrategytodynamicallydeterminethenumberofexaminersrequired,weratedtheindividualexaminersbasedontheirskillset.ThiswasestimatedbasedontheAFISperformanceobtainedonthemarkupstheyprovided.Figure3.11showsperfor-manceimprovementwhenindividualexaminersareselectedindecreasingorderoftheirskillset.Afterfusingthethreemarkupsfromthetopthreeexaminers,additionalmarkupshavenegligibleimpactontheoverallaccuracy.Also,utilizingmorenumberofmarkupsdoesnotnecessarilyimprovetheoverallaccuracy.Overall,151latentsintheNISTSD27requireexam-inermarkupsbasedonthelights-outAFISresults(atalevelof0.05).137,131,126,126,124,and123latentsneedmarkupsafterfusionoflights-outAFISwithbest-1,best-2,best-3,best-4,best-5,andallsixexaminermarkups,respectively.86(a)(b)(c)(d)(e)(f)(g)Figure3.10AnexamplelatentforwhichthematedreferenceprintisretrievedatalowerrankafterfusingthecrowdsourcedmarkupswiththeAFIS.Inthelights-outmode,theAFISretrievedthematedprintshownin(g)atrank-1.Therankofthematedprintin(g)usingtheindividualmarkupsbythesixexaminersshownin(a)-(f)is54,1171,3426,595,22and8450,respectively.Thematedprintisretrievedatrank-26usingthecombinationoftheAFISwiththesixmarkups.3.4ConclusionsMatchingpoorqualitylatentstoreferenceprintsisoneofthemostchallengingproblemsingerprintrecognition.Inordertomatchlatentstoreferenceprintswithhighaccuracy,weproposeacrowdpoweredlatentmatchingparadigmwhichinvolvesasymbiosisofexaminerswithAFIS.Givenalatentprint,itiscomparedagainstreferenceprintsusinganAFIS.Basedontheoutputofthelights-outmatch,anautomaticdecisionismadetodetermineifmanualfeaturemarkupsfromlatentexpertswouldbeIfitisdeterminedthatadditionalmarkupwouldhelp,thelatentprintiscrowdsourcedtoapooloflatentexaminers.ThemanualfeaturemarkupsarefedtotheAFISandthecomparisonscoresfromlights-outAFISandthosefrommanualmarkupsinputtoAFISarecombinedtoboosttheaccuracy.Experimentalresultsobtainedonthreedifferentlatentdatabases(NISTSD27,ELFT-EFSandRS&A),againstareferencedatabase87Figure3.11accuracyoftheAFISusinggreedycrowdsourcingforthe258NISTSD27latents.Startingwiththebestexaminer,alevelof0.05isusedtodecideifmarkupfromthenextbestexaminerisneeded.Numbersoflatentsgiventothenextbestexaminerareindicatedinred.DuetothepreponderanceoflowqualityprintsinNISTSD27,therank-1accuracytapersoffafterthreeexaminermarkups.of250,000rolledprints,demonstratethataperformanceimprovementcanbeobtainedusingtheproposedcrowdpoweredframework.88Chapter4DesignandFabricationof3DSingle-FingerTargets4.1IntroductionUntilabout20yearsago,forensiclabsandlawenforcementagenciesweretheprimaryconsumersofrecognitiontechnologywithbeingutilizedtoidentifyrepeatoffendersandtoassociateacrimetocriminal(s).Inchapters2and3,wedevelopedmethodstoaddressoneofthemostimportantproblemsfacedbytheseagencies,namelymatchinglatentcommonlyencounteredincrimescenestolegacyrolledandslapdatabases.However,therecentpasthaswitnessedlargescaledeploymentsofrecognitiontechnologyincivilian,commercialandpersonalapplications,e.g.,theAadhaarprogramtouniquelyidentifyeachresidentofIndia[38],theUnitedStates'OfofBiometricIdentityandManagement'sprogram(formerlyUSVISIT)topreventillegalimmigrantsandcriminalsfromenteringthecountry[20],andtheTouchIDsystemtounlockApplesmartphonesandmakeonlinepayments[36].Giventhisrapidgrowthinlargescaledeploymentsofsystems,itisessentialtohaveareasonableestimateoftheirmatchingperformanceandrobustnessintheoperationalsettings.89Figure4.1Structural(White-Box)v.Behavorial(Black-Box)evaluationofreaders.Instructuralevaluation,detailsoftheinternalsetupofthereaderareknownandreadercomponentassemblyoperationistested.Ontheotherhand,inbehavorialevaluation,theinternaldetailsofthereaderarenotknownandonlyfunctionalityofthereaderistestedbasedonitsinputandoutput.Forthoroughevaluationofsystems,alargenumberofrepresentativengerprintimagesfromtheoperationalscenarioareneeded.Collectingsuchalargenumberofimageswithdifferentcharacteristicsfromhumansubjectsisbothexpensiveandtedious.Bio-metricsynthesisprovidesasolutiontothisproblem.Alargenumberof2Dcanbegeneratedusing2Dsyntheticgenerators[67][192]thatcanbeutilizedforevaluat-ingfeatureextractorsandmatchers.However,theycannotbeusedforassessmentofreaders.Standardcalibrationtargetsaretypicallyusedforstructuralevaluation1ofreaders,e.g.,measuringtheirgeometricaccuracy,distortionandresolution.Onelimitationofthesetargetsthoughisthattheycannotbeusedforbehavorialevaluation2ofthereadersintheoperationalsettings(seeFigure4.1).Furthermore,thesetargetsarenotsuitableforﬁend-to-endﬂevaluationofrecognitionsystemsfromacquisitiontofeatureextractionandmatching.1Structuralorwhite-boxevaluationtestshowinternalsystemcomponentsandcomponentsub-assembliesshouldoperate,andrequirestechnicalknowledgeofthesystem[59].2Behavioralorblack-boxevaluationtestsfunctionssupportedbythesystemintheoperationalordeploymentscenariobyfocusingontheinputandoutputofthesystem[59].90(a)(b)(c)Figure4.2Examplesofimagingphantomsusedinmedicalimaging:(a)Phannie,aphantomtocalibrateMRImachinesdevelopedatNIST[14],(b)aphantomhandusedforevaluatingX-raymachines[41],and(c)atorsophantomusedtocalibrateCT-Scanmachines[34].Thisisbecausetheprocessofuserinteractionwiththereaderleadingtocapturecannotbemimickedusingthesetargets.Inthischapter,weproposetogenerate3Dtargetsforbehavioralevaluationofreadersinoperationalsettings.4.1.1StructuralEvaluationofFingerprintReadersAsmentionedintheearliersection,structural(white-box)evaluationofimagingsystemsisgener-allydoneusingspeciallydesignedobjectswithknownproperties,calledtargets.Inthebiomedicaldomain,forinstance,suchobjects(calledphantoms)areusedforcalibratingandtestingopticalmeasurementofsensinginstrumentation[180],[60](Figure4.2).Similarly,targets(Figure4.3)havealsobeenusedforcalibrationofreaders.TherearetwoseparatestandardscurrentlyinusebytheFederalBureauofInvestigation(FBI)fortheofreaders,(i)thePIV,whichcaterstoreadersdesignedforapplicationsinvolvingpersonv(one-to-onecomparison),and(ii)theAppendixF,whichappliestoreadersdesignedforuseinlargescaleapplicationsinvolvingperson(one-to-manycomparisons)[8].Togettheirreadersvendorsneedtodemonstratethattheimagescapturedusingtheirreadersmeettheimagequalitylaidoutintherelevantstandard[155][156].Atypicalprocedureis(i)touse2D/3Dcalibrationtargetstoascertainiftheimagesofthetargetscapturedusingthereadermeetthespec-91(a)(b)(c)Figure4.32Dimagesofstandardtargetsusedforcalibratingreaders,(a)ronchi(verticalbar)targetforcalibratingthegeometricaccuracy,(b)sinewavetargetformeasuringtheresolution,and(c)multiplebartargetforestimatingthespatialfrequencyresponseofareader(imagestakenfrom[155]).(ii)modifythereaderifneeded,toensureitcapturesimagesofsufqualitytomeettheand(iii)whenwiththereadersubmittestimagestothetestingagencyforreview3[8].Ifthetestdataisfoundtomeetthedesiredspec-thetestingagencythereaderasbeingcompliantwiththestandard.4.1.2BehavioralEvaluationofFingerprintReadersStandardcalibrationtargets(seeFigure4.3)areusedforstructuralevaluationofreaders.Forexample,thetargetsin[33]areutilizedfortestingfrustratedtotalinternal(FTIR)components(LED,glassprismandplatenassembly)ofanopticalreader.However,thesetargetsarenotsuitableforbehavioral(black-box)evaluationofareaderinthepresenceofoperationalvariations(e.g.,placementandpressureetc.)whenusersinteractwiththereader.Thisisbecausethesetargetsarenotconstructedusingmaterialswithproperties(e.g.,hardnessandelasticity)similartothehumanskin.3ReviewofthesubmittedtestdataisconductedbytheTechnologyEvaluationStandardsTestUnit,apartoftheFBI'sBiometricCenterofExcellence(BCOE)ledbytheCriminalJusticeInformation(CJI)ServicesDivision[9].92(a)(b)(c)(d)(e)(f)Figure4.4Evaluatingaopticalreaderusingthe3Dtargetsdesignedandfabricatedbytheauthors.(a)The3Dtargetiswornona,(b)theisplacedonthereaderplaten,and(c)-(f)multiple2Dimpressions(fourshownhere)ofthe3Dtargetarecapturedtoevaluatethereader.4.1.33DTargetsforBehavioralEvaluationForbehavioralevaluationofareader,onepossibilityistoconductpilotstudiesinvolv-inghumansubjectsintheusingthereader.This,however,isatediousprocessbothintermsoftimeandresourcecommitment,andislimitedbytheamountandpossiblevariationsinthe-printdatathatcanbecollected.Besides,suchaprocedurecannotbeusedforrepeatablebehavioralevaluationofthereaderbecause,inpractice,thesamesetofsubjectsistypicallynotavailableforrepeattesting.Thegoalofthisresearch,therefore,istofabricatestandard3Dtargetswhichcanbeusedforrepeatablebehavioralevaluationofreaders.Wefabricate3Dtargetswithmaterialsimilarinhardnessandelasticitytothehumanskinsuchthattheycanbewornonaandplacedonthereaderplateninanaturalmanner(seeFigure4.4)4.4ThisworkwaspublishedinIEEETransactionsonInformationForensicsandSecurity(TIFS),2016[54].93Table4.1Comparisonofprevailing2Dsyntheticrprintbasedevaluationmethodswiththeproposed3Dtargetgenerationmethod.MethodArtifacts*FingerprintFeaturesEvaluationUseCasesSFinGe[67]2Dsynthetic(electronic)Knownridgewandridgedensityfeatures;uncontrolledminutiaeplacementFingerprintfeatureextractorsandmatchersIBGDHSSBIR[105]2Dsynthetic(electronic)Knownridgewandridgedensityfeatures;partiallycontrolledminutiaeplacementFingerprintfeatureextractorsandmatchersZhaoetal.[192]2Dsynthetic(electronic)Knownridgew,ridgedensityandminutiaeplacementFingerprintfeatureextractorsandmatchersNIST**3Dtargets(electronicandphysical)KnowncalibrationpatternfeaturesContactless3DreadersProposed3Dtargets(electronicandphysical)Knownridgew,ridgedensityandminutiaeplacementEnd-to-endsystems,includingreaders,featureextractorsandmatchers*Thetermelectronicisusedfordigitallygeneratedartifacts,whereasthetermphysicalisusedforphysicallyfabricatedartifactsfromelectronicartifacts.**ThisresearchiscurrentlyunderwayatNISTandhasnotbeenpublishedyet.Theutilityofthefabricated3Dtargetsextendsbeyondbehavioralevaluationofreaders.3Dtargetsgeneratedusing2Dsyntheticimageswithknownfea-tures(e.g.type(loop,whorl,arch),minutiaepositionandorientation,andcoreanddeltacountandlocations)canbeusedtoevaluatefeatureextractionandmatchingalgo-rithms.Suchtargetscan,therefore,beusedforend-to-endevaluationofarecognitionsystemfromplacingtheonthereaderandcapturingthe2Dimpressiontoextractingfea-turesandcomparingthecapturedimagetothegallerytemplates.Further,sincethefabricated3Dtargetsaresimilarincharacteristicstothehumanskin,inouropinion,theycanalsobeusedtoevaluatethenextgenerationtouchlessreaders[21][10][17].Hencetheproposed3Dtargetsarebettersuitedforntsystemevaluationpurposesthantheprevailingmethodswhichonlyuse2Dsynthesizedimages(seeTable4.1).94Figure4.5Generatinga3DtargetA,givena2DcalibrationpatternIanda3DsurfaceS.Aphysical3Dtargetiscreatedbyprojectinganelectronic2Dcalibrationpatternontoagenericelectronic3Dmodelofthesurface5.Theelectronic3Dsurfaceisalignedsuchthatthelengthisalongthey-axis,widthalongthex-axisanddepthalongthez-axis.Theelectronic3Dsurfaceisthenpreprocessedtoensuresufforestablishingthecorrespondencebetweentheelectronic2Dcalibrationpatternandtheelectronic3Dsurface.The2Dcalibrationpatternisthenmappedontothefrontportionoftheelectronic3Dsurfaceandcorrespondencesbetweeneachvertexonthefrontalelectronic3Dsurfaceandthepixellocationsinthe2Dcalibrationpatternareestablished.The2Dcalibrationpatternisengravedontothefrontalelectronic3Dsurfacebydisplacingeachvertexalongthesurfacenormalaccordingtothetexturevaluesatthemappedpixellocations.Finally,theelectronic3Dsurfaceispost-processedtocreateanelectronicmodelofawearable3Dtargetreadyfor3Dprinting.Thephysical3Dtargetsarefabricatedusingastate-of-the-art3Dprinter(StratasysObjet350Connex6)withmaterialsimilarinhardnessandelasticitytothehumanskin.The3Dprintedtargetsarecleanedusinga2MNaOHsolutiontogenerateevaluation-readyphysical3Dtargets.ThecompleteprocessisillustratedinFigure4.5.5The3Dsurfacecouldeitherbetheshapeofthesensedusinga3Dscannerorasyntheticallygeneratedsurfacedescribingtheshapeofthe.Inourcase,thesurfacewasscannedusingtheArtecEva3Dscanner[2].6Thenamingofcompaniesandproductsheredoesnotimplyendorsementorrecommendationofthosecompaniesorproductsbytheauthorsortheorganizationstheyrepresent.95Therearetwokindsoferrorsthatcanbeintroducedduring3Dtargetcreationfrom2Dimage,(i)the2Dto3Dprojectionerrorofthemappingalgorithmusedinelectronic3Dtargetcreation,and(ii)thefabricationerrorintroducedbythe3Dprinterwhenfabricatingthephysical3Dtargetfromtheelectronic3Dtarget.Toassessthey7ofthe3Dtargetgenerationprocess,weesti-matethe2Dto3Dprojectionerror,andthe3Dprintingfabricationerrorbyobservingthetargetsunderadigitalopticalmicroscope(KeyenceDigitalMicroscopeVHX600[11]),(iii)matching2Dcalibrationpatternfeaturesusedfor3Dtargetcreationtoboththeelectronicandphysical3Dtargetimages,and(iv)evaluatingsimilaritybetweendifferentimagesofphysical3Dtargets.Weshowthat(i)featurespresentinthe2Dcalibrationpatternarepreservedduringthecreationofelectronic3Dtarget,(ii)featuresengravedontheelectronic3Dtargetarepreservedduringphysical3Dtargetfabrication,and(iii)intra-classvariabilitybetweenmultipleimpressionsofthesamephysical3Dtargetissufsmallformatchingatfalseacceptrate(FAR)of0.01%.Wealsoshowthatthegenerated3Dtargetsaresuitableforbehavioralevaluationofthreedifferent(500/1000ppi)PIV/AppendixFopticalreadersintheoperationalsettings.Insummary,thecontributionsofthischapterareasfollows:Designofwearable3Dtargetsusinga2Dto3Dprojectionalgorithmthatpreservesdistancesonthe2Dcalibrationpatternwhilemappingitto3Dsurface.Inourpreliminarywork[53],weusedanangle-preserving2Dto3Dmapping[82]whichdidnotpreservepoint-to-pointspacinginthe2Dcalibrationpatternduring3Dprojection,especiallyneartheperipheryofthe3Dsurface.Becauseitisimportanttopreservedistancesonthe2Dcalibrationpatternafter3Dprojection,hereweuseadistancepreservingmapping[177].Fabricationof3Dtargetsusingastate-of-the-art3Dprinterwithmaterialshavingsimilarhardnessandelasticitytothehumanskin.Thesetargetscanbeimagedbythreedif-ferentcommercial(500/1000ppi)opticalreaders.The3Dprinterusedtofabri-7Fidelityreferstothedegreeofexactnesswithwhichthe2Dcalibrationimageisreproducedinthegenerated3Dtarget.96catetargetsin[53]onlyprintedhardplastictargetsthatcouldnotbeimagedbycommercialreaders.Proceduretochemicallycleanthe3Dprintedtargetswithoutimpactingtheengravedtargetpatterns.Estimationof(i)2Dto3Dprojectionand(ii)3Dprintingfabricationerrors;theseerrorsareaccountedforduringreaderevaluations.Comprehensiveexperimentationtoshowtheof2Dcalibrationpatternfeaturesduring3Dtargetgeneration.Preliminaryexperimentationforbehavorialevaluationofreadersusingthegen-erated3Dtargets.4.2Generating3DTargetsA3DtargetAisgeneratedusinganarbitrary2DcalibrationpatternIwithfeatures,andageneric3DsurfaceS.Letthegrayscalevalueinthe2DcalibrationpatternIatspatialcoordinates(u;v)bedenotedbyI(u;v).Also,assumethatthe3DrsurfaceSisatriangularmeshwithasetVofverticesandasetToftriangles.Eachvertex,v,inVhas(x;y;z)coordinatescorrespondingtoitsspatiallocationinS,andatriangleinTconnectsauniquesetofthreevertices.Generatingthe3DtargetAusingIandSthenconsistsofthefollowingmainsteps(Figure4.5).1.Preprocessing3Dsurface:AlignSsuchthatthelengthisalongthey-axisinS.SampleverticesfromthesetVbasedonthecurvatureofS.ThissamplingprocessreducesthedensityofS,therefore,subdivideS(asexplainedinSection2.1)toensuresufduringprojectionofthe2DcalibrationpatternI.DisplaceSoutwardsalongthedirectionofthesurfacenormalscomputedateachvertextocreateanoutersurfaceSO.SeparatethefrontSOFandrearportionSORofSO(seeFig.4.6).ThefrontportionSOFofSOwillbeusedforprojection.RetaintheoriginalsurfaceS.972.Preprocessing2Dcalibrationpattern:IfthepatternIbeingprojectedisa2Dimage,extracttheskeletonISoftheimageI.IncreasetheridgewidthoftheskeletonISusingmorphologicaloperations,andsmooththeimageusingaGaussianbeforeprojectingitontotheelectronicfrontalsurfaceSOF.ThispreprocessingstepisnecessarytoensurethatridgesandvalleyspresentinIareengravedsmoothlyontoSOF.Notethatthispreprocessingstepisnotneededifanyother2Dcalibrationpattern(e.g.sinegrating)isbeingprojected.3.Mapping2Dprintto3Dsurface:ProjectthefrontportionSOFof3Dsur-faceSOto2Dandcorrectforrotationandusingcorrespondingcontrolpointsbetween3DsurfaceSOFandthe2DprojectionofSOF,andtranslationwithrespecttoreferenceco-ordinatescomputedfromI.MakethefrontportionoftheoutersurfaceSOdensedependingontheresolutionofItoensuresufofmappingI.Determinethemappingbetweenthe(x;y;z)spatiallocationsoftheverticesonthefrontportionoftheouter3DsurfaceSOandthe(u;v)imagedomainofI.4.Engraving2Dcalibrationpatternon3Dsurface:Tocreateridgesandvalleys,displacetheverticesonthefrontportionofSOalongthesurfacenormalsaccordingtothetexturevaluesinIatthemapped(u;v)locations.5.Postprocessing3Dsurface:CombinethefrontandrearportionsoftheoutersurfaceSO.MaketheoriginalsurfaceSasdenseastheoutersurfaceSOandthenstitchthetwosurfacestogethertoobtainawatertightsolidtarget.Thisthecreationofthe3DtargetAasanelectronic(virtual)target.6.3DPrinting:Specifythephysicaldimensionsaswellastheprintingmaterialaccordingtothehardnessandelasticityofthehumanskinbeforeprintingthe3DtargetAusinga3Dprinter(StratasysObjet350Connex).987.ChemicalCleaning:Cleanthe3Dprintedtargetsusing2MNaOHsolutionandwatertoremovetheprintersupportmaterialresidueandobtainevaluation-ready3Dtargets.Adetaileddescriptionofeachofthesestepsusedinthe3Dtargetcreationprocessforagiven2DcalibrationpatternIanda3DsurfaceSisgivenbelow.4.2.1Preprocessing3DsurfaceAsequenceofpreprocessingstepsisexecutedonthe3DsurfaceSbeforeprojectingthe2DcalibrationpatternIonS(seeFigure4.6).Thesestepsinclude:(i)alignmentofthe3Dsurface,(ii)remeshingthe3Dsurface,(iii)subdivisionofthe3Dsurface,(iv)creatingoutersurfacefromthegiven3Dsurface,and(v)separatingfrontandrearportionsoftheouter3Dsurface.4.2.1.1AlignmentThe3DsurfaceS,arbitrarilyorientedinthe(x;y;z)coordinateframe,isalignedsuchthatthelengthisalongthey-axis,widthalongthex-axisandheightonthez-axis.Fordoingthis,eachvertexinthesetVistranslatedsuchthatthecenterofthesurfaceScoincideswiththeoriginofthe(x;y;z)coordinateaxes.Principalcomponentanalysis(PCA)[128]isusedtodeterminetheprincipledirectionsofthesurfacespread.ThecomputedprincipalcomponentsarethenusedtoalignthesurfaceS.Notethatthissteponlyalterstheabsolute(x;y;z)coordinatevaluesoftheverticesinVandretainsthegeometryofthesurfaceS.4.2.1.2RemeshingThe3DsurfaceSisremeshedbysamplingverticesfromVusingthemethodin[167].Thevertexv1issampledrandomlyfromV,andthegeodesicdistancemapU(v1)fromv1toeveryothervertexinViscomputedbysolvingtheeikonalequationusingthefastmarchingmethod[129]:99(a)(b)(c)(d)(e)(f)(g)(h)Figure4.6Preprocessing3Dsurface.(a)OriginalsurfaceS,(b)aligningSsuchthatthelengthisalongtheyaxis,(c)alignedS(triangularmesh),(d)remeshingS(triangularmesh),(e)subdividingS(triangularmesh),(f)subdividedSview),(g)creatingoutersurfaceSOfrom(f),and(h)separatingfrontandrearportions,SOFandSOR,ofSO.jj5U(v1)jj=P(v1):(4.1)Here,5isthegradientoperator,andP=1=F,whereFisthespeedoffrontpropagationusedinthefastmarchingmethod.VerticesarethensamplediterativelybyaddingthefarthestvertexamongtheremainingverticesiniterationifromtheverticesinthesampledvertexsetVi1atiterationi1.NotethatthegeodesicdistancemapUi,atiterationi,isupdatedusingthefollowingequation:Ui=min(Ui1;U(vi));(4.2)100whereU(vi)isthegeodesicdistancemapofthevertexsampledatiterationi,andUi1isthegeodesicdistancemapcomputedatiterationi1.Duringthisiterativeprocedureofsamplingvertices,thespeedoffrontpropagationFissetto1=(1+C),whereCistheaggregatecurvatureateachvertexinV.Thisresultsinmoreverticesbeingsampledinthehighercurvatureregionsofthe3Dsurfaceandviceversa.TheaggregatecurvatureCiscalculatedusingthetwoprincipalcurvaturesCminandCmaxasfollows,C=jCminj+jCmaxj:(4.3)Here,j:jistheabsolutevalueoperator,CminandCmaxarecomputedfromthe3DcurvaturetensorCTcalculatedusingthemethodin[49].Inparticular,CminandCmaxcorrespondtothetwohighesteigenvaluesofthecurvaturetensorCT.Finally,Delaunaytriangulationisusedforrecreatingtheremeshedsurfacefromthesetofsampledvertices[51].4.2.1.3SubdivisionAlthoughremeshingmakesthesurfaceSuniformlydensedependingonitscurvature,itreducesthedensityofthevertices.Toensuresufforprojectingthe2DcalibrationpatternIontothesurfaceS,Loop'ssurfacesub-divisionmethod[136]isusedtoincreasedensityofvertices.LetthesetofverticesandtrianglesobtainedafterremeshingbedenotedbyVRandTR,respectively.ThismethodcreatesnewverticesateachedgeofeverytriangleinTRusingaweightedcombinationofneighborhoodvertices,andcreatesnewtrianglesbyconnectingthesampledverticesatedgesadjacenttoeachother.Theoriginalverticesarethentranslatedtomaintainsurfacesmoothnessandcontinuity.4.2.1.4CreatingoutersurfaceLetVSandTSbethesetofverticesandtrianglesobtainedaftersurfacesubdivision.LetthenormalnatavertexvinthesetVSbedenotedby(nx;ny;nz),wherenx,nyandnzrepresentthenormal101componentsalongthex,yandzdirections,respectively.Eachvertexvisthendisplacedbyaedfactordalongthenormalntoobtainthedisplacedcoordinatesofthevertex(v0x;v0y;v0z):266664v0xv0yv0z377775=266664vxvyvz377775+266664nxnynz377775d(4.4)ThisisdonetocreateanoutersurfaceSOwherethe2Dcalibrationpatternwillbeprojected.Theparameterddeterminesthethicknessofthe3Dtarget.Ideally,itisdesirabletosetdtobeassmallaspossible.However,duetothelimitationofthe3Dprinterresolutionusedforfabricatingthetargets,choosingaverysmalldresultsintheprintedmodelbeingfragile.Therefore,disempiricallysetto1:5mminourexperiments.4.2.1.5SeparatingfrontandrearportionsFrontandrearportionsoftheoutersurfaceSOarethenseparatedbycomputingthesurfacenormalsateachtriangleinTS,andthenretainingthetrianglesandcorrespondingverticeswheresurfacenormalshavethez-componentgreaterthan0inthefrontsurface,andtherestintherearsurface.Notethatthealignmentofthesurfacedoneinstep1)facilitatesthisseparationprocess.LetusdenotethefrontportionoftheoutersurfaceSOasSOFhavingthesetofverticesVOFandtrianglesTOF.Similarly,lettherearportionbedenotedasSORwiththesetofverticesVORandtrianglesTOR.WealsoretaintheoriginalsurfaceSwiththesetofverticesVSandtrianglesTS.4.2.2Preprocessing2DcalibrationpatternIfthepatternIbeingprojectedon3DfrontalsurfaceSOFisaimage,thefollowingpreprocessingstepsareexecutedonI(seeFigure4.7):1.Theskeleton,IS,ofI,a1-pixelwideridgepattern,isextractedusingacommercial-printSDK[146].102(a)(b)(c)(d)Figure4.7Preprocessinga2Dpatternbeforeprojectingitonto3Dsurface.(a)OriginalimageI,(b)extractedskeletonISofthein(a),(c)skeletonISin(b)afterapplyingthemorphologicaloperationofdilation,and(d)dilatedskeletonin(c)smoothedusingagaussian.2.TheridgewidthontheskeletonISisincreasedto3pixelsbyperformingthemorphologicaloperationofdilationusinga2pixelradiusdiskstructuredelement.3.ISisredusinga44Gaussianwith˙=2:5toensurethatridgesandvalleysin2DpatternIareengravedsmoothlyontothe3Dsurface.Thispreprocessingisnotneededforothercalibrationpatterns(e.g.sinegratingofcertainorientationandspacing).4.2.3Mapping2Dcalibrationpatternto3DsurfaceThefrontportionSOFoftheoutersurfaceSOisprojectedfrom3D((x;y;z)space)to2D((u;v)space)bycomputingtheISOMAPembedding[177](seeFigure4.8).RecallthattheverticesandtrianglesinSOFareVOFandTOF,respectively.TheISOMAPembeddingiscomputedby:1.Constructingadjacencygraph:AnadjacencygraphGiscreatedbyconnectingallvertexpairsfvi;vjginVOFthatshareanedgeofanytriangleinTOF.TheedgeweightsinGaresettotheeuclideandistanceD(vi;vj)betweenviandvj.Fornon-adjacentvertexpairsthatdonotshareanyedge,Dissettoanarbitrarylargevalue.103(a)(b)(c)(d)(e)Figure4.8Mappingandengraving2Dcalibrationpatternontothefrontportionoftheouter3DsurfaceSOF.(a)3DfrontaloutersurfaceSOF,(b)frontalsurfaceSOFin(a)isprojectedinto2D,(c)the2DprojectedfrontalsurfaceSOFPissubdivided,(d)correspondencesaredeterminedbetweenthe2DprojectedfrontalsurfaceSOFPand2DcalibrationpatternI,(e)3DfrontaloutersurfaceSOFin(a)isdisplacedalongthesurfacenormalstoengravethepattern.2.Computingshortestpaths:Dijkstra'sshortestpathalgorithm[84]isusedtocomputetheshortestpathbetweenallpairsofnodesinG.GeodesicdistancesbetweenallpairsofverticesinVOFareestimatedbytheshortestpathdistancesofthenodesinG.3.Constructing2Dembedding:LetthematrixDGcontaintheshortestpathdistancescom-putedinthepreviousstep.GivenDG,multidimensionalscaling(MDS)[131]isusedtocreatethe2Dembeddingofvertices.ISOMAPembeddingisusedbecauseitminimizesthedistortioninducedwhenprojectingthefrontportionSOFofthe3Dsurfaceto2DbypreservingthegeodesicdistancesbetweenneighborhooodverticesonSOF8.Letthe2Dprojectedfrontalsurfaceinthe(u;v)coordinatespacebedenotedbySOFPwiththesetofverticesVOFPandthesetoftrianglesTOFP.Rotationandduringthe3Dto2Dprojec-tionofSOFarecorrectedusingcorrespondingcontrolpointsbetweenSOFandSOFP.Referencecoordinates[ru;rv]areextractedfromthe2DcalibrationpatternIfortranslationcorrectionduringthe3Dto2DprojectionofSOF:8Discreteconformalmappingwasusedforprojectinga2Dpatternto3Dsurfaceinourpreliminarywork[53].Itwas,however,observedthatdiscreteconformalmappingdidnotpreservethedistancesonthecalibrationpatternneartheperipheryofthe3Dsurfacesinceitisananglepreservingmapping.104IfthepatternIbeingprojectedisasyntheticimage,thenreferencecoordinates[ru;rv]areextractedfromtheimageusingthemethodin[186].Ifanyothercalibrationpatternisbeingprojected(e.g.sinegratings,horizontal/verticalbarpatternsetc.),thenthelocationofthecenterpixelinthe2DcalibrationpatternIisusedasthereferencepointi.e.[ru;rv]=[w=2;h=2],wherewandharethewidthandheightofI.Thenextstepistodeterminetheone-to-onemappingbetweenthepixellocations(u;v)onIandtheverticesVOFonSOFP.Foraccuratelydeterminingtheone-to-onecorrespondence,thedensityofSOFaswellasits2DprojectionSOFPisfurtherincreasedusingmidpointsurfacesubdivision.VerticesaresampledonthemidpointsoftheedgesinTOF,andthesampledverticesontheadjacentedgesarejoinedtocreatenewtriangles.TheresolutionofIbeingprojectedisfactoredintothecomputationswhiledeterminingthecorrespondencebetweenpixellocationsonIandverticesVOFonSOFP.Forexample,ifthecalibrationpatternbeingprojectedhasaresolutionof500ppi,thescaleofprojectionis19:685pixels/mm.Therefore,thecoordinatesofIarescaledbythisfactorbeforedeterminingthecorrespondence.Ideally,thedensityofSOFshouldbeincreasedaccordingtothedimensionsofthecalibrationpatternIbeingprojected.Forexample,ifacalibrationpatternofwidthwandheighthwithwhpixelsisbeingprojected,thenexactlywhverticesarerequiredintheprojectionregionforbuild-ingtheexactcorrespondencebetweenthepixellocationsonIandtheverticesonSOF.However,itwouldresultinaverylargenumberofverticesandtrianglesonthesurfaceandconsiderablyincreasethecomputationalcomplexityofanyfurtheroperationsonthesurface.Therefore,thedensityofSOFisonlyincreasedtotheextentthatitretainstheessentialtopologyofthepatternbeingprojected9.Letthesetofverticesandtrianglesonthe2DprojectedfrontalsurfaceobtainedafterthisstepbedenotedbyVOFPSandTOFPS,andthecorrespondingverticesandtrianglesontheouter3DfrontalsurfacebeVOFSandTOFS,respectively.Theone-to-onecorrespondencebetweenthepixellocationsonthecalibrationpatternIandthesetofverticesVOFPisthenestablished.9Forthesurfaceusedinourexperiments,thedensityisincreasedsothatthereareapproximately250,000verticesand500,000trianglesonthefrontportionofthe3Dsurface.105(a)(b)(c)(d)(e)Figure4.9Postprocessing3Dsurface.(a)Separatedfrontandrearportionsofouter3Dsurface,(b)frontandrearportionsshownin(a)arecombinedtocreatetheouter3Dsurface,(c)outer3Dsurface(bottomview),(d)theretainedoriginal3Dsurface(bottomview),(e)electronic3Dtargetcreatedbystitchingtheouterandoriginalsurfacein(c)and(d).4.2.4Engraving2Dcalibrationpatternon3DsurfaceInthepenultimatestep,surfacenormalsarecomputedateachvertexinthesetVOFPS.TheverticesarethendisplacedalongtheirsurfacenormalstoengravetheridgesandvalleysonSOF(seeFigure4.8(e)).LetthenormalatavertexvinthesetVOFPSbedenotedby(nx;ny;nz),wherenx,nyandnzrepresentthenormalcomponentsalongthex;yandzdirections,respectively.Thedisplacedcoordinatesofthevertex(v0x;v0y;v0z)alongthenormalarethencomputedusingtheprincipleofvertexdisplacementmapping[39]asfollows:266664v0xv0yv0z377775=266664vxvyvz377775+266664nxnynz377775(1I0(u;v))Rd(4.5)Here,I0(u;v)isthescalenormalizedgrayscalevalueintherange[0;1]ofthemappedgrayscalevalueat(u;v)fromthe2Dcalibrationpatternonthevertexv,andRdisthemaximumverticalridgedisplacementwhichissetto0:22mminourexperiments10.10Theaverageridgeheightonanadulthumanisabout0:06mm;howeverwesetRdto0:22mmempiricallyduetolimitationofthestate-of-the-art3Dprinterresolutionusedforfabricatingthetargets.106Table4.2Comparisonofmechanicalpropertiesofthetwo3Dprintermaterialsusedfor3Dtargetfabricationwiththehumanskin.PropertyHumanSkin[88][89]TangoBlackPlusFLX980[31]FLX9840-DM[30]ShoreAhardness20-4126-2835-40TensileStrength(MPa)5-300.8-1.51.3-1.8ElongationatBreak(%)35-115170-220110-1304.2.5Postprocessing3DsurfaceTheengravedSOFandSORarecombinedtogethertorecreatetheoutersurfaceS0O.TheoutersurfaceS0OisthenstitchedtogetherwiththeretainedoriginalsurfaceSOtocreateacontinuouswatertight3DshellSWreadyfor3Dprinting.Fordoingthis,theboundaryofthetwomeshesS0OandtheSOiscomputed.Trianglesarethensyntheticallygeneratedtoconnectthetwoboundariestocreateacontinuousshell(seeFigure4.9).Thiscontinuouswatertightshellisbasicallythe3DtargetAinelectronicform.4.2.63DprintingWeuseastate-of-the-art3Dprinter(StratasysObjet350Connex)thathasXandYresolutionof600dpiandZresolutionof1600dpiforfabricatingthe3DtargetswithUVcurablerubber-likepolymericmaterials.ThisprinterisbasedonPolyJetprintingtechnologywhichslicesa3Dmodelintohorizontallayers,andthenprintsthemodellayerbylayer.The3Dtargetsareprintedinhighspeedmodewhereintheyareslicedinto30micrometer()layersduringtheprintingprocess.Notethattheprinterdoesnotsupportprintingthetargetwithrubber-likematerialsinthehighresolutionmodewhichallowsforeven16micrometerlayerslicing.However,wefoundthat30slicingsufwithridgedisplacementRdof0:22mm.Inthehighspeedmode,thetimetakentofabricateone3Dtargetusingtheprinterisapproximately90minutes.Twodifferentrubber-likematerials,TangoBlackPlusFLX980[31],andFLX9840-DM[30](adigitalmaterialsynthesizedintheprinterbycombiningarubber-likematerialandarigidmaterial)areusedforprintingthe3Dtargets.Thesematerialsareselectedbecausetheyare107Figure4.10The2Dimagesofamanuallycleaned3Dtarget(shownontheleft)andthesametargetafterchemicalcleaning(shownontheright)capturedusingaopticalreader.Chemicalcleaningofthe3Dtargetwith2MNaOHsolutionandwaterremovesthe3Dprintersupportmaterialresidueandprovidesabetterqualityimage.similarinhardnessandelasticitytothehumanskin(seeTable4.2)11.NotethatwearelimitedinthechoiceoftheprintingmaterialpertheprinterEventhoughthechoiceoffabricationmaterialsislimited,ourapproachisbetterthanamanualprocessofcreatinga2.5Dor3Dmouldofaandthencastingthetargets.Thisisbecausethe3Dprintingprocess(i)isautomated,(ii)canaccuratelyreplicatetargets,and(iii)isefbecauseitcanprintseveraltargetsinparallel.4.2.7ChemicalcleaningWhileprintingthe3Dtargets,theprinterusesasupportmaterialtopreventthemodelsbeingfabricatedfrombreaking.Asaresult,oncethetargetsareprintedtheyneedtobecleanedtoremovethesupportmaterial.Manualcleaningremovesthebulkofsupportmaterial,however,stillleavessomeresidue.Therefore,themanuallycleanedtargetsaredippedina2MNaOHsolutionforapprox.3hourstodissolvethesupportmaterialresidue.Subsequently,thetargetsarecleaned11Theprintingmaterialsusedareblackincolor,andtheiropticalcharacteristicsdifferfromthatofhumanskin.Therefore,itmaynotbepossibletoimagethefabricatedtargetsusingsomeopticalreaders(e.g.darkreaders).Toovercomethislimitation,wearecurrentlyexploringthepossibilityofusingalternativefabricationmethods.108Figure4.11Thetwosourcesoferrorin3Dtargetgeneration(showninred)givena2Dcalibrationpatternanda3Dsurface:(i)2Dto3Dmapping,(ii)3Dprintingfabrication.withwatertoobtainevaluation-ready3Dtargets.Figure4.10shows2Dimagesofamanuallycleaned3Dtargetandthesametargetafterchemicalcleaningwith2MNaOHandwatercapturedusingaopticalreader.Thetargetimagequalityimprovesconsiderablypostchemicalcleaning.4.3Fidelityof3DTargetGenerationInordertodeterminetheof3Dtargetgeneration,wemeasuretheerrorintroducedduring(i)projectionof2Dcalibrationpatternto3Dsurfacetocreateelectronic(virtual)target,and(ii)fabricationofphysical3Dtargetfromtheelectronic3Dtargetusing3Dprinting.Wealsoconductexperimentstodeterminetheof2Dpatternfeaturesduring3Dtargetcreation.4.3.12Dto3DProjectionErrorGeodesicdistancesbetweenallpairsofverticesonthefrontal3DsurfaceSOF,andEu-clideandistancesbetweenthecorresponding2Dmappingofvertexpairsafterthefrontalsurfaceisunwrappedto2DusingtheISOMAPalgorithmarecomputed.Ratioofgeodesicdistancestoeu-clideandistancesiscomputedtodeterminetheextenttowhichdistancesarepreservedduring2Dto3Dprojection.Forthesurfaceusedinourexperiments,thegeodesictoeuclideandistanceratioisestimatedas0.942.Thisindicatesthatthereisa5.8%reductioninpairwise(point-to-point)109Table4.3ObservedaveragegratingspacingonthreedifferenttargetswhenviewedundertheKeyenceVHX-600DigitalMicroscopeattwodifferent(50Xand100X).Expectedaveragespacingforeachtargetis0.478mm.Target50X100XHorizontal0.426mm0.427mmVertical0.420mm0.412mmCircular0.415mm0.419mm(a)(b)Figure4.12Estimating3Dprintingfabricationerrorbymeasuringpoint-to-pointdistancesbe-tweenhorizontalgratingsona3Dtargetat(a)50Xand(b)100XusingtheKeyenceDigitalMicroscopeVHX-600.distancesdueto2Dto3Dmappingalgorithm.Weaccountforthiserrorinreadereval-uationexperiments.4.3.23DprintingFabricationErrorThreedifferent3Dtargetsarecreatedbyprojectingsyntheticallygenerated2Dtestpatterns:(i)horizontal,(ii)vertical,and(iii)circulargratings,withaedcenter-to-centerspacingof10pix-els.Spacingof10pixelsintestpatterngratingsshouldcorrespondtospacingof0.508mmingratingsetchedonthefabricatedphysicaltargets(attheprojectionscaleof500ppi).However,theexpectedaveragegratingspacingonthephysicaltargetsis0.478(0.5080.942)mmduetothe2Dto3Dprojectionerror(5.8%).Tomeasuretheobservedaveragegratingspacingonthe110fabricatedtargets,thethreetargetsareviewedunderanopticalmicroscope(KeyenceVHX600DigitalMicroscope[11]).Fivedifferentimagesofeachofthethreetargetsarecapturedattwodifferenttionsof50Xand100Xusingthemicroscope.Atotalof20and10pointpairsaremanuallymarkedonconsecutivegratingsinimagescapturedattheof50Xand100X,respectively.Point-to-pointdistancesaremeasuredbetweenthemarkedpointpairsusingthesoftwareprovidedwiththemicroscope(see,forexample,Figure4.12).Theobservedaveragegratingspacingforthethreetargetsatthetwoisestimatedastheaverageofthepoint-to-pointdistancemeasurementstakenbetweenthemanuallymarkedpointpairs(seeTable4.3).Thesemeasurementsindicatethatthegratingsetchedonthephysicaltargetsbythe3Dprinteraremuchclosertoeachotherthanexpectedor,inotherwords,gratingspacingisreduceduponduring3Dfabrication.Basedonthedifferencebetweentheobservedandtheexpectedaveragegratingspacingforthethreetargets,theaveragereductioningratingspacingduetofabricationisestimatedtobe11.42%.Althoughthiserrorisquiteitisexpectedsincewe3Dprintvery(0.5mm)gratings,andthe3Dprinterisnotveryaccurateinprintingobjectsatsuchascale.Thiserroriscompensatedforinreaderevaluationexperiments.4.3.3Fidelityof2Dpatternfeaturesduring3DtargetcreationToassessifthefeaturesinthe2Dcalibrationpatternareadequatelypreservedduringthe3Dtargetgenerationprocess,wedetermineifthefeaturespresentinthe2Dcalibrationpattern,I,arepreservedduringprojectionto3Dsurfacetocreatetheelectronic(virtual)3Dtarget,featuresengravedontheelectronic3Dsurfacearepreservedafterfabricationofthephysical3Dtarget,featurespresentinthe2Dpattern,I,arepreservedonthephysical3Dtarget,andintra-classvariabilitybetweenthecapturedimpressionsofthe3Dtargetusingreadersisminimal.111Table4.4Similarityscoresbetweentheimages(2D)oftheelectronic3DtargetsinMeshlabandthe2DimagesfromNISTSD4usedfortargetgeneration.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.FingerprintS0005S0010S0017S0083S0096Score171378212116106FivedifferentrolledimpressionsfromtheNISTSpecialDatabase4(NISTSD4)[19]areusedascalibrationpatternsandprojectedontoa3Dsurfacetogenerateelectronic3Dtargets.Thephysical3Dtargetsarefabricatedwitheachofthetwofabricationmaterialsusingastate-of-the-art3Dprinter(seeSection4.2.6).Threeopticalreaders,abbreviatedasOR1,OR2,andOR3areusedforimagingthephysical3Dtargets12.OR1isaPIV500ppiopticalreader,whereasOR2andOR3are1000ppiopticalreaderscomplyingwiththeIAFISAppendixFimagequalityAcommercialrprintSDK[146]isusedforconductingallmatchingexperiments.Thecapturedimagesusingthethreereadersareupsampledbyafactorof1.1toaccountforridgespacingreductiondueto2Dto3Dprojectionand3Dprintingbeforeconductingthematchingexperiments.4.3.3.1Fidelityof2Dpatternfeaturesafterprojectionto3DsurfaceEachelectronic3Dtargetispreviewedinthe3DmeshprocessingsoftwareMeshlab[15],anditsfrontalimageistaken.Thecapturedimageoftheelectronic3Dtargetisrescaledmanuallytothesamescaleastheoriginal2Dimagesusedduringthesynthesisofthetarget.Therescaledfrontalimagesoftheelectronic3Dtargetismatchedtotheoriginal2DimageusingtheSDK.Figure4.13showsasampleimage(calibrationpattern)fromtheNISTSD4andtheimagesofitselectronic3Dtarget.TheminutiaeextractedandmatchedusingtheSDKaremarkedonthetwoimages.Table4.4showsthecorrespondingsimilarityscores.Allsimilarity12Capacitivereaderscouldnotbeusedinourevaluationbecausestate-of-the-art3Dprinterscurrentlydonotallowprintingobjectsusingconductivematerials.Wearecurrentlyexploringthepossibilityofusingalternativefabricationmethodstointroduceconductivity.112Figure4.13Minutiaecorrespondencebetween(a)rolledimage(S0083fromtheNISTSD4),and(b)2Drenderingoftheelectronic3Dtargetgeneratedusing(a).Similarityscoreof116isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.scoresareabovethevscorethresholdof33(@FAR=0.01%)forNISTSD4.Thisdemonstratesthatthefeaturespresentinthe2Dimagesarepreservedduringthesynthesisoftheelectronic3Dtargets.4.3.3.2Fidelityoftheengravedfeaturesonthe3Dsurfaceafter3DprintingTheimageofanelectronic3Dtargetismatchedtocapturedimageofthecorrespondingphysical3Dtargetusingthethreeopticalreadersforeachoftheten3Dtargets.Figure4.14showsminutiaecorrespondencesobtainedusingtheSDKbetweentheimageofoneelectronictargetanditscapturedimageusingopticalreaderOR2.Table4.5showsthesimilarityscoresforthisexperiment.Noticethatthesimilarityscoresareabovethevthresholdscoreof33(@FAR=0.01%)foralltentargets.Thisdemonstratestheoffeaturesengravedonthe3Dsurfaceafter3Dprinting.113Table4.5Similarityscoresbetweentheimages(2D)oftheelectronic3Dtargetsandtheimagescapturedbythethreeopticalreadersofthephysical3Dtargetsfabricatedwithtwodifferentmaterials(TangoBlackPlusFLX980andFLX9840-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.TangoBlackPlusFLX980FingerprintOR1(500ppi)OR2(1000ppi)OR3(1000ppi)S0005165197392S0010192350359S0017143180207S0083372407348S0096165204336FLX9840-DMFingerprintOR1(500ppi)OR2(1000ppi)OR3(1000ppi)S0005201342324S0010194390342S0017143228302S0083326473441S0096120210179Figure4.14Minutiaecorrespondencebetween(a)imageoftheelectronic3Dtarget(ofS0083inNISTSD4),and(b)theimagecapturedbyopticalreader2(1000ppi)ofthephysical3DtargetfabricatedwithFLX9840-DM.Similarityscoreof473isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.114Table4.6Similarityscoresbetweentheimagescapturedbythethreeopticalreadersofthe3Dtargetsfabricatedwithtwodifferentmaterials(TangoBlackPlusFLX980andFLX9840-DM)andthefromNISTSD4usedintheirgeneration.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.TangoBlackPlusFLX980FingerprintOR1(500ppi)OR2(1000ppi)OR3(1000ppi)S000593161171S0010129150183S001793167167S0083174344167S0096131240197FLX9840-DMFingerprintOR1(500ppi)OR2(1000ppi)OR3(1000ppi)S0005114410185S0010113209173S0017122182158S0083140374305S009696177177Figure4.15Minutiaecorrespondencebetween(a)rolledimage(S0083fromtheNISTSD4),and(b)theimagecapturedbyopticalreader2(1000ppi)ofthe3Dtargetgeneratedusing(a)andfabricatedwithFLX9840-DM.Similarityscoreof374isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.115Table4.7Rangeofsimilarityscoresforpairwisecomparisonsbetweenvedifferentimagescap-turedbythethreeopticalreadersofthesame3Dtargetfabricatedwithtwodifferentmaterials(TangoBlackPlusFLX980andFLX9840-DM).V6.3SDKwasusedforgener-atingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.TangoBlackPlusFLX980FingerprintOR1(500ppi)OR2(1000ppi)OR3(1000ppi)S0005431-1017675-1146929-1286S0010638-10491053-14551169-1620S0017464-11551230-1592843-1292S0083890-14401016-1620744-1325S0096726-1286842-1443774-1334FLX9840-DMFingerprintOR1(500ppi)OR2(1000ppi)OR3(1000ppi)S0005597-12951103-1689921-1620S0010647-12391256-16431299-1605S0017534-12981203-14791170-1481S0083614-12621326-16971215-1656S0096807-13441154-14011238-1607Figure4.16Minutiaecorrespondencebetweentwoimages(a)and(b)capturedbyopticalreader2(1000ppi)ofthe3DtargetgeneratedfromS0083inNISTSD4andfabricatedwithFLX9840-DM.Similarityscoreof1494isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.1164.3.3.3End-to-endof2Dcalibrationpatternfeaturesafter3DprintingTable4.6showsthesimilarityscoresobtainedwhencomparingtheimagesofallten3Dtargetscapturedusingthethreereaderstothecorrespondingoriginal2Dimages.Figure4.15showsminutiaecorrespondencebetweentheimageandimagesofthegenerated3DtargetusingtheSDK.Thekeyobservationsandinferencesbasedonthisexperimentare:Imagesofthe3Dtargetscapturedusingallthreeopticalreaderscanbesuc-cessfullymatchedtotheoriginalrprintimagesusedforgeneratingthetargets;allthesimilarityscoresinTable4.6arecantlyabovethevthresholdscoreof33(@FAR=0.01%).Becausetheimagesofthe3Dtargetscanbesuccessfullymatchedtotheoriginalimages(@FAR=0.01%),itcanbeinferredthatthesalientfeaturespresentinthe2Dpatternarepreservedduringthefabricationofthephysical3Dtarget.4.3.3.4Intra-classvariabilitybetween3DtargetimpressionsFivedifferentimpressionsofeachoftheten3Dtargetsarecapturedusingallthreeopticalreaders.PairwisecomparisonsbetweentheveimpressionsobtainedfromareaderareperformedusingtheSDK.Figure4.16showstheminutiaecorrespondencebetweentwodifferentimpressionsofa3DtargetcapturedusingopticalreaderOR2.Table4.7showstherangeofsimilarityscoresallofwhicharehigherthanthethresholdat0.01%FAR.Thisindicatesthattheintra-classvariabilitybetweendifferentimagesofthe3Dtargetissmall.1174.4BehavioralEvaluationofFingerprintReadersusing3DTargetsInbehavioralevaluation,theaimistotestthefunctionalityofthereaderintheopera-tionalscenario.Thereareseveraldifferentparameterswhichcanimpactthequalityoftheimagecapturedbythereaderinafunctionalsetting,e.g.,theamountanddirectionofthepressureap-pliedbyauserandthemovementofhisonthereaderplatenwhencapturingtheimage.Ourendgoalistoassesstheeffectoftheseparametersonthereaderperformancebyex-plicitlycontrollingtheseparameters.Forthis,weplantomount3Dtargetsonarobotichandandconductcontrolledexperimentation.However,toshowtheutilityofthefabricated3Dtargetsforbehavioralevaluationofreaders,weconducttwopreliminaryexperiments(i)using3Dtargetscreatedfromsyntheticallygeneratedtestpatternstoevaluatedirectionalimagingcapability(ExperimentI),and(ii)using3Dtargetscreatedbyprojectingpatternstoevaluatethecapabilitytocapturepatterns(ExperimentII).4.4.1ExperimentI:SyntheticSineGratingTargetsTendifferentimpressionsofeachofthethreetargetscreatedusinghorizontal,verticalandcircularsinegratingsof10pixelspacingarecapturedusingallthreeopticalreaders.Center-to-centerspacingisthenmeasuredineachofthecapturedimpressionsusingthemethodin[104].Directionalimagingcapabilityofreadersissubsequentlyassessedbasedonhowwellthegratingspacingonthethreetargetsisrecoveredbythereaders.Figs.4.17,4.18and4.19showthethreedirectionaltestpatterns,theelectronictargetsgeneratedusingthethreepatterns,andsomesampleimagesofthethreetargetscapturedusingthethreeopticalreaders.Theaverageandthestandarddeviationoftheobservedcenter-to-centergratingspacinginthecapturedimpressionsofthethreetargetsisreportedinTable4.8.Notethattheexpectedgratingspacingonthesetargetsis8.278(100.827)pixelsaftertakingintoaccounttheprojection(5.8%)andfabricationerror(11.42%).Followingaresomeobservationsbasedonthisexperiment:118Table4.8Mean()andstd.deviation(˙)ofcenter-to-centerspacingintheimagesofthethreedi-rectionaltesttargetscapturedusingthethreeopticalreaders(OR).(Expectedgratingspacing=8.278pixels.)TestpatternOR1(500ppi)OR2(1000ppi)OR3(1000ppi)Horizontal=8.307,˙=0.101=8.445,˙=0.085=8.420,˙=0.030Vertical=8.869,˙=0.076=8.561,˙=0.076=8.592,˙=0.098Circular=8.921,˙=0.044=8.823,˙=0.048=8.721,˙=0.053(a)(b)(c)(d)(e)Figure4.17Evaluatingopticalreaderswitha3Dtargetgeneratedusingahorizontalsinegrating.(a)Horizontalsinegrating(10pixelseparationbetweenthegratings);(b)electronic3Dtargetgeneratedusing(a);(c),(d)and(e)aresampleimagesofthefabricatedtargetcapturedusingopticalreaders1,2and3,respectively.Thereisaslightdistortionapparentin(b)thatisduetothe2Dto3Dprojectionerror.Theobservedspacinginimagesofallthreetargetsis,onaverage,greaterthantheexpectedspacing.Sinceweaccountforthe2Dto3Dprojectionand3Dfabricationerrorsinourspacingmeasurements,thisdifferenceshouldbeduetotheof3Dtargetgratingswhenthetargetispressedagainstthereaderplaten.Weperformedone-samplet-test[151]toascertainifthemeanoftheobservedspacingineachcaseisstatisticallydifferentcomparedtotheexpectedspacing.Inallbutonecase,themeanobservedspacingwasdeterminedtobedifferentthantheexpectedmeanatlevelof0.05.Thedeviationfromtheexpectedspacingisfoundtobegreaterforthecirculartargetthanthehorizontalandverticaltargets.Thiscanbeexplainedbythefactthatthecharacteristicinducedwhenthetargetispressedagainstthereaderplatenisradialaroundthe119(a)(b)(c)(d)(e)Figure4.18Evaluatingopticalreaderswitha3Dtargetgeneratedusingaverticalsinegrating.(a)Verticalsinegrating(10pixelseparationbetweenthegratings);(b)electronic3Dtargetgeneratedusing(a);(c),(d)and(e)aresampleimagesofthefabricatedtargetcapturedusingopticalreaders1,2and3,respectively.Thereisaslightdistortionapparentin(b)thatisduetothe2Dto3Dprojectionerror.centralpointofcontact.Inotherwords,targetregionsclosertothecentralpointofcontactwiththereaderplatenoutmorecomparedtosurroundingregions.TherelativeeffectofsuchaisnotasprofoundonbothhorizontalandverticalgratingscomparedtocirculargratingsbecausethecirculargratingsalignsymmetricallywiththeradialThehorizontaltargetspacingcapturedbyallthreereadersisobservedtobeclosesttotheexpectedspacingcomparedtoverticalandcirculartargets.Thismaybeduetothewaypressureisappliedonthereaderplatenwithrespecttotherelativeorientationofthegratingswhilecapturingthetargetimages.Controlledexperimentation,whereboththemagnitudeanddirectionofpressureappliedonthereaderplatenisedbeforecapturingthetargetimpressions,isrequiredtounderstandtheunderlyingcause,whichwillbeundertakeninfuturestudies.4.4.2ExperimentII:FingerprintTargetsTheten3DtargetsgeneratedbyprojectingvedifferentimagesfromNISTSD4andfabricatedusingeachofthetwoprintingmaterialsareusedtoevaluatetheimagingcapabilityofthe120(a)(b)(c)(d)(e)Figure4.19Evaluatingreaderswitha3Dtargetgeneratedusingacircularsinegrating.(a)Circularsinegrating(10pixelseparationbetweenthegratings);(b)electronic3Dtargetgen-eratedusing(a);(c),(d)and(e)aresampleimagesofthefabricatedtargetcapturedusingopticalreaders1,2and3,respectively.Thereisaslightdistortionapparentin(b)thatisduetothe2Dto3Dprojectionerror.threereaderstocapturepatterns.Center-to-centerridgespacingiscomputedontheoriginal2Dpatternthatisusedtocreateeachtargetusingthemethodin[104].AnalogoustoExperimentI,theaverageandvarianceofcenter-to-centerridgespacingvaluesiscomputedforvedifferent2Dimpressionsofeachtargetcapturedusingthethreeopticalreaders.Notethatthesamemethod[104]isusedtocomputeridgespacingonthecaptured2Dplainimpressionsofthetargets.Table4.9showsthecomputedridgespacingmeasurements.Followingarethemainobservationsbasedonthisexperiment:Todetermineiftheyarestatisticallydifferent,themeanobservedspacingineachcasewascomparedtotheexpectedspacingusingone-samplet-test[151].Inallcases,themeanobservedspacingwasfoundtobedifferentthantheexpectedspacingatsignif-icancelevelof0.05.The1000ppireadersOR2andOR3are,onaverage,betterthanthe500ppireaderinpre-servingridgespacing.ThismaybeduetolesserridgebeinginducedbythereaderplatensforthesetworeaderscomparedtoOR1.Amongstthetwo1000ppi121Table4.9Mean()andstd.deviation(˙)ofcenter-to-centerridgespacinginthetargetimagescapturedusingthethreeopticalreaders(OR).Theexpectedaverageridgespacing(inpixels)inthetargetimagesisindicatedinbrackets.TangoBlackPlusFLX980TestpatternOR1(500ppi)OR2(1000ppi)OR3(1000ppi)S0005(7.818)=8.493,˙=0.096=8.250,˙=0.048=8.099,˙=0.054S0010(8.433)=9.215,˙=0.156=9.172,˙=0.024=9.128,˙=0.053S0017(8.932)=9.893,˙=0.118=9.525,˙=0.038=9.523,˙=0.136S0083(8.621)=9.100,˙=0.191=9.111,˙=0.057=9.110,˙=0.190S0096(8.473)=8.817,˙=0.056=8.839,˙=0.075=8.670,˙=0.102FLX9840-DMTestpatternOR1(500ppi)OR2(1000ppi)OR3(1000ppi)S0005(7.818)=8.440,˙=0.129=8.288,˙=0.011=8.135,˙=0.079S0010(8.433)=9.559,˙=0.065=9.168,˙=0.052=9.077,˙=0.048S0017(8.932)=9.988,˙=0.073=9.539,˙=0.032=9.565,˙=0.055S0083(8.621)=9.302,˙=0.061=9.131,˙=0.037=9.080,˙=0.042S0096(8.473)=8.752,˙=0.102=8.772,˙=0.063=8.654,˙=0.063readers,OR3seemstoperformmarginallybetter,onanaverage,thanOR2inpreservingridgespacing.The500ppireaderOR1hasasmallplatenandisonlyabletopartiallyimagethetargets.Therefore,overallfewerspacingmeasurementsareusedforaveragespacingcom-putationsandthevariationinspacingisrelativelyhigherforthereaderOR1thanthereadersOR2andOR3.Thereisnoimpactofthefabricationmaterialontheridgespacingmeasurementsinimagescapturedusingthethreereaders.AllthreeopticalreadersusedforconductingexperimentsarePIV/AppendixFNote,however,thattheerrorsobtainedinourevaluationexperimentsarecomparativelygreaterthanpermittedgeometricerrorsforthePIVandAppendixFstandards.Thisisbecauseoftheofthepatternsonthe3Dtargetswhentheyarepressedagainstthereaderplatens.122Thecurrentstandardsdonotexplicitlyaccountforthiserror.However,itisimportanttoconsiderthiserrorintheoperationalscenariowhereuser-dependentparameters,suchasplacementandpressureappliedonthereaderplaten,directlyimpacttheimageacquiredbyareader.Anotherimportantconsiderationishowmanydifferenttargetsandimagingsamplespertargetareadequateforevaluationofreaders.Tothiseffect,itisimportantthatthesetoftargetsusedforreaderevaluationarerepresentativeofoperationaldata.Targetsgeneratedusingntpatternsofdifferenttypes(whorl,loop,andarch)andusingdifferentshapesaredesirabletotestforvariationsencounteredinthefunctionalenvironment.Itisalsoimportanttocapturemultipleimpressionsofthesetargetstomeasuretheeffectofintra-classvariations.4.5ConclusionsStructuralevaluationofreadersistypicallydoneusing2Dor3Dtargetsdesignedforcalibratingimagingdevices.Whilethesetargetsareusedforstructuralevaluationofreaders,theycannotbeusedforbehavioralevaluationofreadersinoperationalscenar-ios.Inthisresearch,wehavedesignedandfabricatedwearable3Dtargetsthatcanbeplacedonthereaderplaten,andimagedanalogoustooperationalsettingwhereauser'swillinteractwiththereader.The3Dtargetsarecreatedbyprojecting2Dcalibrationpatternsofknowncharacteristics(e.g.sinegratingsofknownspacing)ontoageneric3Dsurfacetogenerateelectronic3Dtargets.Theelectronictargetsarethenfabricatedusingastate-of-the-art3Dprinterwithmaterialsimilarinhardnessandelasticitytothehumanskin.Ourexperi-mentalresultsshowthat(i)featurespresentinthe2Dcalibrationpatternarepreservedduringthecreationofelectronic3Dtarget,(ii)featuresengravedontheelectronic3Dtargetarepreservedduringphysical3Dtargetfabrication,and(iii)intra-classvariabilitybetweenmultipleimagesofthesamephysical3Dtargetissuflysmallformatchingat0.01%FAR.Wealsoshowthat123thegenerated3Dtargetscanbeusedforbehavioralevaluationofthreedifferent(500/1000ppi)PIV/AppendixFopticalreadersintheoperationalsettings.124Chapter53DWholeHandTargets:EvaluatingSlapandContactlessReaders5.1IntroductionInthepreviouschapter,wedesignedandfabricated3Dtargetsforopticalreaderswithskin-likehardnessandelasticitythatcouldbewornonatomimicthecapturepro-cess.Weprojected2Dcalibrationpatternsofknowncharacteristics(e.g.withknownridgew,ridgespacingandminutiaeorsinegratingsoforientationandspacing)ontoa3Dsurfaceofknowndimensionstocreateelectronic3Dtargets.Theelectronic3Dtargetswerefabricatedusingastate-of-the-art3Dprinter(StratasysObjet350Connex);theprintedtargetswerethensuccessfullyusedforevaluatingopticalreaders.Fingerprintrecognitionsystemsdesignedforlarge-scaleapplications(e.g.lawenforcement[160],homelandsecurity[20]andnationalIDprograms[38])generallyrequirecapturingallten(tenprints)ofapersonduringenrolment(see,e.g.,Figure5.1).Tomaintainhighthroughput,tenprintacquisitionisusuallydonebycapturingtwoslapimpressions1ofthefouroftheleftandrighthand,followedbysimultaneouscaptureofthetwothumbprints(also1Afoursimultaneouscapture(index,middle,ringandlittlealtogether)iscalledaslapimpression.125(a)(b)Figure5.1Tenprintcapture(fourcaptureofeachofthetwohands(shownin(a)and(b))followedbysimultaneouscaptureofthetwothumbs)byaUnitedStates(US)CustomsandBorderProtection(CBP)ofataportofentryintheUS.Imagereproducedfrom[158].termedas4-4-2capture)usingaslapreader.Mostslapreadersarecontact-basedopticaldevicesthatcaptureinthefollowingmanner:(i)userplacesfourortwothumbsofhishandonaglassplaten,(ii)hisareilluminatedwithlightofawavelength,(iii)frictionridgesonthetipabsorbtheincidentlightwhilevalleysthelight,and(iv)aglassprismthelightontoaCCDorCMOSarrayforimagingtheThequalityoftheacquiredslapimpressionisafunctionofseveraluser-dependentvariables,e.g.,thepressureappliedonthereaderplatenbyeach,andtherelativeorientationofthewithrespecttoeachotherandthereaderplaten,aswellasthereaderoptics.Contact-basedslapcapture,however,inducesdistortioninthecapturedimageduetoteningoftheskinwhenthearepressedagainstthereaderplaten.Itisalsotypicallyrequiredtocleanthereaderplatenaftereveryfewcapturestopreventaccumulatedresidueduetorepeateduseofthereaderfromimpactingthequalityofthecapturedimage.Further,someusershavehygiene-relatedconcernsinusingcontact-basedreaders.Toalleviatetheseissues,contactlessslapcapturetechnologywasintroduced,andhassincegarneredattention[165].In2007,theNationalInstituteofJustice(NIJ)initiatedthefastcaptureinitiativetocreatenew126(a)(b)(c)Figure5.23Dwholehandtargetforevaluatingslapandcontactlessreaders.(a)Elec-tronic3Dhandtargetcompletewiththefourthumbandglove;theindexandmiddleengravedonthetargetareshownatfullscaleinredandblueboxes,respec-tively.(b)Fabricatedhandtargetwithtranslucentrubber-likematerialTangoPlusFLX930[31].(c)slapcapturebyacontact-basedreaderusingthefabricatedhandtargetin(b).127technologythatwillautomaticallyﬁcapturethesameimagesas10rolledinlessthan15secondsandbothpalmprintsinlessthan1minuteﬂ[7].ThegoalofNIJ'sinitiativewastoimproveimagequality,throughputandthecommercializationofcontactlessreadersforlawenforcementandhomelandsecurityagencies.Giventhatalmostallcriminal-printdatabasescontainrolledprints,anotherobjectiveofthisinitiativewastoimproveaccuracybycomparingrolledprintstorolledprintsratherthanslaptorolledprints.Sincethen,advanceshavebeenmadeinthedesignanddevelopmentofcommercial-gradecontactlessslapreaders.State-of-the-artcontactlessreadersgenerallyuseoneofthefollowingtwoopticalimagingtechniques:(i)structuredlighting,whereaedlightpatternisusedtoestimatethedif-ferenceinthedepthofridgesandvalleysforgeneratinga3Drepresentationofthe,or(ii)multi-viewimagingtechniquewheremultiplecamerasareusedtoimagethefromdifferentviewpointstoconstructa3Drepresentation.Animportantrequirementforacquir-inggoodqualityimagesusingcontactlessslapreadersistheproperpositioningoftheuser'swithrespecttotheimagingcomponentofthereader.Giventhatuser-inducedvariabilitiescanimpactthequalityofimagesacquiredbycontactlessslapreaders,itisimportanttoevaluatethereaderstoensurethatimagequalitysufforrecognition,i.e.,comparingacquiredimagestorolled(orslap)printsinthedatabase.Whileevalua-tionprocedureshavebeendevelopedtoassesscontact-basedreaders[155][156],thereisstillanimpendingneedtodevelopmethods,metricsandartifactsforevaluationofcontactlessreaders.Forthisreason,NISTstartedtheContactlessFingerprintCaptureDeviceMea-surementResearchProgramwiththeaimofﬁdevelopingmethodologiesformeasuringtheimageofcontactlesscapturedevicesﬂ[3].Here,wedesignandfabricatewholehandtargets(bothelectronicandphysical)forevaluatingcontact-basedandcontactlessslapreaders(seeFig.5.2)2.Tocreateawholehand2ThisworkwaspublishedintheproceedingsoftheInternationalConferenceoftheBiometricsSpecialInterestGroup(BIOSIG),2016[55].128(a)(b)(c)(d)Figure5.3Imagesofa3Dtargetfabricatedwithtranslucentrubber-likematerialTan-goPlusFLX930[31](shownin(a))capturedbythreedifferentPIV[155]opticalreadersusingdifferentwavelengthsoflightforcapture:(b)bluewavelength,(c)combinationofblueandredwavelengths,and(d)redwavelength.Targetsprintedwithblackcoloredrubber-likematerials(TangoBlackPlusFLX980[31]andFLX9840-DM[30])couldnotbeimagedusingthesethreereaders.target,wesegmentanelectronic3Dhandsurface3intosixdifferentparts:fourindividualthethumb,andtheremainingmiddleportionofthehandsurface4.Individualtargetsforthefourandthethumbarecreatedbyprojecting2Dcalibrationpatternsonto3Dsurfacesusingthemethoddescribedinthepreviouschapter.Themiddleportionofthehandsurfaceissyntheticallyprocessedtomakeawearableglove.Eachofthesixpartsofthewholehandareprintedusingastate-of-the-art3Dprinter(StratasysObjet350/500Connex45)withmaterialsthataresimilarinhardnessandelasticitytothehumanskinaswellasappropriateforimagingwithopticalreaders.Theprinterslices3Dpartsinto2Dhorizontallayersandprintsthemlayerbylayer.Itusesasupportmaterialtopreventthepartsbeingprintedfrombreaking.Thebulkofthesupportmaterialcanbemanuallyremovedfromtheprintedparts.However,toremoveanysupportmaterialdebrisremainingontheprintedparts,theindividualpartsaresubsequently33Dhandsurfacecaneitherbeobtaineddirectlyusinga3Dscannerorsyntheticallygenerated.Weuseasyntheti-callydesigned3Dhandsurface.4Asingle3DhandtargetmodelwithallvebecomesquitecomplexduetotheresolutionrequirementsforengravingBecausethe3Dprintersoftwaredoesnotacceptlargeelectronicmodel(>100MB),thehandtargetisdesignedandmanufacturedinparts.5ThetwoprintershaveXandYresolutionof600dpiandZresolutionof1600dpi.Thissufforprintingtargetswithmicron-scalegratings,e.g.,129cleanedwith2MNaOHsolutionandwater.Theprintedpartsarethenphysicallyassembledtocreatethewholehandtarget(seeFig.1(b)).Theprinted3Dhandtargetscanbeimagedusingthreedifferentcommercial(500/1000ppi)AppendixFcontact-basedslapreadersandaPIVcontactlessslapreader6.Weextractindividualplainprints7foreachfromslapimpressionsofthewholehandtargetscapturedusingthethreeslapreadersandshowthattheycanbesuccessfullymatchedto(i)theoriginal2Dusedtocreatethewholehandtarget,and(ii)thefrontalimagesofelectronicwholehandtargets.Wealsoconductexperimentstoevaluatethethreeslapreadersandthecontactlessslapreaderusingthegeneratedwholehandtarget.Thecontributionsoftheresearchdetailedinthischapterareasfollows:1.Generationofwholehandtargetforevaluatingcontact-basedandcontactlessslapreaders.Inthepreviouschapter,wehadgeneratedindividualtargetsforevaluatingcontact-basedopticalreadersonly.Wefurtherextendourmethodtogenerateawholehandtargetforusewithopticaldevices,e.g.slapreaders.2.Determinationofopticallycompatible3Dprintingmaterialsforfabricating3Dtargets.Pre-viously,wehadprinted3Dtargetswithmaterialssimilarinhardnessandelasticitytotheskin(TangoBlackPlusFLX980[31]andFLX9840-DM[30]4).However,thesemate-rialswereblackincolor,andcouldnotbeimagedwithopticalreadersusingcertainlightwavelengths(e.g.blue).Toremedythis,wenowusetranslucentwhitishrubberymate-rials(TangoPlusFLX930[31]andFLX9740-DM[30]4)thatprovidethedesiredhardnessandelasticityaswellasappropriateopticalpropertiesforusewithavarietyofcontact-basedopticalreaders(seeFig.5.3).Abluish-graycoloredrigidopaquemate-rial(RGD8520-DM[30])isusedtomanufacturetargetsforthecontactlessslap6Thecontactlessslapreadercapturesfour(index,middle,ringandlittlewithasinglehandmovement.7Thetermplainprintisusedtorefertotheimpressionofanindividualextractedfromtheslapimpression[23].130Figure5.4Generatinga3Dwholehandtargetfromageneric3Dhandsurfaceandasetof2Dcalibrationpatterns.reader.Thismaterialprovidesoptimumcontrastbetweenntridgesandvalleysforimagingthetargetwiththecontactlessslapreader.5.2GeneratingWholeHandTargetLetagenericelectronic3DhandsurfacebedenotedbyH.AssumethattheelectronicsurfaceHisatriangularmeshwithasetofverticesVHandasetoftrianglesTH.Eachvertex,v,inVHhas(x;y;z)coordinatescorrespondingtoitsspatiallocationinH,andeachtriangleinTHconnectsauniquesetofthreeverticesinVH.Asmentionedearlier,thewholehandtargetWisgeneratedfromHinparts.Assumethatthe2DcalibrationpatterntobeprojectedontotheithinHis131Figure5.5Cleaningandassemblingthe3Dprintedandglovestocreateawholehand3Dtarget.denotedbyIi(i=f1:::5g).ThecompleteprocesstocreatethewholehandtargetW,givenHandthesetof2DcalibrationpatternsI,isdescribedbelow(seeFig.5.4).1.Partitioning3Dhandsurface:TheelectronichandsurfaceHisdividedintosixdifferentparts:thefourSi(i=f1...4g),thethumbS5,andtheremainingmiddleportionMofthehandsurface,whichcanbedescribedasaerlessglove.Theselectortoolinopen-source3DmeshprocessingsoftwareMeshlab[15]isusedforselectingthedifferentparts.Anewmeshlayeristhencreatedforeachselectedpart.Theregistrationofthesixpartswithrespecttoeachotherremainsintactwhilepartitioningthehandsurface.Thisfacilitatesas-semblyofthefabricatedpartstocreatethewholehandtarget.Assumethatthesetofverticesandtrianglespresentineach3DsurfaceSiisdenotedbyViandTi,respectively.Also,letIi(u;v)denotethegrayscalevalueatspatialcoordinates(u;v)inthecalibrationpatternIi.1322.Preprocessing3Dsurfaces:ElectronicsurfaceSiisalignedsuchthatthelengthisalongthey-axisinSi.ThesurfaceSiisre-meshedbysamplingverticesfromthesetVibasedonthecurvatureofSi[167].Surfacere-meshingreducesthedensityofSi,therefore,SiissubdividedusingLoop'smethod[136]toensuresufduringprojectionofthe2DcalibrationpatternIi.SiisdisplacedoutwardsalongthedirectionofthesurfacenormalscomputedateachvertexvtocreateanoutersurfaceSOi.Note,however,thattheoriginalelectronicsurfaceSiisretained.ThefrontportionSOFiandtherearportionSORiofSOiareseparatedasonlythefrontportionSOFiisusedforprojection.3.Preprocessing2Dcalibrationpatterns:IfthepatternIibeingprojectedisa2Dimage,skeletonISioftheimageIiiscreated.TheridgewidthoftheskeletonISiisincreasedusingmorphologicaloperations,andtheimageissmoothedusingaGaussianbeforeprojectingitontothefrontalsurfaceSOFi.ThispreprocessingstepisimportanttoensurethatridgesandvalleyspresentinIiareengravedsmoothlyontoSOFi.Notethatpreprocessingisnotneededifanyother2Dcalibrationpattern(e.g.sinegrating)isbeingprojected.4.Mapping2Dcalibrationpatternsto3Dsurfaces:ThefrontportionSOFioftheoutersurfaceSOiisprojectedto2DusingtheISOMAPalgorithm[177].RotationandarecorrectedusingcorrespondingcontrolpointsbetweenfrontportionSOFiandthe2Dpro-jectionofSOFi.TranslationcorrectionisdoneusingthereferencecoordinatescomputedfromIi.ThefrontportionSOFiisfurthersubdivideddependingontheresolutionofIitoensuresufnt(highsimilarityscoresforaFARof0.01%)ofmappingIi.There-after,themappingbetweenthevertexlocations(x;y;z)onthefrontportionSOFiandthegrayscalevaluesatlocations(u;v)inIiisascertained.5.Engraving2Dcalibrationpatternson3Dsurfaces:Ridgesandvalleysareen-gravedonSOFibydisplacingtheverticesonthefrontportionSOFialongthesurfacenormalsaccordingtothetexturevaluesatthemapped(u;v)locationsinIi.133(a)(b)Figure5.6Sample3Dtargetsfabricatedfor(a)contact-basedreaders(usingtranslu-centrubber-likematerialFLX9740-DM[30])and(b)contactlessreaders(usingrigidopaquema-terialRGD8520-DM[30]).6.Postprocessing3Dsurfaces:ThefrontandrearportionsoftheoutersurfaceSOiarecombined.TheoriginalsurfaceSiismadeasdenseastheoutersurfaceSOiandthenthetwosurfacesarestitchedtogethertocreatethe3DtargetAiinelectronic(virtual)form.7.Creatingglove:ThemiddleportionMofthehandisdisplacedoutwardalongthesurfacenormalscomputedateachvertexvtocreateanouterreplicaMOofM.MandMOarethenstitchedtogethertocreateawearablegloveMW.Thisshesthecreationofthesixpartsofthewholehandtargetinelectronicform.8.3DPrinting:ThethumbandfourtargetsAiandthegloveMWarephysicallyfabri-catedusinga3Dprinter(StratasysObjet350/500Connex).Twodifferentprintingmaterials,TangoPlusFLX930[31]andFLX9740-DM[30],areusedtofabricatethethumbandfourtargetsAiaswellasthegloveMWforcontact-basedslapreaders(see,e.g.,Figure5.6(a)).Thesematerialsaresemi-translucentwhitishrubber-likematerialswithsimilarhardnessandelasticityashumanskin(seeTable5.1).Unliketheblackrubber-likematerialsearlier,thesematerialsareopticallysuitableforimagingwithavarietyofcontact-134Table5.1Comparisonofthemechanicalpropertiesofthethreeprintingmaterialsusedfor3Dwholehandtargetfabricationwiththehumanskin.TangoPlusFLX930andFLX9740-DMarerubber-likematerialssimilarinmechanicalpropertiestothehumanskinandaresuitableforusewithcontact-basedslapreaders.RGD8520-DMisarigidopaquematerialthatprovidesoptimumridge-valleycontrastforusewiththecontactlessslapreader.PropertyHumanSkin[88][89]TangoPlusFLX930[31]FLX9740-DM[30]RGD8520-DM[30]ShoreAhardness20-4126-2835-40N.A.TensileStrength(MPa)5-300.8-1.51.3-1.840-60ElongationatBreak(%)35-115170-220110-13015-25basedopticalreaders.Abluish-grayrigidopaquematerial,RGD8520-DM[30],isusedtomanufacturetheindividualthumbandtargetsAiforthecontactlessslapreader(seeTable5.1andFigure5.6(b)).Thismaterialprovidesoptimumcontrastbetweenridgesandvalleysforimagingwiththecontactlessslapreader.ThewearablegloveMWforthecontactlessslapreaderisfabricatedwithTangoPlusFLX930.9.ChemicalCleaning:Themajorityoftheprintersupportmaterialisremovedmanuallyfromthe3Dprintedparts.Afterthis,the3Dprintedpartsaresoakedin2MNaOHsolutionfor3hours,andthenrinsedwithwatertoremovetheprintersupportmaterialresidue(Figure5.5).10.PartAssembling:ThecleanedphysicalpartsAi(i=1:::5)andMWareassembledtogetherwithsupergluetogenerateawearablewholehandtargetW.5.3Fidelityof3DWholeHandTargetGenerationToascertainthe8ofthewholehandtargetcreationprocess,weassesshowwellthefea-turespresentinthe2Dcalibrationpatternsarereplicatedontheelectronic3Dhandtargetafterthe2Dto3Dprojectionofthepatterns,andonthephysical3Dhandtargetpost3Dprintingand8Fidelitymeansthedegreeofexactnesswithwhichthe2Dcalibrationpatternsarereproducedontheelectronicandphysical3Dhandtarget.135cleaning.WecreatearighthandtargetusingvedifferentrolledfromNISTSD4[19].Twosamplesofthewholehandtargetarefabricatedwiththetwoprintingmaterials,TangoPlusFLX930andFLX9740-DM.FivedifferentslapimpressionsofthehandtargetarecapturedusingthreedifferentAppendixFcontact-basedslapreaders,SR1,SR2andSR39(see,e.g.,Fig.5.7).SR1andSR3are500ppireaderswhereasSR2isa1000ppireader.Comparisonsbetween(i)2DfromNISTSD4andthefrontalimagesofcorrespondingengravedontheelectronic3Dhandtarget,(ii)frontalimagesofengravedontheelectronic3Dhandtargettocorrespondingplainprintsextractedfromslapimpressionsofthephysical3Dhandtarget,and(iii)the2Dusedtogeneratethehandtargettocorrespondingplainprintsextractedfromslapimpressionsofthephysical3Dhandtarget,aremadetoascertaintheofthe3Dwholehandtargetgenerationprocess.Furthermore,plainprintsextractedfromvedifferentslapimpressionsofthe3Dhandtargetarecomparedwitheachothertodeterminetheconsistencybe-tweendifferentimpressionsofthetarget(intra-impressionvariability).V6.3SDK[146]isusedforconductingallcomparisonexperiments.Allslapimpressionsareupsampledbyafac-torof1.2usingbicubicinterpolationtoaccountforreductioninridgespacingdueto2Dto3Dprojectionand3Dprintingbeforeconductingmatchingexperiments(seeSections4.3.1and4.3.1,respectively,for2Dto3Dprojectionand3Dprintingfabricationerrormeasurements).5.3.1Replicationof2Dcalibrationpatternfeaturesonelectronic3DhandtargetFrontalimagesofindividualengravedontheelectronic3DhandtargetarecapturedusingMeshlab[15].Theyarerescaledmanuallytoapproximatelythesamescaleasthe2DgerprintsfromNISTSD4.Eachindividualimagefromtheelectronic3Dtargetiscomparedtothecorresponding2DfromNISTSD4.Table5.2showsthesimilarityscoresobtainedforthisexperiment.Allsimilarityscoresareabovethev9Vendornamesarenotprovidedtomaintaintheiranonymityinthisevaluation.136Figure5.7Sampleslapimpressionofthe3Dwholehandtargetcapturedusingacontact-basedslapreader.Table5.2Similarityscoresbetweenfrontalimages(2D)ofindividualengravedontheelectronic3DhandtargetcapturedinMeshlabandthecorresponding2DimagesfromNISTSD4usedfortargetgeneration.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.FingerprintS0005(index)S0043(middle)S0083(ring)S0096(little)S0044(thumb)score203150399183249thresholdof33@FAR=0.01%forNISTSD4.Thisdemonstratesthatthefeaturespresentinthe2Dcalibrationpatternsarereplicatedwithhighontheelectronic3Dhandtarget.5.3.2Replicationofelectronic3Dhandtargetfeaturesonphysical3DhandtargetIndividualplainprintsaremanuallyextracted(forconvenience)fromtheslapimpressionscapturedusingthethreecontact-basedslapreaders.Eachplainprintiscomparedtothefrontalimageofthecorrespondingengravedontheelectronic3Dhandtarget.Allsimilarityscoresarewell137Table5.3Similarityscoresbetweenthefrontalimages(2D)oftheindividualengravedontheelectronic3Dhandtargetandthecorrespondingplainprintsextractedfromaslapimageofthephysical3Dhandtargetscapturedbyeachofthethreecontact-basedslapreaders(SR1,SR2andSR3).Physicaltargetswerefabricatedwithtwodifferentmaterials(TangoPlusFLX930andFLX9740-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.TangoPlusFLX930FingerprintSR1(500ppi)SR2(1000ppi)SR3(500ppi)S0005(index)8768168S0043(middle)6671122S0083(ring)327171158S0096(little)173141108S0044(thumb)656993FLX9740-DMFingerprintSR1(500ppi)SR2(1000ppi)SR3(500ppi)S0005(index)14715978S0043(middle)48201107S0083(ring)362441222S0096(little)140156129S0044(thumb)636250abovethevthresholdof33@FAR=0.01%forNISTSD4(seeTable4.5).Thisshowsthatfeaturesengravedontheelectronic3Dtargetarepreservedpost3Dprintingandcleaning.5.3.3Replicationof2Dcalibrationpatternfeaturesonphysical3DhandtargetPlainprintsextractedfromtheslapimpressionsofthephysical3Dhandtarget,capturedusingthethreecontact-basedslapreaders,arecomparedtocorresponding2DfromNISTSD4.Table5.4showsthesimilarityscoresobtainedforthisexperiment.Becauseallsimilarityscoresarewellabovethevthresholdscoreof33@FAR=0.01%forNISTSD4,itcanbeinferred138Table5.4Similarityscoresbetweentheplainprintsextractedfromslapimpressionscapturedbythethreecontact-basedreaders(SR1,SR2andSR3)ofthephysical3Dhandtargetsandthecor-respondingfromNISTSD4usedintheirgeneration.Physicaltargetswerefabricatedwithtwodifferentmaterials(TangoPlusFLX930andFLX9740-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.TangoPlusFLX930FingerprintSR1(500ppi)SR2(1000ppi)SR3(500ppi)S0005(index)549141321S0043(middle)213161315S0083(ring)441374411S0096(little)308392423S0044(thumb)209422345FLX9740-DMFingerprintSR1(500ppi)SR2(1000ppi)SR3(500ppi)S0005(index)719570510S0043(middle)221579357S0083(ring)426596303S0096(little)419510366S0044(thumb)119404371thatthe2Dcalibrationpatternfeaturesarereplicatedwithhighonthephysical3Dhandtarget.5.3.4Consistencybetweendifferentimpressionsofthephysical3DhandtargetIndividualplainprintsextractedfromdifferentslapimpressionsofthesamephysical3Dhandtargetarecomparedwitheachothertomeasuretheirintra-classsimilarity.SimilarityscoresforthisexperimentarereportedinTable5.5.Allsimilarityscoresareabovethev139Table5.5Rangeofsimilarityscoresforpairwisecomparisonsbetweenplainprintsofthesameextractedfromvedifferentslapprintscapturedbythethreecontact-basedslapreaders(SR1,SR2andSR3)ofthesame3Dwholehandtarget.Resultsareshownfortwophysicalhandtargetsfabricatedwiththetwoprintingmaterials(TangoPlusFLX930andFLX9740-DM).V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.TangoPlusFLX930FingerprintSR1(500ppi)SR2(1000ppi)SR3(500ppi)S0005(index)839-1373603-1193797-1434S0043(middle)551-930501-909581-1206S0083(ring)756-1127843-1290990-1272S0096(little)644-1071344-1133957-1413S0044(thumb)800-1061743-1263989-1160FLX9740-DMFingerprintSR1(500ppi)SR2(1000ppi)SR3(500ppi)S0005(index)980-1359735-1271779-1229S0043(middle)579-1079855-1265539-1190S0083(ring)837-1254924-1467897-1503S0096(little)639-1043710-1059630-1221S0044(thumb)723-1178845-1796822-1469thresholdscoreof33@FAR=0.01%indicatingthatmultipleslapimpressionsofthesame3Dhandtargetarehighlyconsistent.5.4EvaluatingContact-basedSlapFingerprintReadersCenter-to-centerridgespacingmeasurementsarecomputed(usingthemethodproposedin[104])intheplainprintsextractedfromvedifferentslapimpressionscapturedusingthethreecontact-basedslapreaders.Wecomparethesemeasurementsagainsttheexpectedaveragecenter-to-centerridgespacinginthecorresponding2Dusedduringtargetcreation.Theexpectedridgespacingiscomputedtakingintoconsiderationthe2Dto3Dprojectionerror(5.8%)andthe3D140Table5.6Mean()andstd.deviation(˙)ofcenter-to-centerridgespacings(inpixels)intheplainprintsextractedfromvedifferentslapimagesofthe3Dwholehandtargetscapturedusingthethreecontact-basedslapreaders(SR1,SR2andSR3).Expectedaverageridgespacing(inpixels)foreach2DfromNISTSD4isshowninbrackets.Thespacingmeasurementstakeintoconsiderationthereductioninspacingdueto2Dto3Dprojectionand3Dprintingfabricationerrors.TangoPlusFLX930FingerprintSR1(500ppi)SR2(1000ppi)SR3(500ppi)index(7.82)=8.06,˙=0.10=7.87,˙=0.06=7.90,˙=0.05middle(8.33)=8.64,˙=0.03=8.57,˙=0.06=8.35,˙=0.08ring(8.62)=8.58,˙=0.05=8.65,˙=0.10=8.65,˙=0.07little(8.47)=8.49,˙=0.07=8.49,˙=0.10=8.49,˙=0.04thumb(7.67)=7.67,˙=0.04=7.66,˙=0.06=7.67,˙=0.06FLX9740-DMFingerprintSR1(500ppi)SR2(1000ppi)SR3(500ppi)index(7.82)=7.87,˙=0.08=7.80,˙=0.08=8.00,˙=0.08middle(8.33)=8.61,˙=0.09=8.64,˙=0.05=8.36,˙=0.05ring(8.62)=8.63,˙=0.14=8.66,˙=0.03=8.64,˙=0.10little(8.47)=8.52,˙=0.10=8.51,˙=0.14=8.54,˙=0.08thumb(7.67)=7.66,˙=0.07=7.66,˙=0.03=7.67,˙=0.05printingfabricationerror(11.42%)thatwereestimatedinSections4.3.1and4.3.2,respectively.Table5.6liststhemeasurementstakenfromslapimpressionsofthetwohandtargets.Followingaresomeobservationsbasedonthisexperiment:Theestimatedridgespacingsinslapimpressionsofthehandtargetscapturedusingthethreecontact-basedslapreaders,SR1,SR2andSR3are,onaverage,within0.08pixelsofeachother.Inotherwords,allthreeslapreadersSR1,SR2andSR3performequallywellinpreservingridgespacing.Theestimatedridgespacingsintheplainprintsofindex,middle,ringandlittleare,onaverage,marginallygreaterthantheexpectedridgespacing.Althoughtheincreaseinridgespacingsisnotasasthatreportedpreviouslyfor3Dtargets,itis141consistentwithourobservation.ThisincreaseinridgespacingisduetotheoftheskinbecauseofthepressureappliedonthereaderplatenwhilecapturingForthethumb,however,thiseffectisnotobservedtobeasprofoundcomparedtotheotheranddoesnotseemtoimpacttheridgespacingmeasurements.Onepossiblereasoncouldbethedifferenceinpressureonthereaderplatenforeachgerwhilecapturingslapimpressions.Further,usingtheone-samplet-test[151],forallexceptthemiddle,theestimatedridgespacingvaluesarestatisticallysimilartotheexpectedvaluesatlevelof0.05.Thisisincontrasttoourobservationinchapter4foropticalreaderswhereestimatedvalueswerestatisticallydifferentcomparedtoexpectedvalues.Abetterunderstandingoftheunderlyingcausewouldrequirecontrolledexperimentationwhereknowncontactpressureisappliedbyeachduringcapture.Thisisatopicoffutureresearch.Choiceoffabricationmaterialofthehandtargetdoesnotseemtoimpacttheridgespacingmeasurementsinimagescapturedusingthethreeslapreaders.5.5EvaluatingContactlessSlapFingerprintReaderThecontactlessslapreaderusedinourexperimentisaPIV500ppireaderthatcapturesa512512imageofeachfromasinglewaveofthehand.Therefore,forevaluatingthecontactlessslapreader,wegeneratedarightwholehandtargetbyprojectingcircularsinegratingsofedridgespacing(10pixels)suchthattheycovertheentire10.TherigidopaquematerialRGD8520-DMwasusedtofabricatethethumbandfourtargetswhereasrubber-likexiblematerialTangoPlusFLX930wasusedtomanufacturetheglovesothatitiseasytowear.Fivedifferentslapimpressionsofthewholehandtargetwerecapturedusingthecontactlessslapreader(see,e.g.,Fig.5.8).Analogoustotheearlierexperiment,center-to-centerridgespacingmeasurementsarecomputedintheplainprintsextractedfromvedifferent10Wearedesigningamethodtodoasimilarprojectionfor142(a)(b)Figure5.8Circularsinegrating(ridgespacing=10pixels)usedtogeneratethe3Dwholehandtarget(shownin(a))andtheslapimpressionofthecorrespondinghandtargetcapturedusingthecontactlessslapreader(shownin(b)).Thecircularsinegratingappearstoexhibitthemoireeffect[16].143Table5.7Mean()andstd.deviation(˙)ofcenter-to-centerridgespacings(inpixels)intheplainprintsextractedfromvedifferentslapimagesofthecirculargratingwholehandtargetcapturedusingthecontactlessslapreader(CR).Expectedaverageridgespacing(inpixels)ofthecirculargratingengravedonthehandtargetis8.28.Thespacingmeasurementstakeintoconsiderationthereductioninspacingdueto2Dto3Dprojectionand3Dprintingfabricationerrors.RGD8520-DMFingerprintCR(500ppi)index=8.12,˙=0.16middle=8.35,˙=0.10ring=8.28,˙=0.15little=8.03,˙=0.15thumb=7.67,˙=0.08slapimpressionscapturedusingthecontactlessslapreader.Wecomparethesemeasurementsagainsttheexpectedaveragecenter-to-centerridgespacinginthecirculargratingsusedduringtargetcreation.Theexpectedridgespacingtakesintoconsiderationthe2Dto3Dprojectionerror(5.8%)andthe3Dprintingfabricationerror(11.42%).Table5.7liststhemeasurementstakenfromcontactlessslapimpressionsofthehandtarget.Followingaresomeobservationsbasedonthisexperiment:Theaveragedeviationinestimatedcenter-to-centerridgespacingsinslapimpressionsofthecirculargratinghandtargetisabout0.25pixelsfromtheexpectedridgespacing.Us-ingonesamplet-test[151],theestimatedspacingsarestatisticallydifferentthanexpectedspacingsforallbutone.Furtheranalysisisneededtointerpretthismeasurementandunderstand,inmoredetails,theeffectsoftheunconstrainednatureofcontactlesscapture,thesizeofthecapturedareaaswellasthenatureofthematerialusedtocreatethetarget.Theestimatedridgespacingsintheplainprintsofindex,middle,ringandlittleare,onaverage,closertotheexpectedridgespacingcomparedtothethumb.Thismaybebecausethefourarecapturedtogetherinaslapimpressionwhereasthumbis144capturedindividuallyasaseparateimpression,andtheuserdynamicsinvolvedinthetwocaptureprocesses(e.g.alignmentwithrespecttotheopticalcapture,relativemovement)arequitedifferent.Controlledexperimentationwheretherelativepositioningoftheuser'swithrespecttothereaderisedatthetimeofcontactlessslapcaptureisrequiredtoinvestigatethisfurther.Itisatopicoffutureresearch.5.6ConclusionsWehavepresentedamethodtodesignandfabricatewholehand3Dtargetsforevaluatingmulti-capturedevices,e.g.,contact-basedandcontactlessslapreaders.2Dcalibrationpatternsofknowncharacteristics(e.g.ofknownridgewandridgespacing,sinegratingsofknownorientationandcenter-to-centerspacing)areprojectedontoageneric3Dhandmodeltocreateanelectronic3Dhandtarget.Physical3Dhandtargetisfabricatedfromtheelectronictargetusingastate-of-the-art3Dprinter.Material(s)similarinhardnessandelasticitytothehumanskinaswellasopticallysuitableforusewithavarietyofreadersareusedfor3Dhandtargetfabrication.Ourexperimentalresultsshowthatfeaturespresentinthe2Dcalibrationpatternsarereplicatedwithhighbothontheelectronicandphysical3Dhandtargetduringthe3Dhandtargetgenerationprocess.WealsoconductexperimentstoevaluatethreeAppendixFslapreadersandaPIVcontactlessslapreaderusingthefabricated3Dhandtargets.Tothebestofourknowledge,thisisthestudythatdemonstratestheutility11ofthewearable3Dhandtargetsforevaluationofslapreaders,bothcontactandcontactless.11WearablespoofswereusedtospoofabasedsecuritysystemintheCBScrimethrillerTVshowPersonofInterest'sSeason3episode14titledProvenance(https://www.youtube.com/watch?v=mzfDG2wmqc4).Ourresearchhasshownthatthisisnowpossibleinreality!145Chapter6Generating3DConductiveFingerprintTargets6.1IntroductionThereare,atpresent,over2billionsmartphoneusersworldwide,anditisestimatedthattheuserbasewillgrowtoaround2.66billionby2019,i.e.,withinthenextfewyears,everythirdpersonintheworldwillbeusingasmartphone[28].Followingtheintroductionofsmartphoneunlockandpaymenttechnologybymajorvendors(e.g.,Apple,SamsungandGoogle),alargenumberoftheseusersnowusesmartphonesequippedwithreaders.Oneoftheprimaryreasonsoftheincreasinguseofauthenticationonsmartphonesistherelativeeaseofuseandhighersecurityoftechnologycomparedtotraditionalauthenticationmechanismssuchaspasscodes.In2016,about29%ofthesmartphonesthatwereshippedhadareader,andthisnumberisexpectedtomorethandoubleinthenexttwoyears.Itisestimatedthatin2018,twooutofeverythreeshippedsmartphoneswillhaveareader[29].Further,thenumberofusersusingNearFieldCommunication(NFC)technologyformobilephonepaymentsisexpectedtotriplefromabout54millionin2016to166millionin2018[27].146(a)(b)Figure6.13Dtargetsforevaluatingcapacitivereaders.(a)Sample(er)target,and(b)animpressionofthein(a)capturedusingaPIV500ppicapacitivereader.Theembeddedreadersonmostsmartphones,includingthosefrommajorvendors(AppleiPhone[36],SamsungGalaxyS[25]andGoogleNexus1),usecapacitivesensing.Capac-itivereaderstypicallyconsistofasiliconplate,whereeachelementoftheplateisamini-sensorinitselfthatsensesthecapacitancedifferencebetweenridgesandvalleys.Typically,theyhaveasmallformfactor,withthesensingareabetween5-8cm2,tokeepthereadercostlow.Thesmallformfactorcoupledwithlowcostmakescapacitivereaderssuitableforembeddingonmobilede-vicesincludingsmartphones,laptopsandtablets.Inaddition,capacitivereadershavealsobeenembeddedinstandaloneterminalsforaccesscontrol.Inchapter4,wedesignedandfabricated3Dtargetsforoperationalevaluationofsinglopticalreaders.Weprojected2Dcalibrationpatternswithknownfeatures,e.g.,sinegratingsgeneratedwithorientationandspacing,syntheticwithknowngerprinttype,ridgew,ridgespacingandminutiaepoints,ontoageneric3Dsurfacetocreateelectronic3Dtargets.Astate-of-the-art3Dprinterwasusedforphysicalfabricationofthe1www.google.com/nexus/147Table6.1Mechanicalandelectricalpropertiesofhumanskin.PropertyHumanSkinHardness[88][89]Properpresentationonthereaderplaten20-41(ShoreA)TensileStrength[88]Durabilityforrepeatableoperationalevaluation5-30MPaElongationatBreak[88]Pertinentdistortiononcontactwiththereaderplaten35-115%ElectricalResistance[72][127]ResistancetowofelectricchargeŸ1-2Mm3Dtargetswithmaterialssimilarinhardnessandelasticitytothehumanskin.Weshowedthatthe3Dtargetsynthesisandfabricationprocesswasabletoreproducecalibrationpatternswithhighonbothelectronicandphysical3Dtargets.Wealsoperformedevaluationofthreedifferentopticalreadersusingthefabricatedtargets.Subsequently,inchapter5,weextendedthetargetgenerationmethodtocreatewholehand3Dtargetsforevaluatingcontact-basedandcontactlessslapreaders.Wesegmented3Dsurfacespertainingtoeachofthefourandthethumbfroma3Dhandsurfaceandprojectedcalibrationpatternsontoeachsurfacetogenerateelectronic3Dwholehandtarget.Weusedahigh-resolution3Dprintertomanufacturephysical3Dhandtargetwithmaterialsthatweresimilarinmechanicalpropertiestothehumanskinaswellascompatibleforimagingwithavarietyofopticalreaders.Furthermore,weusedthegeneratedtargetstoevaluatethreecontact-basedslapreadersandonecontactlessslapreader.Althoughsuitableforusewithopticalreaders,the3Dtargetsdesignedearlierwerenotcom-patiblewithcapacitivereadersasthefabricationmaterialsusedtoprintthe3Dtargets(UV-curablerubber-likepolymericmaterials:TangoBlackPlusFLX980,FLX9840-DM,TangoPlusFLX930andFLX9740-DM)werenon-conductive.Thisisbecausestate-of-the-arthigh-resolution3Dprintersonlysupportprintingwithlimitedrubber-likepolymermaterialsandthesupportedmate-rialsareelectricinsulators.148Giventhatalargenumberofcapacitivereadersarebeingusedinconsumerandaccesscontrolapplications,e.g.,mobilephoneunlockandpayments(ApplePay[1],SamsungPay[26]),inthischapterweaddresstheaforementionedlimitationbydesigningandfabricatingtargetssuitableforimagingwithcapacitivereaders(seeFigure6.1;alsoseeTable6.1forthemechanicalandelectricalpropertiesofthehumanskin).Weusethemethodproposedinchapter4tocreate3Dtargetswithmaterialssimilarinhardnessandelasticitytothehumanskin.Wethenuseasputterdepositiontechniquetocoatthesurfaceof3Dtargetswiththinlayersofmaterialswithconductiveproperties(titanium(Ti)+gold(Au)).WerefertotheTi-Aucoated3Dtargetsasers.Weshowthatthesputterdepositionof30nmTi+300nmAudoesnotimpactthefeaturesetchedonthe3Dtargets.Further,weshowthatthecoated3Dtargetscanbeimagedwithtwodifferenttypesofcapacitivereaders:smallareareaders(Ÿ0.5cm1.2cm)designedforsmartphones,andrelativelylargerarea(Ÿ1.3cm1.8cm)readersdesignedforaccesscontrolapplications.Insummary,thecontributionsofthischapterareasfollows:Methodtocoat3Dprintedtargetswithathinlayerofconductivematerials(Ÿ300nm)toimpartappropriateelectricalconductivityforthetargetstobesensedbycapacitivereaders.Demonstratebothqualitativelyandquantitativelythatthecoatingprocessdoesnotimpactfeaturesextractedfromthe3Dprintedtargets.Showtheutilityofthecoatedtargetsforevaluatingstandalonecapacitiveread-ers,aswellasreadersembeddedinaccesscontrolterminalsandsmartphones.Investigatethepotentialuseofconductive3Dspoofstoevaluatespoofvulnerabilityofca-pacitivereaders.149Figure6.2Mainstepsinvolvedincreatingagivena2Dcalibrationpatternanda3Dsurface.6.2SputterCoating3DTargetsWegenerate3Dtargetsusingthemethoddetailedinchapter4.2Dcalibrationpatternswithknownfeatures(e.g.withknownminutiaelocations,sinegratingswithorientationandspacing)aremappedontoanelectronic3Dsurfacetocreateelectronic3Dtargets.Wearablephysical3DtargetsarefabricatedusingtheStratasysObjetConnex350withmaterialssimilarinhardnessandelasticitytothehumanskin(TangoBlackPlusFLX980[31]andFLX9840-DM[30]).Bulkoftheprintersupportmaterialismanuallyremovedfromtheprinted3Dtargets.Thetargetsarethendippedina2MsolutionofNaOHforapproximately3hrs.andrinsedwithwaterforcompleteremovalofthesupportmaterialresidue.Giventhecleaned3Dtargets,theDCsputterdepositiontechnique[175]isusedtocoattheirsurfacewiththinlayersofconductivematerials.Figure6.2illustratesthemainstepsinvolvedingeneratingagivena2Dcalibrationpatternanda3Dsurface.Sputterdeposi-tionisoneofthemostpopulartechniquesfordepositingthinconductiveoninsulatorsandsemi-conductors[171].Itiswidelyusedinthesemi-conductorindustrytodepositthinonintegratedcircuitcomponents,foranti-glarecoatingsonglassinopticalapplications,andtode-positthinmetalliclayersonCDs,DVDsandsolarcells[171].Differenttypesofsputterdepositionmethods,e.g.,ionbeamsputtering,DCsputtering,RFsputtering,canbeuseddependingonthecharacteristicsofthesubstrateandthetargetmaterialtobedepositedandthedesiredcoatingthick-150(a)(b)Figure6.3DCsputterdepositiontocoatthe3Dtargetswiththinlayersofconductivematerials.(a)representationoftheDCsputteringprocess(imagereproducedfrom[175]),and(b)theDentonVacuumDCsputteringsystem[4]usedforDCsputtering.Titanium(Ti)andGold(Au)ionsfromthecathodetargetaredepositedontheanodesubstrateusingArgon(Ar)astheprocessgas.151ness.Here,weuseDCsputteringbecausethismethodisbothsuitableandefforapplyingconductivematerialcoatingson3Dprintedtargets.6.2.1DCSputteringProcessFigure6.3(a)illustratestheDCsputteringprocess[175].Thesputteringchamberisvacuumedtoevacuatepotentialcontaminants,e.g,watervapourandatmosphericgases,thatcouldinterferewiththesputteringprocess.Thesputteringtargetmadeofthematerialtobedeposited(e.g.silver(Au),Copper(Cu),orGold(Au))isplacedatthecathode,andthesubstrateonwhichthethinlayerhastobedepositedisplacedattheanode.Aprocessgas(typicallyArgon(Ar))isthenaddedtothevacuumedchamberatapressure,typicallybetween1-100mTorr.Anegativepotentialbiasthatissufforelectronemissionfromthesputteringtarget(generallybetween500-5000VDC)isappliedtothecathode.Electronsemittedfromthetargetduetothisnegativebiasstrikethemoleculesoftheprocessgasintheneighborhoodofthecathode(sputteringtarget)andproducepositivelychargedprocessgasions.Thegeneratedpositivegasionstraveltowardsthecathodeduetothenegativepotentialbias.Whentheprocessgasionscollidewiththecathode(sputteringtarget),theirkineticenergyistransferredtothetargetresultingintheemissionofsputteringmaterialatoms.Theejectedmaterialatomsmovetowardstheanodewheretheycondensetoformathinlayeronthesubstratesurface.6.2.2ChoiceofSputteringMaterialsWeinitiallysputtercoatedothermetals,e.g.,silver(Ag),copper(Cu),andchromium(Cr))onthe3DtargetsposttheTicoating(see,e.g,Figure6.4(a)and(b)).Althoughcoatingsofthesemetalswerefoundtobesufconductiveforthe3Dtargetstoregisteroncapacitivereaders,themetalcoatingswouldreactwithatmosphericgasesandwatervapoursovertimetoformcompoundswithlowelectricalconductivity(e.g.coppercarbonate(CuCO3)andchromiumoxide(Cr2O3))onthe3Dtargetsurface.Thiswouldrenderthe3Dtargetsunusablewithcapacitivereaders.152(a)(b)(c)Figure6.43Dtargetscoatedwith(a)and(b)athinlayer(300nm)ofsilver(Ag)andcopper(Cu),respectivelyoverathinlayer(30nm)oftitanium(Ti),and(c)100nmoftin(Sn)dopedindiumoxide(ITO).Thetargetsin(a)and(b)wereprintedwithTangoBlackPlusFLX980[31]andthetargetin(c)wasprintedwithTangoPlusFLX930[31].Targetscoatedwithotherconductivetransparentoxidesarenotshownherebecausetheyarevisuallysimilarto(c).Wealsoattemptedtocoattransparentconductiveoxides,e.g,Tin(Sn)dopedIndiumOxide(ITO)[62],Zinc(Zn)andAldopedIndiumOxide(IZAO)[73],andSn,ZnandAldopedIndiumOxide(IZATO)[106]on3DprintedtargetsusingDCsputtering(see,e.g,Figure6.4(c)).Thepri-maryadvantageofusingtransparentconductiveoxidecoatingsovermetalliccoatingsistheirhightransparencywhichpreservestheunderlyingopticalproperties[58]ofthe3Dtargets.However,thewearandtear(abrasionresistance)ofconductiveoxidecoatingswasfoundtobeinadequateforrepeatableevaluationofcapacitivereadersovertime.Inourtests,thecoatingswerefoundtowearoutaftertakingabout5-10impressionsofthecoatedtargetswithcapacitivereaders.Basedonthisexperimentalobservation,weformulatedthefollowingtwohypothesisforlowabrasionresistanceofconductiveoxidecoatings:(i)surfaceofthe3Dprintedtargetsisnotreceptiveofconductiveoxidecoatingsandrequiressomepre-treatmentbeforesputtering,or(ii)thecoatingscannotbecured,e.g.,usinghightemperatureannealingpostDCsputteringbecausethe3Dprintingmaterialsaresensitivetohightemperatures(60C).Toincreasethereceptivityofthe3Dtargetsurfaceto153Table6.2Qualitativecomparisonofdifferentthincoatingsappliedon3DtargetsusingDCsputtering.CoatingThicknessConductivityStabilityinairAbrasionResistanceTi+Au30nm+300nmAdequateHigh(doesn'treact)ModerateTi+Ag30nm+300nmAdequateLow(˘1week)ModerateTi+Cu30nm+300nmAdequateLow(˘2weeks)ModerateCr300nmAdequateLow(˘1-2days)ModerateITO100nmAdequateHigh(doesn'treact)LowIZAO300nmAdequateHigh(doesn'treact)LowIZATO300nmAdequateHigh(doesn'treact)Lowconductiveoxidecoatings,weattemptedtopre-treatthe3DtargetsurfacewithhighenergyplasmaforashortdurationoftimebeforesputteringITO.However,the3Dprintingmaterialswerefoundtobehighlysensitivetothispre-treatmentandthehighenergyplasmaimpactedthecalibrationpatternsetchedonthe3Dsurface.Furtherinvestigationisrequiredtoidentifytheexactunderlyingcausewhichisatopicoffutureresearch.Wealsoexperimentedwiththeapplicationofathinlayer(50nm)ofpoly(3,4-ethylenedioxythiophene):poly(styrenesulfonate)(PEDOT:PSS)colloidalsolutionona3Dtargetusingspincoating(at2000rpm).However,theconductivityofthesamplewasfoundtobeinad-equateforregistrationoncapacitivereaders.ExplorationofmethodstoimprovetheconductivityofPEDOT:PSS(e.g.,[176],[48])beforeitscoatingon3Dtargetsisatopicoffutureresearch.Basedontheresultsofourinitialexperimentation,Auwaschosenforcoating3DtargetsbecauseitisaninertmetalthatdoesnotreactwithatmosphericgasesandAucoatinghasrelativelyhighabrasionresistance.Table6.2liststheadvantagesanddisadvantagesofapplyingdifferentconductivecoatingson3DtargetsusingDCsputtering.6.2.3SputteringTi+AuTheDentonVacuumDesktopPro[4]whichisacompact,highvacuumsputteringsystemisusedforDCsputtering(seeFigure6.3(b)).Thesputteringsystemhasarotaryplatformwherethe154(a)(b)(c)Figure6.53Dmountfabricatedtoholda3Dtargetforstableplacementonthesputteringsystem'srotaryplatform.(a)Electronic3Dmodel,(b)3Dprintedphysicalmodeland(c)3Dtargetonthemountshownin(b)aftergoldcoating.substratetobecoatedneedstobeplacedandrotatedinordertouniformlycoatthesubstratesurfacewiththetargetmaterial.Becausedirectlyplacingthe3Dtargetsontherotaryplatformwasfoundtobeunstable,wefabricatedastable3Dmounttoholdthe3Dtargetsbeforeplacingthemontheplatform.The3DmountisdesignedinMeshlab[15]bycombiningageneric3Dmodelofawitharectangularbasewithdimensionsof35mm37mm10mm(seeFigure6.5(a)).Themountis3DprintedusingtheStratasysObjetConnex350withtherigidopaquewhitematerial,VeroWhite[31](Figure6.5(b)).Each3Dtargetismountedonthismountbeforesputtering.Further,the3Dtargetregionwithouttheetchingsiscoveredwithtapetoonlysputtertargetmaterialonregionsthatcontainedtheetchedpattern,e.g,pattern.Thistapeisremovedaftersputteringtoobtainthecoated3DtargetanalogoustothatshowninFigure6.1(a).Table6.3liststheexperimentalparametersusedforDCsputtering.Highpurity(>99%)thickgold(Au)andtitanium(Ti)sputteringtargetswith2.00ﬂdiameter0.125ﬂ[12][13]areused.Argon(Ar)isusedastheprocessgasatapressureof4mTorr.Powersourceof125Wisused.30nmofTitanium(Ti)issputterdepositedonthe3Dprintedsamplesbecauseithasgoodadhesion/bindingpropertiestothe3Dprintingmaterialaswellasgold(Au).Thisisfollowedby155Table6.3ParametersettingsforTi+AuDCSputtering.ParameterValueArgaspressure4mTorr1Powersource125W2Tisputteringrate0.21nm/sTilayerthickness30nm3Tisputteringtime2.4minAusputteringrate1.1nm/sAulayerthickness300nmAusputteringtime5min1milliTorr;2Watts;3nanometerssputterdepositionof300nmofAu2onthe3Dtargets.ThesputteringratesforTiandAuat125Vnegativebiasare0.21nm/sand1.1nm/s,respectively.Atthesesputteringrates,ittakesabout2.4minutestosputter30nmTiandabout5minutestosputter300nmAu.Theestimatedin-housecostofsputtercoatingeach3Dtargetwith30nmTiand300nmAu,includinglabor,isapproximatelyUS$2.Giventhatthecosttogenerateaphysical3DtargetisapproximatelyUS$10,thetotalestimatedcosttofabricateaisaboutUS$12.6.3ImpactofSputterCoatingon3DTargetFeaturesInchapter4,wehaddemonstratedthatthe2Dcalibrationpatternfeaturesarereplicatedwithhighonelectronic3Dtargetsafter2Dto3Dprojectionandonfabricatedphysical3Dtargetspost3Dprintingandcleaning.Becausewesputtercoatthecleaned3Dtargetsgeneratedusingthesamemethod,weconductassessmentoffrictionridgeetchingsontheerspostsputterdepositionwithTiandAu.Toconducttheexperiments,wegeneratefourdifferent3Dtargetsbyprojectingdif-ferent(S0005,S0010,S0083,S0096)fromNISTSD4[19]ontoa3Dsurface.2Incontrast,thediameterofahumanhairisanorderofmagnitudethicker(typicallybetween17-181m[5]).156(a)(b)Figure6.6Sampleimpressionsshownin(b)ofacapturedusingtheembeddedcapaci-tivereaderdesignedforsmartphonesin(a).Twoofthesetargets(S0005,S0010)arefabricatedwithTangoBlackPlusFLX980[31],andtheothertwo(S0083S0096)arefabricatedwithFLX9840-DM[30].Inchapter4,wehadreportedthereductioninetchingspacingsonphysical3Dtargetsdueto2Dto3Dprojection(5.8%)and3Dprinting(11.42%).Wesettheprojectionscaleto16.79pixels/mmduring2Dto3Dprojectiontoaccountfortheseerrors.Unlikeourearliermethodofaccountingfortheseerrorsindistancemeasurementsbyupsamplingthetargetimagescapturedusingreaders,thisaprioripro-jectionscaleadjustmentensuresthatspacingsintheoriginal2Dcalibrationpatternsaremaintainedinthe3Dtargetetchingspost2Dto3Dprojectionand3Dprinting.Further,thedepthbetweenridgesandvalleysonthe3Dtargetsissetto0.24mm.Acommercial500ppiAppendixFopticalreader3isusedtocaptureplainimpressionsofthephysical3Dtargetspost3Dprintingandcleaning,whereasacommercial500ppiPIVcapacitivereaderisusedforcapturingtheplainimpressionsoftheerspostsputterdeposition.V6.3[146],whichisacommerciallyavailableSDK,isusedforconductingallcomparisonexperiments.3Themakeandmodelofthereadersusedintheexperimentscannotbedisclosedbecauseofproprietaryreasons.157Table6.4Similarityscoresbetween500ppiplainimpressionoffabricatedphysical3Dtargetscapturedbytheopticalreaderto500ppiplainimpressionofthecorrespondingsputtercoatedcapturedbythecapacitivereader.Physical3DtargetsS0005andS0010werefabri-catedwithTangoBlackPlusFLX980,andS0083andS0096werefabricatedwithFLX9840-DM.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.FingerprintS0005S0010S0083S0096Score764810680708Figure6.7Minutiaecorrespondencebetween(a)plainimpressionofthe3DtargetgeneratedusingimageS0083fromNISTSD4capturedbytheopticalreader,and(b)plainimpressionofthesametargetcapturedbythecapacitivereader(a).Similarityscoreof680isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.6.3.1Fidelityofphysical3DtargetfeaturesonPlainimpressionsofthephysical3Dtargetscapturedusingtheopticalreaderbeforesputtercoatingarecomparedtotheplainimpressionsofthecorrespondingercapturedusingthesingle-capacitivereaderaftersputterdeposition.Thiscomparisonofpreandpostsputterdepositiontargetimagesisusedtoassesshowwellthefeaturesonthephysical3Dtargetsarepreservedonersaftersputterdeposition.Table6.4showsthecomparisonresultsofthisexperiment.Thesimilarityscoresobtainedforallcomparisonsareabovethevthresholdscoreof33computedforNISTSD4ataedfalseacceptrate(FAR)of0.01%.Thisindicates158Table6.5Similarityscoresbetweenplainimpressionsofthesputtercoatedcapturedusinga500ppicapacitivereadertothecorresponding2DntsfromNISTSD4usedintheirgeneration.Physical3DtargetsS0005andS0010werefabricatedwithTangoBlackPlusFLX980,andS0083andS0096werefabricatedwithFLX9840-DM.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.FingerprintS0005S0010S0083S0096Score471333183203Figure6.8Minutiaecorrespondencebetween(a)rolledimageS0083fromNISTSD4,and(b)plainimpressionofthe3Dtargetgeneratedusing(a)capturedbythecapacitivereader.Similarityscoreof183isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.thatfeaturespresentonthephysical3Dtargetsarereplicatedwithhighontheerspostsputterdeposition.6.3.2Fidelityof2DcalibrationpatternfeaturesonPlainimpressionsoftheerscapturedusingthecapacitivereaderarecom-paredtothecorresponding2DimagesfromNISTSD4usedintheirgeneration.End-to-endof2Dcalibrationpatternfeaturesonisassessedbasedonhowwellthe2Dpatternfeaturesarereplicatedontheerspost3Dprinting,cleaningandsputterdeposi-tion.Table6.5showsthecomparisonresults.Forallcomparisons,thesimilarityscoresgenerated159Table6.6Rangeofsimilarityscoresbetweenvedifferent500ppiplainimpressionsofeachsput-tercoated.Physical3DtargetsS0005andS0010werefabricatedwithTangoBlackPlusFLX980,andS0083andS0096werefabricatedwithFLX9840-DM.V6.3SDKwasusedforgeneratingsimilarityscores.Thethresholdonscores@FAR=0.01%is33.FingerprintS0005S0010S0083S0096Scorerange926-1251884-1164824-1215462-1008Figure6.9Minutiaecorrespondencebetweentwodifferentplainimpressions(a)and(b)ofthesame3DtargetgeneratedusingimageS0083fromNISTSD4capturedbythecapacitivereader.Similarityscoreof1164isobtainedbetween(a)and(b)whichisabovethethresholdof33at0.01%FAR.areabovethevthresholdscoreof33forNISTSD4atFARof0.01%.Thisdemonstratesthatthe2Dcalibrationpatternfeatureswerereplicatedwithhighontheers.6.3.3Intra-classvariabilitybetweenimpressionsofFivedifferentplainimpressionsofeacherarecapturedusingthecapacitivereaderandcomparedagainsteachotherinordertoassesstheconsistencybetweendifferentim-pressionsofthesameer.Table6.6showstherangeofsimilarityscoresobtainedforthisexperiment.Allsimilarityscoresarehigherthanthevthresholdscoreof33forNISTSD4atFARof0.01%.Thisshowsthatdifferentimpressionsofthesameerarehighlyconsistent.Inotherwords,thereislowintra-classvariabilitybetweendifferentimpressionsofthesameer.160Table6.7Mean()andstd.deviation(˙)ofcenter-to-centerspacing(inpixels)intheimagesoftheerscapturedusingthe500ppicapacitivereader(CR).Expectedgratingspacing(inpixels)isshowninbrackets.TestpatternCR(500ppi)S0005(9.45)=9.57,˙=0.14S0010(10.20)=10.34,˙=0.21S0083(10.44)=10.60,˙=0.14S0096(10.24)=10.28,˙=0.116.4EvaluationofCapacitiveReadersInthissection,wedescribethepreliminaryexperimentstoevaluate(i)alargeareaPIV500ppistandalonereader,and(ii)smallareacapacitivereadersembeddedinsmart-phonesusing6.4.1LargeareareaderFivedifferentplainimpressionsofeacherarecapturedwiththecapacitivereader.Theridgespacingineachcapturedimpressionismeasuredusingthemethodproposedin[104].Theaveragemeasuredridgespacingfromtheveimpressionsofeacheriscomparedtotheridgespacingofthecorrespondingoriginal(computedusingthesamemethod[104])usedinthegenerationoftheer.Table6.7showstheaverageandvariationinridgespacingmeasurementsfromtheveimpressionsofeachcapturedusingthereader.Followingarethekeyobservationsbasedonthisexperiment:Thecomputedridgespacinginimagesofallfouris,onaverage,observedtobegreaterthan(butwithin0.15pixelsof)theexpectedspacing.Thisismostlikelyduetotheofgratingswhentheyarepressedagainstthecapacitivereaderplatenandisconsistentwithourearlierobservationregardingcontact-basedopticalreaders.Usingtheone-samplet-test[151],thecomputedridgespacingvaluesarestatisticallydifferentthantheexpectedvaluesforallatalevelof0.05.Note,however,that161(a)(b)Figure6.10Evaluationofcapacitivereadersembeddedinsmartphonesusing(a)En-rolmentofaonanAppleiPhone6s,and(b)unlockingoftheiPhone6susingthesame.theincreaseinridgespacingobservedhereisnotasasthatreportedwithopticalreadersinchapter4.Tobetterunderstandthesedifferences,controlledexperimentationwithknowncontactpressureduringcaptureisrequired.Theaveragedeviationinridgespacingbetweendifferentimpressionsofthesameisbetween0.1-0.2pixels.Thesearecomparabletotheridgespacingdeviationmeasure-mentsusing3Dtargetsforoneoftheopticalreaders,butslightlygreaterthanspacingmeasurementsreportedfortheothertwoopticalreadersinchapter4.Thisisbe-causethecapacitivereaderhasasmallerplatencomparedtothetwoopticalreaderswhichresultsinonlypartiallyimagesoftheTherefore,overallfewerridgespacingmeasurementsareusedforspacingcomputations.6.4.2EmbeddedsmallareareadersWeperformfeasibilityexperimentsusingtwodifferentsmartphones,theAppleiPhone6sandtheSamsungGalaxyS74,andacapacitivereadermoduledesignedforsmartphones.Figure6.6showstheimpressionsacquiredwithcapacitivereadermoduledesignedforsmartphones.Weenrollaerusingtheenrolmentprocedureonthetwophones(seeFigure6.104Commercialsmartphonesdonotprovideaccesstoimages.162(a)(b)Figure6.11Sample3Dspoof.(a)Electronic3Dspoof,and(b)physical3Dspoofafterconductivecarboncoating.(a)).Subsequently,wemaketenindependentattemptstounlockthetwophonesusingtheenrolleder.Theenrolledertemplateisthendeletedandwerepeatthisprocedureusingadifferenter.Wewereabletosuccessfullyunlockthetwophonesinallourattemptsusingeachofthefourers(seeFigure6.10(b)).Thisindicatesthepotentialfeasibilityofusingersastargetsforevaluatingcapacitivereadersembeddedinsmartphones.6.5PresentationAttacksonCapacitiveReadersAlthoughtheprimarygoalofmanufacturingconductive3Dartifactsisevaluationofcapacitivereaders,aby-productisthepotentialuseofsuchartifactsinperformingpresentationattacksoncapacitivereaders.Below,wedescribeasimpleprocedurethatcanbeusedtocreateaconductive3Dspooffroma2Dplainofaknownsubject.1.Usea3Dmodelingsoftware,e.g,Meshlab[15]tosyntheticallygenerateacuboidalsurfaceforprojectingthe2Dprint.Setthelengthandwidthofthecuboidtoatleast3.5cmand1632.5cm,respectively.Thisensuresthatthereisadequatesurfaceareaforprojectingtheplainprint.Also,settheheight(wallthickness)ofthecuboidtoatleast1mmfor3Dprintingthecuboidasasolidobject.2.Usethemethoddescribedinchapter4tocreatethe3Dspoofbyetchingtheplainprintontothecuboidalsurface.Whilecreatingtheelectronicspoof,setthe2Dto3Dprojectionscaleappropriately(16.79pixels/mm)toaccountfor2Dto3Dprojectionerrorand3Dprintingfabricationerror.Printthespoofusingahigh-resolution3Dprinter(e.g.StratasysObjetConnex350)withmaterialssimilarinhardnessandelasticitytothehumanskin(e.g.Tan-goBlackPlusFLX980).3.Dipthe3Dprintedspoofin2MNaOHsolutionforapprox.3hrs.andthenrinseitwithwater.Onceitdries,spraycoatconductivecarbon(e.g.[35])ontothe3Dspoof.Thisimpartstherequiredconductivityforregisteringthespoofwithcapacitivereaders5.Wegeneratedve3Dspoofsfromindexofvedifferentsubjectsusingtheafore-mentionedprocedure.ThespoofswerefabricatedwithTangoBlackPlusFLX980.Figure6.11showsanexample.Forconductingexperiments,theindexofthesubjectswereenrolledontwodifferentcapacitivereaders,asinglcapacitivereaderandanembed-dedcapacitivereaderinanaccesscontrolterminal.V6.3SDKwasinterfacedwiththecapacitivereaderforperformingcomparisons.Fortheembeddedreader,thecomparisonalgorithmbuiltintotheaccesscontrolterminalwasusedforcomparisons.Fiveseparateattemptsweremadeoneachofthereadersusingthevespoofs.Inallattempts,wewereabletosuccessfullyspoofthethreereaders.Note,however,thatthegenerated3Dspoofswerenotabletospoofthecapacitivereadersembeddedinsmartphones.Webelievethisisbecausecomparisonalgorithmsinsmart-phonesusetexture-basedfeaturesinadditiontominutiaefeatures.Giventhatthetexturecharac-5Spraycoatingofconductivecarbonisnon-uniform.Furthermore,thecarboncoatingonlyimpartsconductivityforalimitedtime(1-2hours),andhaslowabrasionresistance.Thisprocess,therefore,cannotbeusedforcreatingcapacitive3Dtargets.164teristicsofthecreated3Dspoofsdifferfromthehumanskin,thesespoofsarenoteffectiveforcapacitivereadersinsmartphones.6.6ConclusionsCapacitivereadersarenowbeingincreasinglyusedinconsumerandaccesscontrolapplications,e.g.,forsmartphoneunlockandpointofsale(POS)payments.Giventhewidespreaddeploymentofcapacitivereaders,animportantrequirementistodevelopstandardartifactsandproceduresforrepeatableevaluationofthesereaders.Inthischapter,wedescribedaproceduretogenerate3Dtargets,termeders,forcapacitivereaderevaluation.Weusedastate-of-the-artmethodtocreateelectronic3Dtargetsbymapping2Dcalibrationpatternswithknownfeaturesontoa3Dsurface.Physical3Dtargetswerefabricatedfromelectronic3Dmodelsusingastate-of-the-arthigh-resolution3Dprinter.Asputterdepositiontechniquewassubsequentlyusedtocoatthesurfaceof3Dprintedtargetswithathinlayeroftitaniumandgoldparticles.Wedemonstratedthatthe2Dcalibrationpatternfeaturescanbereplicatedwithhighlityontheers.Furthermore,weevaluatedacommerciallyavailable500ppicapacitivereaderusingers.Weshowedthattheerscanbeusedastargetsfortestingcapacitivereadersembeddedinsmartphones.Thespoofvulnerabilityofcommerciallyavailablecapacitivereaderstopresentationattacksusing3Dprintedspoofswasalsoassessed.165Chapter7SummaryInthisdissertation,wehaveproposedimprovementsto(i)thedesignofrecognitionsystemsforlatentmatching,and(ii)theoperationalevaluationmethodsforreadersusing3Dandwholehandtargets.Ourcontributionsaresummarizedbelow:Designofatop-downmatchingparadigmforautomaticmatchingoflatentstoexemplarprints.Thisframeworktakesfeedbackfromthetop-Kcandidateprintsfromthereferencedatabase,outputbyalatentmatcher,toimprovetheoveralllatentmatchingaccuracy.Ourapproachcanbewrappedaroundanybaselinelatentmatcherinordertoimproveitsmatchingperformance.Todeterminetheadequacyoffeedback,wedeveloped(i)astatisticaltestbasedonthedistributionofsimilarityscoresbetweenthelatentandthetop-Kcandidateexemplarstodecidewhenfeedbackisrequired,and(ii)alocalqualitybasedmetrictodeterminewhichlatentregionswouldfromfeedbackinordertoimprovethesimilarityscorewiththetruemate.Usingtheproposedparadigm,thematchingperformanceofastate-of-the-artlatentmatcherimprovedby0.5-3.5%ontwolatentdatabases.DesignofalatentmarkupcrowdsourcingframeworkwheremultiplehumanexaminersandtheAFISworkinsynergytoboostlatentmatchingperformance.Givenalatent,anAFISisusedtoautomaticallymatchthelatentagainstexemplarsinthereferencedatabase.BasedontheoutputoftheAFIS,astatisticaltestisusedtodecideifadditionalmarkupfrom166examinersisrequired.Ifso,thelatentiscrowdsourcedtoapoolofexaminersforprovidingmarkups.ThesetofmarkupsprovidedbyexaminersarethenindividuallyfedtotheAFIStoobtainthenewsetofsimilarityscores.TheoutputoftheAFISfordif-ferentexaminersisfusedtogetherviascorelevelfusionwiththeinitialautomaticmatchoutputtoimprovetheoveralllatentmatchingaccuracy.Agreedycrowdsourcingframeworkisalsoproposedwhereinsteadofcrowdsourcingthelatenttoallexaminersatonce,exam-inermarkupsareobtainedincrementally,asdeterminedbyourstatisticaltest,toboosttheoverallaccuracy.performanceimprovement(2.5-11.5%)isobtainedusingthecrowdsourcedmarkupsinconjunctionwithastate-of-the-artlatentmatcher.Designandfabricationof3Dtargetsforoperationalevaluationofopticalread-ers.2Dcalibrationpatterns(e.g.,sinegratingsand2Dngerprintswithknownsingularpointsandminutiae)areprojectedontoa3Dsurfacetocreateelectronic3Dtargets.Astate-of-the-art3Dprinterisusedtofabricatephysical3Dtargetsfromtheelectronicmod-els.Aproceduretochemicallycleanthe3Dprintedtargetsusing2MNaOHsolutionandwaterisalsodeveloped.Wemeasurethe(i)2Dto3Dprojectionerror,and(ii)fabricationerrorintroducedbythe3Dprintertoassesstheof3Dtargetsynthesisandfabricationprocess.Weshowthatthe2Dcalibrationpatternfeaturesarereplicatedwithhighbothonelectronicandphysical3Dtargets.Wealsoconductexperimentstoestimatetheerrorintroducedbythreedifferent500/1000ppicommercialreadersusing3Dtargetscreatedusingsinegratingsaswellaspatterns.Designandfabricationof3Dwholehandtargetscompletewithallfourandthethumbprintforevaluatingslapandcontactlessreaders.2Dcalibrationpatternsaremappedto3Dsurfacescorrespondingtoeachoftheveandagloveiscreatedtogenerateelectronic3Dhandtarget.3Dphysicaltargetsaresubsequentlyprintedusingastate-of-the-art3Dprinterandcleanedwith2MNaOHandwater.Weshowthatthe2Dcalibrationpatternfeaturesarereplicatedwithhighbothontheelectronicand167physical3Dwholehandtargets.Wealsodemonstratethatthemanufactured3Dwholehandtargetscanbeusedforevaluatingthreedifferent500/1000ppicontact-basedslapreadersandacontactlessslapreader.Fabricationofconductive3Dtargetsforevaluationofcapacitivereaders.The3Dprintedtargetsareelectricallynon-conductive.Toimpartconductivity,thesurfaceof3Dprintedtargetsiscoatedwiththinlayersofmetals(titanium+gold)usingDCsputtering.Weshowthatthisprocessimpartsconductivitytoimagethetargetswithcapacitivereaderswithoutdegradingtheofengravedfeaturesonthetarget.Thefabricatedconductive3Dtargetsareusedforevaluatingstandaloneaswellasembeddedcapacitivereaders.Asimpleproceduretocreate3Dprintedspoofsforperformingpresentationattacksoncapac-itivereadersisalsodescribed.Weshowthatthe3Dprintedspoofscansuccessfullyspoofa500ppicommercialcapacitivereaderandacapacitivereaderembeddedonanaccesscontrolterminal.7.1FutureWorkFollowingaresomepossiblefutureresearchdirectionsfortheproblemsinvestigatedinthisthesis:Feedbackparadigmforlatentmatching:Incorporatelevel-2featuressuchasridgeskeletonandminutiaeintothefeedbackparadigmtofurtherimprovelatentmatchingaccuracy.Latentmarkupcrowdsourcingframework:Validatetheproposedframeworkagainstalargerreferencedatabaseofafewmillionreferenceprints.Exploreensemble-basedmetaalgorithmssuchasbaggingandboostingtofurtherimprovethematchingperformanceofAFIS.3Dandwholehandtargets:Exploremethodstobuilduniversal3Dtargetswiththeopticalandelectricalpropertiessimilartohumanskinforusewithbothoptical168andcapacitivereaders,e.g.,bycreatinganegative(mold)ofthe3Dtargetsandthencastinguniversaltargetswithappropriatematerials.Investigateprocedurestoimpartconductivitytotargetsmanufacturedviamoldingandcasting,e.g,mixingconductiveinkstocastingmaterials.Thiswouldfacilitatebenchmarkingofdifferentopticalandcapacitivereadersusingthesametarget,aswellasinvestigationofreaderinteroperabilityusing3Dtargets.Further,experimentalproceduresbasedon3Dtargetscanpotentiallybeusefultorevisetheexistingevaluationstandards(PIV/AppendixF).Simulatetheeffectsofdryandwornoutusing3Dtargetswithdifferentdepthsofengravingstofurtherstudytheimagingcapabilitiesofdifferentreaders.Studyhowuser-inducedvariabilities,e.g.contact-pressureappliedonthereaderplatenandrelativeorientationswithrespecttoeachotheraswellasthereaderplaten,impactthequalityofthecapturedimages.Todothis,controlledexperimentationwhereknowncontact-pressureisappliedonthereaderplatenwhilecapturingimpressionsisrequired.Developsolutionstopreventpresentationattacksusing3Dprintedspoofs.169BIBLIOGRAPHY170BIBLIOGRAPHY[1]ApplePay.http://www.apple.com/apple-pay/.[2]ArtecEva3Dscanner.http://www.artec3d.com/hardware/artec-eva/.[3]ContactlessFingerprintCapture,NationalInstituteofStandardsandTechnology.http://www.nist.gov/itl/iad/ig/contactless[4]DentonVacuumDesktopProSputteringSystem.http://www.dentonvacuum.com/technologies/production-solutions/desktop-pro/.[5]DiameterofaHumanHair.http://hypertextbook.com/facts/1999/BrianLey.shtml.[6]ELFT-EFSpublicchallengedataset.http://biometrics.nist.gov/cslinks/latent/elft-efs/ELFT-EFSPublicChallenge2009-04-23.pdf.[7]FasterandBetterFingerprinting:TheFastCaptureInitiative,NationalInstituteofJustice.http://www.nij.gov/topics/technology/biometrics/pages/fast-capture-initiative.aspx.[8]FBIBiospecsIAFISFAQs.https://www.fbibiospecs.orfaq.html.[9]FBIIAFISproductlisting.http://www.fbi.gov/about-cert.[10]FlashScan3Dtouchlesssensor.http://www[11]KeyenceDigitalMicroscopeVHX-600.http://www1.keyence.eu/products/microscope/microscope/vhx6002/vhx60021.php.[12]KurtJ.LeskerGold(Au)SputteringTargets.http://www.lesker.com/newweb/depositionmaterials/depositionmaterialssputtertargets1.cfm?pgid=au1.[13]KurtJ.LeskerTitanium(Ti)SputteringTargets.http://www.lesker.com/newweb/depositionmaterials/depositionmaterialssputtertargets1.cfm?pgid=ti1.[14]MeetPhannie,NIST'sStandardﬁPhantomﬂforCalibratingMRIMachines.http://www.nist.gov/pml/electromagnetics/phannie051110.cfm.[15]Meshlab.http://sourceforge.net/projects/meshlab/.[16]Moireeffect.http://whatis.techtareffect.[17]MorphoFingerontheFly.http://www-control-242/semi-automated--on-the-r-946/?lang=en.[18]NISTSD27LatentDatabase.http://www.nist.gov/srd/nistsd27.cfm.[19]NISTSpecialDatabase4.http://www.nist.gov/srd/nistsd4.cfm.171[20]OfofBiometricIdentityManagementServices.http://www.dhs.gov/obim-biometric-services.[21]ONEprinttouchlessreader.http://www.idairco.com/products/.[22]OperationASHA'shealthcaree-compliancebiometricsystem.http://www.opasha.org/our-work/ecompliance-innovation-and-health/.[23]RecordingLegibleFingerprints,FederalBureauofInvestigation.https://www.fbi.gov/about-biometrics/recording-legible-[24]RonSmithandAssociates.http://www.ronsmithandassociates.com/.[25]SamsungFingerprintSensor.http://www.samsung.com/us/support/answer/ANS00039905/997339942/SM-G920TZKATMB.[26]SamsungPay.http://www.samsung.com/us/samsung-pay/.[27]Statista:NFCmobilepaymentusersworldwidefrom2012to2018.http://www.statista.com/statistics/461512/nfc-mobile-payment-users-worldwide/.[28]Statista:Numberofsmartphoneusersworldwidefrom2014to2019.http://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/.[29]Statista:Penetrationofsmartphoneswithsensorsfrom2014to2018.http://www.statista.com/statistics/522058/global-smartphone-penetration/.[30]StratasysDigitalMaterials(DMs)DataSheet.http://www.stratasys.com/˘/media/Main/Secure/Material%20Specs%20MS/PolyJet-Material-Specs/DigitalMaterialsDatasheet.pdf.[31]StratasysPolyjetMaterialDataSheet.http://www.stratasys.com/˘/media/Main/Secure/Material%20Specs%20MS/PolyJet-Material-Specs/PolyJetMaterialsDataSheet.pdf.[32]TBScontactless3Dsensor.http://www.tbs-biometrics.com/en/hardware/3d-enroll.html.[33]ThorLabsCalibrationTargets.http://www.thorlabs.com/newgrouppage9.cfm?objectgroupid=7502.[34]TorsoPhantom.https://www.kyotokagaku.com/products/detail03/ph-4.html.[35]TotalGroundCarbonConductiveCoating.http://www.unicornelex.com/ProductDetails.asp?ProductCode=26-0801&gclid=CN7Nh5qmv88CFQMdaQodkNMNNA.[36]TouchIDSystem.http://support.apple.com/kb/ht5949.[37]UIDBiometricDevicehttp://www.uidai.gov.in/biometric-device-[38]UniqueIdeAuthorityofIndia.http://uidai.gov.in/.172[39]VertexDisplacementMappingusingGLSL.http://www.ozone3d.net/tutorials/vertexdisplacementmappingp02.php.[40]WaltDisneyWorld.https://disneyworld.disney.go.com/.[41]X-RayPhantomHand.http://www.anatomicalmodel.eu/x-ray-phantom-hand-transparent.[42]BiometricDesignStandardsforUIDApplicationsv1.0.UIDAICommitteeonBiometrics,Dec2009.[43]GuidancetoapplicantforbiometricdevicesforUIDApplication(BDCS-03-02).STQCITServices,Dec2010.[44]LatentPrintExaminationandHumanFactors:ImprovingthePracticethroughaSystemsApproach.Technicalreport,U.S.DepartmentofCommerce,NationalInstituteofTechnol-ogy,2012.[45]ForensicScienceinCriminalCourts:EnsuringValidityofFeature-ComparisonMethods.Technicalreport,PresidentsCouncilofAdvisorsonScienceandTechnology(PCAST),2016.[46]QualcommSnapdragonSense.https://www.qualcomm.com/products/snapdragon/security/sense-id,2016.[47]A.AbhyankarandS.Schuckers.Fingerprintlivenessdetectionusinglocalridgefrequenciesandmultiresolutiontextureanalysistechniques.InIEEEInternationalConferenceonImageProcessing(ICIP),pages321Œ324,2006.[48]D.Alemu,H.Wei,K.Ho,andC.Chu.HighlyconductivePEDOT:PSSelectrodebysimpletreatmentwithmethanolforito-freepolymersolarcells.Energy&EnvironmentalScience,5(11):9662Œ9671,2012.[49]P.Alliez,D.Cohen-Steiner,O.Devillers,B.L´evy,andM.Desbrun.Anisotropicpolygonalremeshing.InACMTransactionsonGraphics(TOG),volume22,pages485Œ493,2003.[50]F.Alonso-Fernandez,J.Bigun,J.Fierrez,H.Fronthaler,K.Kollreider,andJ.Ortega-Garcia.FingerprintRecognition.InGuidetoBiometricReferenceSystemsandPerformanceEval-uation,pages51Œ88.Springer,2009.[51]N.Amenta,M.Bern,andD.Eppstein.Thecrustandthe-skeleton:Combinatorialcurvereconstruction.GraphicalModelsandImageProcessing,60(2):125Œ135,1998.[52]S.S.Arora,K.Cao,A.K.Jain,andG.Michaud.Crowdpoweredlatentntidenti-fusingAFISwithexaminermarkups.InInternationalConferenceonBiometrics(ICB),pages363Œ370,2015.[53]S.S.Arora,K.Cao,A.K.Jain,andN.G.PaulterJr.3DFingerprintPhantoms.In22ndInternationalConferenceonPatternRecognition(ICPR),pages684Œ689,2014.173[54]S.S.Arora,K.Cao,A.K.Jain,andN.G.PaulterJr.DesignandFabricationof3DFinger-printTargets.IEEETransactionsonInformationForensicsandSecurity,11(10):2284Œ2297,2016.[55]S.S.Arora,A.K.Jain,andN.G.PaulterJr.3dWholeHandTargets:EvaluatingSlapandContactlessFingerprintReaders.InInternationalConferenceoftheBiometricsSpecialInterestGroup(BIOSIG),2016.[56]S.S.Arora,E.Liu,K.Cao,andA.K.Jain.LatentFingerprintMatching:PerformanceGainviaFeedbackfromExemplarPrints.IEEETransactionsonPatternAnalysisandMachineIntelligence,36(12):2452Œ2465,2014.[57]D.R.Ashbaugh.Quantitative-QualitativeFrictionRidgeAnalysis:AnIntroductiontoBasicandAdvancedRidgeology.CRCPress,1999.[58]C.A.Bishop.TransparentConductingCoatingsonPolymerSubstratesforTouchscreensandDisplays.InHandbookofVisualDisplayTechnology,pages975Œ988.Springer,2012.[59]R.Black.ManagingtheTestingProcess.JohnWiley&Sons,2002.[60]D.A.Boas,C.Pitris,andN.Ramanujam.HandbookofBiomedicalOptics.Taylor&Francis,2011.[61]M.Bod´en.Aguidetorecurrentneuralnetworksandbackpropagation.TheDALLASproject.ReportfromtheNUTEK-supportedprojectAIS-8,SICS.Holst:Applicationofdataanalysiswithlearningsystems,(2):1Œ10,2001.[62]M.BoehmeandC.Charton.PropertiesofITOonPETindependenceonthecoatingconditionsandthermalprocessing.SurfaceandCoatingsTechnology,200(1):932Œ935,2005.[63]E.BorensteinandS.Ullman.Combinedtop-down/bottom-upsegmentation.IEEETrans-actionsonPatternAnalysisandMachineIntelligence,30(12):2109Œ2125,2008.[64]K.CaoandA.KJain.LatentOrientationFieldEstimationviaConvolutionalNeuralNet-work.InIAPRInternationalConferenceonBiometrics,pages1Œ8,2015.[65]K.CaoandA.K.Jain.Hackingmobilephonesusing2DprintedMichiganStateUniversityTechnicalReport,MSU-CSE-16-2,2016.[66]K.Cao,E.Liu,andA.K.Jain.SegmentationandenhancementoflatentAcoarsetoridgestructuredictionary.IEEETransactionsonPatternAnalysisandMa-chineIntelligence,36(9):1847Œ1859,2014.[67]R.Cappelli.SFinGe:anapproachtosyntheticgeneration.InInternationalWorkshoponBiometricTechnologies(BT2004),pages147Œ154,2004.[68]R.Cappelli,M.Ferrara,andD.Maltoni.Minutiacylinder-code:Anewrepresentationandmatchingtechniqueforrecognition.IEEETransactionsonPatternAnalysisandMachineIntelligence,32(12):2128Œ2141,2010.174[69]R.Cappelli,D.Maio,andD.Maltoni.Syntheticbasegeneration.In16thIEEEInternationalConferenceonPatternRecognition,volume3,pages744Œ747,2002.[70]G.CasellaandR.L.Berger.StatisticalInference,volume70.DuxburyPressBelmont,CA,1990.[71]C.Champod,C.J.Lennard,P.Margot,andM.Stoilovic.FingerprintsandOtherRidgeSkinImpressions.CRCpress,2004.[72]Y.A.Chizmadzhev,A.V.Indenbom,P.I.Kuzmin,S.V.Galichenko,J.C.Weaver,andR.O.Potts.Electricalpropertiesofskinatmoderatevoltages:contributionofappendagealmacropores.BiophysicalJournal,74(2):843Œ856,1998.[73]D.Cho,K.Kim,T.Kim,Y.Noh,S.Na,K.Chung,andH.Kim.TransparentandxibleamorphousInZnAlOgrownbyroll-to-rollsputteringforacidicbuffer-freexibleorganicsolarcells.OrganicElectronics,24:227Œ233,2015.[74]H.Choi,M.Boaventura,I.A.G.Boaventura,andA.K.Jain.AutomaticsegmentationoflatentInIEEEFifthInternationalConferenceonBiometrics:Theory,Applica-tionsandSystems,pages303Œ310,2012.[75]H.Choi,A.Nagar,andA.K.Jain.OntheevidentialvalueofInInternationalJointConferenceonBiometrics(IJCB),pages1Œ8,2011.[76]S.A.Cole.SuspectIdentities:AHistoryofFingerprintingandCriminal.HarvardUniversityPress,2009.[77]B.Coppin.IntelligenceIlluminated.Jones&BartlettLearning,2004.[78]NationalResearchCouncil.StrengtheningForensicScienceintheUnitedStates:APathForward.2009.[79]PolkCountySherrif'sofFingerprintingService.http://www.polksheriff.org/INSIDEPCSO/OBA/HR/Pages/FingerprintingService.aspx.[80]J.Daugman.600millioncitizensofIndiaarenowenrolledwithbiometricID.http://spie.org/x108321.xml,2014.[81]DeltaID-MobileSecurity.http://www.deltaid.com/.[82]M.Desbrun,M.Meyer,andP.Alliez.Intrinsicparameterizationsofsurfacemeshes.InComputerGraphicsForum,volume21,pages209Œ218,2002.[83]T.G.Dietterich.Ensemblemethodsinmachinelearning.InMultipleSystems,pages1Œ15.Springer,2000.[84]E.W.Dijkstra.Anoteontwoproblemsinconnexionwithgraphs.NumerischeMathematik,1(1):269Œ271,1959.175[85]I.E.Dror,A.E.Peron,S.L.Hind,andD.Charlton.Whenemotionsgetthebetterofus:theeffectofcontextualtop-downprocessingonmatchingAppliedCognitivePsychology,19(6):799Œ809,2005.[86]I.E.Dror,K.Wertheim,P.Fraser-Mackenzie,andJ.Walajtys.TheImpactofHumanŒTechnologyCooperationandDistributedCognitioninForensicScience:BiasingEffectsofAFISContextualInformationonHumanExperts.JournalofForensicSciences,57(2):343Œ352,2012.[87]R.O.Duda,P.E.Hart,andD.G.Stork.Pattern.JohnWileyandSonsInc.,secondedition,2001.[88]C.EdwardsandR.Marks.Evaluationofbiomechanicalpropertiesofhumanskin.ClinicsinDermatology,13(4):375Œ380,1995.[89]V.FalangaandB.Bucalo.Useofadurometertoassessskinhardness.JournaloftheAmericanAcademyofDermatology,29(1):47Œ51,1993.[90]H.Faulds.OntheSkin-FurrowsoftheHand.Nature,22:605,1880.[91]FederalBureauofInvestigation.IntegratedAutomatedFingerprintSystem(IAFIS).http://www.fbi.gov/about-us/c[92]J.FengandA.K.Jain.Fingerprintreconstruction:Fromminutiaetophase.IEEETransac-tionsonPatternAnalysisandMachineIntelligence,33(2):209Œ223,2011.[93]J.Feng,J.Zhou,andA.K.Jain.Orientationestimationforlatentenhance-ment.IEEETransactionsonPatternAnalysisandMachineIntelligence,35(4):925Œ940,2013.[94]H.Fronthaler,K.Kollreider,andJ.Bigun.LocalfeaturesforenhancementandminutiaeextractioninIEEETransactionsonImageProcessing,17(3):354Œ363,2008.[95]J.Galbally,F.Alonso-Fernandez,J.Fierrez,andJ.Ortega-Garcia.Ahighperformancelivenessdetectionmethodbasedonqualityrelatedfeatures.FutureGenerationComputerSystems,28(1):311Œ321,2012.[96]F.Galton.FingerPrints.MacMillan,1892.[97]FrancisGalton.Personalanddescription.JournalofAnthropologicalInstituteofGreatBritainandIreland,pages177Œ191,1889.[98]E.German.LatentPrintExamination.http://www.onin.com/fp/,2007.[99]L.Ghiani,A.Hadid,G.L.Marcialis,andF.Roli.Fingerprintlivenessdetectionusingbinarizedstatisticalimagefeatures.InIEEESixthInternationalConferenceonBiometrics:Theory,ApplicationsandSystems(BTAS),pages1Œ6,2013.[100]L.K.HansenandP.Salamon.Neuralnetworkensembles.IEEETransactionsonPatternAnalysisandMachineIntelligence,12(10):993Œ1001,1990.176[101]M.R.Hawthorne.Fingerprints:AnalysisandUnderstanding.CRCPress,2009.[102]W.J.Herschel.TheOriginsofFingerprinting.OxfordUniversityPress,1916.[103]J.G.HollandsandC.D.Wickens.EngineeringPsychologyandHumanPerformance.Pren-ticeHall,NewJersey,1999.[104]L.Hong,Y.Wan,andA.K.Jain.Fingerprintimageenhancement:algorithmandper-formanceevaluation.IEEETransactionsonPatternAnalysisandMachineIntelligence,20(8):777Œ789,1998.[105]IBG.Syntheticbiometricimagegenerator.DepartmentofHomelandSecuritySmallBusi-nessInnovationResearchFinalReport,2012.[106]H.Im,H.Hong,J.Lee,J.Lee,Y.Heo,andJ.Kim.Structuralandelectricalpropertiesofalandbco-dopedznothinJournalofNanoelectronicsandOptoelectronics,6(3):301Œ305,2011.[107]AppliedImageInc.Seimens/starsectortarget.https://www.appliedimage.com/products/test-targets-and-charts-1/rit-siemens-sayce-and-other-test-targets/siemens-star-sector-target-t-50.[108]M.Indovina,V.Dvornychenko,R.A.Hicklin,andG.I.Kiebuzinski.ELFT-EFSEvaluationofLatentFingerprintTechnologies:ExtendedFeatureSets[Evaluation#2].NISTIR,7859,2012.[109]M.Indovina,R.A.Hicklin,andG.I.Kiebuzinski.ELFT-EFSEvaluationofLatentFinger-printTechnologies:ExtendedFeatureSets[Evaluation#1].NISTIR,7775,2011.[110]A.K.Jain.Fingerprints:Provinggroundforpatternrecognition.InternationalConferenceonPatternRecognition(ICPR),2006.[111]A.K.Jain.Technology:BiometricRecognition.Nature,449(7158):38Œ40,2007.[112]A.K.Jain,S.S.Arora,L.Best-Rowden,K.Cao,P.S.Sudhish,andA.Bhatnagar.Biomet-ricsforchildvaccinationandwelfare:Persistenceofntrecognitionforinfantsandtoddlers.arXivpreprintarXiv:1504.04651,2015.[113]A.KJain,S.S.Arora,L.Best-Rowden,K.Cao,P.S.Sudhish,A.Bhatnagar,andY.Koda.GivingInfantsanIdentity:FingerprintSensingandRecognition.In8thInternationalCon-ferenceonInformationandCommunicationTechnologiesandDevelopment,page29,2016.[114]A.K.Jain,K.Cao,andS.S.Arora.RecognizinginfantsandtoddlersusingIncreasingthevaccinationcoverage.InIEEEInternationalJointConferenceonBiometrics(IJCB),pages1Œ8,2014.[115]A.K.Jain,Y.Chen,andM.Demirkus.Poresandridges:high-resolutionmatch-ingusinglevel3features.IEEETransactionsonPatternAnalysisandMachineIntelligence,29(1):15Œ27,2007.177[116]A.K.JainandJ.Feng.Latentpalmprintmatching.IEEETransactionsonPatternAnalysisandMachineIntelligence,31(6):1032Œ1047,2009.[117]A.K.JainandJ.Feng.Latentngerprintmatching.IEEETransactionsonPatternAnalysisandMachineIntelligence,33(1):88Œ100,2011.[118]A.K.Jain,P.Flynn,andA.A.Ross.HandbookofBiometrics.Springer,2007.[119]A.K.Jain,L.Hong,andR.Bolle.On-linevIEEETransactionsonPatternAnalysisandMachineIntelligence,19(4):302Œ314,1997.[120]A.K.JainandS.Z.Li.HandbookofFaceRecognition.Springer,2011.[121]A.K.Jain,K.Nandakumar,andA.Ross.Scorenormalizationinmultimodalbiometricsystems.PatternRecognition,38(12):2270Œ2285,2005.[122]A.K.Jain,K.Nandakumar,andA.Ross.50yearsofBiometricResearch:Accomplish-ments,Challenges,andOpportunitiesﬂ.PatternRecognitionLetters,2016.[123]A.K.Jain,S.Prabhakar,andS.Pankanti.OntheSimilarityofIdenticalTwinFingerprints.PatternRecognition,35(11):2653Œ2663,2002.[124]A.K.JainandA.Ross.Bridgingthegap:Frombiometricstoforensics.PhilosophicalTransactionsoftheRoyalSocietyB,370(1674):20140254,2015.[125]A.K.JainandD.Zongker.Featureselection:Evaluation,application,andsmallsampleperformance.IEEETransactionsonPatternAnalysisandMachineIntelligence,19(2):153Œ158,1997.[126]Z.Jin,A.B.J.Teoh,T.S.Ong,andC.Tee.Fingerprinttemplateprotectionwithminutiae-basedbit-stringforsecurityandprivacypreserving.ExpertSystemswithApplications,39(6):6157Œ6167,2012.[127]G.K.Johnsen.Skinelectricalpropertiesandphysicalaspectsofhydrationofkeratinizedtissues.PhDthesis,UniversityofOslo,2010.[128]I.Jolliffe.PrincipalComponentAnalysis.WileyOnlineLibrary,2005.[129]R.KimmelandJ.A.Sethian.Computinggeodesicpathsonmanifolds.ProceedingsoftheNationalAcademyofSciences,95(15):8431Œ8435,1998.[130]P.Komarinski.AutomatedFingerprintSystems(AFIS).AcademicPress,2005.[131]J.B.KruskalandM.Wish.MultidimensionalScaling.Sage,1978.[132]S.LalithaandN.Kumar.Multipleoutliertestforupperoutliersinanexponentialsample.JournalofAppliedStatistics,39(6),2012.[133]H.O.LancasterandE.Seneta.Chi-SquareDistribution.WileyOnlineLibrary,1969.178[134]T.LewisandN.R.J.Fieller.Arecursivealgorithmfornulldistributionsforoutliers:I.gammasamples.Technometrics,21(3):371Œ376,1979.[135]E.Liu,S.S.Arora,K.Cao,andA.K.Jain.Afeedbackparadigmforlatentmatching.InInternationalConferenceonBiometrics(ICB),pages1Œ8,2013.[136]C.Loop.Smoothsubdivisionsurfacesbasedontriangles.Master'sthesis,UniversityofUtah,1987.[137]DongguanHongchengOpticalProductsLtd.Visioncalibrationgridtargets.http://hcgxcy.en.ec21.com/OpencvVisionCalibrationGridTargets--85915708603700.html.[138]Y.Lu,H.Tang,S.Fung,Q.Wang,J.M.Tsai,M.Daneman,B.E.Boser,andD.A.Horsley.Ultrasonicsensorusingapiezoelectricmicromachinedultrasonictransducerar-rayintegratedwithcomplementarymetaloxidesemiconductorelectronics.AppliedPhysicsLetters,106(26):263503,2015.[139]A.V.Maceo.Anatomyandphysiologyofadultfrictionridgeskin.TheFingerprintSource-book,2011.[140]V.M¨akinenandE.Ukkonen.Pointpatternmatching.InEncyclopediaofAlgorithms,pages657Œ660.Springer,2008.[141]D.Maltoni,D.Maio,A.K.Jain,andS.Prabhakar.HandbookofFingerprintRecognition.Springer,2009.[142]DelMarLiveScan.SanDiegoFingerprinting.http://www.delmarlivescan.com/san-diego-services/.[143]E.MarascoandA.Ross.Asurveyonschemesforrecognitionsystems.ACMComputingSurveys,47(2):28,2015.[144]T.Matsumoto.Gummyandconductivesiliconerubberimportanceofvulnerabilityanalysis.InAdvancesinCryptology-ASIACRYPT2002,pages574Œ575.Springer,2002.[145]T.Matsumoto,H.Matsumoto,K.Yamada,andS.Hoshino.Impactofgummyonsystems.InElectronicImaging,pages275Œ289,2002.[146]NeurotechnologyInc.Vhttp://www.neurotechnology.com/v.html.[147]NISTSpecialDatabase14.http://www.nist.gov/srd/nistsd14.cfm.[148]NISTSpecialDatabase27.http://www.nist.gov/srd/nistsd27.cfm.[149]U.S.MarshalsServiceforStudents.http://www.usmarshals.gohistory.htm.[150]WestVirginiaUniversity(WVU)latentdatabase.http://www.cse.msu.edu/˘rossarun/.179[151]D.S.Moore.TheBasicPracticeofStatistics,volume2.WHFreeman,2007.[152]A.NagarandA.K.Jain.Onthesecurityofnon-invertibletemplatetransforms.InInternationalWorkshoponInformationForensicsandSecurity(WIFS),pages81Œ85,2009.[153]A.Nagar,K.Nandakumar,andA.K.Jain.Securingtemplate:Fuzzyvaultwithminutiaedescriptors.In19thInternationalConferenceonPatternRecognition,pages1Œ4,2008.[154]K.NandakumarandA.K.Jain.Biometrictemplateprotection:Bridgingtheperformancegapbetweentheoryandpractice.IEEESignalProcessingMagazine,32(5):88Œ100,2015.[155]N.B.Nill.Testproceduresforverifyingimagequalityrequirementsforpersonalidentityv(PIV)singlecapturedevices.TechnicalReportMTR060170,MITRE,2006.[156]N.B.Nill.TestproceduresforverifyingIAFISimagequalityrequirementsforscannersandprintersv1.4.TechnicalReportMTR05B0016R7,MITRE,2013.[157]NIST.SummaryofresultsfromELFT07PhaseITesting.v/latent/elft07/phase1aggregate.pdf,2007.[158]U.S.DepartmentofHomelandSecurity.NewBiometricTechnologyimprovessecurityandfacilitatesU.S.entryprocessforInternationalTravelers.https://www.dhs.gov/xlibrary/assets/usvisit/usvisitedu10-consumerfriendlycontent1400words.pdf.[159]FederalBureauofInvestigation.TheScienceofFingerprints:andUses.UnitedStatesDepartmentofJustice,1984.[160]FederalBureauofInvestigation.NextGenerationhttp://www.fbi.gov/about-biometrics/ngi,2012.[161]A.Oliva,A.Torralba,M.S.Castelhano,andJ.M.Henderson.Top-downcontrolofvi-sualattentioninobjectdetection.InProceedingsofInternationalConferenceonImageProcessing,volume1,pagesIŒ253,2003.[162]N.M.Oliver,B.Rosario,andA.P.Pentland.Abayesiancomputervisionsystemformod-elinghumaninteractions.IEEETransactionsonPatternAnalysisandMachineIntelligence,22(8):831Œ843,2000.[163]PrecisionOptics.Calibrationtargets.http://www.pog.eu/en/ms/sms-calibration-targets.html.[164]S.Pankanti,S.Prabhakar,andA.K.Jain.OntheindividualityofIEEETrans-actionsonPatternAnalysisandMachineIntelligence,24(8):1010Œ1025,2002.[165]G.ParzialeandY.Chen.Advancedtechnologiesfortouchlessrecognition.InM.Tistarelli,S.Z.Li,andR.Chellappa,editors,HandbookofRemoteBiometrics,AdvancesinPatternRecognition,pages83Œ109.Springer,2009.180[166]A.A.Paulino,J.Feng,andA.K.Jain.Latentmatchingusingdescriptor-basedHoughtransform.IEEETransactionsonInformationForensicsandSecurity,8(1):31Œ45,2013.[167]G.Peyr´eandL.D.Cohen.Geodesicremeshingusingfrontpropagation.InternationalJournalofComputerVision,69(1):145Œ156,2006.[168]D.Retelny,S.Robaszkiewicz,A.To,W.Lasecki,J.Patel,N.Rahmati,T.Doshi,M.Valen-tine,andM.S.Bernstein.Expertcrowdsourcingwithteams.InACMSymposiumonUserInterfaceSoftwareandTechnology,2014.[169]R.K.Rowe,S.P.Corcoran,K.A.Nixon,andR.E.Ostrom.Multispectralimagingforbiometrics.InBiomedicalOptics,pages90Œ99,2005.[170]S.Sangiorgi,A.Manelli,T.Congiu,A.Bini,G.Pilato,M.Reguzzoni,andM.Raspanti.MicrovascularizationoftheHumanDigitasStudiedbyCorrosionCasting.JournalofAnatomy,204(2):123Œ131,2004.[171]K.Seshan.HandbookofThinFilmDeposition.WilliamAndrew,2012.[172]N.C.SicklerandS.J.Elliott.Anevaluationofimagequalityacrossanelderlypopulationvis-a-visan18-25yearoldpopulation.InInternationalCarnahanConferenceonSecurityTechnology,pages68Œ73,2005.[173]C.SuandS.N.Srihari.Generativemodelsforindividualityusingridgemodels.In19thInternationalConferenceonPatternRecognition(ICPR),pages1Œ4,2008.[174]C.SuandS.N.Srihari.ProbabilityofrandomcorrespondenceforInCompu-tationalforensics,pages55Œ66.Springer,2009.[175]T.Swain.VacuumTechnology60A&60B:ThinFilmDepositionProcesses.http://lpc1.clpccd.cc.ca.us/lpc/tswain/chapt14.pdf.[176]T.Takano,H.Masunaga,A.Fujiwara,H.Okuzaki,andT.Sasaki.PEDOTnanocrystalinhighlyconductivePEDOT:PSSpolymerMacromolecules,45(9):3859Œ3865,2012.[177]J.B.Tenenbaum,V.DeSilva,andJ.C.Langford.Aglobalgeometricframeworkfornonlineardimensionalityreduction.Science,290(5500):2319Œ2323,2000.[178]M.TicoandP.Kuosmanen.Fingerprintmatchingusinganorientation-basedminutiade-scriptor.IEEETransactionsonPatternAnalysisandMachineIntelligence,25(8):1009Œ1014,2003.[179]M.Trauring.AutomaticComparisonofFinger-ridgePatterns.Nature,197:938Œ940,1963.[180]V.V.Tuchin,A.N.Bashkatov,E.A.Genina,V.I.Kochubey,V.V.Lychagov,S.A.Portnov,N.A.Trunina,D.R.Miller,S.Cho,andH.Oh.Fingertissuemodelandbloodperfusedskintissuephantom.Proc.SPIE,7898:78980ZŒ78980ZŒ11,2011.181[181]B.T.Ulery,R.A.Hicklin,J.Buscaglia,andM.A.Roberts.Repeatabilityandreproducibil-ityofdecisionsbylatentexaminers.PloSone,7(3):e32800,2012.[182]U.UludagandA.Jain.Securingtemplate:Fuzzyvaultwithhelperdata.InInternationalConferenceonComputerVisionandPatternRecognitionWorkshop,pages163Œ169,2006.[183]M.Vatsa,R.Singh,S.Bharadwaj,H.Bhatt,andR.Mashruwala.AnalyzingofIndianpopulationusingimagequality:aUIDAIcasestudy.InInternationalWorkshoponEmergingTechniquesandChallengesforHand-BasedBiometrics,pages1Œ5,2010.[184]C.Watson,G.Fiumara,E.Tabassi,S.L.Cheng,P.Flanagan,andW.Salamon.Fin-gerprintVendorTechnologyEvaluation,NISTIR8034.http://nvlpubs.nist.gov/nistpubs/ir/2014/NIST.IR.8034.pdf,2012.[185]C.Wilson.FingerprintVendorTechnologyEvaluation2003:SummaryofResultsandAnal-ysisReport,NISTIR7123.http://fpvte.nist.gov/report/ir7123analysis.pdf,2004.[186]S.Yoon,K.Cao,E.Liu,andA.K.Jain.LFIQ:Latentimagequality.InIEEEIn-ternationalConferenceonBiometrics:Theory,ApplicationsandSystems,pages1Œ8,2013.[187]S.Yoon,J.Feng,andA.K.Jain.Onlatentenhancement.InSPIEConferenceSeries,volume7667,pages76670701Œ76670710,2010.[188]S.Yoon,J.Feng,andA.K.Jain.AlteredAnalysisanddetection.IEEETransactionsonPatternAnalysisandMachineIntelligence,34(3):451Œ464,2012.[189]S.YoonandA.K.Jain.Longitudinalstudyofrecognition.ProceedingsoftheNationalAcademyofSciences,112(28):8555Œ8560,2015.[190]S.Yoon,E.Liu,andA.K.Jain.Onlatentimagequality.InProceedingsofInternationalWorkshoponComputationalForensics,2012.[191]Z.Zhai,P.Sempolinski,D.Thain,G.Madey,D.Wei,andA.Kareem.Expert-citizenen-gineering:ﬁcrowdsourcingﬂskilledcitizens.InIEEEDigitalAvionicsSystemsConference,2011.[192]Q.Zhao,A.K.Jain,N.G.PaulterJr,andM.Taylor.Fingerprintimagesynthesisbasedonstatisticalfeaturemodels.InIEEEFifthInternationalConferenceonBiometrics:Theory,ApplicationsandSystems(BTAS),pages23Œ30,2012.[193]Y.Zhu,S.C.Dass,andA.K.Jain.Statisticalmodelsforassessingtheindividualityofgerprints.IEEETransactionsonInformationForensicsandSecurity,2(3):391Œ401,2007.182