AUTOMATICALLYADDRESSINGUNCERTAINTYINAUTONOMOUSROBOTSWITHCOMPUTATIONALEVOLUTIONByAnthonyJosephClarkADISSERTATIONSubmittedtoMichiganStateUniversityinpartialful˝llmentoftherequirementsforthedegreeofComputerctorOfPhilosophy2016ABSTRACTAUTOMATICALLYADDRESSINGUNCERTAINTYINAUTONOMOUSROBOTSWITHCOMPUTATIONALEVOLUTIONByAnthonyJosephClarkAutonomousroboticsystemsarebecomingprevalentinourdailylives.Manyrobotsarestillrestrictedtomanufacturingsettingswhereprecisionandrepetitionareparamount.However,autonomousdevicesareincreasinglybeingdesignedforapplicationssuchassearchandrescue,remotesensing,andtasksconsideredtoodangerousforpeople.Inthesecases,itiscrucialtocontinueoperationevenwhensomeunforeseenadversitydecreasesperformancelevrobotwithdiminishedperformanceisstillsuccessfulifitisabletodealwithuncertainty,whichincludesanyunexpectedchangeduetounmodeleddynamics,changingcontrolstrategies,orchangesinfunctionalityresultingfromdamageoraging.Theresearchpresentedinthisdissertationseekstoimprovesuchautonomoussystemsthroughthreeevolution-basedtechniques.First,robotsareoptimizedo˜inesothattheybestexploitavailablematerialcharacteristics,forinstance˛exiblematerials,withrespecttomultipleobjectives(e.g.,speedande˚ciency).Second,adaptivecontrollersareevolved,whichenablerobotstobetterrespondtounforeseenchangestothemselvesandtheirenviron-ments.Finally,adaptationlimitsarediscoveredusingaproposedmodediscoveryalgorithm.Oncetheboundariesofadaptationareknown,self-modelingisappliedonlinetodeterminethecurrentoperatingmodeandselect/generateanappropriatecontroller.Thesethreetechniquesworktogethertocreateaholisticmethod,whichwillenableautonomousroboticsystemstoautomaticallyhandleuncertainty.Theproposedmethodsareevaluatedusingrobotic˝shasatestplatform.Suchsystemscanbene˝tinmultiplewaysfromtheintegrationof˛exiblematerials.Moreover,robotic˝shoperateincomplex,nonlinearenvironments,enablingthoroughtestingoftheproposedmethods.CopyrightbyANTHONYJOSEPHCLARK2016ThisdissertationisdedicatedtoMegan,Trapp,andBeckett.ivACKNOWLEDGEMENTSManythankstomydoctoraladvisor,Dr.PhilipMcKinley,aswellasthemembersofmycommittee:Drs.ErikGoodman,CharlesOfria,BillPunch,andXiaoboTan.IwouldalsoliketoacknowledgethesupportandfeedbackprovidedbymycoworkersandmembersoftheBEACONCenteratMichiganStateUniversity.vTABLEOFCONTENTSLISTOFTABLES....................................ixLISTOFFIGURES...................................xChapter1Introduction................................11.1ProblemDescription................................41.2ThesisStatement..................................51.3ResearchContributions..............................51.4DocumentLayout.................................6Chapter2Background................................72.1FlexibleMaterialsinRobots...........................72.2EvolutionaryComputation............................82.2.1MultiobjectiveOptimization........................102.2.2Modeling..................................112.2.3Realitygap.................................132.3AdaptiveControl.................................142.3.1Model-basedAdaptiveControl......................162.3.2Model-freeAdaptiveControl.......................172.3.3EvolvingAdaptiveControl........................182.4ModeDiscoveryandSelf-modeling.......................192.5RoboticFishPlatform..............................21Chapter3EvolvingCaudalFinMorphologyandControlPatterns....243.1Methods......................................243.1.13D-PrintedFins..............................273.1.2OscillatoryControllers...........................273.1.3GeneticAlgorithm.............................293.2EvolvingFinMorphology............................303.2.1MathematicalModelValidation......................303.2.2PhysicalValidation.............................333.2.3EvolutionaryOptimization........................353.3EvolvingFinMorphologyandControlPatterns................383.3.1FixedControl,EvolvedMorphology...................393.3.2EvolvedControl,FixedMorphology...................403.3.3EvolutionofControlandMorphology..................423.3.4EvolutionofTurning............................443.3.5EvolutionofVelocityandTurning....................443.4FinsWithNonuniformFlexibilityandNon-RectangularShapes.......463.5Conclusions....................................51Chapter4EvolvingSwimmingPerformancewithMechanicalE˚ciency.534.1ModelingandSimulation.............................54vi4.1.1EvolutionaryOptimization........................554.1.2Constraints.................................564.1.3FitnessEvaluation.............................574.2FinFabricationandTesting...........................584.3ExperimentsandResults.............................614.4PhysicalValidation................................674.5Conclusion.....................................69Chapter5EvolvingSwimmingPerformancewithElectricalE˚ciency..715.1DesignofaSmallRoboticFish.........................725.1.1RoboticFish................................735.1.2BodyandFinFabrication.........................735.1.3CustomPCB................................745.1.4ElectromagneticActuator.........................745.1.5ControlSignal...............................765.1.6EvolutionaryOptimization........................775.1.7NSGA-IICon˝guration..........................785.1.8FinConstraint...............................795.1.9FitnessEvaluation.............................795.2EMOSimulationResults.............................805.3PhysicalValidation................................885.4Conclusion.....................................89Chapter6EvolvingAdaptiveControl.......................916.1Model-freeAdaptiveControl...........................926.2Methods......................................966.2.1MFACforRoboticFish..........................966.2.2SimulationDynamics...........................1006.2.3Di˙erentialEvolution...........................1006.3Single-EvaluationExperimentsandResults...................1016.4Multi-EvaluationExperimentsandResults...................1036.5Conclusions....................................109Chapter7DiscoveringAdaptationBoundaries.................1117.1EvolvingaBaseMorphology...........................1127.2ModeDiscoveryAlgorithm............................1137.2.1ScenarioParameters............................1147.2.2DeterminingScenarioFeasibility.....................1157.2.3EvolvableParameters...........................1177.2.4Algorithm..................................1177.3ExperimentsandResults.............................1197.3.1BoundaryScenarioSelection.......................1197.3.2SimultaneousParameterSweep......................1217.3.3Volume-BasedScenarioSelection.....................1227.4Self-Modeling...................................125vii7.4.1TheAquaticRobot............................1267.4.2Self-ModelingAlgorithm..........................1277.4.3InferenceTechnique............................1287.4.4ExperimentalResults...........................1297.5Discussion.....................................131Chapter8Conclusions................................1328.1O˜ineOptimizationofMorphologyandControl................1328.2MultiobjectiveOptimization............................1338.3EnhancingAdaptiveControl...........................1338.4ModeDiscovery...................................1348.5PotentialforFutureInvestigations........................135BIBLIOGRAPHY....................................137viiiLISTOFTABLESTable3.1:Fixed-ControllerExperimentalResults.................39Table3.2:SpeedComparisonAmongExperiments.................48Table4.1:RangeofEvolvedParameters......................56Table4.2:Simulation-RealitySpeedComparison..................68Table5.1:RangeofEvolvedParameters.......................78Table5.2:ParameterSetsSelectedforFurtherInvestigation............85Table5.3:ComparisonBetweenSpeedsAcquiredinSimulationandAttainedbytheRobotinaWaterTank........................88Table6.1:EvolutionaryRangeofMFACParameters,asWellastheTypicalValues.101Table6.2:FinCharacteristicsforthe9-EvaluationsExperiment..........104Table7.1:EvolvableMFACParameters.......................117Table7.2:BoundaryScenarios(FinParameters)..................120Table7.3:VolumeScenarioLimits..........................122Table7.4:MFACPerformanceComparison.....................125ixLISTOFFIGURESFigure2.1:Ablockdiagramforafeedbackcontrolsystem.Theshadedblockshowstheadditionalcomponentsnecessaryforadaptivecontrol.Thesignalsr,x,y,andearethereferencesignal(desiredoutput),thecurrentoutputofthesystem,themeasuredoutputofthesystem(includinganymeasurementerrorduetod),andtheerrorbetweentheinputandthemeasuredoutput,respectively,whileuisthecontrolsignal.Allofthesesignalsvaryasafunctionoftime..................15Figure2.2:AgraphicalrepresentationoftheMRACmethod.Areferenceinputrisfedtoboththereferencemodelandtheadaptivecontroller.Thecontrollerproducesanoutputsignaluandisadaptedusing,whichisbasedontheerrorebetweenthereferencemodelandthecontrolledsystemoutputs,ypandyrespectively...................16Figure2.3:AgraphicalrepresentationoftheMFACANN.Acontinuoustimeerrorsignaleis˝rstnormalized(viatheNrmblock)andthenpropagatedthroughtheinputneuronsIi.TheANNisthenactivatedasafeed-forwardnetworktoproduceanoutputV.The˝nalcontrolleroutputuisanampli˝edsummationofVande.................18Figure3.1:Graphicalrepresentationofthesimulatedhydrodynamics.Linearve-locityvandangularvelocitywaretheresultofthrustforceFT,dragforceFD,liftforceFL,anddragmomentMD.FTiscalculatedasthesumofallforcesactingonthe˝nsegments...............25Figure3.2:(a)Therobotic˝shprototype.(b)Depictionofthevirtual˝shmodelwithathree-segmentrigid-bodycaudal˝n.Movementofthe3D-printedrectangularcaudal˝nisaccomplishedusingaservomotorwithasetrangeofmotionandperiodofoscillation..............26Figure3.3:TheMNOcomprisestwomutually-inhibitingneurons.TheoutputofthetheMNOisequaltotheoutputofneuron1minustheoutputofneuron2(Adaptedfrom[89]).......................29Figure3.4:Predictedvelocitiesfordi˙erentYoung'sModulusvaluesfromthemathematicalmodelcalculations.Notethatthisassumesthatthebodyisanchored..............................32Figure3.5:Resultsofthehillclimberandevolutionaryrunsfordeterminingtheoptimumsti˙nessofa˝xeddimension˝n.Bothmethodsconvergedonacommonsti˙nessyieldingthehighestaveragevelocity.Darkershadesindicateclusteredresultsfromdi˙erenttrials..........33xFigure3.6:Observedaveragevelocityfordi˙erentmaterialsusedinprinted˝ns.Sti˙nessincreasesfromlefttorightintheplot..............34Figure3.7:Visualperformanceoftheevolved˛exible˝ninsimulation(left)versusafabricated˛exible˝ntestedontheprototyperobot(right)......35Figure3.8:Visualizationofthe˝tnesslandscapefordi˙erentshapeandsti˙ness˝ns.Notethatheightisdependentuponlengthindeterminingshape,therefore,heighthasbeenomittedfromthedata.Asthelengthofthe˝nincreases,theYoung'sModulusincreasesaswelltomaintainsimilarsti˙ness˝nsfordi˙erentlengths......................37Figure3.9:Anexampleofacontrollersignalthatisbeyondthecapabilitiesofthesimulatedmotor.Thesolidredlinerepresentsthesignalgeneratedbythe0.9Hzsinusoidalcontroller,andthedashedbluelinethesimulatedmotorhinge.................................40Figure3.10:BoxplotsforeachMNOparameterfromthebestperformingMNOsofeachof30independentruns........................41Figure3.11:Thefrequency-domainresponseoftheoverallbestsinusoidal(A)andMNOcontrollers(B)............................42Figure3.12:Pathsoftherobotic˝shwhilebeingcontrolledbydi˙erentoscilla-torycontrollersandhavingdi˙erent˛exibilities.Note:plotsoverlapsigni˝cantlyattheorigin..........................43Figure3.13:Evolvedturningbehavior:(a)pathtakenbythebestsolution,and(b)controlsignalandresultingservoangle..................45Figure3.14:Thepathsproducedbytwoevolvedcontrollers:in(a)˝n˛exibilityismoreoptimalforturningratherthanvelocity,andin(b)˛exibilityismoreconducivetoachievinglinearvelocity................46Figure3.15:Diagramforirregular˝ns.Eachsegmenthasauniquelengthandheight,andthejointbetweeneachsegmenthasitsownvaluefor˛exibility.47Figure3.16:Boxplotsforthebaselineexperiment.Parameters(fromleft-to-right)includeL1(thelengthofeachsegment),D1(thedepthofeachseg-ment),E1(˛exibilityateachjoint),andthecontrolpattern'sampli-tudeandfrequency.ThemorphologyparametersareasdepictedinFigure3.15.................................48xiFigure3.17:Boxplotsforthelengthexperiment(a),thedepthexperiment(b),andthe˛exibilityexperiment(c).Foreachsetofboxplots,ifaparametervalueisunspeci˝ed(forexample,L2in(b))thenallsegmentsorjointssharethesamevalue............................49Figure3.18:Boxplotsfortheirregularexperiment.Forthisexperimentallpossibleparametersaresubjecttoevolution....................50Figure4.1:(top)Therobotic˝shprototypeusedinthisstudy,and(bottom)thevirtualrepresentationevolvedduringsimulation.............55Figure4.2:Anillustrationofthedynamicsinvolvedincalculating˝tness.Thepathoftherobotic˝shincludestwoparts:alight-bluesegmentfrom0to5seconds,whichdoesnotdirectlya˙ect˝tness,andadark-bluesegmentfrom5to10secondsusedtoevaluate˝tness.Thepathsettlestoanaverageheading,whichisnotinlinewiththeX-axis,duetoabiascausedbytheinitialrotationofthecaudal˝n.Thedashed,orangeline(d5to10s)representsdisplacementoverthe˝nal5secondsofsimulation..................................58Figure4.3:(a)Diagramofacompositematerialfora3D-printed˛exiblecaudal˝n(note:thecaudal˝nwouldbeonitsside).(b)Top-viewphotographofa3D-printedcaudal˝nwithaninnerthicknessof0.38mm.Theoverallthicknessisaconstant1.2mm.Thee˙ectiveYoung'smodu-lusvalueforthecompositematerialdependsontherelativethickness(tinner)oftheinnerVeroWhitePluslayerwithrespecttothetwo˛exibleTangoBlackPlusouterlayers........................59Figure4.4:Testingofphysical˝ns.(a)Diagramoftheexperimentshowingthetestingprocess.(b)Photographoftheexperimentalsetupformeasur-ingthee˙ectiveYoung'smodulusof3D-printedcompositematerials.60Figure4.5:E˙ectiveYoung'smodulusofcompositematerialsfordi˙erentvaluesoftinner...................................61Figure4.6:AcombinedParetofrontincludingthebestsolutionsfromeachofthe20replicateevolutionarysimulations.Thelabeled,redsymbolsdenotesolutionsthatwerephysicallyfabricatedandvalidated.....62Figure4.7:Thecompleteevolutionaryhistoryforeachofthe20replicateexperi-ments.Everyfeasible,evolvedsolutionisplotted,andthePareto-frontishighlightedindarkblue.........................63xiiFigure4.8:Plotsofevolvedsolutionsshowingrelationshipsbetween˝nlength(x-axis)andthetwoobjectives,speed(top)ande˚ciency(bottom).AllPareto-optimalsolutionshavea˝nlengthbetween6to12cm,thusthex-axisdoesnotincludetheentireevolvablerange(3to15cm).Thestraight,orangelinesindicatethebest-˝tforeachsetofvalues.....64Figure4.9:Theevolvedparametersscaledbetween0and1forthecombinedParetofrontsolutions................................65Figure4.10:Comparisonbetweenthemotionofacaudal˝nforphysicalandvirtualexperiments.Toincreasevisibility,apurplelinetracesthelengthofthecaudal˝nforthephysicaldevice.Thedashed,blackreferencelinesprovideacommonanglewithwhichtheside-by-sideimagescanbecompared................................68Figure5.1:(a)Smallrobotic˝shcastfromliquidrubber;thebodyhasbeenpaintedgrayandtheprintedcaudal˝nsaredetachable.(b)Thede-vice'scustomPCBandrechargeablebattery...............72Figure5.2:(a)ASolidworksmodeloftherobotic˝shmold.(b,c)Twoimagesofthe3D-printed,clearplasticmoldusedduringthecastingprocess...73Figure5.3:Aphotographofseveral3D-printedcaudal˝ns.Each˝nhasdi˙erentmorphologicalcharacteristics:length,height,and˛exibility.......74Figure5.4:(a)Top-viewdiagramoftheactuator,and(b,c)photographsoftheelectromagneticactuator..........................75Figure5.5:Anexampleofthecontrolvoltagesignalwiththeresultingsimulatedmotorangle.Thedashed,bluelinedepictsthecontrolsignalwithafrequencyof0.5HzandaPWRof0.4(PWRindicatesthefractionoftimespentateither3.3or-3.3V).Thesolid,orangelineistheangleofthemotor,whichhasarisetimerelatedtotheappliedtorque,andafalltimerelatingtothecenteringtorque,duetothecenteringpermanentmagnet.............................77Figure5.6:Theevolutionaryhistoryofasinglereplicatesimulationrun.Everyindividual(includingdominatedindividuals)inthepopulationatagivengenerationisplottedineach˝gure.Forall˝gures,theunitsforspeedandpowerarecm/sandmW,respectively.............80xiiiFigure5.7:(a)Everyevolvedindividualamongall25replicatesimulations(exclud-inginfeasiblesolutions).Graymarkersdenoteindividualsandnon-graymarkersdenoteindividualsthatwerePareto-optimalforagivenreplicateexperiment.Threeclustershavebeenidenti˝edandgivendi˙erentcolors.Theremainingplotsdisplayindividualsbelong-ingtotheenergycluster(b),speedcluster(c),andlocal-optimumcluster(d).Pleasenotethattheclusterplotsdonotsharethesamerangesfortheiraxes............................81Figure5.8:Box-plots(toprow)andhistograms(bottomrow)ofthedistributionsofeachevolvedparameter,scaledbetween0and1,forallofthePareto-optimalindividuals.Foreachbox-plot,thecentralredlineindicatesthemedian,thelightblueboxoutlinesthe25ththrough75thper-centiles,thedashedlightblueverticallinesdenotetheboxwhiskers(non-outliers),andthedarkbluecirclesdenoteoutliers.Ineachhis-togram,thehorizontalaxesarescaledparametervaluesandtheheightofeachbinindicatesthedensityaroundagivenvalue..........82Figure5.9:AcomparisonofcontrolparametersamongthethreePareto-optimalclusters.Frequency(right)andthePWR(left)arebothincreasedtoachievefasterswimmingspeedsanddecreasedtoattainlowerpowerconsumption.Foreachbox-plot,acentralredlineindicatestheme-dian,aboxoutlinesthe25ththrough75thpercentiles,avertical,dashedlinedenotetheboxwhiskers(non-outliers),andredmarkersdenoteoutliers...............................84Figure5.10:Controlparametersweepsfortwosetsofparameters:evolvedindivid-ualswithhighspeed(a,b)andwithlowenergyconsumption(c,d)...86Figure5.11:Morphologicalparametersweepsfortwosetsofparameters:(a,b)anevolvedindividualwithhighspeed,and(c,d)anevolvedindividualwithlowenergyconsumption.......................87Figure6.1:AgraphicalrepresentationoftheMFACANN.Acontinuoustimeerrorsignaleis˝rstnormalized(Nrmblock)andthenpropagatedthroughtheinputneuronsIi.TheANNisthenactivatedasafeed-forwardnetworktoproduceanoutputV.The˝nalcontrolleroutputuisanampli˝edsummationofVande......................93Figure6.2:Examplesofarobotic˝shcontrollertrackinganinputreferencesignalrepresentingdesiredspeed.Ineachplot,thedashed,blacklinedenotetheinputreferencesignal(r),thebluelinerepresentsthemeasuredsystemsoutput(y),andtheredlinerepresentstheerror(e)betweenthesetwosignals.Plot(a)isanexampleofrelativelypoortracking,whereasplot(b)exhibitse˙ectivetracking................97xivFigure6.3:AblockdiagramoftheMFACcontrollerandtherobotic˝sh.Signalsrandydenotethereferenceandmeasuredspeeds,respectively,eisthedi˙erencebetweenreferenceandmeasuredspeeds,anduisthecontrolleroutput..............................98Figure6.4:Inboth(a)and(b),the˝nsu˙ersdamageatt=30s.In(a)therobotic˝shiscontrolledbyanon-adaptivecontroller,andin(b)therobotiscontrolledbyanadaptivecontroller.Theadaptivecontrollerisabletosuccessfullyregaintheabilitytotrack..................99Figure6.5:ResultsforanMFACcontrollerwithtypicalparameterscontrollingarobotic˝sh.Thedashedorangelinedenotesthereferencespeedr,theactualspeedyoftherobotic˝shistheblueline,andtheerrorebetweenthesesignalsisred........................102Figure6.6:Resultsfortheoverallbest(acrossallreplicateexperiments)single-evaluationsolutionsimulatedwithdefault˝ncharacteristicsandthesamereferencesignalutilizedduringevolution.Thecontrollershowspoorperformancestartingatthe80secondmark,andsimilarresultswerefoundinallreplicateexperiments..................103Figure6.7:Theoverallbest9-evaluationsolutionevaluatedonsim1.TheMFACcontrollerisabletodrivetherobotic˝shatthedesiredreferencespeed(r)......................................104Figure6.8:Controlsignaluandtheresultingmotoranglefortheoverallbest9-evaluationsolutionevaluatedonsim1.Thecontrolsignaltrajectoryroughlyfollowsthereferencesignal....................105Figure6.9:Theoverallbest9-evaluationsolutiontestedagainst˝nlengthsthatwerenotencounteredduringanyoftheevolutionarysimulations.In(a)˝nlengthisshortenedto80%ofthedefaultlength,andin(b)˝nlengthislengthenedto120%ofthedefaultlength.Inbothcases,theevolvedcontrollerisabletoadapttoanovel˝nlength.........105Figure6.10:Theoverallbest9-evaluationsolutionevaluatedwitha˝nthatis150%ofthedefault˝n˛exibility.Evolvedcontrollerswereabletoadapttothisnovelvaluefor˛exibility.......................106Figure6.11:Theoverallbest9-evaluationsolutionsimulatedwithdefault˝nchar-acteristicsandareferencesignalnotencounteredduringevolution...107xvFigure6.12:Theoverallbest9-evaluationsolutiontestedagainst˝nlengthsand˝n˛exibilitiesthatwerenotencounteredduringevolutionarysimulations.In(a)˝nlengthisshortenedto80%ofthedefaultlengthandthe˛exibilityisincreasedto120%ofthedefault,andin(b)˝nlengthislengthenedto120%ofthedefaultlengthand˝n˛exibilityisreducedto80%ofthedefaultvalue........................108Figure6.13:Performanceofthebestevolvedsolutionfromthealtered9-evaluationsexperimentstestedwitha˝nlength120%ofthedefault........108Figure6.14:Speedvs.oscillatingfrequencyforseveraldi˙erent˝ncharacteristics.Forcertainconditionsincreasingthefrequencyresultsinslowerspeeds.109Figure7.1:(a)Renderingofthesimulatedrobotic˝sh.Individual˝nsegmentsappearindi˙erentcolors,andthe˝nappearstoextendabovethesurfaceofwaterforvisualizationpurposesonly.(b)Aprototype3D-printedrobotic˝shwitha˛exiblecaudal˝n(topcoverremovedforillustration).................................113Figure7.2:Evolutionofbasemorphology:(a)Meanvaluesforthebestperformersacrossall30replicateevolutionaryruns.Theshadedregionrepresentsthe95%con˝denceinterval.(b)Box-plotsofdistributionsofthe˝nalevolvedparameters,acrossthereplicateruns.Themedianvaluesarerepresentedbyahorizontalredline,blueboxesrepresentonestandarddeviationeithersideofthemean(theblueboxesaresometimescoveredbythemedianline),andblackcirclesdenoteoutliers..........114Figure7.3:Referencesignalscenarioparametersincludevaluestodescribefourtimesegments.Inthe˝rstsegment(fromt=0tot=t1)speedrampsfrom0toS1,inthesecondsegmentthereferencespeedremainssteadyatS1.Inthethirdsegment(fromt=t2tot=t3)thespeedramps(upordown)tothe˝nalspeed,whichisheldsteadyduringthe˝naltimesegment...................................115Figure7.4:Threebehaviorsareshown.Theblueandgreenlinesdenotesystemsthataredirectandreverseacting,respectively.Theredlineshowsasystemthatswitchesactingmodesandisdeemedinfeasible......116Figure7.5:FlowchartoftheModeDiscoveryAlgorithmusedtodiscoverexecutionmodeboundariesandproduceaMFACparametervaluesforthatmode.118Figure7.6:Boundaryscenariovaluesforthethree˝nparameters..........119xviFigure7.7:Parametersweepplots:(a)lengthvs.depth;(b)lengthvs.˛exibility(c)depthvs.˛exibility.Theredboxesdenotethelimitationsofadapt-abilityfoundbytheboundaryselectionmethod,andtheblacklinein(a)correspondstothelength-widthlimitationofthesimulationmodel.121Figure7.8:Plotsformodediscoveryusingvolumeselection:(a)lengthvs.depth;(b)lengthvs.˛exibility;(c)depthvs.˛exibility.............123Figure7.9:AnexampleofanevolvedMFACadaptingtosuddendamage.At60seconds,eachofthe˝nmorphologyparametersareabruptlychanged.124Figure7.10:Modelingandfabricationofanaquaticrobot[92]:(a)simulationmodelinOpenDynamicsEngine(ODE);(b)correspondingSolidWorksmodelforfabricatingprototype;(c)3D-printedpassivecomponentsofproto-type;(d)integrationofelectroniccomponentsandbatteryintotheprototype;(e)assembled,paintedandwaterproofedprototypeinanelliptical˛owtank.Themainbodyofthephysicalprototypeis13cmlongand8cmindiameterwithpectoral˛ippersthatare8cmlongand2cmwide..................................126Figure7.11:BasicoperationoftheEstimation-ExplorationAlgorithm[11].....127Figure7.12:Simulatedrobotsfromthecasestudy:(a)damagedtargetrobotwithapectoral˝nhalfthelengthofanormalone;(b)morphologicalmodelproducedusingtheunmodi˝edEEAapproach;(c)morphologi-calmodelproducedbytheInference-basedEEAapproach.......130xviiChapter1IntroductionIncreasingly,roboticdevicesarebeingdeployedinuncontrolled,unstructuredenviron-ments.Unlikeindustrialmanufacturingsettings,whererobotsoftenperformasinglerepet-itivetask,autonomousrobotsmustoperateinremote,hazardous,andhighlydynamicenvi-ronmentsinwhichtheyareexpectedtoperformmultiplecomplextasks.Examplesincludesearchandrescue,remotesensing,packagedelivery,andassistinghumansindangerousoc-cupationssuchasminingand˝re˝ghting.Withthisincreaseinrequiredautonomyandcom-plexitycomesanincreaseindesigndi˚culty.Accordingly,inthepastdecadetheengineeringcommunityhasseenmanyadvancesincyber-physicaltechnologies,includingMEMS-basedphysicalsensing(accelerometers,gyroscopes,etc.),rechargeablebatteries(e.g.,lithiumpoly-mer),andincreasedcomputingpowerofembeddedsystems.Allofthesetechnologieshavebecomereadilyavailableascommodityhardwareforresearchers,hobbyists,andstudents.Despitetheseadvances,however,mostautonomoussystemsstilllacktheabilitiesex-hibitedbyevensimplebiologicalorganisms.First,thematerials,sensors,andactuatorscomprisingroboticsystemsarenotase˙ectiveasbiologicaltissue,andsecond,thecon-trolsystemsgoverningroboticsystemsdonotyetexhibittheadaptabilityandrobustnessofbiologicalsystems.Inthisdissertation,wedevelopandapplyevolutionarycomputation(EC)methodstoimprovesuchautonomousroboticsystems.ECcodi˝esthebasicprinci-1plesofgeneticevolutionincomputersoftware[35].Theopen-endednatureofevolutionaryalgorithmshasbeenshowntodiscovernovel,oftencounter-intuitive,solutionstocomplexengineeringproblems[19],sometimessurpassinghumandesigners[1].Oneapproachtomimickingthe˛uidnatureofnaturalorganismsinroboticsystemsistointegratesoft,˛exiblematerialsintorobots[126,15].Inthiscase,˛exiblecomponentsareintendedtopartiallycompensatefortherelativelyprimitiveactuationcapabilitiesofrobotscomparedtobiologicalorganisms.Anexampleistheactionexhibitedbythe˛exible˝nofarobotic˝sh[26],whichcanincreasethrustcomparedtothatgeneratedbyarigid˝n.However,theintegrationof˛exiblematerialscomesatthecostofincreaseddesigncomplexity,speci˝callyindiscoveringhowtobestexploittheirproperties.Forexample,howshouldthe˛exible˝nbeactuated/controlledtotakeadvantageofitsspring-likena-ture?Evolutionarymethodshavebeenshowntoworkwellwithsuchproblems,andproduceafavorablematchingbetweenmorphology(i.e.,thephysicalcomponentsofarobot)andcontrol[93].Amajorityofevolution-basedroboticsstudies,referredtoasevolutionaryrobotics(ER),focusonasingleobjective;forexample,theobjectiveisoftentomoveasfastaspossible.Inpracticehowever,autonomousrobotsoftenhavelimitedenergycapacity.Thismeansthatifthemoste˙ectivewaytomovefastconsumesalotofpowertherobotwillsoonbeunabletomoveatall.Forthisreason,itcanbebene˝cialtoconsiderasecondobjectivesuchaspowere˚ciency.Arobotcanthenbeoptimizedwithamultiobjectiveapproachinorderto,forexample,bothmovefastandbee˚cient.Multiobjectivetechniquesareparticularlyhelpfulwhentwoormoreoftheobjectivesareincompetition,meaningthattoimprovethe˝rstobjective,thesecondobjectivemustbeperformedatadeterioratedlevel.Insteadofasingleoptimalsolution,evolutionarymultiobjectivealgorithms(EMOs)˝ndasetofPareto-optimalsolutionswhereeachindividualcanbeconsideredoptimaldependingonhowmuchweightisgiventoeachobjective[37].Moreover,theinformationattainedfromanEMOcanbeutilizedbotho˜ineduringdesignandon-boardduringruntime.Thisenables2asystemtoadjusttherelativeimportanceofcompetingobjectivesasitsinternalstateanditsenvironmentevolvethroughtime.Althoughevolutionarymethodsoften˝nde˙ective,ifnotoptimal,designsforroboticsystems,thesystemsthemselvescanchangeovertime.Forexample,motorperformancemaydegradeorthecharacteristicsof˛exiblematerialsmaychangeasacomponentages.Thus,itisunlikelythatanystaticcontrollerwillremaine˙ectiveforthelifeofanautonomoussystem.The˝eldofcontroltheoryhasproducedasetoftechniquesspeci˝callyaimedatalleviatingthisissueofchangingdynamics.Adaptivecontrolmethodscombineparameteres-timationwithcontrollawinordertocontrolsystemswithdynamicsthatareeitherunknownorchangingunpredictably[63].Traditionaladaptivecontrolreliesonsu˚cientlyaccuratemodelsofthetargetsystem(andmodelingthedesiredresponseforthesystem),allowingforvariationsintheunderlyingdynamicsthroughasetofadaptivelaws.Overthepastsixtyyearssuchadaptivecontrolmethodshaveproventobee˙ective,particularlyinthedomainsofaircraftcontrol,powersystems,andprocesscontrol[63,96].Morerecently,however,researchershavestartedtodevelopandre˝nemodel-freeanddata-driventechniques,whichdonotrelyonthedevelopmentofaspeci˝cdynamicmodel,butratheronlyoninput-outputdata[59].Thesetechniquesstillrequiresigni˝cantdomainknowledgeanddesignexpertise.Inthisdissertation,however,weuseevolutionarymethodstodevelopadaptivecontrollersandenhancetheiradaptability.Adaptivecontrollersenablecyber-physicalsystems,suchasautonomousrobots,toman-ageuncertainconditionsduringexecution.However,thereisalimittotherangeofconditionsthatcanbehandledbyagivencontroller.Whenthislimitisexceeded,acontrollermightfailtorespondasexpected,notonlyrenderingitine˙ectivebutpossiblyputtingtheentiresystematrisk.Forthisreason,itisimportanttodiscovertheboundariesofagivenadap-tivecontroller.Collectively,theseboundariesde˝neanexecutionmodeforthatcontroller.Explicitspeci˝cationofmodeboundariesfacilitatesthedevelopmentofdecisionlogicthatdetermines,basedonsystemstateandsensedconditions,whentoswitchtoadi˙erentexecu-3tionmode(andtypicallyadi˙erentcontroller),suchasoneforprovidingfail-safeoperation.Oncemodeboundarieshavebeendetermined,however,itisstillanopenquestionastohowaroboticsystemshoulddetermineitscurrentmode.Howdoesthesystemidentifycompen-satorybehaviors?Oneapproachtohandlingsuchacaseisforthesystemtodevelopanewrepresentationofitsprincipledynamics.Suchtechniquesarereferredtoasmodel-learningorself-modelingmethods.Thesetechniquesareintendedtoenableautonomousrobotstoau-tomaticallygeneratemodelsbasedsolelyoninformationextractedfromtheaccessibledatastreams(i.e.,datafromsensors)[97].Model-learningtechniquescommonlyusestandardregressiontechniquesincludingleast-squares[79].However,Bongardetal.havealsoshownthatanon-boardevolutionaryalgorithmiscapableofdiscoveringaself-modelthroughaseriesofevolvedactionsandmeasuredresponses[12].Ingeneral,theideaisto˝rstlearnthebehaviorofthesystemfromobservations,andthendeterminehowtomanipulatethesystem(viathenewlygeneratedself-model)toachieveadesiredresponse[97].1.1ProblemDescription.Thisdissertationaddressesthreeissuesofincreasingcomplexity.(1)Howcanthebody-planandcontrolpatternofroboticsystemsbesimultaneouslyoptimizedwhiletakingintoaccount˛exiblecomponents?Itisadvantageoustouseanoptimizationprocessthatau-tomaticallymatchesmorphologywithcontrolpatterns.(2)Howcanroboticsystemsbeoptimizedwithrespecttomultiple,competingobjectives?Mostcomplexsystemsmustneces-sarilybalancetheimportanceofmultipleobjectives(e.g.,speedandenergyusage).(3)Howcanautonomousroboticsystemsadapttomodelinginaccuraciesandtosubtlevariationsinphysicaldynamics,andthenuseknowledgeofadaptiveboundariestoswitchto(orgeneratenew)behaviorsviaaself-modelingprocess?Anyo˜ine(simulation-based)optimizationwillinfuseerrorduetomodelinginaccuracies,andfurthermore,degradationofmaterialsandmotorscancauseconventional(nonadaptive)controllerstodetune.Beyondadaptingto4subtlechanges,anautonomoussystemmayencountermorecatastrophicchanges,suchasphysicaldamageandmotorfailure,thatrequiremoredrasticresponse.Suchmeasures,whileperhapsnotrestoringcompletefunctionality,shouldatleastenabletherobottoreturntoarepairstation.1.2ThesisStatement.Employingevolutionaryoptimizationmethodscanenableautonomousroboticsystemstoleveragethebene˝tsof˛exiblecomponentswithrespecttocontrolpatternsandmultipleobjectives,automaticallyadapttosubtlechangesinoperatingdynamics,andovercomemoresigni˝cantchangesthatmightotherwisecauseacompletefailure.Themethodsdevelopedinthisresearchareappliedtorobotic˝sh.Robotic˝shbene˝tfromtheuseof˛exiblecomponents,andtheyoperateinahighlynonlinearenvironmentthatiswellsuitedfortestingadaptivecontrol.Furthermore,self-modelingandself-healingtechniquescanbetestedbychangingthecaudal˝nactuator(i.e.,thetail˝n)orstructureofthecaudal˝nitself.1.3ResearchContributions.Thebroadcontributionofourresearchisaholisticmethodfordevelopingautonomoussystemscapableofmitigatingtheuncertaintyofanunstructuredenvironment.Morespecif-ically,thisresearchhasproducedresultsregardingthedesignofcyber-physicalsystems,includingtheco-developmentofrobustembeddedcomputingsystemsandrobotmorpholo-gieswith˛exiblecomponents.Thisworkcanbebrokenintothreeparts.First,weleverageddynamicmodelsof˛exiblecomponentssuchthatmaterialpropertieswereoptimizedinsim-ulation.Thegoalsweretoincreasethemaneuverability,speedandagility,ofautonomousrobotswhileconsideringbothmorphologyandcontrolpatterns.Second,weexploredopti-mizationprocessesthatbalancetrade-o˙samongthecompetingobjectivesofperformance5ande˚ciency.Finally,weenhancedcontrolsystemscapableofadaptingtochangesinen-vironmentalconditionsandtotheelectricalandmechanicalcomponents;anexampleofthelatterisdegradingperformanceof˛exiblecomponents.Duringthisprocess,wealsode-velopedmethodsfordeterminingmodeboundariesofadaptivebehaviors,real-timefailurediagnosis,andswitchingtocompensatorybehaviors.1.4DocumentLayout.Theremainingchaptersareorganizedasfollows:InChapter2wecovertherelevantbackgroundandrelatedmaterials.Chapter3describesourinvestigationsinwhichmorphol-ogyandcontrolpatternsaresimultaneouslyevolved.Chapters4and5discussourresearchinvolvingmultiobjectiveevolution.Likewise,inChapters6and7wedescribestudiesin-volvingadaptivecontrol,modediscovery,andself-modeling.Finally,Chapter8containsconclusionsforthisdocument.6Chapter2BackgroundResearchintoautonomousroboticsystemsrequiresmodeling,simulation,anddesignofbothcontrolandmorphology.Inadditiontothesetopics,ourresearchincludesintegrating˛exiblematerialsandtheuseofevolutionaryoptimization.Moreover,toproducesystemscapableofhandlingadverseconditionswemakeuseofadaptivecontrol(includingthedis-coveryofboundariesofadaptation),self-modeling,andself-healingtechniques.Thekeytheoriesandmethodsrelatedtotheaforementionedtopicsareintroducedinthischapter.2.1FlexibleMaterialsinRobotsAlthoughmanyrecentrobotdesignsaresaidtobebio-inspiredorbiomimetic,manyoftheseelectro-mechanicalsystemsfailtoreachlevelsofperformance,adaptability,androbustnesscomparabletothoseoftheirnaturalcounterparts.Onereasonisthatmostroboticsystemscompriseonlyhard,rigidmaterials,whilenaturalorganismsincludesofttissuesthatenhanceperformanceandinteractionwiththeenvironment.Integrating˛exiblecomponentsintorobotmorphologiesisonemethodforbridgingthisperformancegapbetweenarti˝cialandnaturalcreatures.Withtheproliferationof3Dprinters,thedesign,rapidfabrication,andtestingof˛exiblecomponentshasbecomemucheasier.Forexample,Richteretal.havefabricatedwingsofa7roboticdragon˛ycomposedofboth˛exibleandrigidcomponents[108],andinourworkwehaveshownthat,whencomparedtoarigid˝n,apassively˛exiblecaudal˝ncanimprovethrustofarobotic˝sh[26].Bothofthesestudiesrelyon3Dprintersforconstructionofroboticdevices.Asidefromautonomousrobots,˛exibleandcompliantmaterialscanbene˝tapplicationdomainssuchasthedesignofprostheses,safetyinrobot-humaninteraction,andautomatedmanufacturing[101,15].Despitethemanyadvantagesofincorporating˛exiblecomponentsintorobots,theydocomeatthecostofincreaseddesigncomplexityandincreaseddesigne˙ortforthecontrolschemesgoverningtheirmotion.Thisisparticularlyimportantifthecontrollerisrequiredtohandlechangesin˛exibilityovertime.Forexample,thecharacteristicsofa˛exible˝nmaychangedependingonwatertemperatureandthee˙ectsofmaterialaging.Insuchcases,itmightbeappropriatetouseanautomateddesignprocesscapableoftestingmanydi˙erentparametercombinations.However,designingaroboticsysteminsuchawaycanleadtoaverylargesearchossiblytoomanyparametercombinationstoperformabrute-force,exhaustivesearch.Likewise,withcomplexparameterinteractionsasimplegradient-basedsearchalgorithmislikelyto˝ndonlylocaloptima.Intheresearchpresentedinthisdissertation,weapplyevolutionarysearchtechniques,whichareknownfortheirabilitytowidelysearchthespaceofpossibleanswersaswellas˝ndnovelsolutionsbeyondthepreconceptionsofhumandesigners[57,50,36].2.2EvolutionaryComputationEvolutionarycomputation(EC)methodshaveproventobee˙ectiveforavarietyofproblems[77].Themostwell-knownvariantisthegeneticalgorithm(GA)[57],astochasticsearchtechniqueinwhichapopulationevolvestowardasolutiontoanoptimizationproblem.Thegenomeofeachindividualinthepopulationisanencodingofacandidatesolution.Inoptimizingarobot,forexample,thegenomemightincludevaluesfortherobot'scontroller8aswellascharacteristicsoftherobot'smorphology(e.g.,oscillatingfrequencyofthemotorandthedimensionsofthe˝n).Aninitialpopulationofindividualsiseithergeneratedran-domlyorseededaroundapointofinterest,suchasasolutiontotheproblemthatisknowntobee˙ective.Genomeperformanceisquanti˝edviaa˝tnessfunction,andevolutionaryoperatorssuchascrossoverandmutationareappliedtohigh-performinggenomes,producinganewgenerationofpossiblesolutions.Thisprocedureisrepeateduntilamaximumnumberofgenerationsisreachedoruntilanoptimal(oradequate)solutionhasbeenfound.Geneticprogramming(GP)[50,76]isarelatedmethodwheretheindividualsareactualcomputerprograms.BothGAsandGPshavebeenshowntobee˙ectiveinawidevarietyofscienceandengineeringdomains,rivalingandevensurpassinghumandesigners[1],includingthedesignofanantennaforaNASAsatellite[84]andtoautomatically˝ndandrepairsoftwarebugs[48].AnotherformofECisneuroevolution[131],amachinelearningmethodinwhichevolutionaryalgorithms(EAs)areusedtoevolvearti˝cialneuralnetworks(ANNs).Neu-roevolutionhasbeenappliedtothedesignofcontrollersfor˝nlessrockets[51]andmobilerobots[17,45].Evolutionaryrobotics(ER)[17,114,98,81,45,13],isasub˝eldofECconcernedwiththedesignofroboticsystems.Oneoftheprimarybene˝tsofERmethodsisthattheycanconcurrentlyoptimizebothcontrolandmorphologyoftherobot,whichtypicallyproducesbettercoupling(orsynergisticrelationship)betweenbehaviorandphysicalform[21,90,58,127,103],andhencebetterperformance.Evolvedcontrollersusuallytaketheformofanarrayofcontrolparameters,centralpatterngenerators[107,9],orgeneticprograms[77,67].ANNsinparticularhavebeenextensivelystudiedbytheERcommunity[114,131,51,29],primarilyduetotheirbiologicalnatureandtheirabilitytomodelanynonlinearmappingfrominputtooutput.NEAT[119]andHyperNEAT[118],developedbyStanelyetal.,aretwoofthemostcommonalgorithmsforevolvingANNs.InNEAT,ANNsaregraduallybystartingwithasimplenetworkandrandomlyaddingnewneuronsand9connectionsbetweenneurons.Incontrast,HyperNEATANNsaregenerally˝xedintopology,butthevaluesforconnectionsamongneuronsisbasedonthephysicallayoutofthenetwork.Recently,ERhasbeenappliedtoproblemsinsoftrobotics[56].Similartotherobotic˝shdevelopedforthisstudy,softrobotscomprise(orcontain)malleable,˛exiblecompo-nents,whicharemeanttoimproveperformanceand/orsafety[104,86,70,82,109].Forexample,Cheneyetal.[19]evolvedlocomotionforvoxel-basedsimulatedrobots,whereeachrobotismadeupofa3D-gridofcubicvoxelsandeachvoxelcanbeevolvedwithdi˙erentmaterialproperties.Thatstudydemonstratedthatevolvingsoftrobotswithagenerativeencoding,basedonprinciplesfromdevelopmentalbiology,dramaticallyimproveslocomotionwhencomparedtodirectencodings.Theresearchdiscussedinthisdissertationleveragesthebene˝tsofbothERandtraditionalcontrolparadigms.Speci˝cally,inChapter3weinvesti-gatehowECcanbeusedtodiscoveroptimalcontrolpatternsforthe˛exiblecaudal˝nsofarobotic˝sh,andinChapter6weenhancetheadaptabilityofanadaptivecontroller.2.2.1MultiobjectiveOptimization.InmanyERstudies,algorithmvariantsaredemonstratedonrelativelysimpletasks.Forexample,arobotmaybeoptimizedtosimplymoveasfastorasfaraspossible[26].However,inareal-worldapplicationthetaskislikelytobemorecomplex,suchasway-pointfollowingorstationholding.Furthermore,multiplecompetingobjectivesmightneedtobesimultaneouslyoptimized.Onesuchexampleisanautonomoussystemthatneedstobebothe˙ectiveatlocomotionandabletocarryouttaskswithlimitedpower(i.e.,itneedstobeenergye˚cient).Insuchcases,usingaevolutionarymultiobjectivealgorithm(EMO)canbeadvanta-geous.EMOalgorithmsoperateusingsimilarprinciplesastheirsingleobjectivecounter-parts.However,unliketraditionalapproachestomanagingmultipleobjectivesinwhich˝tnessvaluesareaweightedsumofdi˙erentgoals(atechniquesometimesreferredtoasscalarization)[39],EMO˝tnessfunctionsreturnasequenceofvalues,whereeachvaluerep-10resents˝tnesswithrespecttoadi˙erentobjective.Inaddition,insteadoflocatingasingleoptimalsetofparametervalues,EMOsconvergetoasetofPareto-optimalsolutions.Indi-vidualsbelongingtoaParetofrontaresaidtobenondominated;thatis,eachofthesolutionsisoptimalwithrespecttosomecombinationoftheobjectives.ThemostcommonEMOalgo-rithms,suchasNSGA-II/III[37,112]andSPEA2[140],useanelitismapproachfordrivingsolutionstowardtheoptimalParetofront,andanichingorcrowdingmechanismtoensurethattheentiresetofPareto-optimalsolutionscanbefound.Indicator-basedEMOs[139,66]alsoproduceasetofsolutions,butdosobymaximizinganindicatorvariable,whichactsasasingleobjectiveforevolvingtheentirepopulation.Oneofthemostpopularindicatorvariablesishypervolume[8].Bymaximizingthehypervolumeofapopulation,thealgorithme˙ectivelydrivessearchtowardtheuser-de˝nedgoalswhilemaintainingadistancebetweenevolvedindividualsinthesolutionspace.TheadvantagesofEMOalgorithms,whencom-paredtosingleobjectiveEAsandparametersweeps,include:(1)locatingaParetofrontwithfewerevaluations,(2)automaticallyhandlingofconstraints,(3)notneedingtospecifytherelativeimportanceamongmultipleobjectives,and(4)automaticallysortingsolutionsaccordingtofeasibilityanddomination.InChapters4and5wedemonstratethatEMOsaree˙ectiveinoptimizing˛exilerobotic˝shcaudal˝nsforboththrust(speed)ande˚ciency(mechanicalandelectrical).2.2.2Modeling.ToapplyECalgorithmstorobotdesign,solutionsaretypicallyevolvedentirelyinsimulation.Theprimaryadvantageistimesaulationscaneasilybeexecutedinparallelandmuchfasterthanphysicalexperiments.ManyERsimulationsareconductedwitharigid-bodydynamicsengine.Forexample,theOpenDynamicsEngine(ODE)[116]iswidelyusedbytheERcommunity.ODE,whichwasdevelopedbythecomputergamingcommunity,hastheadvantageofbeinge˚cient,albeitatthecostofaccuracy.Otherphysicsengines,suchastheBulletPhysicsLibrary[32],Chrono::Engine[22],andNewtonGame11Dynamics[65],arealsousedbytheERcommunity,dependingonspeci˝cfeaturesofthetargetsystemorenvironmentunderstudy.Forexample,BulletPhysicsincludessupportforsimulatingsoftbodies,butatthistimeusesalessstablealgorithmforhandlingtheconstraintsbetweenrigidbodies,whichcanbeimportantforarticulatedroboticcomponents.Useofsuchphysicsenginesisreferredtoasphysicalsimulation.Inphysicalsimulation,asimulatedworldisconstructedfrombasicprimitives(ellipsoids,cuboids,cylinders,etc.)andthesecomponentsinteractwithoneanotherthroughcollisionsandappliedforces/torques.Analternativetophysicalsimulationiscallednumerical(ormathematical)simulation,whereallcomputationsarebasedonmathematicalformula(includingdi˙erentialequations).Nu-mericalsimulationenvironments(e.g.,MatlabandSimulink[115])areoftenmoreaccuratethanphysicalsimulations,however,theyhaveincreasedcomputationalcosts,aremoredif-˝culttoparallelize,andarelimitedintermsofmakingtheenvironmentinteractiveanddynamic.Forexample,simulatingcollisionsandaddingnewobstaclesmid-simulationisstraightforwardinaphysicalsimulation,butisofteninfeasiblefornumericalsimulation.AlthoughalloftheabovesimulationtoolsarehelpfultoERresearch,noneofthemincludessu˚cientsupportfor˛exiblematerialsorhydrodynamics.Whensuchfeaturesarerequired,dynamicmodelsmustbedeveloped.Forexample,Gomezetal.[51]reliedonanaccurateaerodynamicsenginetooptimizeaguidancesystemfor˝nlessrocketsinsimulation.AsdescribedinChapter3,weincorporatedahydrodynamicsmodel,aswellasamodelfor˛exiblecaudal˝ns,intoaphysicalsimulationinordertooptimizethepropulsionofarobotic˝sh[26].Animportantfeatureofsuchmodelsisthattheyincludeonlycriticalaspectsoftheunderlyingdynamics.Simplifyingassumptionsmakethemodelsmoree˚cienttodevelopandexecute.Thelatterisparticularlyimportantinevolutionaryalgorithms,wherethecostof˝tnessevaluationoftendominatesexecutiontime.However,theseassumptions,alsogiverisetodiscrepanciesbetweenbehaviorsseeninsimulationandinreality,theso-calledrealitygap.122.2.3Realitygap.Despitenumerousimpressiveresultsinthe˝eldofER,thegeneralapplicabilityofECtoroboticsisstilllimited,asmostERtechniquesaredevelopedanddemonstratedinsimulation.Therealitygapoftenleadstodi˚cultieswhentryingtorealizeanevolvedsolutioninaphysicalsystem.Overthepasttwodecades,severalmethodshavebeendevelopedaimedatcircumventingtherealitygap.Themostobviousapproachistoperformevolutiononthedevicesthemselves.Thisisreferredtoason-board(orevolvingphysicalsystems)evolutionandhasproventobequitee˙ective[46,16].However,on-boardevolutionisbothtimeconsumingandpotentiallyhazardoustothephysicaldevice.Forinstance,randomlyevolvedsolutionsmayputtoomuchstressonarobot'smotors,orcauseittomoveintoadangerouslocation(e.g.,driveo˙ofaledge).Animprovementistobeginevolutioninsimulationandthen˝nishonthedevice[106].Asimilarapproachistoincorporatephysicaltrials,atsetintervals,intotheevolutionaryalgorithm[75].Thesemethodsassumethattherealitygapissmallenoughsuchthatsimulation-evolvedindividualscancrossthegap;thatis,theperformanceinsimulationiscloseenoughtothatinreality.Additionally,itisgenerallyinfeasibletoevolvemorphologiesusinganon-boardprocess.Sinceitisdesirabletodoasmuchworkinsimulationaspossible,somemethodsat-tempttoaccountfortherealitygapthroughadditionalmodelingorcontrolcomplexity.Forexample,Miglinoetal.[91]modeledsimulationinaccuraciesasenvironmentalnoise;doingsoplacesanevolutionarypressureonevolvedrobotstohandlecertainlevelsoferrorbetweensimulationandreality.However,anopenquestionishowmuchnoiseshouldbemodeled.Anotherapproach,proposedbyFloreanoetal.[45],istoevolve˛exiblecontrollerscapableofadaptingonline.Thesecontrollersusearti˝cialneuralnetworks[47,94],meaningthatsomeneurons(andsomeconnections)areallowedtochangetheirbehavioraccordingtoanevolved,staticsetofrules.However,suchnetworksstillhavetroublewhenfacinglargedisparitiesbetweensimulationandreality.13Othermethodsforcrossingtherealitygapusephysicaltrialsasawaytoimprovethesimulationitself[74,134].Bongardetal.[14]developedtheestimation-explorationalgorithm(EEA)toiteratebetweenevolvinganeuralcontrollerandimprovingthesimulation.IntheexplorationphaseofEEA,anewtestisfoundthatbestexposesadisparitybetweenrealityandthecurrentmodel.Whileintheestimationphase,thechosentestisusedtoimproveapopulationofmodelssothattheybettermatchreality.Abene˝tofthisapproachisthatitdoesnotrequireanymorphologicalmodeltobespeci˝edapriori,(althoughsuchamodeldoesimproveperformance).ThemaindrawbackofEEAisthatitassumesthesimulationcanbeparameterizedandsubsequentlyimproved,whichmaybecometoodi˚cultasrobotandenvironmentalcomplexityareincreased.ThesestudiesdemonstratetheimportanceoftherealitygaptoER.Regardlessofhowaccuratethesimulation,therewilllikelyexistdisparitiesbetweensimulationandreality.Themoste˙ectiveERapproachesincludeamixtureofbothsimulatedandphysicalevalu-ations.Inthisdissertation,werelyoncontroller-basedtechniquestocrosstherealitygap.Speci˝cally,onebene˝tofusingtheadaptivecontrollersdescribedinthenextsectionisthattheytreatanyunmodeled(orpoorlymodeled)dynamicsassomethingtowhichtheycanautomaticallyadapt.2.3AdaptiveControlDi˙erentfromthetechniquesaddressedintheprevioussection,ourresearchreliesinpartonadaptivecontroltoaidincrossingtherealitygap.Adaptivecontroltechniqueshavebeenstudiedsincetheearly1950s,startingwiththedesignofautopilotsforhigh-performanceaircraft[63].Thesemethodscombineparameterestimationwithacontrollawinordertocontroltargetsystemswithdynamicsthatareeitherunknownorchangingunpredictably.Forthepurposeofthisdocument,controltechniquescanbedividedintotwocategories:model-basedandmodel-free(alsoreferredtoasdata-driven).Astheirnamessuggest,model-14basedmethodsrelyonadynamicmodeldevelopedwith˝rstprinciples,whereasmodel-freecontroltechniquesdonotnecessitateanymodeling.Figure2.1illustratesthedi˙erencebetweenaconventionalfeedbackcontrollerandanadaptivecontroller.Bothusetheerrorebetweenthereferencesignalandmeasuredoutputinordertoproduceaninputuforthetargetsystem(orplant).Whileanon-adaptivecontrollerenablesthesystemtorespondtodynamicconditionsbyadjustingthevalueofu,themannerinwhichitdoessois˝xed(speci˝cally,oncetunedforaparticularsystemandexpectedsetofconditions,theparameterstotheequationsde˝ninguareconstants).AnexampleisthewidelyusedPIDcontroller,whereuistheweightedsumoftermsinvolvinge(Proportionalterm),itsintegral(Integralterm),anditsderivative(Derivativeterm).Figure2.1:Ablockdiagramforafeedbackcontrolsystem.Theshadedblockshowstheadditionalcomponentsnecessaryforadaptivecontrol.Thesignalsr,x,y,andearethereferencesignal(desiredoutput),thecurrentoutputofthesystem,themeasuredoutputofthesystem(includinganymeasurementerrorduetod),andtheerrorbetweentheinputandthemeasuredoutput,respectively,whileuisthecontrolsignal.Allofthesesignalsvaryasafunctionoftime.ThemainshortcomingsofPIDcontrolarethatitrequiresparametertuningandtypi-callycannotrespondtochangesinthecontrolledsystem.Incontrast,anadaptivecontrollerautomaticallyadjustscontrolparametersonlineaccordingtoadaptivelaws(shownbytheshadedboxinthe˝gure).E˙ectively,anadaptivecontrollercanadjusthowitrespondstoaparticularsetofinputsbasedonthesituationsthatithasalreadyencountered.152.3.1Model-basedAdaptiveControl.Twoofthemostwell-studiedadaptivecontroltechniquesaremodel-referenceadaptivecontrol(MRAC)andadaptivepole-placementcontrol(APPC).AdiagramofMRACcontrolisshowninFigure2.2.ForbothMRACandAPPC,thedesiredbehaviorisbasedonahand-designedreferencemodel(outlinedbyared,dashedboxinthe˝gure),andacontrollerisadaptedonlinesuchthatthetargetsystembehavessimilarlytothereferencemodel.Speci˝cally,errorbetweenthemodelandsystemoutputsisusedbytheadaptivelawstodrivethesystemtowardthebehaviorofthemodel.Adaptivelawsaredi˙erentialequationsthatincludeerrorasaterm;theseequationsaretypicallyderivedusingLyapunovstabilitycriteriasuchthattheyguaranteeconvergencewithinsomepredeterminedamountoftime[63].Figure2.2:AgraphicalrepresentationoftheMRACmethod.Areferenceinputrisfedtoboththereferencemodelandtheadaptivecontroller.Thecontrollerproducesanoutputsignaluandisadaptedusing,whichisbasedontheerrorebetweenthereferencemodelandthecontrolledsystemoutputs,ypandyrespectively.MRACapproacheshavearestrictionthattheplantmustbeminimum-phase(i.e.,allzeroshavetobestable).Practically,thisrestrictionmeansthatMRACcannotbeappliedinmanyrealisticsettings,includingsystemswithtimedelays(whenasystemisslowtorespondtovaryinginputcommands).APPCrelaxestheassumption,anddoesnotrequireunstablezero-polecancellationduringcontrollerdesign.Accordingly,MRACcanbeconsidereda16specialcaseofAPPC.Ataminimum,however,bothofthesetechniquesrequireadesignertospecifythepolesofareferencemodel.Althoughtherearemanytechniquesandtoolsthataidinpoleplacement,muchoftheprocessisstilldependentupontheexpertiseofadesigner.Anotherapproachtoadaptivecontrol,unrelatedtoAPPCorMRAC,isneuroadaptivecontrol[63].Thissetoftechniquesisparticularlyusefulfornonlinearsystems,especiallywhenanynonlinearitiesaredi˚culttoidentify.Inneuroadaptivecontrol,nonlinearitiesinthesystemareapproximatedwithANNswhoseweightsareadaptive.Anoverviewofthesetechniquescanbefoundin[63,64].Althoughtheabovecontrolmethodshavebeenextensivelystudiedandapplied,theyallrequirethedevelopmentofasuitablemodelofthedesiredbehavior.Relianceonaspe-ci˝creferencemodelmakesitdi˚culttoaccommodateunexpectedconditionsthatinduceachangetotheunderlyingdynamics.Additionally,thehighorderassociatedwithmanyphysicalsystems,suchasrobotswith˛exiblecomponents,canrenderthedesignandimple-mentationofamodel-basedcontrollerintractable.Analternativetomodel-basedcontrolistousemoregeneralpurposemodel-freecontroltechniques.2.3.2Model-freeAdaptiveControl.Amodel-freeadaptivecontrollerismeanttobeasgeneral-purposeasPIDwhilealsobeingcapableofadaptingtochangesonline.Model-freeadaptivecontrol(MFAC)tech-niquesareintendedforybosituationswhereonlypartialandpossiblyinaccurateinformationaboutthesystemisknown.Aswithtraditionalapproachestoadaptivecon-trol[64],MFACattemptstominimizetheerrorbetweendesiredandactualoutcomes.HouandHuang˝rstproposedtheideaofMFACbasedontheconceptofapseudo-partial-derivativeandrelianceononlysysteminputsandoutputs[60].Later,Aidongetal.[3]andKaro«[68]proposedmodi˝cationstoMFACthatenhancethe˛exibilityandresponsivenessoftheMFACalgorithm,respectively.TheMFACapproachemployedinthisdissertation,17developedbyCheng[20],combinesatraditionalproportionalcontrollerwithanadaptiveANN.TheMFACANN,showninFigure2.3,computesitsoutputinthesamemannerasaconventionalfeed-forwardneuralnetwork,andoperatessimilarlytoastandardfeedbackcontroller(seeFigure2.1)wherethegoalistominimizetheinputerrorsignal.Unlikeanonadaptivecontroller,however,thevaluesoflinksconnectingnodesareadjustedduringoperationaccordingtoasetofadaptationlaws,whicharediscussedinChapter6.Figure2.3:AgraphicalrepresentationoftheMFACANN.Acontinuoustimeerrorsignaleis˝rstnormalized(viatheNrmblock)andthenpropagatedthroughtheinputneuronsIi.TheANNisthenactivatedasafeed-forwardnetworktoproduceanoutputV.The˝nalcontrolleroutputuisanampli˝edsummationofVande.2.3.3EvolvingAdaptiveControl.TheERcommunityhasexploredtheevolutionofadaptivecontrol[47,124],however,theseapproachestypicallyuseANNswithsynapticplasticitytohandleonlineadaptation.Anevolutionarypressureforadaptivebehaviorisincorporatedbysubjectingevolvedindividualstoadverseconditionsduringsimulation.Incontrast,traditionaladaptivecontroltechniquessuchasMRACandAPPCaregenerallynotoptimizedinsimulationandadaptaccordingtodesignedadaptivelaws.AnexceptionistheworkofCoelhoetal.[30],whichextendstraditionaladaptivecontrolmethodsbyincorporatinganevolvedneuralcompensator,which18actsasanadaptivegainmoduleandimprovesMFAC'sabilitytohandlenonlinearsystems.Inourresearch,describedinChapter6,wefurtherexpanduponestablishedmodel-freeadaptivecontrolmethodsbyoptimizingadaptivecontrolparametersinsimulation.Speci˝cally,wesubjectadaptivecontrollerstoavarietyofdi˙erentconditionsduring˝tnessevaluationinordertoputanevolutionarypressureonadaptivecontrollerstohandleawiderangeofscenarios.2.4ModeDiscoveryandSelf-modelingAdaptivecontrolmethodsaree˙ectiveforhandlingthedynamicconditionsthatroboticsystemsexperiencethroughoutthelifetimeofaphysicaldevice.Thisincludesdegradationofmaterialsandchangingenvironmentalconditionssuchastemperature.However,oncetheseconditionsvarybeyondacertainthreshold(e.g.,duetoadamagedcomponent)anadaptivecontrollerwillfail.Forexample,anadaptivecontrollercantypicallyadapttoaservomotoractingdi˙erentlyunderlow-powerconditions,butisnotabletocontroladevicewithaseizedservo.Insuchcases,moreextrememeasuresmustbetakentotopreventtheroboticsystemfromfailingcompletely.Thisscenarioisparticularlydi˚cultforanysystemthatisnotalwaysunderdirecthumansupervision.Thus,ifthesystemisexpectedtohandlemoreextremechanges,itisessentialtoexplic-itlyde˝netheboundariesofanadaptivecontroller.Morespeci˝cally,itisimportanttoknowbyhowmuchcananygivenphysicalpropertychangebeforeacontrollerfailsandthesystemneedstoswitchtoadi˙erentmodeofexecutiondrivenbyadi˙erentcontroller.Explicitknowledgeofsuchmodeboundariesfacilitatesthedevelopmentofhigher-leveldecisionlogic,whereeachmodecanhaveitsownuniquecontrolstrategy(e.g.,adaptivecontroller).Theau-tomateddiscoveryofexecutionmodesreducestherelianceonasystemdesignertomanuallyanticipatefailurecases.Alternativebehaviorsandvmodescanbepre-programmed(e.g.,usingevolutionarymethods),whichprovidesane˚cientmeansofaddressingproblems19duringoperation.Traditionalsolutionstothisproblemincludecontroltheorytechniquessuchasgainschedulingandmodeladaptivecontrolwithswitching[49,95,111,53,55,54]inwhichasetofcontrollersaredesignedtohandlea˝nitesetofpossibleconditions.Forthisdissertation,however,werelyonEC-basedmethods.Eveniftheexecutionmodeshavebeende˝ned,however,thesystemstillmustbeabletodeterminethecurrentmode.Basedpurelyonsystemstateinformationandsensedcondi-tions,howshouldacyber-physicalsystemdecidewhichexecutionmodeiscorrect(orbest-˝t)?Theproblemcanalsobestatedasfollows:howcanthesystemdiscoveranewmodelforitselfduringruntime?Withanewmodel,thesystemcanswitchtothecorrectmode(andaccompanyingcontroller).Oneapproachtoachievingsuchruntimemodeldiscoveryisself-modeling,wherethesystemmaintainsaninternaltalofitself[12].Thesearchcapabilityofcomputationalevolutionhasbeenshowntobee˙ectiveindiscoveringbothmodelsandcorrespondingbehaviorsforrobots.BongardandLipson[14]introducedtheEstimation-ExplorationAlgorithm(EEA),ageneralpurposealgorithmforreverseengineeringcomplex,nonlinearsystems.TheauthorsdemonstratedtheuseofEEAtoautomaticallydiagnosefailuresinterrestrialrobots(quadrupedsandhexapods)andgen-eratecompensatorybehaviorswithouthumanintervention[83].However,thereareareasofpotentialimprovement.First,thealgorithmiscomputationallyexpensivetorunonarobot,whichmayhavelimitedresources.Second,evenwithanaccurateself-modelitislikelythattherealitygapwillmakeitdi˚culttoevolvecompensatorybehaviors.Amorerecentalgorithm,theT-ResiliencealgorithmbyKoosetal.[73],hasbeendevelopedtoaddresstheseshortcomingsaswellasotherconcerns.Thisnewalgorithmincorporatestechniquesspeci˝callydevelopedtocrosstherealitygap,includingtheKoos'spreviousworkinvolvingamultiobjectiveapproachtopromotetransferablecontrollers[75].Ourapproach,combiningself-modelingwitho˜ineoptimizationandadaptivecontrolwillallowanautonomousrobottobothquicklyrecoverfromminorissuesandcompensateforpotentiallycripplingdamage.However,unliketherelatedstudiesdescribedabove,our20workdoesnotrelyonexplicitlyaddressingtherealitygap.Instead,ouruseofanadap-tivecontrollerwillallowroboticsystemstoadapttobothunforeseenchangesthatoccurthroughoutthelifetimeofarobotaswellasanypoorlymodeledorunmodeleddynamicsthattypicallyaccountfortherealitygap.Inaddition,theself-modelingprocesshasanaddedbene˝tofprovidingameanstoautomaticallygenerateanewcompensatorybehav-iorsevenwhenfacedwithcompletelyunanticipatedscenarios(i.e.,scenariosnotconsideredduringthemodediscoveryprocess).Inthecaseofarobot,ifthesystemhasanaccuratesimulationmodelofitsownmorphology,includingsensorsandactuators,itcanderivenewbehaviorsdynamicallyusingthemodelratherthanthephysicalsystem.Ourresearchintomodediscoveryandself-modelingarediscussedinChapter7.2.5RoboticFishPlatformTechniquesdiscussedintheprevioussectionsofthischapterprovideafoundationuponwhichtobuildaholisticapproachforautonomousrobotstohandleadverseconditions.However,eachofthetechniquesmustbeextendedtoenablethisintegration.Toconductourresearch,weusedrobotic˝shasthetargetdevice.Similartolive˝sh,robotic˝shaccomplishswimmingbydeformingtheirbodiesor˝n-likeappendages.Thisformoflocomotiono˙erscertainkeyadvantagesrelativetotra-ditionalpropeller-drivenaquaticvehicles.First,robotic˝sharepotentiallymoremaneu-verable,whichiscriticalwhenoperatinginclutteredunderwaterenvironments[2,123].Second,sincerobotic˝shproduceverylowacousticnoisesandexhibitwakesignaturessim-ilartothoseoflive˝sh,theyarelessintrusivetoaquaticecosystemsando˙erstealthinsecurity-relatedapplications.Third,with˝n/bodymovementsoccurringatrelativelylowfrequencies(typicallyonlyafewHz),thesesystemsarelesslikelytoharmaquaticanimalsorbecomejammedwithforeignobjects.Giventhesecharacteristics,robotic˝shareantici-patedtoplayanimportantroleinenvironmentalmonitoring[123],inspectionofunderwater21structures[61],trackingofhazardouswastesandoilspills[130],andthestudyofnaturalsystems[84,43,122,105].Severalrecentstudieshaveaddressedthedesignofaquaticrobotswithcompliantcom-ponents[87,5,62,85,18,42,102].Forexample,LowandChong[85]usedstatisticalmethodstoinvestigatethee˙ectofcontrolandmorphologicaldesignparametersontheresultingthrustofarobotic˝shwithacompliantcaudal˝n.Espositoetal.[42]performedasimilaranalysisofacaudal˝nwithsixindependentlyactuated˝nrays.Whilestudyingasinglekinematicparameter,thephasedi˙erencebetweenthedrivingangleatthebaseofa˛exiblecaudal˝nandthe˝n-bendingangle,Parketal.[102]discoveredthatmaximalthrustoccursataspeci˝cphasedi˙erence,evenwhenthemorphologyofthecaudal˝nischanged.Althoughmanyrecentworksinthisareafocuson˛exiblecaudal˝ns,asevidencedbythestudiesmentionedabove,Daouetal.[40]haveinvestigatedacompliantbody,whereboththeheadandcaudal˝nwererigid.Autonomyandadaptationareparticularlyimportantinaquaticenvironments,wherehumanoversightisoftenlimited,ifnotimpossible.However,whileinvestigationsofrobotic˝shhaveproducedmanyadvancesoverthepasttwodecades[125,69,6,133,41,78,85,132],robotic˝shstilldonotrivaltheirbiologicalcounterpartsintermsofrobustswimmingabilities.Di˙erentfromtheabovestudies,ourworkisconcernedwithimproving,throughevolutionaryoptimization,thrust(whichisnecessaryformaximizingaveragespeed),energye˚ciency(whichisimportanttodeployedsystems),andadaptability.InChapter3wedemonstratehowevolutionaryalgorithmscanbeutilizedtooptimizemorphologicalcharacteristicsandcontrolpatterns.Theinteractionbetweengeometryand˛exibilitycomplicatestheunderlyingdynamics.Forexample,determiningtheso-calledfoil-shapeforrobotic˝shtail˝nsdependsonagivenperformancemetric(e.g.,speed,power)[44].InChapters4and5weextendthisworktoincludemultiobjectiveoptimizationofswimmingabilitiesandenergye˚ciency.InChapter6,wedescribeourinvestigationsintoevolvingadaptivecontrolandapplyingittorobotic˝sh.Furthermore,22inChapter7weshowthataddingalayeraboveadaptiveconwhichwede˝nemodeboundariesandmonitorformoreseverechanges,triggeringaself-modelingroutinewhenaquaticrobotstoselect/evolvenewbehaviorsthatcompensateforpossibledamage.Bycombiningo˜ineoptimization,adaptivecontrol,andself-modelingonrobotic˝sh,wearebeabletostudyhowautonomousroboticsystemscanbemademorerobustinthefaceofuncertainenvironments.23Chapter3EvolvingCaudalFinMorphologyandControlPatternsWebeginourstudybyexploringanevolution-basedmethodologyforthedesignofarobotic˝shcaudal˝nandthecontrolpatternthatgovernsitsmotion.Evolutionaryoptimizationoccursinarigid-bodydynamicsenginethatincorporatesamathematicalmodelofthehydrodynamicsassociatedwitharobotic˝sh.Thechiefcontributionofthischapterisanevolution-baseddesignmethod,integratingrecentlydevelopeddynamicmodels,of˛exiblematerials,thatcanbeadaptedintoageneralroboticsengineeringprocess.3.1MethodsTocreateahydrodynamicsenvironment,webuiltasimulatoratopamathematicalmodelof˛exiblematerialsproposedbyWangetal.[128]andanopensourcerigid-bodydynamicsengine,theOpenDynamicsEngine(ODE)[116].Usingrigid-bodydynamics,nat-uralcaudal˝nmotionisapproximatedbydividingthe˝nintomultiplediscretesegmentsconnectedbyaspringanddampingsystem[128].Themathematicalmodelweusetocom-Someoftheresultsanddescriptionsinthischapterwerepublishedin[26]and[23].24putetheforcesproducedby˝nsisbasedonLighthill'stheory[80],forwhichthemotionofeachpointonabodycanbeusedtodeterminehowmuchpropulsionisgenerated.Formostofthischapter,weconsideronlyrectangular˝ns;however,laterweshowanextensiontoWang'smodelinwhichweinvestigatethesimulationof˝ns(i.e.,˝nsthatarenotrectangularandhavenonuniform˛exibility).Figure3.1:Graphicalrepresentationofthesimulatedhydrodynamics.LinearvelocityvandangularvelocitywaretheresultofthrustforceFT,dragforceFD,liftforceFL,anddragmomentMD.FTiscalculatedasthesumofallforcesactingonthe˝nsegments.ThehydrodynamicsandspringforcesareshowninFigure3.1.Theforceactingoneach˝nsegmentficanbecalculatedindependently,andtheresultingthrustforceFTisasummationofallsegmentforces,includinganadditionalforcethatactsatthetipofthe˝nalsegmentfL.Inthe˝gure,threesegmentsareshown(althoughthemodelgeneralizestoanynumberofsegments),alongwiththeforcesthatapplytoeachindividualsegment.Accordingtothemathematicalmodel,each˝nsegmentgeneratestwocomponentforces,aresistivecomponentandapropulsivecomponent,bothofwhicharedescribedbyEquation3.1:~fi(˝)=0BB@fi;x(˝);fi;y(˝)1CCA=mddt(v?^n);(3.1)25wheremdenotesthemassperunitlength(typicallytakenas14ˇˆd2whereˆisthedensityofwateranddistheheightofthe˝n),˝isthelocationonthe˝nwheretheforceacts,and^nandv?,respectively,aretheunitdirectionandvelocityperpendiculartothe˝n.Thetipofthe˝nalsegmentexperiencesanadditionalforcedescribedbyEquation3.2:~fL=0BB@FL;xFL;y1CCA=12mv2?^m+mv?vk^n˝=L;(3.2)where˝=Lrepresentstheposteriorendofthe˝n,and^mandvk,respectively,aretheunitdirectionandvelocityparalleltothe˝n.ThesehydrodynamicforcescanbecalculatedgiventheXandYpositionsofeach˝nsegmentovertime.Atthebaseofthe˝n,whichisattachedtothebody,amotordrivestherhythmicmotioninaoscillatorypattern.Alongwithcaudal˝ndimensions,theYoung'smodulusofelasticity,whichiscapturedintheparametersforthespringsanddampers,determines˛exibility.Thisrelationshipprovidesameansoftransferringsimulateddesignsintorealmaterialsusingknownandinferredmaterialproperties.ThistopicisdiscussedfurtherinChapters4and5.Thesimulatedrobotic˝shismodeledafteraphysicalrobotic˝shprototype(showninFigure3.2(a)),whichwasoriginallyconstructedtotesttheperformanceofdi˙erent˝ndimensionsandmaterialsti˙nesses.ArepresentationofthevirtualmodelcanbeseeninFigure3.2(b),showingthemainbodyandathree-segmentcaudal˝n.(a)(b)Figure3.2:(a)Therobotic˝shprototype.(b)Depictionofthevirtual˝shmodelwithathree-segmentrigid-bodycaudal˝n.Movementofthe3D-printedrectangularcaudal˝nisaccomplishedusingaservomotorwithasetrangeofmotionandperiodofoscillation.263.1.13D-PrintedFins.Test˝nswerefabricatedusinganObjetConnex350multi-material3Dprinter.Finswereprintedwithacombinationofdi˙erentphysicalmaterialstoyield˛exibilitiesthatresemblethemotionobservedinsimulation.Asdemonstratedin[108],a3Dprintercanconsiderablyimprovethee˚ciencyofanexperimentaldesignprocess.Severaliterationsofprintedpartscanbefabricatedinamatterofhours.Theprinted˝nswereattachedtotherobotic˝shprototypeandevaluatedinanaquatictestenvironment.Animageofthephysicalrobotwithattached˝nisshowninFigure3.2(a).Timetrialswereusedtodeterminetheaveragevelocityachievedbyeach˝n,whilevisualobservationshelpeddeterminethe˛exibilityof˝nsduringmovement.Inthesephysicaltrials,theheight,length,andthicknessofeach˝nwere˝xedat2.5,8.0,and0.1cm,respectively,andtheYoung'smodulusofelasticitywasprovidedbythemanufacturerdatasheets.Foreachoftheprinted˝ns,therobotwasplacedinatesttankandallowedtoreachastableswimmingspeedbeforetheaveragevelocitywascomputed.Thesti˙nessofeach˝ncanbecalculatedwithEquation3.3:Ks=Edh312l;(3.3)whereKsrepresentsamaterial'storsionspringconstant,dandldenotetheheightandlengthofthe˝n,respectively,ErepresentsYoung'smodulusofelasticityforthematerialitself,andhisthethicknessofa˝n.Thesevaluescanbedirectlyusedinsimulationduringoptimizationtrialsandprovideameansofcomparingsimulationandphysicalresults.3.1.2OscillatoryControllers.CPGs[71,135,33]andotheroscillatorycontrollershavebeenstudiedforcontrollingrobotic˝sh[2].CPGsareabletocoordinatemultipleactuationdevices,animportantfeatureforrobotswithmorethanoneservomotor[2,137,138].Crespietal.[33]demonstratedthis27conceptwhentheycontrolledanamphibiousrobotbyproducinglocomotionpatternsforbothswimmingandcrawlingbehaviors.InthischapterwedescribeexperimentsinwhichweimplementedtheneuraloscillatorproposedbyMatsuoka[88,89],referredtoasaMatsuokaNeuralOscillator(MNO),andcomparedittosinusoidalcontrollers.AnMNOiscomposedoftwoidentical,coupledarti˝cialneuronsthatgenerateanoscillationbasedonafourth-ordersystemofdi˙erentialequations.TheamplitudeandfrequencyofanMNOoutputcanbemodulatedbyadjustingthedynamicparameters.Toproducerhythmiccaudal˝nmotion,wetestedbothsinusoidalwavesandMNOs.Withbothofthesecontrollers,theamplitudeandfrequencyofthepatterndeterminetheaveragevelocityoftherobotic˝sh,whileabias(i.e.,verticalshift)a˙ectstheturningrate.Equation3.4showshowasinusoidalcontrollergovernstheangleofbetweenabodyandcaudal˝n:(t)=Asin(2ˇft)+b;(3.4)whererepresentsthe˝nangle,Aandbaretheamplitudeandbias,respectively,andfisthefrequency.Figure3.3depictstheblockdiagramofanMNO.Incontrasttoapuresinusoid,theoutputofanMNOcontainsaninitialtransientresponsepriortosteady-stateoscillations.Furthermore,asignalgeneratedbyanMNOwillcompriseamorediversefrequencyresponse.ThesetwoMNOcharacteristicsmayprovideanadvantageoverasimplepuresinusoid.Figure3.3depictsthetwoidentical,interconnectedneurons.Theneuronsprovideamutuallyinhibitingfeedback,whichisthesourceofstableoscillation.ForanMNO,˝andTrepresenttherisingandtheinhibitorytimeconstantsrespectively,arepresentsthestrengthofmutualinhibition,bdenotesthedegreeofadaptation,andcisavaryingcontinuousinput.Anin-depthoverviewoftheseparametersandthegoverningdi˙erentialequationscanbefoundin[89].28Figure3.3:TheMNOcomprisestwomutually-inhibitingneurons.TheoutputofthetheMNOisequaltotheoutputofneuron1minustheoutputofneuron2(Adaptedfrom[89]).Itisimportanttonotethattheoutputgeneratedbyeithertypeofcontrollercanexceedthephysicallimitsofthesimulatedrobotic˝shmotor.Whenthecontrolsignalisabovethemotoranglelimits(55degrees),themotorwillreachthelimitandhold.Likewise,whentherequiredangularvelocityistoohigh,themotorwillreachitsmaximumangularvelocityandmaintainthatvelocityuntilthecontrolsignalrate-of-changedecreasesbelowtheangularvelocitythreshold.Bynotlimitingcontrolsignalstophysicalconstraints,evolvedcontrollerswillbeabletoproducenon-sinusoidalcaudal˝nmotion.Thisbehaviorcanbeseenintheexperimentsdescribedbelow(seeFigure3.9foranexample).3.1.3GeneticAlgorithm.The˝rstexperimentswereperformedusingavariantoftheconventionalGA,anddif-ferentialevolutionisusedfortheirregular˝ns.Ingeneral,everyindividualinthepopulationcanencodethecaudal˝nmorphologyaswellasthecontrollerparametersdependingonthefocusoftheexperiment.Eachindividualgenomeisrepresentedasasetofgenes,whereeachgeneencodesarealvalueintherange[0,1].Thetranslationfromgenetoparameterisasfollows:MNOparametersa,b,c,˝,andTaremappedintotherange[0,10],sinusoidal29controlparametersamplitudeandbiasinto[0,720]degrees,frequencyinto[0,20]Hz,and˝n˛exibilitysismappedintotherange[0,1.8e-2]Nm.AswithmostGAs,westartbyrandomlyinitializingthepopulation,andthencontinuethroughasequenceofgenerationsinwhicheachindividualisevaluatedandstochasticallyallowedtoreproduce.Di˙erentfromaconventionalGA,ourvariantemploysa1Dtrivialgeography[117]inwhicheverylocationinthepopulationhasgeographicalmeaning(i.e.,thepopulationisdividedintooverlappingsubpopulations).OurGAusesasize3tournamentselectionforrecombination.Thisimplementationmaintainsthepopulation'sdiversitywithlittleoverhead.Additionally,weimplementedsingle-pointcrossoverandmutation.Intheeventofamutation,thea˙ectedgeneisrandomlydisplacedbyaddingavaluedrawnuni-formlyfromaGaussiandistributionwithazeromeanandavarianceof0.01;valuesaboveorbelowtherange[0,1]aftermutationaresetto0or1,respectively.3.2EvolvingFinMorphologyThissectioncanbedividedintothreeparts:mathematicalmodelvalidation,physicalvalidation,andevolutionaryoptimization.We˝rstcompareoursimulationresultswithdataderiveddirectlyfromthemathematicalmodel.Next,weperformasimilarcomparisonbetweensimulationanddatagatheredfromphysicalexperiments.Onceoursimulationenvironmentwasvalidated,weappliedevolutionaryalgorithmstooptimize˝nforshapeand˛exibility.Forallexperimentsinthissection,thebody-˝nmotoroscillatesata˝xedrateof0.9Hzina30degreesymmetricalrangeofmotion.Inlatersections,wealsoevolvetheoscillatorymotion.3.2.1MathematicalModelValidation.Priortophysicalvalidationandevolutionaryexperiments,itwasimportanttoensurethatoursimulationenvironmentmatchedthemathematicalmodel.Anydisparitybetween30simulationandmodelcouldsignifyanerrorthatwouldmakeevolutionaryresultsmeaning-less.Twoalgorithmswereemployedtooptimizethesti˙nessofthesimulatedcaudal˝n.Inbothexperiments,onlytheYoung'smodulusvaluewasaltered.The˝rstalgorithmwasabasichill-climber.Forthisexperiment,100independentrunswereconducted.Everyrunwasinitializedwithadi˙erentseedandaYoung'smodulusvaluechosenuniformlyatrandomfromtherange[0,5GPa].EveryYoung'smodulusvaluewasevaluatedbytranslatingit,withEquation3.3,tothespringcoe˚cientsthatgoverncaudal˝n˛exibility.Forevaluation,thesimulatedrobotic˝shwasallowedtoswimfor10seconds.The˝tnessofeachYoung'smoduluswascomputedastheaveragevelocityachievedoverthisevaluationperiod.Eachhill-climberrunbeganwiththeevaluationofarandomly-choseninitialYoung'smodulusvalue.SubsequentvaluesweregeneratedbyshiftingthecurrentvaluebyarandomnumberchosenuniformlyfromaGaussiandistributionwithameanof0andavarianceof0.1.TheresultingYoung'smoduluswasthenevaluated,andthebetterperforming(higheraveragevelocity)valuewaskeptandusedtogeneratethenexttestcase.Ineachrun,thisprocesswasrepeateduntil100candidatevalueshadbeenevaluated.Everyhill-climberinstanceconvergedtoanoptimumYoung'smodulusofroughly1.9GPa.Thesecondalgorithmdeployedwastheaforementionedgeneticalgorithm.Theprimarypurposeofthisexperimentwastocon˝rmthatthesimulationenvironmentcouldbeusede˙ectivelywithanevolutionaryalgorithm.Thisexperimentcomprised30independentruns.Eachrunwasseededwithadi˙erentvalueandapopulationof125randomlygeneratedindividuals.Everyindividualwasevaluatedinaprocessidenticaltothatusedinthehill-climberexperiment,andthepopulationswereevolvedfor100generations.Resultsfromtheevolutionaryexperimentcloselyresembledthoseofthehill-climber,withthemost˝tindividualsineveryrunhavingaYoung'smodulusnear1.9GPa.DatageneratedfromthemathematicalmodelcanbeseeninFigure3.4,andresultsfromthetwosimulationexperimentsareshownFigure3.5.Theexperimentalresultsshowthatboththehillclimberandevolutionaryapproachesyieldnearidenticalsolutions(i.e.aYoung's31modulusof1.9GPa).Thisisanexpectedresult,asbothexperimentsrelyonthesamesimulationenvironment.Figure3.4:Predictedvelocitiesfordi˙erentYoung'sModulusvaluesfromthemathematicalmodelcalculations.Notethatthisassumesthatthebodyisanchored.ComparingFigures3.4and3.5,adisparitybetweenmodelandsimulationresultsisap-parent.Speci˝cally,themodelpredictsamaximumvelocityofroughly5.1cm/sataYoung'smodulusnear0.9GPa,whilesimulationresultsachieveamaximumaveragevelocitycloserto1.4cm/sataYoung'smodulusnear1.9GPa.Despitethedi˙erences,both˝guresshowthesametrend,inwhichintermediatevaluesoftheYoung'smodulusproducethefastestrobotic˝sh.Additionally,thedisparitybetween˝gurescanbeexplainedbycloserexami-nationofthemodelandsimulator.Themostmarkeddi˙erencesarethatthemathematicalmodelassumestherobotic˝shbodydoesnota˙ectcaudal˝nmotion,andthecaudal˝nsegmentsarewithoutmass.Neitheroftheseassumptionsiscarriedoverintothesimulationenvironment,andbothofthesefactorswouldcausesimulatedrobotic˝shtoappearslowerthanmodeldatawouldpredict.32Figure3.5:Resultsofthehillclimberandevolutionaryrunsfordeterminingtheoptimumsti˙nessofa˝xeddimension˝n.Bothmethodsconvergedonacommonsti˙nessyieldingthehighestaveragevelocity.Darkershadesindicateclusteredresultsfromdi˙erenttrials.3.2.2PhysicalValidation.Next,wefabricatedcaudal˝nswitha3Dprinterandtestedthemontherobotic˝shprototypedescribedearlier.Sixunique˝nswereprinted,eachwithadi˙erentYoung'smod-ulus.Thematerialsrangedfromextremely˛exible(TangoBlackPlus)tonearlyin˛exible(VeroWhite).Eachprinted˝nwasattachedtotherobotandtestedintheaquaticenviron-ment;theaveragevelocitywasmeasuredover5separatetrials.TheresultsofthisexperimentareplottedinFigure3.6.Consistentwiththepredictedperformance,theplotshowsthatanintermediate˛exibilityproducesthehighestaveragevelocity.However,directcomparisonsbetweensimulationandrealityarenotpossibleduetocurrentlimitationsofthe3Dprintedmaterials.Speci˝cally,thematerialsdonothaveanexactYoung'smodulusvalue,butratherthemanufacturerprovidesarangeofpossiblevaluesforeachmaterial(materialspropertiesarenotguaranteedtoremainconstantbetweenprintjobs).Forexample,VeroWhitehasamodulusintherangeof2-3GPa,whiletheothermaterialshavelower-valueranges.33However,sincethemathematicalmodel,simulation,andphysicaldataareallfor˝nsofidenticalshape,somecomparisonscanbemade.First,thevelocityvaluesofthephysicalrobotic˝shareclosertomathematicalmodelpredictionsthantheyaretosimulationresults.Inaddition,theoptimalYoung'smodulusforallresultsisintherangeof1-2GPa.Thereasonforthedisparityinthemodelpredictionswasdiscussedintheprevioussection,howeveritisalsoapparentthatsimulationresultsdonotmatchreality.Themaximumvelocityof3.7cm/sinthephysicalexperimentsisnearlytwicethemaximumsimulationvelocity.Aswiththemathematicalmodel,certainapproximationsweremadeinthesimulationenvironment.Forinstance,distributedforcesweretreatedassinglepointforces,andthe˛exible˝nwassplitintojustthreesegments.Bydecreasingthesizeofeachsegmentandincreasingthenumberofsegments,themotionanddiscretizationofforceswilllikelybemorerealisticandincreasetheaccuracyofthesimulation.Figure3.6:Observedaveragevelocityfordi˙erentmaterialsusedinprinted˝ns.Sti˙nessincreasesfromlefttorightintheplot.Asasecondarymeasureofperformancebetweenthesimulationandphysicalexperi-ments,werecordedtheshapeof˝nsastheyoscillated.Figure3.7presentsasidebyside34comparisonbetweenasimulated˛exiblecaudal˝nandthe3Dprintedversionontherobot.Thisvisualobservationhelpstoreinforcetheviabilityofsimulating˛exiblecaudal˝ns.Figure3.7:Visualperformanceoftheevolved˛exible˝ninsimulation(left)versusafabri-cated˛exible˝ntestedontheprototyperobot(right).3.2.3EvolutionaryOptimization.Uponcompletionofcomparisonsbetweenthemathematicalmodel,simulation,andphysicalresults,optimizationwasexpandedintoafullevolutionarycomputationruninwhichtheYoung'smodulusanddimensionsofarectangularcaudal˝nweresimultaneouslyevolved.Finshapewasallowedtoevolveundertheconstraintthattheoverallareaofthelength-heightfaceandthethicknessofthe˝nremain˝xed.Thiscreatedastateinwhichtheheightofthe˝nwasdependentuponthelengthofthe˝n.Assuch,thetwoparameterstoevolveweretheYoung'smodulusandlengthofa˝n.Practicalconsiderationsontheoveralldimensionsofthe˝nwerealsotakenintoaccountasamaximumlengthof14cm(lengthoftherobotic˝shbody)andaminimumlengthof4cm(halfthelengthofpreviousexperiments)35wereimposeduponevolution.Valuesoutsideofthisrangecouldsu˙erfromtransferabilityissuesgivenelectromechanicalconstraintssuchasthemaximumtorqueexertedbyaservo.Again,anindividualrunconsistedof125individualsevolvingfor100generations.Similartothepreviousevolutionaryexperiments,tournamentselectionofsize3andelitismwereusedtoselecttheparentsforthenextgeneration.Unlikeearlierexperiments,however,singlepointcrossoverwasaddedsothatindividualscouldbegeneratedasacombinationoftwoselectedparents.Intotal,30replicaterunswereconductedto˝ndtherelationshipbetween˝nsti˙ness,˝nshape,andaveragevelocity.Fromtheevolutionaryruns,asetofoptimumvalueswasfoundforboththeYoung'smodulusanddimensionsofthe˝n.TheYoung'smodulusfoundinthetrialwas7.55GPa,andthecaudal˝nlengthandheightwere14and1.43cmrespectively.Hence,the˝ttestsolutionsreachedthemaximum˝nlengthallowedatacostof˝nwidth.Thisresultwasexpected,asalonger˝nwillbeabletogeneratelargerpropulsiveforces,whilewidthhasalessere˙ectonthisforce.ThischaracteristiccanbeseenbycloseexaminationofEqua-tion3.2,wherethelengthofa˝nisalinearfactor,andlonger˝nswillhaveahigherangularvelocityneartheposteriorofthe˝n.WhiletheYoung'smodulusfoundinthetrialislargerthanthatfoundinpriorexperi-ments,theresultingmaterialsti˙nessissimilar:1:35103Nmfortheoriginalexperiments,and1:73103Nmforthefullevolutionaryexperiments.Thisresultsuggeststhatasinglesti˙nessvaluemaybeadequateforanyrectangularcaudal˝ndimensions.Thereasonthesesti˙nessvaluesaresimilaristhataslengthincreased,theYoung'smodulusalsoincreasedtomaintainafairlyconstantvalue.Figure3.8presentsthethreedimensional˝tnesslandscapefoundintheevolutionaryrun.Asshown,apeakislocatedatamodulusofelasticityof7.55GPaandalengthof14cm.Thiscombinationyieldedanaveragevelocityof2.2cm/s.Thislandscapewouldsuggestthatforeachsetofdimensionsthereisaspeci˝cYoung'smodulusthatcorrelatestotheoverallbestperformancefora˝n.36Figure3.8:Visualizationofthe˝tnesslandscapefordi˙erentshapeandsti˙ness˝ns.Notethatheightisdependentuponlengthindeterminingshape,therefore,heighthasbeenomit-tedfromthedata.Asthelengthofthe˝nincreases,theYoung'sModulusincreasesaswelltomaintainsimilarsti˙ness˝nsfordi˙erentlengths.Thecomplexdynamicsofanunderwaterenvironmentmakedesigninge˚cientrobotic˝shachallengingengineeringendeavor.Consideringthedi˚culty,itisdesirabletocreateanautomateddesignprocessbywhichrobotic˝shcanbeoptimizedforaspeci˝ctask.Makinguseofthehydrodynamicmodelforarobotic˝shcaudal˝n,wehaveshownthataninsilicoprocesscanbeusedtooptimizetheYoung'smodulusofa˛exible˝n.Insimulation,weobservedthattheoptimumYoung'smodulusisdependentonboththecaudal˝nmotionanddimensions.Speci˝cally,foranycombinationof˝nfrequency,amplitude,height,widthandlengththerewillbeauniqueYoung'smodulusoptimum.However,whentheYoung'smoduluswassimultaneouslyevolvedwith˝nshape,wefoundthattheoverallresulting˝nsti˙nessexhibitedcomparablecharacteristics.Generally,highervaluesoflengthandYoung'smodulusproducedfasterswimmers.373.3EvolvingFinMorphologyandControlPatternsIntheprecedingsection,weinvestigated˛exiblecaudal˝nsusinga˝xedsinusoidalcontroller.Inthissection,evolutionisresponsibleforoptimizingbothcontrolandmorphol-ogyinordertoimprovelocomotion.Throughcontroller-morphologyevolution,thecomplexinternalinteractionsbetweenactuationandmaterialpropertiescanbeexploitedtoproduceasystemthatmatchestheneedsofitsenvironment.GivinganEAthefreedomtoadjustbothmorphologyandcontrolallowssolutionstohaveatightcouplingbetweenmotionandphysicalform.We˝rstperformedaseriesofexperimentsinwhichwecompareMNOstopuresinusoidalsignals.Inthesepreliminaryexperiments,thetargetofevolutionistoreachamaximalaveragevelocity.ToensurethatinitialMNOtransientsandstartingbiasdonota˙ectthestableaveragevelocitymeasurement,allevaluationsbeganaftera5secondstart-upperiod;thetotalevaluationperiodis15seconds.Toprovideabasisfortheevolutionexperimentswe,˝rstseparatelyevolvedmorphologyandcontrol.Followingthesevelocity-onlyexperiments,weproceededtoevolvingsinusoidalcontrollersforminimalturningradius.Intheprevioussection,weobservedthatforagivencontrolpatternthereislikelyauniqueoptimalmorphology.Thisphenomenonisduetoamatchingbetweenthenaturalfrequencyofthe˝nandthefundamentalfrequencyoftherhythmiccontroloscillations.Foragiven˝n˛exibility,thenaturalfrequencycanbede˝nedasthefrequencyatwhichthe˝nwouldvibrateonceplacedintomotionandisindependentofactuation.Thefundamentalfrequencyofacontrolsignalisbestdescribedasthefrequencywiththehighestpowerinthefrequencydomain.Tocon˝rmthis˝nding,weperformedexperimentsinwhichthe˛exibilityofthecaudal˝nisactedonbyevolution,butcontrolpatternsare˝xed.Additionally,weperformedcomplementaryexperimentsinwhich˝n-˛exibilityis˝xed,butcontrolpatternsareactedonbyevolution.Eachisdescribedinturn.383.3.1FixedControl,EvolvedMorphology.Intheseexperiments,puresinusoidalcontrollersgovernthemotionofthecaudal-˝nwhilethespringcoe˚cientisadjustedbyevolution.Theamplitudeofsinusoidalcontrollerswassetat30degrees,whilethefrequencywassetat0.3Hz,0.6Hz,and0.9Hzinthreeindependentexperiments.Thesefrequencieswerechosenbasedontypicalroboticoperatingmodes.TheresultsfromtheseexperimentsaresummarizedinTable3.1.Table3.1:Fixed-ControllerExperimentalResultsControllerSpringVelocityFreq.Ampl.0.3Hz303.58e-4Nm0.92cm/s0.6Hz307.80e-4Nm2.37cm/s0.9Hz301.38e-3Nm3.02cm/sFromTable3.1itcanbeseenthatasthefrequencyofthecontrolsignalincreasesfrom0.3Hzto0.9Hz,theoptimalspringcoe˚cientalsoincreases.Fromthisdata,wecandeterminethatthenaturalfrequencyofthecaudal˝nisgreaterthanorequalto0.9Hz.Anothertakeawayfromthisdataisthatasthefrequencyincreasessotodoestheresultingvelocity.Itislikelythatfrequencywillincreaseonlyuntilthenaturalfrequencyofthe˝nisreached,atwhichpointtheaveragevelocitywillbegintodecreaseonceagain.Fundamentally,asthesti˙nessofa˝nincreasesitcansustainmoreangularmomentum,andthefastera˝noscillatesthehighertheresultingthrustforce.Controlsignalsarenotdependentupontheperformanceoftheactuatedhinge.Thisbehaviorisdisplayedinthe0.9Hzexperiment.Duetotheangularvelocityconstraint,theactuatedhingeisnotabletoreachthe30degreeamplitudebeforethecontrollersignalbeginsthenextcycle.ThisbehaviorisdepictedinFigure3.9.Aninterestingsidee˙ectofthehingelaggingbehindthesignalisthatresultinghingemotiondoesnotnecessarilyresembleapuresinusoid,butratheratriangularwave.39Figure3.9:Anexampleofacontrollersignalthatisbeyondthecapabilitiesofthesimulatedmotor.Thesolidredlinerepresentsthesignalgeneratedbythe0.9Hzsinusoidalcontroller,andthedashedbluelinethesimulatedmotorhinge.3.3.2EvolvedControl,FixedMorphology.Inthenextsetofexperimentsweevolvedcontrolsignalsfora˝xedcaudal˝n˛exibility.Thisvalue(1.8e-3Nm)representsthe˛exibilityofthedefaultrobotic˝shprototypecaudal˝n.Thepurposeofthisexperimentistocon˝rmthatforagivenspringcoe˚cient,thecau-dal˝nwillhaveanaturalfrequency,basedonmaterialproperties,atwhichitwillproducethemostthrust.First,weconducted30replicaterunsinwhichsinusoidalcontrollerswereevolved,suchthattheamplitudeandfrequencyofthesinusoidalsignalwereadjustedbythegeneticalgorithm.Themost˝tsinusoidalcontrollersproducedanaveragevelocityof3.9cm/s,andconvergedtoasignalwithanamplitudeandfrequencyofroughly110degreesand1.26Hz.Afterthesinusoidexperiments,weconducted30morereplicaterunswiththeMNOcontroller.Unlikethesinusoidalcontrollers,thebestsolutionsvariedamongthedi˙erentrunsproducingsimilarsignalswithdi˙erentparametersets.Figure3.10showshowwidely40MNOparametersvaried.Undersuchdynamics,itisimpracticaltohand-selectparametersandexpecttoproduceanoscillation,muchlessanoscillationcapableofgeneratingoptimalbehavior.Thisisanimportantbene˝tofgeneticalgorithms:manyuniqueparametercom-binationscanbee˚cientlytestedinamannernotpossibleforatraditionalgradient-descentalgorithm.Figure3.10:BoxplotsforeachMNOparameterfromthebestperformingMNOsofeachof30independentruns.ThefundamentalfrequenciesofthebestMNOsolutionswereinthesamerange(1.26Hz)andproducedsimilarvelocities(3.9cm/s).Frequencydomaindataforthebestsolu-tionsfrombothexperimentsareshowninFigure3.11.Thisplotdemonstratesthatforthegivenmorphologicalcharacteristics,includingthespringcoe˚cient,thetwocontrollertypesconvergedonasimilarsolution.Thisisanexpectedresult,asasinglecontrolpatternwillmatchthegivencaudal˝nnaturalfrequency.41Figure3.11:Thefrequency-domainresponseoftheoverallbestsinusoidal(A)andMNOcontrollers(B).3.3.3EvolutionofControlandMorphology.Next,weturntothemainobjectiveofourstudy:toinvestigatetheoptimizationofrobotic˝shthroughthesimultaneousevolutionofmorphologyandcontrol.Assuch,allfollowingexperimentsevolvebothcontroland˝n˛exibility.First,weevolvedsinusoidalcontrollerswiththespringcoe˚cientgoverning˛exibility.Again,thesinusoidalcontrollersconvergedtoasingleoptimalsolution.Inthiscase,evolvedsinusoidshadafrequencyof4.2Hzandanamplitudeof100degrees.Alloptimalsolutionsalsoconvergedonasinglespringcoe˚cientof1.7e-2Nm,whichisnearthemaximumvaluesetfortheseexperiments(1.8e-2Nm).Giventhehigherfrequency,itisnotsurprisingthatahigherspringcoe˚cientevolved.Asshowninthe˛exibility-onlyexperiments,higherfrequenciesrequirealower˝n˛exibility.Maximalaveragevelocityalsocontinuedthesametrend,asthemost˝tsolutionsproduceenoughthrusttoaverage11cm/s,whichissigni˝cantlyhigherthanthetestsinwhichonlymorphologyoronlycontrolwereevolved.Thisresultdemonstratestheimportanceofsimultaneouslyoptimizingcontrolandmorphology.The˝nalvelocity-targetedexperimentsinvolvethejointevolutionofMNOparameterswithcaudal˝n˛exibility.Asbefore,theparametersandbestsolutionsfromoneruntoanothervariedwidely(similartotherangesdisplayedinFigure3.10).Themost˝tsolutions42fromthe30replicaterunsalsodi˙eredin˝nal˝tness.Thisresultspeakstothedi˚cultyinselectingthecorrectMNOparameters.Aswiththesinusoidalcontroller,˛exibilitywaslowerthaninpreviousexperiments,withtheoverallbestsolutionhavingaspringcoe˚cientof9.2e-3Nm.However,thisMNOcontrollerwascapableofproducingonlyenoughthrustforanaveragevelocityof8.3cm/s.Figure3.12plotsaveragetrajectoriesforeachoftheevolutionaryexperimentsinwhichmaximalvelocityisthetargetofevolution.Fromthe˝gure,itcanbeseenthatsystemsproducedthroughthesimultaneousevolutionofmorphologyandcontrolclearlyoutperformsystemsinwhichonlycontrolormorphologyareevolved.AsomewhatsurprisingresultisthatsinusoidalcontrollersoutperformMNO-basedcontrollers.Evenwhenthepopulationsizeandnumberofgenerationsareincreased,MNOcontrollersonlycomeclosertomatchingthesinusoidalperformance;theydonotsurpassit.Foropen-loopsingleactuationdevices,itappearsthatcontrollersbasedonMNOsdonothaveanyadvantageovertraditionalsinusoids.Inlightofthisresult,thefollowingexperimentsontheevolutionofturningmotionsfocusonsinusoidal-basedcontrollers.Figure3.12:Pathsoftherobotic˝shwhilebeingcontrolledbydi˙erentoscillatorycontrollersandhavingdi˙erent˛exibilities.Note:plotsoverlapsigni˝cantlyattheorigin.433.3.4EvolutionofTurning.Experimentstargetedatturningweresimilartothevelocityexperimentswiththe˝t-nessbeingevaluatedasaverageangularvelocityinplaceoflinearvelocity.Solutionswererewardedforhavingahighaverageangularvelocityandwereevaluatedoveraperiodof30seconds.Theincreaseintimeallowsacircularpatterntodevelop,whichishelpfulfortheanalysis.Evolvedsinusoidalcontrollersproducelocomotioninwhichtherobotic˝shbeginsmovingstraightforwardduringatransientphase,andthenbeginstoturninatightcircle.Thebestevolvedsinusoidalcontrollerproducedapathofradius4.5cm,asshowninFigure3.13.Bothfrequencyandthespringcoe˚cientwerereducedcomparedtosinusoidalvelocityexperiments;thefrequencyandspringcoe˚cientwere2.2Hzand1.2e-2Nmcom-paredto4.2and1.7e-2.AlsoshowninFigure3.13,theevolvedcontrolsignalforcesthe˝ntothemaximumhingeanglewithregularchangesoflessthan10degrees.InFigure3.13(b),thecontrolsignal(reddashedline)andresultinghingemotion(solidblueline)demonstratethatapuresinusoidalcontrolsignalcanresultinnon-sinusoidalmotion.3.3.5EvolutionofVelocityandTurning.Toexploremoregeneralrobotic˝shbehavior,our˝nalexperimentistoevolveevolvingcontrolandmorphologyforbothbothvelocityandturning.Thisisdonebyevolvingtwoseparatesinusoidalcontrollers:onespeci˝ctoforwardlocomotionandanothertoturning.Thechallengefortheevolutionaryalgorithmistostrikeabalancebetweenthehigher˛exi-bilityconducivetotightturning,andthehighersti˙nessvaluesthatproducegreaterthrustformaximalaveragevelocity.Otherthanasharedspringcoe˚cientforthedrivencaudal˝n,thetwocontrollersareindependentofeachother,andareeachevaluatedfor20secondsunderidenticalinitialconditions.Formultipleobjectives,typicallyaspecializedalgorithm,suchasthemultiobjectiveevolutionaryalgorithmNSGA-II[37],isutilizedtoformacompleteParetofrontofnon-dominatedsolutions.WeperformsuchexperimentsinChapters4and5.However,forthis44Figure3.13:Evolvedturningbehavior:(a)pathtakenbythebestsolution,and(b)controlsignalandresultingservoangle.studywefocusedonanalyzingasimplesetofdatato˝ndthetrade-o˙sbetweenoptimizingturningandvelocity.Somesolutionsmaybeoptimalforturning,othersforvelocity,andstillmorewhiche˙ectacompromisebetweenthesetwoobjectives.Figure3.14showstheperformanceoftwohigh-˝tnesssolutions.Thetopportionofthe˝gureshowstheturningandvelocitypathsofasolutionthatiscomparativelybetteratturningthanachievinghighvelocity.Theradiusoftheturningpathisapproximately7cmandtheaveragevelocityis1cm/s.Therobotic˝shperformsadequatelywhencomparedtoourpreliminarystudies,howeveritislesse˚cientthansolutionsevolvedexclusivelyforturningorvelocity.Theevolvedspringcoe˚cientof1.5e-2Nmisinbetweenpreviouslyfoundoptimalvaluesforturning(1.2e-2Nm)andvelocity(1.7e-2Nm).Thisresultindicatesthatthesolutionisacompromise.ThebottomportionofFigure3.14showsthepathsofadi˙erentsolution,whichwasbetteroptimizedformaximalaveragevelocityratherthanturning.Thissolutionisvery45similartothehighest˝tnesssolutionofthevelocityevolutionexperiment.Thefrequencyandspringcoe˚cientwere4.6Hzand1.7e-2Nmcomparedtothe4.2and1.7seenbefore.Asshowninthe˝gure,however,theturningbehaviorhasasigni˝cantwobble,thatis,thecenteroftheturningradiusisconstantlymoving.Thisshiftisduetoapoormatchingbetweenthematerialsnaturalfrequency,andtheoptimalfrequencyofthesinusoidalcon-trollerresponsibleforturning.This˝nalexperimentdemonstratesthatevolutionarysearchiscapableofexploringthecomplexinteractionsbetweenmorphologyandcontrol,whileatthesametimebalancingcompetingobjectives,allinanonlinearenvironment.Figure3.14:Thepathsproducedbytwoevolvedcontrollers:in(a)˝n˛exibilityismoreoptimalforturningratherthanvelocity,andin(b)˛exibilityismoreconducivetoachievinglinearvelocity.3.4FinsWithNonuniformFlexibilityandNon-RectangularShapesThesimulationmodeldescribedinSection3.1,basedonworkdonebyWangetal.[128],hasseverallimitations.First,allmotionoccursinthetwo-dimensionalsurfaceofwater.E˙ectively,themodeldoesnotconsidertheabilitytodiveorrise.Second,˝nsarerequiredtobeThatis,˝nsmustbeatleastthreetimeslongerthantheyaretall.Finally,themodelonlyconsiders˝nsofuniform˛exibilityandrectangularshape.Whileeliminating46the˝rsttwoitemswouldrequirethedevelopmentofanewmodel,herewebrie˛yexplore˝nsofnonuniform˛exibilityandnon-rectangularshape.Figure3.15depictsthevariablesassociatedwithsimulatingtheirregular˝ns.Inthediagramweshowacaudal˝nwith˝vesegmenealsouse˝vesegmentsinallexperi-mentsdiscussedinthissection.Eachsegmenthasauniquevalueforlength(L1..L5)anddepth(D1..D5),andthejointbetweeneachsegmenthasavaluefor˛exibility(E1..E4).Weperformed˝veexperimentsinwhichweevolvedasimulatedrobotic˝shformaximumaveragevelocityusingthedi˙erentialevolutionalgorithm[120](detailsregardingDEwillbepresentedinChapter6).Weconducted40replicatesimulationsforeachofthe˝veex-perimentsdescribedinthissection.Eachreplicateranfor100generationsandincluded100individuals.Evolvedparametersincludethoselistedinthe˝gure,aswellasthefrequencyandamplitudeofthesinusoidalcaudal˝nmotion.Fortheseexperiments,weusedanumeri-calsimulation,inSimulink[115],basedonthedynamicsdescribedinSection3.1.Inthesimulation,eachsegmentandeachjointcanbecon˝guredseparately.Figure3.15:Diagramforirregular˝ns.Eachsegmenthasauniquelengthandheight,andthejointbetweeneachsegmenthasitsownvaluefor˛exibility.The˝veexperimentsinclude:(1)baseline(segmentsareidenticaland˛exibilityisuniform),(2)length(segmentlengthsareevolvedseparately),(3)depth(segmentdepths47areevolvedseparately),(4)˛exibility(joint˛exibilitiesareevolvedseparately),and(5)irregular(lengths,depths,and˛exibilitiesareevolvedseparately).ThebestspeedsfortheseexperimentsareshownTable3.2,andthebestresultsareachievedwhenallparametersareevolvedindividual(theirregularexperiment).Table3.2:SpeedComparisonAmongExperimentsBaseline22.1cm/sDepth33.5cm/sLength38.9cm/sFlexibility22.1cm/sIrregular51.8cm/sFigure3.16showsboxplotsforthe˝veparametersevolvedinthebaselineexperiment.Forthisboxplot(andintheremainingboxplotsdescribedinthissection)whenonlyonevalueisgivenforatypeofparameter(e.g.,L1isprovided,butnotL2,L3,L4,orL5)theremainingsegments(orjoints)arecon˝guredwiththesamevalue.Forrectangular˝ns,anintermediatetotal˝nlengthof7.8cm(thesumofallsegmentlengths),aYoung'smodulusof3GPa,andanamplitudeofapproximately20are˝xedinthe˝nalpopulation.Valuesfor˝ndepthandcontrolfrequencyconvergetorelativelysmallrangesofvalues.Figure3.16:Boxplotsforthebaselineexperiment.Parameters(fromleft-to-right)includeL1(thelengthofeachsegment),D1(thedepthofeachsegment),E1(˛exibilityateachjoint),andthecontrolpattern'samplitudeandfrequency.ThemorphologyparametersareasdepictedinFigure3.15.Figure3.17showsboxplotsforthelength,depth,and˛exibilityexperiments.Intheseexperiments,onlyonetypeofmorphologicalparameter(i.e.,length,depth,and˛exibility)48(a)(b)(c)Figure3.17:Boxplotsforthelengthexperiment(a),thedepthexperiment(b),andthe˛exibilityexperiment(c).Foreachsetofboxplots,ifaparametervalueisunspeci˝ed(forexample,L2in(b))thenallsegmentsorjointssharethesamevalue.isevolvedindependentlyforeachsegmentorjoint.Amongtheseexperimentswe˝ndthattheamplitude,frequency,and˛exibilityvalueshavesimilartrends.Speci˝cally,amplitudeevolvestorelativelylowvalues,frequencyevolvestohighervalues,and˛exibilityvaluesarerelativelyhighacrossallreplicates.Asidefromthesetrends,however,itappearsthateachexperimentevolvesfundamentallydi˙erentvaluesforlengthanddepth.Forexample,inthe49lengthexperimentmostsegmentshaverelativelylowvalues,butinthedepthexperimentthevalueforlengthevolvesnearthemaximumvalue.Figure3.18:Boxplotsfortheirregularexperiment.Forthisexperimentallpossibleparam-etersaresubjecttoevolution.Figure3.18showboxplotsfortheirregularexperiment.Inthisexperiment,eachsegmenthasauniquelyevolvedlengthanddepthvalue,andeachjointhasitsownvaluefor˛exibility.Additionally,thecontrolamplitudeandfrequencyareevolved.The˝nalspeedsachievedinthisexperiment(51.8cm/s)showsthatformaximumspeeditisbettertoevolveallavailablemorphologicalparameters.Evenso,theboxplotsshowthatnoneoftheevolvedparametersconvergeto˝nalvalues,thusitisdi˚culttodrawanyfurtherconclusions.ItappearsthatDEwasunabletosolvethisproblemwithintheallowednumberofgenerations.Increasingthenumberofgenerationsandthepopulationsizemayimproveconvergence.Whiletheresultspresentedinthissectionarepromising,fortheremainderofthisdocumentwefocusonrectangular˝nswithuniform˛exibility.Wemadethisdecisionfortworeasons:(1)itenablesustomaintainconsistencybetweenstudies,and(2)theadditionalcomplexity(andparameters)requiredforthemorecomplex˝nshapesmakestheanalysis50ofourmultiobjectiveexperimentsandcontrolalgorithmslessstraightforward.Speci˝cally,whenweareconsideringadaptivecontrol,itisworthkeepingthesimplerectangular˝nsdiscussedinSection3.1sothatwecanfocusontheprocessofevolvingcontrolparameters.3.5ConclusionsInthischapter,wedemonstratedanevolutionarydesignmethodforrobotic˝shcaudal˝nsandcontrolpatterns.We˝rstdevelopedasimulationenvironmentinwhichunique˝ncon˝gurationscouldbetested.Thesimulationenvironmentwascreatedbycombiningarigid-bodydynamicsenginewithamathematicalmodelofa˛exiblecaudal˝n'shydrody-namics.Totestthesimulationenvironment,we˝rstimplementedahill-climberalgorithm.Givena˝xed˝nshapeandcontrolpattern,thehill-climberalgorithmmappedoutthe˝t-nesslandscapefor˝nsti˙nessvs.velocity.Theseresultswerecomparedtodatagenerateddirectlyfromthemodel,whichcon˝rmedthatthesimulationandthemathematicalmodelhavecomparabledynamics,althoughtheabsolutevaluesdi˙er.Hill-climberresultswerefurthervalidatedthroughcomparisonswithphysicalexperi-ments.Withtheaidofa3Dprinter,anaquatictestenvironment,andarobotic˝shproto-type,weconductedaseriesofvelocitytestsforseveral3D-printed˝ns.All˝nswereidenticalinshape,buthadYoung'smodulusvaluesrangingfromverylowtonearlyin˛exible.Plotsofsti˙nessvs.velocityforthemathematicalmodel,simulation,andphysicalexperimentsallshowedasimilartrendinwhichaveragevelocitywasmaximalforintermediatecaudal˝n˛exibility.Thisresultdemonstratesthatitispossibleforasimulationenvironmenttocapturekeyaspectsofthedynamicsof˛exiblematerials.Tosimultaneouslyoptimizeseveral˝nparameters,weprogressedfromthehill-climberexperimentstoanevolutionaryalgorithm.AgeneticalgorithmwasusedtoevolveboththeYoung'smodulusandshapeofa˝n.Fromthisseriesofexperiments,wefoundthatthemost˝t˝nsgenerallyevolvedtobeaslongaspossiblewhilemaintainingafairlyconstant51sti˙nessvalue.Thisresultisconsistentwiththefactthatlonger˝nsgenerallyproducelargerpropulsiveforces.Additionally,ourresultsshowthatforeach˝nshape,andpresumablyforeachcontrolpattern,thereisanassociatedoptimalYoung'smodulus.Thesimulatedandphysicalresultspresentedinthisstudydemonstratethee˙ectivenessofanevolutionarybasedapproachgiventhehighdimensionalityofthesolutionspace.Next,wesimultaneouslyevolvedofmorphologyandcontrol.Evolutionarypressureswerebasedonachievingmaximalaveragevelocityormaximalturningratebyadjustingcontrolparametersandthe˛exibilityofacaudal˝n.Weassessedtheutilityoftheproposedmethodbyevolvingbehaviorsforbothturningandachievinghighvelocity,producingtwocontrollers.Thealgorithmsuccessfullyfoundsolutionsabletoperformbothbehaviors.Astheseresultsindicate,anevolutionaryoptimizationapproachisparticularlywellsuitedtothenonlinearityoftheaquaticdomain.Speci˝cally,thisstudydemonstratesthatevolutionaryalgorithmscanhandlethecomplexinteractionsfoundbetweenmaterialproperties,physicalform,andcontrolpatterns.Infuturechapters,weexpandonevolvingbothmorphologies(byconsideringmultipleobjectives)andcontrollers(byusingfeedbackcontrollersthatareadaptive).52Chapter4EvolvingSwimmingPerformancewithMechanicalE˚ciencyManystudiesinevolutionaryroboticsfocusonoptimizingthesystemforasingletask.Likewise,existingworkonoptimizingperformanceofrobotic˝shwith˛exible˝norbodyhastypicallydealtwithasingleoptimizationobjective,forexamplespeedorthrust.However,inpracticalapplicationsrobotic˝sh(andothertypesofrobots)areoftenrequiredtomeetmultipleobjectives,eithersimultaneouslyorwithindi˙erenttasksorenvironments.Forexample,whilespeedisingeneralanimportantspeci˝cation,energye˚ciency(andthusoperatingduration)isoftenequallyasimportant.Maneuverability,theabilityoftherobottomaketightturnsordealwithdisturbances,isalsoaparticularlysigni˝cantobjective.Inthischapter,weconsiderenergye˚ciencyastheratiobetweentheandtotalmechanicalpowerexertedbythecaudal˝n(thesetermsarediscussedindetailinSection4.1).Weapplyevolutionarymultiobjectiveoptimization(EMO)tothedesignofarobotic˝shwitha˛exiblecaudal˝n,showninFigure4.1.EMOalgorithmstypicallyuseanelitismapproachfordrivingsolutionstowardtheoptimalParetofront,andanichingorcrowdingSomeoftheresultsanddescriptionsinthischapterwerepublishedin[28].53mechanismtoensurethattheentiresetofPareto-optimalsolutionscanbefound[140,37].TheadvantagesofEMOalgorithmsinclude:(1)locatingaParetofrontwithfewerevaluationswhencomparedtoaparametersweep,(2)automaticallyhandlingconstraints,and(3)automaticallysortingsolutionsaccordingtofeasibilityanddomination.Forthisstudy,weapplytheNSGA-IIalgorithm[37],whichiswidelyusedinbothresearchandreal-worldapplications.ComparedwithotherEMOalgorithms,themainadvantagesofNSGA-IIincludeafastersortingoperationandamoree˙ectivemethodformaintainingdiversity(i.e.,reducingprematureconvergence)[140,31,139,72,136].Theresultsofsimulationsrevealseveralgeneralprinciplesthatcanbeappliedinthedesignofrobotic˝shmorphologyandcontrol.Toverifythatthesimulationresultsarephysicallyrelevant,weselectedseveraloftheevolvedsolutions,fabricated˛exiblecaudal˝nsusingamulti-material3Dprinter,andattachedthemtoarobotic˝shprototype.Ex-perimentalresults,conductedinalargewatertank,correspondreasonablywelltosimulationresultsinbothswimmingperformanceandpowere˚ciency,demonstratingtheusefulnessofevolutionarycomputationmethodstothisapplicationdomain.Thecontributionofthischapterisademonstrationthatmultipleobjectivescanbesimultaneouslyoptimizedforarobotic˝sh,andthatinperformingsuchanoptimizationwecanextractmeaningfuldesigncriteriathatwillbeapplicabletosimilarroboticsystems.4.1ModelingandSimulationFindingcombinationsofmorphologicalandcontrolparametersthate˙ectivelybalancespeedandenergyconsumptionischallenging.AsdemonstratedinChapter3,inrobotic˝shthatincorporatecompliantmaterials,morphologicalandcontrolparametersarehighlyinterrelated.Speci˝cally,anychangetothe˛exibilityofthecaudal˝nwillrequireacorre-spondingchangetothecontrolsignaltoensurethatthefrequencyofoscillationmatchesthenaturalmotionofthe˝n.54Figure4.1:(top)Therobotic˝shprototypeusedinthisstudy,and(bottom)thevirtualrepresentationevolvedduringsimulation.Inthisstudy,weagainuseadynamicsmodeldevelopedbyWangetal.[128]basedonLighthill'sLarge-AmplitudeElongatedBodyTheoryofLocomotion[80].UnliketheworkpresentedinChapter3,however,wedonotutilizearigidbodydynamicsengine(i.e.,physicalsimulation)forthiswork.Instead,simulationoftherobotic˝shisconductedinSimulink[115],whichallowsforastraightforwardtranslationofdynamicequationsintosimulation(i.e.,numerical/mathematicalsimulation).Numericalsimulationprovidesgreateraccuracyforoursimulatedrobotic˝sh.DetailsregardingthemodeleddynamicscanbefoundinSection3.1.4.1.1EvolutionaryOptimization.Thenumericalsimulationoftherobotic˝shaboveisparameterizedbyseveralterms,includingtheamplitudeandfrequencyofasinusoidalcontrolsignalandthelength,height,and˛exibility(springanddampingcoe˚cients)ofthecaudal˝n.Therangesforeachofthesevalues,asidefromthespringcoe˚cient,arelistedinTable4.1.55Table4.1:RangeofEvolvedParametersMinMaxAmplitude(rad)0.080.5Frequency(Hz)0.53.0FinLength(cm)3.015.0FinHeight(cm)1.05.0Young'sModulus(GPa)0.13.0Unliketheotherparameters,therangeofspringcoe˚cientvaluesisnotchosenbyadesigner.Instead,itislimitedbythepropertiesofavailablematerials.Forthisstudy,thespringcoe˚cientisrestrictedtowhatour3Dprinter,anObjetConnex350,iscapableoffabricating.Moreover,forphysicalmaterialsitismorecommontoconsidertheirYoung'smodulus,aninherentmaterialpropertyrelatingtoelasticityand˛exibilitymeasuredinPascals(Pa).Consequently,thisquantityisdisplayedinthetableinplaceofarangeforspringcoe˚cients.Theprintedmaterials,andresultingYoung'smodulusrange,arediscussedinthenextsection(Section4.2).Forevolvinggenomescomprisingrealvalues,NSGA-IIrequiresausertosetthefourfollowingparameters(selectedvaluesareinparenthesis):theprobabilitiesofcrossover(90%)andmutation(33%),andthedistributionindexforbothsimulatedbinarycrossover(10)andpolynomialmutation(10).4.1.2Constraints.Alongwithevolvingtheparametersasreal-valuednumbers,NSGA-IIalsoaccommo-datesthefollowingtwolimitationsonthedynamicmodel.First,thedynamicmodelisonlyvalidforanelongated˝ninwhich˝nlengthisatleastroughlythreetimes˝nheight,whichcorrespondstoaminimumlength-heightratioof3:1.Second,sincethedynamicmodelitselfdoesnotlimitpowersuppliedatthebaseofthecaudal˝n,presumablybyaservomotor,we56imposeamaximumpowerconstraint.Practically,thismaximumpowerconstraintlimitsthetopspeedoftherobotic˝sh.Thesetwoconstraintsaregivenbythefollowingequations:length3height0(4.1)MAX_POWERpower0(4.2)wherelength,height,andpowerrefertopropertiesoftherobotic˝shcaudal˝n,andthemaximumpowerconstantMAX_POWERwasdeterminedexperimentally.InNSGA-IItheselimitationsaretreatedasconstraints,whichenablesthealgorithmtosmoothlyfollowagradientfrominfeasible(i.e.,asolutionthatviolatesaconstraint)tofeasiblesolutions.Fortheconstraintequations,anegativevaluedenotesaninfeasiblesolution.4.1.3FitnessEvaluation.Eachevolvedindividualisevaluatedfor10secondsofsimulationtime,however,onlythesecondhalfofthisperioddetermines˝tness.Thisallowstherobotic˝shtoreachaspand˝nalheading,withaveragespeedandpowere˚ciencycalculatedoverthe˝nal5seconds.Powere˚ciencyistheratiooftwoterms:e˙ectivepowerasthenumerator,andtotalpowerasthedenominator.Bothofthesetermsaremechanicallybased(asopposedtotheelectricalpowerofthemotor)andarecalculatedusingEquations(3.1),(3.2),andthefollowingde˝nitionofmechanicalpower:~P(t)=~F~v;(4.3)where~Pismechanicalpower,and~Fand~vcanbetakenastheinstantaneousforceandvelocity,respectively,ofapointonthecaudal˝n.57E˙ectivepower,sometimescalledusefulpower,isillustratedinFigure4.2.E˙ectivepowertakestherobotic˝sh'svelocityandtotalforceandprojectsitalongtheaveragedirectionofmotion(labeledd5to10s)beforeusingEquation(4.3)tocalculateanaveragepower.Practically,e˙ectivepowerincludesonlythesurge(forward-to-back)forceproducedbythecaudal˝n.Totalpower,ontheotherhand,includestheforceexertedtocreatebothsway(side-to-side)andsurgemotions.Theresultingpowere˚ciencyratiohasarangefrom0to100%,withhighernumbersbeingdesirable.Figure4.2:Anillustrationofthedynamicsinvolvedincalculating˝tness.Thepathoftherobotic˝shincludestwoparts:alight-bluesegmentfrom0to5seconds,whichdoesnotdirectlya˙ect˝tness,andadark-bluesegmentfrom5to10secondsusedtoevaluate˝tness.Thepathsettlestoanaverageheading,whichisnotinlinewiththeX-axis,duetoabiascausedbytheinitialrotationofthecaudal˝n.Thedashed,orangeline(d5to10s)representsdisplacementoverthe˝nal5secondsofsimulation.4.2FinFabricationandTestingInsimulation,˛exibilityofthecaudal˝nisdeterminedbyspringcoe˚cients.However,the˛exibilityforactualmaterialsisexpressedasaphysicalpropertysuchastheYoung'smodulus.Therefore,werequirethefollowingequation,whichrelatesaspringcoe˚cienttotheYoung'smodulusofa3D-printedcomponent:Ks=Edh312l;(4.4)whereKsandErefertothespringcoe˚cientandYoung'smodulusvalues,respectively,andd,h,andlrepresenttheheight,thickness,andlengthofarectangular˝n,respectively.58TomatchtheYoung'smodulusofprintedmaterialswithevolvedspringcoe˚cients,weneededawaytofabricatea˝nforagivenYoung'smodulusvalue.Todoso,wedesignedcomposite˝nsinwhich˛exibilityisadjustedbyvaryingtherelativethicknessoftwodi˙erentmaterials,asshowninFigure4.3.The˝ncomprisestwoouterlayersofarubber-likepolymer,andaninnerlayerofamorerigidplastic.Whendiscussingcompositematerials,˛exibilityisoftenreferredtoasane˙ectiveYoung'smodulustodistinguishfromuniformlyfabricatedmaterials.Thus,specifyingthethicknessoftheinnerlayer,tinner,and˝xingtheoverallthicknessto1.2mmlimitstherangeofpossiblee˙ectiveYoung'smodulusvalues.(a)(b)Figure4.3:(a)Diagramofacompositematerialfora3D-printed˛exiblecaudal˝n(note:thecaudal˝nwouldbeonitsside).(b)Top-viewphotographofa3D-printedcaudal˝nwithaninnerthicknessof0.38mm.Theoverallthicknessisaconstant1.2mm.Thee˙ectiveYoung'smodulusvalueforthecompositematerialdependsontherelativethickness(tinner)oftheinnerVeroWhitePluslayerwithrespecttothetwo˛exibleTangoBlackPlusouterlayers.Todeterminethe3D-printablerangeofe˙ectiveYoung'smodulusvalues,wesetuptheexperimentshowninFigure4.4andmeasuredtheYoung'smodulusvaluesforaseriesofcompositematerialswithdi˙erentvaluesoftinner.Forthisexperimentalsetup,theYoung'smodulusEisevaluatedwith:59E=L3bPL3IbwL(4.5)whereLbandIbarethelengthandareaofmomentinertiaofthetestcomposite,respectively,andPLandwLaretheloadanddisplacementatthetipofthecomposite,respectively.AsshowninFigure4.4,asampleofcompositematerialis˝xedtoaharnesswhileitstiprestsagainstaloadcell.Displacementatthetipisadjustedusingaslidingrailandmeasuredwithalasersensor.Threereplicatesetsofloadanddisplacementdataaregatheredforeachcompositematerial,inwhicheachsetcomprises˝vedatapointsatdi˙erentdisplacements.Aleastsquareerrormethodisadoptedto˝ndtheslopebetweenforceanddisplacementforeachofthethreesets,andthee˙ectiveYoung'smodulusforeachcompositeisevaluatedastheaverageofthethreereplicateexperiments.(a)(b)Figure4.4:Testingofphysical˝ns.(a)Diagramoftheexperimentshowingthetestingprocess.(b)Photographoftheexperimentalsetupformeasuringthee˙ectiveYoung'smodulusof3D-printedcompositematerials.Figure4.5plotstheresultsoftheseexperimentsfordi˙erentvaluesoftinner.Theevolvablerangeofspringcoe˚cientscorrespondstoane˙ectiveYoung'smodulusofapprox-imately100MPato3GPa.Thesevaluescorrespondtoarangeofmaterialsroughlyfromrubber,whichtypicallyhasaYoung'smodulusof10to100MPa,tohardplastics,which60haveaYoung'smodulusof1to5GPa.Here,weusethebest˝tlineinFigure4.5to˝ndtherequiredtinnervalueforagivenYoung'smodulus.Figure4.5:E˙ectiveYoung'smodulusofcompositematerialsfordi˙erentvaluesoftinner.4.3ExperimentsandResultsWeconducted20replicateevolutionaryruns,eachcontaining200individualsevolvingfor500generations.Allofthe20replicatesconvergedtoasimilarParetofront.Intotal,eachreplicatesimulationrequiresroughly5104evaluationstoconvergetoa˝nalParetofront,whichissigni˝cantlylessthanwhatcouldberequiredforaparametersweepoverthesameparameterranges.Forexample,testingeachofthe6parameters(i.e.,the5parametersshowninTable4.1andanadditionalspringdampingcoe˚cient)at10evenlydistributedpointswouldrequireexactly106evaluations.Furthermore,unlikeanEMOalgorithm,suchaparametersweepwouldnotconsidervaluesinbetweenthe10˝xedpoints,resultinginamorecoarseoptimization.61Figure4.6showsthecombinedParetofrontforthe20replicateruns;eachofthesubse-quent˝guresinthissectionwillcorrespondtothedatafoundinFigure4.6.ThecombinedParetofrontrevealsthatfortherobotic˝shprototypeanevolvedcaudal˝nandcontrolsignalcanproduceamaximumaveragespeed(giventhemaximumpowerconstraint)intherangeofapproximately4.8to5.8cm/sandane˚ciencyintherangeof35to42%.Asex-pected,weseethatanincreaseinspeedisaccompaniedbyadecreaseine˚ciency.AlthoughtheParetofrontisclearlyformed,weemphasizethatitcoversonlyasmallrangeofvalues(4.8-5.8cm/sand35-42%e˚ciency).TheParetofrontismoreclearlyvisibleinFigure4.7,whichplotseveryfeasiblesolutionevolvedineachofthe20replicateexperiments.Figure4.6:AcombinedParetofrontincludingthebestsolutionsfromeachofthe20replicateevolutionarysimulations.Thelabeled,redsymbolsdenotesolutionsthatwerephysicallyfabricatedandvalidated.Inspectingthea˙ectsofeachparameteroneachobjectivecangiveinsightintohowparametervaluesshouldbeselected.Forexample,thetwoplotsinFigure4.8showhowthelengthofacaudal˝na˙ectseachofthetwoobjectives.Forlength,allvaluesarebetween6and12cm,andhighervaluesforlengthwithinthisrangeproducehigherspeed,but62Figure4.7:Thecompleteevolutionaryhistoryforeachofthe20replicateexperiments.Everyfeasible,evolvedsolutionisplotted,andthePareto-frontishighlightedindarkblue.decreasede˚ciency.Alengthinthisrangecorrespondstoroughlyhalfthelengthofthebody(14cm),whichisacommonratioforbiological˝sh.Althoughthesevaluesforlengthdependontheotherparameters,intuitivelyincreasingthelengthofthe˝nshouldincreasethrust,andthereforespeed,atthecostofincreasedpowerusage.Figure4.9plotsparametervaluesforthesolutionsinthecombinedPareto(seeTable4.1fortherangeofeachparameter).Inthe˝gure,eachparameterisscaledbetween0and1foraneasiercomparison.Thelinesbetweenparametersconnectvaluesthatbelongtothesamegenome(evolvedsolution).The˝gureillustratestwoimportantpoints.First,eachparameterconvergestoarelativelysmallrangeofvalues,particularly˝n˛exibility(˝nswithascaledvaluenear0arevery˛exible).Theevolutionofrelatively˛exible˝nsisnotunexpected,astheyproducehigherthrustforlowervaluesofcontrolfrequencyandamplitude[23].Essentially,amore˛exible˝nisinherentlymoree˙ectiveande˚cient.Second,mostofthevariationinthe˝nalpopulationisinthelengthandheightofthe63Figure4.8:Plotsofevolvedsolutionsshowingrelationshipsbetween˝nlength(x-axis)andthetwoobjectives,speed(top)ande˚ciency(bottom).AllPareto-optimalsolutionshavea˝nlengthbetween6to12cm,thusthex-axisdoesnotincludetheentireevolvablerange(3to15cm).Thestraight,orangelinesindicatethebest-˝tforeachsetofvalues.caudal˝n,meaningthatthecontrolparametersand˝n˛exibilityconvergetovaluesthatareusefulfor˝nsofdi˙erentshapes.Inspectingtherelationshipbetweenevolvedsolutionsandconstraints(Equations(4.1)and(4.2))canprovidefurtherinsightintowhatconstitutesagooddesign.Forexample,theconstrainton˝ndimensionssetsaminimumlength-heightratioof3:1,andalthoughtheheightandlengthparametersexhibitarelativelyhighvariationinthePareto-optimalset,64Figure4.9:Theevolvedparametersscaledbetween0and1forthecombinedParetofrontsolutions.mostconvergetoasimilarratio.Morespeci˝cally,thebestperforming˝nshavearatiolessthanroughly3.5:1(meanof4.1andmedianof3.2),whichsigni˝esthatwhilechangingtheheightandlengthofthe˝ncanalterthespeedande˚ciency,itisbesttokeeptherationear3:1forbetterperformance.Sincesimulationenvironmentdoesnotallowfor˝nswitharatiooflessthan3:1,physicaltestingwillhavetobeconductedtodeterminewhetherlowervaluesaremoreorlessbene˝cial.Likewise,inspectingtheconstraintonmaximumpowershowsthatthebestsolutionsutilizeasmuchpowerasisallowed,whichisimportantforgeneratingthemostthrust.Asforcontrolofthecaudal˝n,Figure4.9indicatesthatitisbettertoincreasetheamplitudeanddecreasefrequencyofthesinusoidalmotion.Thiscanbestbeexplainedbyexaminingtheequationsofmotionforthecaudal˝n:65(t)=Asin(2ˇFt);(4.6)_(t)=2ˇAFcos(2ˇFt);(4.7)(t)=4ˇ2AF2sin(2ˇFt);(4.8)whereAandFaretheamplitudeandfrequencyofthesinusoidalmotion,andistheangleofthecaudal˝nwithrespecttothebody.Equation(4.8)demonstratesthatcaudal˝naccelerationisproportionaltothesquareoffrequency,butonlylinearlyproportionaltoamplitude.AsshownbyEquations(3.1)and(3.2),theresultingthrust,andthusthetotalpower,isproportionaltothissameacceleration.Thus,theevolutionarypreferenceforhighamplitudesandlowfrequenciessuggeststhathigherfrequenciesrequiretoomuchpower,whichviolatesthesecondconstraint,andthatitismoree˙ectivetoincreasespeedbyincreasingamplitude.OurinterpretationofFigure4.9,alongwiththecalculatedcaudal˝naspectratio,providesthefollowinggeneralguidelinesonhowtoproduceane˙ective,e˚cientrobotic˝sh.Speci˝cally,thecaudal˝nshould:(1)berelatively˛exible(havealowYoung'smodulus),(2)havea˝nlength-heightratiocloseto3:1,(3)havea˝nlengthofroughlyone-halfthelengthofthebody,and(4)increasethrustandspeedbyincreasingtheamplitudeofmotionratherthanfrequency.SinceeverysolutionontheParetofrontisnondominated(i.e.,optimalinsomerespect),itisdi˚culttoclaimthatonesolutionisbetterthananotherbasedontheir˝tnessvalues.Tocomparesolutions,however,wecandeviseametricthatcaptureswhatisdeemedimportant.Forexample,wemayconsiderhowmuchtotalpowerisneededfortherobotic˝shtotravel1meter,asinEquation4.9:Ptotal=t1meterPaverage(4.9)66wherePtotalisthetotalpoweraccumulated,t1meteristhetimethatittakestotravel1meter,andPaverageistheaveragepowerexertedbytheevolvedcontrolpattern.Thismetricincludesaspectsofbothobjectives:itisbene˝cialtodecreasetimespenttraveling,t1meter,byincreasingspeed,andbene˝cialtodecreasePaveragebybeinge˚cient.WithrespecttoEquation4.9,we˝ndthatsolutionsnearthemiddleoftheParetofront,asopposedtothetwotails,performbest.Theworstperformingsolutionsarethosethatarefasterbutlesse˚cient.Whilethisisasimpletest,asimilarprocesscanbefollowedtoselecta˝nalevolvedsolutioninmorecomplexscenarios.Thechosenmetricneedonlyrelatetobehaviorsanticipatedforarequiredtask.Forexample,ifmaneuverabilityisimportantforavoidingobstacles,themetriccanincludeturninge˚ciency.4.4PhysicalValidationToverifythatsolutionsevolvedinsimulationarephysicallymeaningful,weselectedfoursolutions(indicatedbythelabeledsymbolsinFigure4.6)fromthecombinedParetofronttofabricateandtest.Makinguseofthebest˝tlinefromFigure4.5enablesthe3D-printingofcompositecaudal˝nswithaspeci˝cevolved˛exibility.Foreachchosensolution,atinnervalueiscalculatedanda˝nofthecorrect˛exibility,lengthandheightisprinted.Tocomparethespeedofvirtualandphysical˝ns,weattachedtheprintedcomposite˝ntotherobotic˝shprototypeshowninFigure4.1.Thisrobotic˝shwasplacedinawatertankandspeedwasmeasuredandaveragedover5trials,whereeachtrialwasconductedinthesamemanneras˝tnessiscalculatedforthevirtualrobot.ResultsofthesephysicalexperimentsaresummarizedinTable4.2,wherelabelsmatchthosefoundinFigure4.6.Forthreeofthetested˝ns,theseresultsshowagoodcorrespondencebetweensimulationandreality.Fin1,however,isnotablyfasterinrealitygiventhesameparametersassimulation,likelyduetothenatureof˛exible˝nswhereperformancecanchangedrasticallyforslightchangesin˛exibility.67Table4.2:Simulation-RealitySpeedComparisonSimulationRealityFin15.17(cm/s)7.43(cm/s)Fin25.39(cm/s)4.00(cm/s)Fin35.62(cm/s)5.00(cm/s)Fin44.97(cm/s)4.90(cm/s)Wenotethat,whilethephysicalmeasurementsroughlycorrespondwiththesimulationresults,asshowninFigure4.7,thePareto-solutionsareclusteredwithinarelativelysmallrangeofvaluesforspeed(1cm/sdi˙erencebetweenminimumandmaximumvalues).Ef-fectively,allthefabricated˝nsproducegoodperformance.Asaconsequenceofthistightclustering,however,thetrendsamongresultsforthephysicalexperimentsdi˙erfromthoseofthesimulations.Additionally,imperfectionsintroducedbythe3Dprintingprocesswillamplifyanydisparitybetweensimulationandreality.Figure4.10:Comparisonbetweenthemotionofacaudal˝nforphysicalandvirtualex-periments.Toincreasevisibility,apurplelinetracesthelengthofthecaudal˝nforthephysicaldevice.Thedashed,blackreferencelinesprovideacommonanglewithwhichtheside-by-sideimagescanbecompared.Whencomparingvirtualandphysicalresultsitissimpletocomparetheirspeeds,yetitisnotasstraightforwardtovalidatepowere˚ciency.However,sinceweareusingmechanicalpowere˚ciency,thetotalpower(denominatorofpowere˚ciency)ofvirtualandphysical68resultswillbeapproximatelyequalwhenbotharecontrolledwiththesamefrequencyandamplitudeandthecaudal˝nmotionsmatch.AssupportedbyFigure4.10,thedynamicmodelutilizedinthisstudyisaccuratewithrespecttothemotionofthecaudal˝n.The˝guredemonstratesthatforsimilarconditions(i.e.,thesametimepoint,controlpattern,and˝nmorphology)thevirtualandphysicalroboticcaudal˝nshavethesamemotion.Thus,itcanbeexpectedthattotalpowerbetweenvirtualandphysicalwillbeconsistentwithoneanother.Tocompareonthebasisofpowere˚ciency,wealsorequiree˙ectivepower(numeratorofpowere˚ciency).Sincee˙ectivepowerisproportionaltospeed,thee˚ciencyofvirtualandphysicaltrialscanbecomparedbyinspectingtheiraveragespeeds.Accordingly,inTable4.2anincreaseinspeedcorrespondstoanincreaseine˚ciency,andviceversa.4.5ConclusionInthisChapter,wepresentedour˝rststudyinwhichweappliedevolutionarymultiob-jectiveoptimizationtotheproblemofbalancingswimmingperformanceandpowere˚ciencyinarobotic˝sh.Swimmingperformanceisconsideredasmaximizingtheaveragespeed,andpowere˚ciencyiscalculatedastheratiobetweene˙ectivepowerandtotalpowerresultingfromcaudal˝nactuation.ResultsfromNSGA-IIevolutionaryexperimentsprovideinsightintohowarobotic˝shshouldbedesignedforincreasedperformanceande˚ciency.First,thecontrolparameters(amplitudeandfrequency)shouldbesetsuchthatfrequencyisonthelowerendofacceptablevaluesandamplitudeisonthehigherendofitsrange,whilestillprovidingenoughthrusttoreachthedesiredspeed.Settingtheseparametersinsuchawaywillleadtoincreasede˚ciencycomparedtousinghighervaluesforfrequencywithloweramplitudes.Second,thelength-heightratioofthecaudal˝nshouldbeapproximately3:1,andthelengthofthe˝nshouldberoughlyhalfofthelengthofthebody.Finally,caudal˝nsshouldbehave69a˛exibilitysimilartorubber-likematerials,asopposedtohardplastics.Theconclusionsdrawnfromtheseresultsareexpectedtogeneralizetoanyrobotic˝shofsimilardesign,regardlessofscale.Toverifythatresultsfromevolutionarysimulationarephysicallymeaningful,wefab-ricatedseveralevolved˝nsandattachedthemtoarobotic˝shprototype.Weutilizedamulti-material3Dprintertofabricate˝nsmadeofcompositematerials,andbasedtheevo-lutionaryrangeof˛exibilitiesonthecapabilitiesofthisprinter.Experimentsconductedinawatertankcon˝rmthattheevolvedspeedscorrespondreasonablywellwithphysicalresults.Further,throughavisualcomparison,weconcludethatthetotalpowerofevolvedsolutionscloselymatchreality,andthus,thesolutionsgeneratedbyNSGA-IIconstitutevalidrobotic˝shdesigns.However,duetotheclusteringofevolvedsolutions,andtheissuescausedbytherealitygap,trendsamongevolvedsolutionswerelostwhentransferedtothephysicaldevice.InChapters6and7,wediscussourmethodofaddressingthisconcernbycombin-ingevolutionarycomputationwithadaptivecontrol.Inthenextchapter,wecontinuethisresearchwithamorecomplexsystemandadi˙erentmechanismforcalculatinge˚ciency.70Chapter5EvolvingSwimmingPerformancewithElectricalE˚ciencyInthischapterwediscussoursecondstudyinvolvingourevolutionarymultiobjectiveoptimizationapproachtothedesignandcontrolof˛exible˝nsforrobotic˝sh.Forthepur-poseofdemonstration,weagainfocusonthecaudal˝nandtwocompetingobjectives:speedandpowerconsumption.Inparticular,weexploretheinteractionsbetweenthesti˙ness,size,andthecontrolpatternforthe˛exible˝n,andweinvestigatehowthespeedperformanceandenergyusagecanbebalanced.Somesituationsmayrequirespeedbesacri˝cedforef-˝ciency(e.g.,thesystemmayneedtoreturntoportduetoalowbattery),whileinothersituationsspeedmaybeparamount(e.g.,thesystemmayneedtoescapeahazard).ThestudypresentedinthischapterexpandsontheworkdescribedinChapters3and4.InChapter3,weexploredhowaconventionalgeneticalgorithmcouldbeappliedtoopti-mizemorphologicalcharacteristics,includingcaudal˝n˛exibility,andcontrolpatternsforarobotic˝shprototype.However,thatsystemhadlimitedcapabilities(i.e.,nosensoryfeed-back,nocommunicationabilities,andalow-powermicro-controller),andoptimizationdidSomeoftheresultsanddescriptionsinthischapterwerepublishedin[27].71notaddressmultipleobjectives.InChapter4,wedescribedaninitialapproachtoapplyingEMOmethodstorobotic˝sh.However,thatinvestigationusedaplatformwithlimitedsensingcapabilities.Theelectromagneticallydrivenrobotic˝shinthischapter,showninFigure5.1,issmaller,hasmorecomputationalpowerandsensingcapabilities,andenablesadirectcalculationofenergyusage,asdiscussedinSection5.1.Wehavealsore˝nedandenhancedthesimulationmodelingfor˛exible˝ndynamics,andwehaveconductedmorephysicalvalidationtrialswiththenewrobotic˝shprototype.Finally,whereasinChapter4wefocusedonusefulmechanicalpower,hereweareabletooptimizeforboththerobot'sspeedandtheaverageelectricalpowerexpended.Asaresult,theanalysisofPareto-optimalsolutionsyieldsresultsnotapparentintheearlierstudies.(a)(b)Figure5.1:(a)Smallrobotic˝shcastfromliquidrubber;thebodyhasbeenpaintedgrayandtheprintedcaudal˝nsaredetachable.(b)Thedevice'scustomPCBandrechargeablebattery.5.1DesignofaSmallRoboticFishInthissectionweprovidedetailsofthetargetrobotic˝sh,thesimulationmodel,andtheevolutionaryalgorithm.725.1.1RoboticFish.Figure5.1showstherobotic˝shusedtotestandvalidatethemethodsproposedinthischapter.Thecaudal˝nisdetachable,enablingustotestmany˝ndesigns,andtherobotic˝shisintendedtooperateeitherautonomouslyorviaremotecontrolforuptothreehoursundernormalconditions(i.e.,non-continuouscommunication,averageactuatorusage).Thedeviceispoweredbya150mAhlithium-ionpolymerbattery,whichprovidesovertwohoursofcontinuousoperationundermaximumload(includingwirelesscommunication,sensing,andactuation).5.1.2BodyandFinFabrication.Thebodyoftherobotic˝shisdesignedtobeassmallaspossible,whileincorporatingallcomponentsnecessaryforuntetheredoperation.Thebodyiscastfromliquidrubber(Smooth-OnEco˛exR00-30),whichresultsinasoft,hform.Allelectricalandmechanicalcomponentsareplacedina3D-printedmoldandtheliquidrubberispouredaroundthem.Themold(seeFigure5.2)wasproducedwithanObjet350Connexprinter.Aphotographofseveral3D-printedcaudal˝nscanbeseeninFigure5.3.Detailsregarding˝ndesignandfabricationprocessarethesameasinChapter4.(a)(b)(c)Figure5.2:(a)ASolidworksmodeloftherobotic˝shmold.(b,c)Twoimagesofthe3D-printed,clearplasticmoldusedduringthecastingprocess.73Figure5.3:Aphotographofseveral3D-printedcaudal˝ns.Each˝nhasdi˙erentmorpho-logicalcharacteristics:length,height,and˛exibility.5.1.3CustomPCB.Tocontroltherobotic˝sh,wedesignedacustomprintedcircuitboard(PCB),picturedinFigure5.1(b).ThePCBincludesa32-bitARMmicrocontroller(AtmelSAMD20),a6-axisinertialmeasurementunit(IMU)(InvenSenseMPU-6050),twolightsensors(IntersilISL29101),andwirelesscommunication(NordicSemiconductornRF24L01+).Themicro-controlleriscapableofexecutingcomplexadaptivecontrolalgorithmswhile˝lteringsensorydata.TheIMUenablesthedevicetomeasureitslinearandangularaccelerations,whichcanbe˝lteredtoprovideestimatesofvelocity.Wirelesscommunicationallowsthedevicetobecontrolledandhaveitssoftwareupdatedremotely,aswellasdeliversensedinformationtoabasecomputer.5.1.4ElectromagneticActuator.TheelectromagneticactuatorisdepictedinFigure5.4.Theactuatorcomprisesacoilofmagnetwire(9.5mmoutsidediameter),aneodymiumpermanentmagnet,anexternalcenteringmagnet,andanattachmentpointforacaudal˝n.Tooperatetheactuator,avoltageisappliedacrossthecoil'sterminals.Thecoilcreatesanelectromagnetic˝eldthatexertsatorqueonthepermanentmagnet,causingitspolestoalignwiththemagnetic˝eld.74Theactuatorusedinthisstudyhasa˝xedmaximumamplitudeof38andacenteringmagnetthatcausestheactuatortoreturntoitscenterwhennovoltageisappliedtothecoil.Equation(5.1)isusedtocalculatetheangularaccelerationofthepermanentmagnetresultingfromanappliedvoltage:=V0NlRIeffective;(5.1)whereistheangularacceleration,Visthevoltageappliedacrossthecoil,0isthemagneticconstant(orthepermeabilityoffreespace),N,l,andRarethenumberofturns,length,andresistanceofthecoil,andIeffectiveisthemomentofinertiaforthepermanentmagnetwithanattached˝n.Thisdeviceexhibitsatorqueofroughly500Nmwithavolumeof35mm3,whichyieldsatorque-per-volumeof15Npermm3.Incontrast,atypicalcommoditymicroservomotorprovidesatorqueofapproximately75mNmwithavolumeof2200mm3resultingin35Npermm3torque-per-volume.Forourpurposes,alternativestotheelectromagneticactuatorareeithertoolarge(commodityservomotors),requirehighervoltages(piezoelectricmotors),orgeneratelesstorque(electropolymersandshapememoryalloys)[4].Likewise,systemsthatneedexternalmagnetic˝elds,suchasthemicrorobotsdevelopedformedicalapplications[99],aretoolimitingintermsofthetypesofenvironmentsinwhicharobotcouldbedeployed.(a)(b)(c)Figure5.4:(a)Top-viewdiagramoftheactuator,and(b,c)photographsoftheelectromag-neticactuator.75Electromagneticactuatorshavebeenusedasmotorsforrobotic˝shbyafewotherresearchgroups.Tosupporttheirfuelcellstudies,Takadaetal.[121]usedasimilaractuatorforarobotic˝sh;thisdevicewas10cminlength,haddivingcapabilities,butdidnotincludeanysensing.Shinetal.[113]usedacomparableelectromagneticmechanismforrobotictadpoles,whichwerelessthan3cminlengthbutdidnotincludeanysensingorcomplexcontrolcapabilities.5.1.5ControlSignal.Theactuatoriscontrolledbysupplyingapositive,negative,orzerovoltagetothecoil.Forthisstudy,wedonotconsidervoltagesotherthan3.3or-3.3volts.Thisleavestwocontrolparametersforforwardthrust:anoscillatingfrequencyandapulse-width-ratio(PWR).Similartoaduty-cycle,PWRde˝nesthefractionoftimethecontrolsignalisactiveduringagivenperiod.Wenote,however,thatPWRhasnegativecomponents,asdepictedinFigure5.5.Forthecontrolsignalexampleshown,PWRissetto0.4,whichresultsinanappliedvoltagethatisactiveonly40percentoftheperiod.Withthissetup,adjustingPWRistheonlywaytoalterenergyconsumption.Forexample,aPWRof0.8willresultintwicetheamountofenergyconsumedwhencomparedtotheexamplesignal.PWRcanrangefrom0to1andtheresultingcontrolsignalwillbeaconstantzeroorasquarewave,respectively.Howtochoosethesetwoparameters(frequencyandPWR)dependsonthedimensionsofthecaudal˝nandthedesiredbalancebetweenspeedandenergyconsumption.Forinstance,itisrarelyusefultosetthePWRto1,asdoingsowillresultinwastedenergy.Speci˝cally,theactuatorwillconsumeenergywhileactivelythe˝ntooneside,whichdoesnotgenerateanyadditionalthrust.Moreover,thetimerequiredforthe˝ntoreachitsmaximumamplitudedependsoncaudal˝ndimensions.Sincetheactuatorwillalwaysgeneratethesametorque,usingalarger˝nwillresultinlowerangularaccelerationcomparedto˝nswithlesssurfaceareabecausethe˝nison76Figure5.5:Anexampleofthecontrolvoltagesignalwiththeresultingsimulatedmotorangle.Thedashed,bluelinedepictsthecontrolsignalwithafrequencyof0.5HzandaPWRof0.4(PWRindicatesthefractionoftimespentateither3.3or-3.3V).Thesolid,orangelineistheangleofthemotor,whichhasarisetimerelatedtotheappliedtorque,andafalltimerelatingtothecenteringtorque,duetothecenteringpermanentmagnet.alargervolumeofwater.Whileconductinginitialtestsoftherobotic˝sh,wefoundthattheactuatorwase˙ectiveonlyfor˝nswithasurfacearealessthan5cm2.5.1.6EvolutionaryOptimization.AsdoneforthestudypresentedinChapter4,weareagainusingtheSimulinkenvi-ronmenttoperformnumericalsimulations.Thedynamicsimulationmodeltakesseveralparameters,includingthefrequencyandPWRofthecontrolsignalandthelength,height,and˛exibility(whichdictatesthespringanddampingcoe˚cientsfor˝nsegments)ofthecaudal˝n.Evolvedgenomescomprisevaluesfortheseparameters,andtheallowablerangeforeachislistedinTable5.1.Valuesinthetableweredeterminedbytestingthephysicallimitationsofthephysicalrobotic˝sh.Forinstance,ifthefrequencyisnearorabove5Hz,thecaudal˝nwillnothaveasu˚cientamountoftimetorotatebeforetheactuatorreversesdirection.Ine˙ect,thecaudal˝nsimplyvibrates,producingverylittlethrust.The77limitsonspringcoe˚cientsallowforcaudal˝nstobehavesimilarlytomaterialsas˛exibleasrubberorassti˙ashardplastic.Table5.1:RangeofEvolvedParameters.MinMaxFrequency(Hz)0.15.0Pulse-Width-Ratio0.11.0FinLength(cm)1.04.0FinHeight(cm)0.31.3SpringConstant(Nm/rad)5e-51e-15.1.7NSGA-IICon˝guration.Asmentionedpreviously,theNSGA-IIalgorithm[37]waschosentoconducttheevolu-tionarymultiobjectivedesign.NSGA-IIe˚cientlysortsacombinedpopulationofparentandchildren(createdusinggeneticoperators)intodi˙erentranksofnondominatedParetofronts.SelectionproceedsbyacceptingindividualsfromeachrankinsequenceuntilNindividualsareselectedintotal.Crowdingdistanceisusedasatie-breakerwhenanentirerankcannotbeselected(becauseitwouldrequireacceptingmorethanNindividuals).Crowdingdis-tanceiscalculatedastheEuclideandistancebetweenthe˝tnessvectorsoftwoindividuals.Thismethodensuresthattheelite(best)individualsareretainedbecauseallindividualsinthe˝rstrankarePareto-optimal.Forevolvinggenomescomprisingrealvalues,NSGA-IIre-quirestheusertosetthefollowingfourparameters(valueschosenforourexperimentsareinparentheses):theprobabilitiesofcrossover(90%)andmutation(20%),andthedistributionindexforbothsimulatedbinarycrossover(20)andpolynomialmutation(20).StartingfromvaluesrecommendedbyDebetal.[37],thesevaluesweredeterminedexperimentallyandcausethepopulationstoconvergeinrelativelyfewgenerations(i.e.,within100generations,whichcorrespondstoapproximately6400˝tnessevaluations).785.1.8FinConstraint.Alongwithevolvingtheaboveparametersasreal-valuednumbers,NSGA-IIalsoac-commodatesconstraints.Inourstudy,thedynamicmodelisonlyvalidforanelongated˝ninwhich˝nlengthisatleastthreetimesthe˝nheight:length3height0;(5.2)wherelengthandheightrefertotheevolveddimensionsoftherobotic˝shcaudal˝n.InNSGA-IIthislimitationiscon˝guredasaconstraint,whichenablesthealgorithmtosmoothlyfollowagradientfrominfeasible(i.e.,asolutionthatviolatesaconstraint)tofeasiblesolutions.5.1.9FitnessEvaluation.Eachindividualinthepopulationisevaluatedfor10secondsofsimulationtime;how-ever,onlythesecondhalfofthisperioddetermines˝tness.Thissetupallowstherobotic˝shtoreachacruisingspeedand˝nalheading,withaveragespeedcalculatedoverthe˝nal5seconds.Inourpreliminarystudy[28](Chapter4),e˚ciencywasde˝nedastheratiobetweenusefulandtotalpower,whereusefulpowerwascalculatedusingtheproductofthetotalpropulsiveforceprojectedontothetrajectoryoftravelandthetravelspeed,andthetotalpowerwascalculatedusingthesumofallmechanicalpowerexertedbythecaudal˝n.Thesecalculationsrequireinstantaneouspowertobecalculatedateverysimulationtimestep.Incontrast,sincePWRofthecontrolsignalisknown,inthecurrentstudyaverageelectricalpowerisdirectlycalculatedusingthefollowingequation:Pavg=PMAXPWR;(5.3)79wherePavgandPMAXrefertotheaverageandmaximuminstantaneouselectricalpowerdeliveredtothemotor,respectively.SincePMAXandPWRareknownvalues,acontrolsignal'senergyconsumptioncanbecalculateddirectly.5.2EMOSimulationResultsWeconducted25replicateEMOsimulationstrials,eachwithpopulationsof64indi-vidualsevolvingfor100generations.Simulationparametersfortherobotic˝shdynamics,otherthanthoserelatedtothe˛exible˝nandthecontrolsignal,arebasedontherobotic˝shprototype(seeSection5.1.1)tobeusedforphysicalvalidation.Mostreplicates(23of25)convergedtonearlyidenticalParetofronts.Figure5.6displaystheevolutionaryhistoryofonereplicatesimulation.Thepopulationisrandomlyinitialized(Figure5.6(a)),andthenevolvestowardthe˝nal,optimalParetofront(Figure5.6(f)).(a)(b)(c)(d)(e)(f)Figure5.6:Theevolutionaryhistoryofasinglereplicatesimulationrun.Everyindividual(includingdominatedindividuals)inthepopulationatagivengenerationisplottedineach˝gure.Forall˝gures,theunitsforspeedandpowerarecm/sandmW,respectively.Figure5.7(a)plotstheaveragepowerandspeedofeveryevolved(feasible)individualfromallreplicatesimulations,withthePareto-optimalindividualsappearingincolorsotherthanlightgray.Jitterappearsinthe˝nalParetofronts(somenon-graypointsappearto80bedominated)becausereplicatesimulationsdonot˝ndidenticalsetsofPareto-optimalsolutions.Theresultsaregroupedintothreeclusters.FirstistheEnergyCluster,whichincludesindividualsthatonaverageconsumelowamountsofpower.Theseindividualsappearinthelower,left-handsectionofthe˝gure,andhavespeedslessthan5cm/sandconsumelessthan60mWofpoweronaverage.Second,theSpeedClusterincludesindividualsthatareinthemiddle,right-handsectionofthe˝gure,andwhichconsumeroughlytwicetheamountofpowerbutalsoswimtwiceasfast.TheLocal-OptimumClusteristhesetofindividualsthathaveconvergedtoalocaloptimumandappearintheupper,centersectionoftheplot.ThereplicaterundepictedinFigure5.6appearstohaveencounteredthesamelocaloptimum(showninplot5.6(c))butwasableto˝ndanevolutionarypathtowardthe˝nalParetofront.(a)(b)(c)(d)Figure5.7:(a)Everyevolvedindividualamongall25replicatesimulations(excludinginfea-siblesolutions).Graymarkersdenoteindividualsandnon-graymarkersdenoteindividualsthatwerePareto-optimalforagivenreplicateexperiment.Threeclustershavebeenidenti˝edandgivendi˙erentcolors.Theremainingplotsdisplayindividualsbelongingtotheenergycluster(b),speedcluster(c),andlocal-optimumcluster(d).Pleasenotethattheclusterplotsdonotsharethesamerangesfortheiraxes.81ThesethreeclustersarehighlightedinFigure5.7(b-d).ThecloserviewsshowthatwithineachclusterasmallerParetofrontisformed(maximizingspeedandminimizingpowerusage).Inspectinghoweachoftheparametersevolvedgivesinsightintowhatcreatedthesedistinctclusters.Forexample,Figure5.8showsthatcaudal˝nmorphology(length,height,and˛exibility)remainsfairlyconsistentforallPareto-optimalindividualsevenacrossclusters;thedatashowninthis˝gureisscaledbetween0and1sothateachparametercanbeplottedonthesameaxis.(a)Figure5.8:Box-plots(toprow)andhistograms(bottomrow)ofthedistributionsofeachevolvedparameter,scaledbetween0and1,forallofthePareto-optimalindividuals.Foreachbox-plot,thecentralredlineindicatesthemedian,thelightblueboxoutlinesthe25ththrough75thpercentiles,thedashedlightblueverticallinesdenotetheboxwhiskers(non-outliers),andthedarkbluecirclesdenoteoutliers.Ineachhistogram,thehorizontalaxesarescaledparametervaluesandtheheightofeachbinindicatesthedensityaroundagivenvalue.Convergenceofmorphologyparametersindicatesthatforthisparticularrobotic˝shanidealcaudal˝nisapproximately3.4cminlength,1cminheight,andhastheminimumallowedspringconstant(i.e.,a˛exibilityresemblingarubbermaterial).Sincethe˝nsdi˙eronlyslightlyamongsolutions,wecaninferthatthecontrolparametersaccountformostofthediversityinthe˝nalpopulations.Apairwisecomparisonoftheparameters'variancesusingtheBrown-Forsytheequalityofvariancestest(andtheBonferronimultiplicity82correctionresultinginaPvalueof0.005)showsthatthevariancesinthecontrolparameters(frequencyandPWR)aresigni˝cantlyhigherthanthoseofthemorphologicalparameters(length,height,and˛exibility).ReferringagaintoFigure5.7(b-d),weconcludethatthreedistinctcontrolstrategieshavebeenevolved,twoofwhicharepractical.Speci˝cally,theenergy-basedcontrolstrategy,whichhasamaximumspeedofroughly5cm/s,andthespeed-basedstrategy,whichhasaminimumaveragepowerexertedofapproximately100mW.Thisresultsuggeststhat,atleastfortherobotusedinthisstudy,morphologycanbe˝xedandthetrade-o˙sbetweenthetwoobjectivescanbeadjustedonlinebychoosingadi˙erentsetofparametersfromtheParetofront.Figure5.9comparesparametersamongthethreedi˙erentcontrolstrategies.ThePWRofevolvedcontrolpatternsislowerforlowerpower-consumingindividuals(medianvalueof0.2),andhigherforfasterindividuals(medianvalueof0.5).Thisresultisexpected,astheonlywaytominimizeenergyusageistoreducetheamountofactivetimefortheactuator.Activetimereferstothedurationoftimethatavoltageisappliedtotheelectromagneticcoiloftherobot'sactuator.Additionally,fasterswimmingindividualsexhibithigherfrequencies(medianvalueof4.0)thanlowpower-consumingindividuals(medianvalueof1.5).Thefrequenciesforlowerpower-consumingindividualssuggestthattheyaresacri˝cingspeedbyevolvinglowerfrequencies(i.e.,sincehigherfrequenciesgenerallyleadtohigherspeedsbutincreasingfrequencydoesnota˙ectpowerconsumption).However,aswillbediscussedlater,increasingfrequencyoftheseindividualsactuallyreducesspeed.Asforthelocaloptimumcluster,itappearsthatNSGA-IIlocatedaregioninthesearchspacewherespeedwasmaximizedbyincreasingthePWRandsacri˝cingpowerconsumption.Likelyduetoitsexploitativenature,thealgorithmwasunabletoescapethisregionofthesearchspaceinsomereplicates.Thatis,NSGA-IIexpandsitscurrentParetofrontbysearchingtheneighborhoodaroundcurrentPareto-optimalindividuals.Therefore,itseemsthatitisdi˚cultto˝ndanevolutionarytrajectoryfromthislocaloptimumtotheactualoptimalPareto-front.83Figure5.9:AcomparisonofcontrolparametersamongthethreePareto-optimalclusters.Frequency(right)andthePWR(left)arebothincreasedtoachievefasterswimmingspeedsanddecreasedtoattainlowerpowerconsumption.Foreachbox-plot,acentralredlineindicatesthemedian,aboxoutlinesthe25ththrough75thpercentiles,avertical,dashedlinedenotetheboxwhiskers(non-outliers),andredmarkersdenoteoutliers..Additionally,Pareto-optimalsolutionstendtohaveeitherhighfrequencyandhighPWRorrelativelylowvaluesforbothparameters.ThistrendoccursbecauseahighfrequencywithalowPWRwillresultinverylittle˝nmotion,duetothedurationoftimethattheactuatorwouldremainactive.Forexample,withafrequencyof5Hz(themaximumallowedvalue)andaPWRof0.4,theactuatorwillremainactiveforonly40msatatime,whichisnotenoughtimetorotatethecaudal˝nforgeneratingthrust.Forcomparison,themedianvalueforactivationdurationforallPareto-optimalsolutions(excludingthelocaloptima)is64mswithastandarddeviationof10ms,andtheminimumandmaximumpossiblevaluesare10and1000ms,respectively.Thus,althoughmostsolutionsrequiretheactuatortobeactiveforrelativelyshortdurations,thereappearstobealowerlimitofapproximately50ms.Toexplorethegeneralityofevolvedsolutions,weselectedthreeevolvedindividualsandasolutionwithhand-chosenparameterstoexaminefurther.ThevaluesfortheseparametersarelistedinTable5.2.Theseparametersetsincludemorphologiesneartheoptimalvalues84listedpreviously,andexhibitarangeofcontrolparametervalues.TheBestSpeedandBestEnergyparametersrefertoevolvedindividualsneartheendpointsoftheParetofront(thehighestspeedandlowestpowerconsumption),theDominatedparametersrefertoarandomlyselectedindividualfromanearlygenerationthatisnotneartheParetofront,andHandChosenrepresentsparametervaluesthatwehavechosenusingexpertknowledge.Table5.2:ParameterSetsSelectedforFurtherInvestigation.FrequencyPWRLengthHeightSpringConstantBestSpeed3.8Hz0.503.8cm1.0cm50uNm/radBestEnergy1.1Hz0.123.0cm1.0cm94uNm/radDominated3.0Hz0.643.9cm1.3cm48uNm/radHandChosen1.0Hz1.003.3cm1.1cm100uNm/radForeach˝ndesignshowninTable5.2,we˝rstconductedaparametersweepoverthecontrolparameters.Thepurposeofthesecontrolparametersweepsistoevaluateindividualsunderarangeofoperatingconditions,becauseformanypracticalapplicationsitwillbeusefultodynamicallyadjusttherelativeimportanceofthetwoobjectives.Insuchasituation,themorphologyis˝xedandonlythecontrolpatternscanbeadjusted.Forexample,ifarobotic˝shisabletochargeitsbatteriesviaasolarcell,itwillbemoreimportanttoconserveenergywhenoperatinginlimitedsunlight.However,underidealconditionstherobotic˝shmaybeabletosacri˝cepowerconsumptionforbetterperformance(e.g.,swimmingspeed).Furthermore,evenunderidealconditionstheremaybevalidreasonstoswimatspeedslowerthanthemaximumvalue(e.g.,iftherobotic˝shistrackinganotherobject).Thus,duringthesesweepsallparametersrelatingto˝nmorphologywere˝xed.Resultsfromtwoexperiments(BestSpeedandBestEnergy)areplottedinFigure5.10.PlotsforDominatedandHandChosenarenotshownastheydisplaycharacteristicssimilartothosedemonstratedbyBestEnergy.Wehavenotprovideddatafortheparametersweepsagainstaveragepower,asaveragepowerisdirectlyproportionaltoPWR.Consideringtheparametersweepplots,adesirabletraitistheabilitytoadjustspeedtospeci˝cvalues.TheonlysetofparameterstodemonstratethistraitistheBestSpeed85(a)(b)(c)(d)Figure5.10:Controlparametersweepsfortwosetsofparameters:evolvedindividualswithhighspeed(a,b)andwithlowenergyconsumption(c,d).individual,asshowninFigure5.10(a,b).Thiscon˝gurationallowstherobotic˝shtoswimatspeedsrangingfrom1to9cm/sbyadjustingthefrequencyorPWR.Additionally,asdepictedinFigure5.10(b),therobotic˝shiscanswimatarangeofspeedsbyadjustingthePWRvalue,whichmeansthattherobotic˝shcane˙ectivelyadjustitsaveragepowerconsumptionfrom20to220mWanditsaveragespeedfrom0.2cm/sto9cm/s.More-over,theapparentlinearrelationshipbetweenPWRandspeedallowsustoconsiderthetrade-o˙sbetweenpowerconsumptionandswimmingspeedwithoutworryingaboutdrasticperformancechangesthatcanoccurundercertainconditions.Speci˝cally,wedonotneedtoavoidcertainvaluesforPWRduetoanonlinearrelationshipoverasmallerinterval.86However,anyvaluesover120mWofaveragedeliveredpower(correspondingtoaPWRof0.5)appearstowasteenergy,asaveragepowerincreasesbutspeeddoesnot.Theremainingthreesetsofparameters(BestEnergy,Dominated,andHandChosen)donotexhibitthesameabilitytoadjustspeed.However,di˙erentspeedsmaybeachievedbytheseindividualsbyadjustingbothfrequencyandPWRsimultaneously.Wealsoconductedsimilarsweepsoverthemorphologicalparameters:˝nlength,height,and˛exibility.Inthesesweeps,thecontrolparameterswere˝xed.ResultsareagainplottedfortheBestSpeedandBestEnergyparametervalues;thisdatacanbefoundinFigure5.11.Forthebestspeedindividual(Figure5.11(a-c))itisapparentthatthemorphologyisop-timizedforspeed.Speci˝cally,anychangeinmorphologyresultsinaspeedreduction.However,speedsattainedbyparametervaluesthatleadtolowerenergyconsumption(Fig-ure5.11(d-f))arelessa˙ectedbythemorphologicalparameters.Thatis,changingthemorphologyresultsinsmallerchangestospeed.(a)(b)(c)(d)(e)(f)Figure5.11:Morphologicalparametersweepsfortwosetsofparameters:(a,b)anevolvedindividualwithhighspeed,and(c,d)anevolvedindividualwithlowenergyconsumption.875.3PhysicalValidationOnegoalofthisstudyistoevolvesolutionsthatcanbeappliedtoaphysicalrobot.Tocon˝rmwhatweseeinsimulationcarriesintorealityweselectedthreeevolveddesignsandonehand-chosenparametersettovalidateexperimentally.Theprocessusedtodesignandfabricatecaudal˝nsisthatsameasthatdiscussedinSection4.2.Speci˝cally,wedesignedacompositematerialthatallowsusto3Dprintcaudal˝nswithspeci˝edcharacteristics(i.e.,dimensionsand˛exibility).Thephysicalexperimentsusethesamecontrolandmorphologicalparametersastheirsimulatedcounterparts(thesevaluesarelistedinTable5.2).Makinguseofthecomposite˝ns,describedpreviously,enables3Dprintingofcaudal˝nswithaspeci˝edmorphology(˝ndimensionsand˛exibility).Thefabricated˝nsareattachedtothesmallrobotic˝shpicturedinFigure5.1.Therobotic˝shisthenplacedinalargewatertank,andspeedismeasuredusingamethodsimilartothatdescribedinthemethodssectionregardingtheevolutionary˝tnessevaluation.Speci˝cally,therobotic˝shspeedismeasuredafteritreachesasteadymaximumspeed.Forphysicalexperiments,eachreportedspeedvalueistheresultofaveragingdatafrom˝vetrials.ResultsfromtheseexperimentscanbefoundintheTable5.3.Table5.3:ComparisonBetweenSpeedsAcquiredinSimulationandAttainedbytheRobotinaWaterTank.SimulationSpeedMeasuredSpeedAbsoluteErrorBestSpeed8.6cm/s3.8cm/s4.8cm/sBestEnergy2.2cm/s1.9cm/s0.3cm/sDominated3.2cm/s3.5cm/s0.2cm/sHandChosen2.7cm/s2.9cm/s0.2cm/sForthreeofthefourvalidatedparametersets,theresultsshowacloserelationshipbetweensimulationandreality.Andforallfoursetsofparameters,theachievedspeedsdemonstrateaconsistentorderbetweenthesimulationandtheexperimentalmeasurement;inotherwords,rankingthedesignsbytheirachievedspeedswillresultinthesameorderingin88simulationasinexperiments.Theseobservationsprovidesupportforthee˙ectivenessofthesimulationmodelandthedesignapproach.Ontheotherhand,wenotethattheBestSpeedevolvedparametersresultinaconsiderablyfasterspeedinsimulationthaninexperiments.Analyzinghigh-speedvideooftherobotic˝shrevealsthatthephysicalcaudal˝ncannotreachaslargeanamplitudeasinsimulation.Thisbehaviorcanlikelybeattributedtothehighfrequencyatwhichthedeviceisoperating.AslistedinTable5.2,thefrequencyforthisindividualis3.8Hz,whichisneartheupperlimitwefoundtobephysicallypossibleduringourinitialprototyping.Modelingoftheelectromagneticactuatorwillneedtobere˝nedtoensurethatthesimulationenvironmentcannotbeexploitedbytheevolutionaryalgorithm,asoccursfortheBestSpeedexperiment.Evenwithahigheraccuracysimulationtherealitygapcanstillbeanissue.Forthisreason,manyresearchgroupsareinvestigatingexplicitmethodsforcounteringtherealitygap.Twoofthemostprominentarethetelligenttrial-algorithm[34]andself-modeling[10].Asanalternativetothesemethods,inthenextchapterwediscussourinvestigationsregardingadaptivecontrolasamethodformitigatinge˙ectsoftherealitygap[25].Inessence,thecontrollerwouldadapttothedi˙erencesbetweensimulationandreality.5.4ConclusionInthischapter,weinvestigatedanevolutionarymultiobjectiveoptimizationapproachtothedesignofmorphologyandcontrolforarobotic˝shwitha˛exiblecaudal.Asawayofillustration,wechosetwocommonobjectives(maximizingswimmingspeedandminimizingenergyconsumption),althoughourapproachwouldworkwithanycombinationofobjectives(forexample,agility,speed,andenergye˚ciency).Therobotic˝shusedduringvalidationexperimentsutilizesanelectromagneticactuator,whichenablesadirectcalculationofenergyconsumption.Simulation-basedoptimizationallowedustoexploretrade-o˙sbetweenthe89twocompetingobjectives.Certainevolvedandhand-chosensetsofparameterswerethenselectedforphysicalvalidation.Withtheaidofa3Dprinter,simulatedcaudal˝nswerefabricatedtomatchspeci˝cations,andsimulationresultswerevalidatedinalargewatertank.Asidefromindividualsexhibitingthefastestspeeds,physicaltrialssupportthee˙ectivenessoftheproposedevolutionarydesignapproach.Surprisingly,controlparameters(frequencyofoscillationandpulse-width-ratio)werefoundtoaccountformostofthevariationamongPareto-optimalindividuals.Thisdiscoveryisintriguing,asitindicatesthatatleastfortherobotic˝shandobjectivesemployedinthisstudy,mostoftheengineeringe˙ortcanbeinvestedincontrollerdesign,whilelesstimeneedstobespentonmorphology.Speci˝cally,morphologycanbe˝xedandcontrolstrategiescanbedesignedsuchthatmostofthePareto-optimalsolutionspacecanbereached.Asaresult,therobotic˝shcanmovefromhighspeed,highenergystatestolowspeed,lowerenergystatesthatarenearthe˝nalPareto-front.Fixingthemorphologicalcharacteristicsandperformingparameterssweepsonthecontrolparametersenabledthediscoveryofindividualsthatexhibitsuchgenerality.Thepurposeofthisstudywastoinvestigatehowcompliantcaudal˝nscouldbematchedwithcontrolparameters(asinChapter3)whileconsideringperformanceandenergycon-sumption.Morebroadly,thetechniquespresentedinthisstudyareintendedtobeasteptowardourgoalofproducingoptimizationmethodsthatcanbeusedforanyrobotsincor-poratingsoft/˛exiblecomponents.Althoughphysicaltestsyieldedresultssimilartothosefoundinsimulation,duetotherealitygapthereweresomediscrepancies.Furthermore,thediscrepancybetweensimulationandrealitytendstoincreaseastherobotoperatesforlongerperiodsoftime;primarilyduetobatterydepletionandchangestothe3Dprintedcaudal˝ndynamicscausedbyaging.Inthenextchapter,weintroduceourresearchaimedatmitigatingthesemodelingerrorsandruntimechanges.Namely,weuseadaptivecontrollerstoautomaticallyadapttounexpectedvariationsthatoccurduringoperation.90Chapter6EvolvingAdaptiveControlThepreviouschaptersdemonstratedthatevolutionarycomputationmethodscanbeusedtodiscovercombinationsofmorphologicalandcontrolpatternvaluesthatresultinbothhighperformanceandenergye˚ciency.However,thosemethodsareappliedo˜ine,priortodeployment.Yet,manyroboticsystemsexperience˛uctuatingdynamicsduringtheirlifetime.Variationscanbeattributedinparttomaterialdegradationanddecayofmechanicalhardware.Oneapproachtomitigatingtheseproblems,aswellashelptocrosstherealitygap,istoutilizeanadaptivecontroller.Forexample,inmodel-freeadaptivecontrol(MFAC)acontrollerlearnshowtodriveasystembycontinuallyupdatinglinkweightsofanarti˝cialneuralnetwork(ANN).However,determiningtheoptimalcontrolparametersforMFAC,includingthestructureoftheunderlyingANN,isachallengingprocess.InthischapterweinvestigatehowtoenhancetheonlineadaptabilityofMFAC-basedsystemsthroughcomputationalevolution.Someoftheresultsanddescriptionsinthischapterwerepublishedin[24]and[25].916.1Model-freeAdaptiveControlAsdescribedinChapter2,theMFACusedinourresearchisbaseduponanadaptiveANN,whoseweightsaredynamicallyupdated.AdiagramoftheMFACANNisshowninFigure6.1.Asaninput,theANNreceivesacontinuouserrorsignale,whichisdiscretizedatasamplingrateTs.Samplingtheerrorsignalisnecessaryforcomputersystems,whichareunabletohandlecontinuousdata.Thediscretizederrorsignalisnormalizedbetween-1and1usingacon˝gurableerrorboundeb.Thisnormalized,discretizederrorsignal,E,ispassedtothe˝rstinputneuron,I1,andthenpropagatedtoeachsubsequentinputneuronatsuccessivesamplingtimes.ThisprocessisrepeatedsuchthattheNIinputneuronsstoretheNmostrecenterrorsignals(E1::ENI).BystoringthesevaluesandusingthemasadditionalinputstotheANN,theMFACtakesadvantageofstateinformationandcanbecalledadynamicsystem.Additionally,ateachsamplingtime,theinputneuronvalues(E1::ENI)arefedforwardfromtheinputneurons(I1::INI)totheNHANNhiddenneurons(H1::HNH),whichinturnfeedtheirvaluestotheoutputneuron(V).The˝naloutputoftheMFAC,u,isasummationoftheoutputneuronvalueandthecurrenterrorsignalampli˝edbythecontrollergainKc.Thenumberofinputneurons,NI,thenumberofhiddenneurons,NH,aswellastherateatwhichlinkweightsareupdated(calledtheadaptivelearningrate,),arecon˝gurable.ResponsivenessofanMFACisalsoadjustablethroughacontrollergainvalueKc.AnMFACANNactivatessimilarlytoatypicalfeed-forwardneuralnetworkasfollows:pj(n)=NIXi=1wij(n)Ei(n)+biasj;(6.1)qj(n)=˙(pj(n));(6.2)where(n)denotesthesampletime,pjisanintermediateoutputofthejthhiddenneuron,wijandEiaretheweightandvalueofaconnectionfromtheithinputneurontothejth92Figure6.1:AgraphicalrepresentationoftheMFACANN.Acontinuoustimeerrorsignaleis˝rstnormalized(Nrmblock)andthenpropagatedthroughtheinputneuronsIi.TheANNisthenactivatedasafeed-forwardnetworktoproduceanoutputV.The˝nalcontrolleroutputuisanampli˝edsummationofVande.hiddenneuron,respectively,biasjisthebiasvalueforthejthhiddenneuron,qjistheoutputofthejthhiddenneuron,and˙()isthesigmoidfunction.Thesigmoidfunctionanditsderivativeareasfollows:˙(x)=11+ex;(6.3)_˙(x)=˙(x)(1˙(x)):(6.4)Thesimplenatureofthesigmoid'sderivativeisoneoftheprimaryreasonsthatsigmoidsareusedforneuralnetworks.Theoutputoftheneuralnetwork,andoutputoftheentireMFACareasfollows:o(n)=NHXj=1hjqj+biaso;(6.5)u(t)=Kc(o(t)+e(t));(6.6)93whereoistheoutputoftheoutputneuron,hjisthevalueofaconnectionfromthejthhiddenneuron,biasoisthebiasoftheoutputneuron,uistheoutputoftheMFAC,Kcisthecontrollergain,andeisthecontinuouserrorvalue(betweenthesetpointrandtheoutputy).WhatsetsanMFAapartfromtypicalfeed-forwardneuralnetworksishowtheweights(wijandhj)areupdated.Theconnectionweightupdateequationsarederivedbyminimizingerror.Theobjectivefunctionis:Es(t)=12(r(t)y(t))2=12e(t)2;(6.7)Tominimizetheobjectivefunctionwithrespecttotheweightsbetweentheinputandhiddenlayers(wij),wetakethepartialderivativeasfollows:wij(n)/@Es@wij;(6.8)=@Es@y@y@wij;(6.9)=@Es@y@y@u@u@wij;(6.10)=@Es@y@y@u@u@o@o@wij;(6.11)=@Es@y@y@u@u@o@o@q@q@wij;(6.12)=@Es@y@y@u@u@o@o@q@q@p@p@wij:(6.13)Similarly,fortheweightsfoundbetweenthehiddenandoutputlayers(hj)weendupwiththefollowingpartialderivatives:94hj(n)/@Es@hj;(6.14)=@Es@y@y@hj;(6.15)=@Es@y@y@u@u@hj;(6.16)=@Es@y@y@u@u@o@o@hj:(6.17)Thesolutiontothesepartialfractionsareasfollows:@Es@y=e;(6.18)@y@u=Sf;(6.19)@u@o=Kc;(6.20)@o@q=NXj=1hj;(6.21)@q@p=qj(1qj);(6.22)@p@wij=Ei:(6.23)whereSfisaproblemspeci˝csensitivityfunction.GivenEquations6.13,6.17,andtheabovesolutionstothepartialderivatives,we˝ndthefollowingupdateequationsfortheneuralnetworkweights:wij(n)=KcSf(n)e(n)qj(n)(1qj(n))Ei(n)NXk=1hk(n);(6.24)hj(n)=KcSf(n)e(n)qj:(6.25)95Despitethebene˝tsoftheMFACapproach,determiningtheoptimalvaluesoftheparameters(NI,NH,Sf,,Kc,Ts,andaboundontheerroreb)ischallenginganddependsontheapplicationdomain.6.2MethodsInstudyinganimalbehavior,zoologistsrecognizetheimportanceoftheevolvedrela-tionshipsamongmorphology,perception,action,andenvironment[38].Thisconceptisalsorelevanttocyber-physicalsystems,whereembeddedcomputersystemsneedtointerpretsensedinformationandrespondaccordinglythroughactuators.Controltheoryfocusesonhowtomodifythebehaviorofdynamicsystemsusingfeedback.Thecontrolofacyber-physicalsystemtypicallyinvolvesmonitoringtheoutputofasystem(e.g.,speed,tempera-ture,˛ow-rate,etc.)andadjustingthesysteminputaccordingly.Oftenthegoalofacontrolsystemcanbereferredtoastracking.Theobjectiveforatrackingcontrolleristogenerateasignalthatdrivesthecontrolledsystemtobehaveinasimilarmannertoaninputreferencesignal.Twoexamplesofcontrollingthespeedofarobotic˝shareshowninFigure6.2.Thesysteminputreferencesignal(theblack,dashedline)andthesystemoutput(blueline)arethedesiredandmeasuredspeeds,respectively,oftherobot.Theerrorbetweenthesetwovalues,whichactsasaninputtothecontroller,isshowninred.Thegoalofatrackingcontrolleristominimizetheerror.InFigure6.2(a),thecontrollerexhibitspoortracking,thatis,thesystemoutputdoesnotcloselytrackthereferencesignal(noticethattheoutputspeedoscillatesaroundthedesiredspeed).Incontrast,theFigure6.2(b)showsacontrollerthatise˙ectiveintracking.6.2.1MFACforRoboticFish.Unlikecontrollersthatadaptbyupdatingparametersspeci˝ctoareferencemodelofthesystem,anMFAChowtocontrolthesystembycontinuallyupdatinglink96(a)(b)Figure6.2:Examplesofarobotic˝shcontrollertrackinganinputreferencesignalrepre-sentingdesiredspeed.Ineachplot,thedashed,blacklinedenotetheinputreferencesignal(r),thebluelinerepresentsthemeasuredsystemsoutput(y),andtheredlinerepresentstheerror(e)betweenthesetwosignals.Plot(a)isanexampleofrelativelypoortracking,whereasplot(b)exhibitse˙ectivetracking.weightsinanadaptiveANN.Asaninputtotheneuralnetwork,MFACtakesacontinuouserrorsignalanddiscretizesitbasedonacon˝guredsamplingrate.Usinganerrorsignalasinputtothecontrolsystemisacommonapproachtoattainingatrackingbehavior.Forexample,PIDcontrollersalsouseadiscretizederrorvalue;however,ourexperiencewithsuchsystemsshowthat,whiletheyareabletoadequatelycontrolrobotic˝sh,theybegintoexhibitpoorerperformanceovertimeascharacteristicsofthephysicaldevicebegintochange.BysavingrecenterrorsignalsandusingthemasadditionalinputstotheANN(labeledI1toINIinFigure6.3),theMFACcantakeadvantageofstateinformation,orso-calledneuralnetworkmemory[52].Figure6.3showsablockdiagramoftheMFACcontrollerwiththerobotic˝sh.Theinputtotheentiresystemisareferencesignalr,whichcanbeanyphysicalsignalrelatingtotherobotic˝sh.Forthisstudy,rreferstoadesiredspeed,andtheoutputoftherobotic˝shyistheactual(measured)speed.Forphysicalexperiments,speedcanbemeasuredby˝lteringandintegratingaccelerometerdata.Generally,referencesignalsaregeneratedbyahigher-levelmodule.97Figure6.3:AblockdiagramoftheMFACcontrollerandtherobotic˝sh.Signalsrandydenotethereferenceandmeasuredspeeds,respectively,eisthedi˙erencebetweenreferenceandmeasuredspeeds,anduisthecontrolleroutput.Thecontroller'sobjectiveistoproduceacontrolsignalusuchthatycloselytracksr.Thatis,ane˙ectivecontrollerwillforcetherobotic˝shtocloselymatchthedesiredspeed,andhavelittleerrorebetweenyandr.Fortherobotic˝sh,uisafrequencyofoscillationforthecaudal˝nmotor,andanumericallycontrolledoscillator(NCO)generatesasinusoidalpatternatthegivenfrequency.Forthisstudy,wehave˝xedthesinusoid'samplitudeto20.Figure6.4showsthedi˙erencebetweenalackofadaptiveabilityandacontrollerthatisabletoadapt.Inboth˝guresthe˝nsu˙ersdamageatthe30secondpoint(denotedbytheverticalorangeline);speci˝cally,the˝nisshortenedfrom8to5.2cmandthe˛exibilityischangedfrom3.0to2.1GPa.Figure6.4(a)showstheresultingbehaviorwhenthesystemcannothandlethechange;thesystemstartsoscillatingaroundthetargetspeed,butdoesnotachievegoodtracking.Figure6.4(b)showstheresultsforanadaptivecontrollerthatisabletoregaintrackingafterdamage.ThemotivatingproblemforthisstudyishowtospecifytheMFACparametersinsuchawaythattheyallowthecontrollertoadapttovariationsincaudal˝nbehavior.Asarobotic˝sh'scaudal˝nundergoesregularwearanddegradationitwillbegintoresponddi˙erentlytomotorcommands,particularlyifthe˝nisfabricatedfrom˛exiblemateri-98(a)(b)Figure6.4:Inboth(a)and(b),the˝nsu˙ersdamageatt=30s.In(a)therobotic˝shiscontrolledbyanon-adaptivecontroller,andin(b)therobotiscontrolledbyanadaptivecontroller.Theadaptivecontrollerisabletosuccessfullyregaintheabilitytotrack.als.Consequently,astatic,non-adaptivefeedbackcontrollerwillbegintodetune,leadingtodeterioratedperformance.Additionally,anMFACcontrollershouldbebetterabletohandlenoisyaccelerometerdata,althoughwedonotconsidernoisymeasurementsinoursimulations.MFACparametersaretypicallysetbasedonexpertknowledge.Forexample,therobotic˝shthatthisstudyisbasedonhasamaximumtailfrequencyofroughly3.5Hz.Chengetal.[20]recommendforthesamplingratetobelessthanone-thirdoftheperiodofthecontrolledsystem,orroughly0.1secondsfortherobotic˝sh.Theerrorboundusedfornormalizationebcanbesettonearthemaximumspeedatwhichtherobotic˝shisexpectedtotravel(i.e.,15cm/sforthemodeledrobotic˝sh).AgoodstartingpointforthenumberofinputandhiddenneuronsNis3,astheANNcaninterpretinputsasthecurrenterrorandits˝rstandsecondderivatives.ThissetupwouldroughlycorrespondtoaPIDalgorithm,awidelyusedgeneral-purposefeedbackcontroller.TypicalinitialvaluesforthegainKcandthelearningrateare1.0and0.8,respectively.InSection6.3,wecomparetheperformanceofanMFACcontrollerimplementedwiththesevaluestothatofonewithevolvedvalues.996.2.2SimulationDynamics.Simulationoftherobotic˝shisconductedinSimulink[115],enablingastraightforwardtranslationofdynamicequationsintosimulation.MoredetailsregardingthesimulationenvironmentcanbefoundinChapters2and4.Therobotic˝shprototypeuponwhichthissimulationisbased(similartothatinFigure7.1)isapproximately20cminlength,includinga7.6cmlongtail˝nofmoderate˛exibilitymadefroma3D-printedABSplastic.6.2.3Di˙erentialEvolution.OtherstudieshaveinvestigatedhowECcanbeusedtoenhancemoretraditionalen-gineeringmethods.Forexample,Coelhoetal.[30]usedevolutionarymethodstoimprovetheperformanceofanadaptivecontrollerbyevolvinganeuralcompensator.Forthiswork,weuseddi˙erentialevolution(DE)[120],aglobaloptimizationalgorithmthatoperatesinasimilarmannertootherevolutionaryalgorithms,butwhichhasbeenshowntoconvergefasterthanreal-valuedgeneticalgorithmsforcertainclassesofproblems[100].DEprogressesinafashionsimilartootherevolutionaryalgorithms.First,apopula-tionisrandomlyinitialized.Forthisstudy,thepopulationsizeissetto50,whichistherecommendedvalueforaDEexperimentwith5evolvingparameters(Ts,eb,N,Kc,and).Next,eachindividualisevaluatedwithaproblem-speci˝c˝tnessfunction.Inthisstudy,weemploytwodi˙erent˝tnessschemes.Inthe˝rst,individualsaresimulatedforonlyonesetofconditions,and˝tnessisassignedasthemeanabsoluteerror(MAE)(i.e.,theaverageerrorbetweenrandy).Inthesecond,eachindividualissimulatedunderavarietyofdi˙erentconditions(varyingcaudal˝ncharacteristics).FitnessisthenassignedasthesumoftheMAEforeachsetofconditions.Onceeachindividualhasbeenassigned˝tness,theDEalgorithmproducesanewgenerationofindividuals.MutationandcrossoveroperatorsarewhereDEdi˙ersfromconventionalreal-valuedgeneticalgorithms.DEfocusesoncreatingnewindividualsnearthebestmemberoftheparentpopulation.Eachchildisinitializedasalinearcombinationofthebestandatleast100twootherrandomlyselectedparents.Duringthisrecombination,therelativeweightofthebestparenttotherandomparentsisreferredtoasthemutationfactorandiscon˝gurable(0.8inthisstudy).Thechildisthencrossedwiththebaseparent(eachparentistakenasthebaseinturn)usingacon˝gurablecrossoverrate(0.7inthisstudy).DEalgorithmspeci˝cs,aswellasacomparisonwithotherevolutionaryoptimizationalgorithms,canbefoundin[100].UsingStorn'sDEnotation,thealgorithmutilizedforthisstudyisdenotedasDE/best/2/bin,wherebestsigni˝esthatallchildrenarecreatedaroundthepreviousgenerationsbestindividual,2denotesthatmutationisbasedontwoindividuals,andbinreferstoabinarycrossoveroperation.MFACparametersareallowedtoevolveonlywithinacertainrange.TherangeforeachparameterislistedinTable6.1.TheserangesarebasedonthetypicalMFACparametersdiscussedearlier.Table6.1:EvolutionaryRangeofMFACParameters,asWellastheTypicalValues.MinimumMaximumTypicalTs(s)0.00.170.1eb(cm/s)5.050.015.0N183Kc0.14.01.00.14.00.86.3Single-EvaluationExperimentsandResultsInthissetofevolutionaryexperimentsweevolveMFACparametersunderasinglesetofconditions.However,toprovideasetofbaselineresultswe˝rstconductasimulationoftherobotic˝shincorporatingtypicalMFACparameters(aslistedinTable6.1).Figure6.5showsresultsfromthissimulation.ThetaskfortheMFACcontrolleristotrackareferencespeedr(theorange,dashedlineinFigure6.5),whichvariesovertimeaccordingtoaprede˝nedpattern.Thisreferencespeed,utilizedduringevolutionandmosttestcases,isdesignedtocontainperiodsrequiringacceleration,deceleration,andsustainingaconstantspeed.101Despitechoosingparametersbasedonexpertknowledge,thecontrollerstrugglestotrackthereferencespeed.Ideally,inFigure6.5(andallsimilar˝gures)thesolidblueline(y)wouldmatchthedashedorangeline(r),andtheerrorline(e)wouldremainatzero.Figure6.5:ResultsforanMFACcontrollerwithtypicalparameterscontrollingarobotic˝sh.Thedashedorangelinedenotesthereferencespeedr,theactualspeedyoftherobotic˝shistheblueline,andtheerrorebetweenthesesignalsisred.Aftertheinitialsimulationusingtypicalparameters,weconducted20replicateDEexperiments.Replicatesareseededwithauniquenumber,andDEalgorithmparametersarecon˝guredasdescribedinSection6.1.EachsetofMFACparameters(i.e.,individualsolutions)isevaluatedunderidenticalcircumstances:itissimulatedfor60secondswiththesamereferencespeedsignal,and˝tnessismeasuredasthemeanabsoluteerror(MAE).Allreplicateexperimentsconvergetosimilar˝tnessvalueswithin150generations.AsshowninFigure6.6,solutionsfromthesingle-evaluationexperimentsperformtheevolutionarytaskwell(i.e.,trackingthereferencespeedencounteredduringevolutionfor60seconds).However,evenaslightchangetothistask,suchasdoublingthesimulationto120seconds,causesalargechangeinperformance.Thisbehaviorcanbeseenduringthe˝nal60secondsofFigure6.6,wheresimplyrepeatingthereferencesignalresultsinpoorertrackingandincreasederror.Thisexperimentdemonstratesthatevolvedsolutionsareincapableofadaptingtonewconditionswhilemaintainingthesamelevelofperformance.102Morespeci˝cally,theevolvedparametersappeartobeover˝t.Thebestsolutionsonlyworkfortheconditionsencounteredduringevolution.Figure6.6:Resultsfortheoverallbest(acrossallreplicateexperiments)single-evaluationsolutionsimulatedwithdefault˝ncharacteristicsandthesamereferencesignalutilizedduringevolution.Thecontrollershowspoorperformancestartingatthe80secondmark,andsimilarresultswerefoundinallreplicateexperiments.6.4Multi-EvaluationExperimentsandResultsGiventheresultsfromSection6.3,weconductedasecondsetofexperimentsinwhich˝tnessofeachindividualisbasedonitsperformanceundermultipledi˙erentconditions.Thesettingsforthesesimulations,referredtoasthe9-evaluationexperiments,arelistedinTable6.2.Inthetable,sim1correspondstotheconditionsusedinthepreviousexperiments.Foreachsimulation,˝n˛exibilityissetto:100%,increasedto200%,ordecreasedto50%ofthedefaultvalue.Likewise,thecaudal˝nlengthissetto:100%,lengthenedto110%,orcontractedto90%ofthedefaultvalue.Evaluatingindividualsunderavarietyofconditionsisintendedtoeliminatethetendencyofevolvingover˝tsolutions.Experiencingmultipleconditionsalsosimulateshow˝nsmaychangeoncedeployed.Forexample,˝ndynamicscanchangeifthe˝nisdamaged(e.g.,cut)orencumberedbyenvironmentalentities(e.g.,seaweed).FitnessiscalculatedasthesummationoftheMAEfromeachofthe960-secondsimulations.Evolvingwiththis˝tness103Table6.2:FinCharacteristicsforthe9-EvaluationsExperiment.FlexibilityLengthsim1100%100%sim2200%100%sim350%100%sim4100%110%sim5200%110%sim650%110%sim7100%90%sim8200%90%sim950%90%functionismeanttoaddanimplicitobjectivetothe˝tnessfunction:bettersolutionsmustbemoreadaptable.Figure6.7showsthebestsolutionfrom20replicatesofthe9-evaluationexperiments.Here,thecontrollercontinuestocloselytrackthereferencefor120seconds,eventhoughevolutionaryevaluationsareonly60secondsinlength.Figure6.8showsthattrackingisaccomplishedbyadjustingthe˝n'soscillatingfrequencyinapatternroughlymatchingthatofthereferencesignal.Althoughthistestindicatesimprovementoverthesingle-evaluationexperiment,itdoesnotaddresstheissueofadaptingtodi˙erentconditions.Figure6.7:Theoverallbest9-evaluationsolutionevaluatedonsim1.TheMFACcontrollerisabletodrivetherobotic˝shatthedesiredreferencespeed(r).104Figure6.8:Controlsignaluandtheresultingmotoranglefortheoverallbest9-evaluationsolutionevaluatedonsim1.Thecontrolsignaltrajectoryroughlyfollowsthereferencesignal.Figure6.9depictsperformanceofthesamesolutionwhenconfrontedwithconditionsthatwerenotencounteredduringevolution.Notably,thecontrollerisabletoadapttothenovel˝nlengths.Signi˝cantly,thisevolvedMFACcontrollershouldallowarobotic˝shtomaintainacertainlevelofperformanceevenifthe˝nlengthchangesduringoperation.(a)(b)Figure6.9:Theoverallbest9-evaluationsolutiontestedagainst˝nlengthsthatwerenotencounteredduringanyoftheevolutionarysimulations.In(a)˝nlengthisshortenedto80%ofthedefaultlength,andin(b)˝nlengthislengthenedto120%ofthedefaultlength.Inbothcases,theevolvedcontrollerisabletoadapttoanovel˝nlength.The˝ncan,however,reachlengthsthatcausethecontrollertoloseitstrackingability.While˝xing˝n˛exibilityandthereferencesignal,weperformedasweepoverawiderangeofdi˙erent˝nlengthsandfoundthatthecontrollercanmaintainperformancewhilecaudal˝nlengthiswithinarangeof60%to137%ofthedefaultvalue.E˙ectivelythecaudal˝n105canbecutfrom7.6to4.5cm(orlengthenedto10.4cm)withoutthecontrollerlosingitsabilitytodrivetherobotic˝shatadesiredspeed.Valuesoutsideofthisrangecauseanoticeableincreaseintheerrorsignal.Figure6.10showsthatevolvedcontrollersarealsoabletoadapttochangesin˝n˛exibility.Similartothe˝nlengthparametersweep,wefoundupperandlowerlimitsfor˝n˛exibilitychanges.Whilekeepingallotherfactorsconstant,theevolvedMFACcontrollerscanmaintainperformanceaslongas˛exibilityremainswithinarangefrom90%to160%ofthedefaultvalue.Figure6.10:Theoverallbest9-evaluationsolutionevaluatedwitha˝nthatis150%ofthedefault˝n˛exibility.Evolvedcontrollerswereabletoadapttothisnovelvaluefor˛exibility.Inadditiontochangesin˝ncharacteristics,evolvedcontrollershavetheabilitytoadapttodi˙erentreferencesignals.Figure6.11demonstratesthatanevolvedcontrolleriscapableoftrackinganovelpatternforthereferencespeed,inthiscasealternatingperiodsoffastaccelerationanddeceleration.Additionaltestresults(notshown)demonstratethatlimitsonthereferencesignaldependonlyonthelimitsoftherobotic˝sh.Speci˝cally,theadaptivecontrollerwillremaine˙ectiveaslongasthereferencesignaldoesnotrequirespeeds,accelerations,ordecelerationsthatareimpossiblefortherobotic˝sh.Forexample,ifthereferencesignalchangestooquickly,therobotic˝shmaynotbephysicallycapableofacceleratingfastenough.106Figure6.11:Theoverallbest9-evaluationsolutionsimulatedwithdefault˝ncharacteristicsandareferencesignalnotencounteredduringevolution.Figure6.12showshowtheevolvedcontrollerhandlessimultaneouschangestoboth˝nlengthand˝n˛exibility.Forthistest,˝nlengthissettovaluesoutsideoftherangeencounteredduringallevolutionarysimulations.Forthetestin6.4,increasingthe˝n'slengthactuallyallowstheevolvedcontrollertoadapttoa˛exibility(80%)thatwouldotherwisecauseperformancedegradation.Speci˝cally,a˛exibilityof80%(ofthedefaultvalue)isbeyondthelowerlimitfoundwhen˛exibilitywasalteredinisolation(i.e.,therangeof90%to160%mentionedpreviously).Thisisindicativeofthecomplexinteractionsamongmaterialproperties(e.g.,˛exibilityanddimensions).Suchinteractionscausedi˚cultieswhendesigningasimplefeedbackcontroller,suchasaPIDcontroller,oramodel-basedcontrollerthatmustaccountforallofthenecessarydynamics.Anevolvedadaptivecontrollercanautomaticallyhandlethesecomplexinteractions.TofurtherincreaseadaptabilityofanevolvedMFACcontroller(i.e.,increasetherangeof˝ncharacteristicvariationwhilemaintainingthesameperformancelevels),the9-evaluationsexperimentswererepeatedwithlargervariationsfromthedefaultvalues.Forinstance,insim5(refertoTable6.2)the˛exibilityissetto1000%ofthedefaultvalue,andlengthisincreasedto200%ofthedefaultvalue.Likewise,insim9˛exibilityissetto10%ofthedefaultvalue,whilelengthisdecreasedto67%ofthedefaultvalue.107(a)(b)Figure6.12:Theoverallbest9-evaluationsolutiontestedagainst˝nlengthsand˝n˛exibili-tiesthatwerenotencounteredduringevolutionarysimulations.In(a)˝nlengthisshortenedto80%ofthedefaultlengthandthe˛exibilityisincreasedto120%ofthedefault,andin(b)˝nlengthislengthenedto120%ofthedefaultlengthand˝n˛exibilityisreducedto80%ofthedefaultvalue.Althoughtheevolvedcontrollerisgenerallystillabletotrackr,thebestsolutionsfromallreplicatesperformedworse,onalltestcases,thanprevioussolutions.Figure6.13showsanindividualfromthealteredexperiments.Figure6.13:Performanceofthebestevolvedsolutionfromthealtered9-evaluationsexper-imentstestedwitha˝nlength120%ofthedefault.TheprimaryreasonfortheMFAC'sinabilitytoadapttolargevariationsliesinhowthedynamicschange.Certain˝ncharacteristicsorcombinationsofcharacteristicscausethephysicalsystemtobehavefundamentallydi˙erently.ThebasicMFACpresentedinthisstudyreliesontheroboticsystemtobedirect-acting.Thatis,astheMFACcontrolleroutput108increases,theoutputfromthecontrolleddevicemustmonotonicallyincrease.Figure6.14depictshowchangesto˝ncharacteristicscanaltertherobotic˝sh'sresponsetocommandsfromthecontroller.Essentially,if˝ncharacteristicsvarybeyondacertainthreshold,therobotic˝shmaynolongerbehaveasadirect-actingroboticsystem.Thisissueismostclearlydemonstratedbythegreen,dottedcurve(200%),whichchangesfromdirecttoreverse-actingnear1.5Hz,andthenbacktodirect-actingnear3.5Hz.Asimilare˙ectcanbeseen,toalesserextent,foreachofthecurves.EvenanoptimizedMFACcontrollerwillbeunabletocopewiththesehighlyvarieddynamics.Suchlimits,orboundaries,de˝neanexecutionmode.Thatis,anadaptivecontrollercanhandleconditionswithinthemode,butislikelytofailifconditionsgooutsidethemode.Figure6.14:Speedvs.oscillatingfrequencyforseveraldi˙erent˝ncharacteristics.Forcertainconditionsincreasingthefrequencyresultsinslowerspeeds.6.5ConclusionsInthischapter,weexploredtheintegrationofevolutionarycomputationandadaptivecontrol.Speci˝cally,weapplieddi˙erentialevolutiontooptimizetheparametersofamodel-freeadaptivecontroller.Thegoalofevolutionisto˝ndasetofparametersthatenableanMFACcontrollertoadapttochangesin˝ncharacteristics(i.e.,lengthand˛exibility)109andchangestothereferencesignal(e.g.,faster/sloweraccelerations).Additionally,evolvedMFACcontrollersshouldbeabletohandlechangestothereferencesignal(i.e.,di˙erentdesiredreferencespeeds).ResultsshowthatevolvingMFACparametersagainstasinglesetof˝ncharacteristicscanproduceacontrollercapableofachievinggood˝tness(i.e.,speci˝cally,lowmeanabsoluteerror).However,thesesolutionsdonotproduceacontrollercapableofadaptingtochanging˝ncharacteristics.Next,the˝tnessfunctionwasmodi˝edtoincludeavarietyofdi˙erent˝ncharacteristics.Thenewlyevolvedcontrollersweretestedunderseveraldi˙erentconditions,includingscenariosinwhichthe˝ncharacteristicswereoutsidetherangeexperiencedduringevolution.Thismethodsucceededingeneratingmoreadaptablecontrollers.Speci˝cally,thebestMFACcontrollerswereabletomaintainclosetrackingaslongas˝nlengthremainedwithin60%to137%ofthedefaultand˝n˛exibilityremainedwithin90%to160%ofthedefault.Toexplorethelimitsofthisapproach,the˝tnessfunctionwasagainmodi˝edtoin-cludealargervarietyofdi˙erentcharacteristics.However,theseexperimentsresultedinpoorerperformingcontrollers.Drasticallyvaryingthe˝ncharacteristicsessentiallycreatesafundamentallydi˙erentsetofgoverningdynamics.Evolvedcontrollersrequirethesys-temtobeeitherdirect-orreverse-acting,whichisnotalwaysthecasewhen,forexample,the˝nismadetobetoo˛exible;insuchcases,increasingthecontrolfrequencyresultsinslowerspeedsratherthanfaster.Eventhemost˝tindividualswereincapableofsuccessfuladaptationwhensubjectedtosuchconditions.Inthenextchapter,wediscussamethodforautomaticallydiscoveringlimitsforagivenadaptivecontroller.110Chapter7DiscoveringAdaptationBoundariesInthecontroltheorydomain,adaptationreferstoacontroller'sabilitytoautomaticallyadjustcontrolparametersduringoperationinresponsetosensedfeedback.Acontroller'senvironmentincludesanyvariablesnotdirectlycontrolledbythesystem.Aswehaveseenintheprecedingchapter,fromtheperspectiveofarobot'scontrolsoftware,theenvironmentincludesnotonlythephysicalsurroundingsinwhichtherobotoperates,butalsocharac-teristicsofthebody(morphology)oftherobot.Aswiththephysicalenvironment,themorphologyissubjecttochange.Inthischapter,weusethetermscenariotodescribeaspeci˝csetoftheseenvironmentalparametersandtheirrespectivevalues.Comparedtoastatic(i.e.,non-adaptive)controller,theabilitytoadaptensuresthatasinglecontrollerwillremaine˙ectiveformanyscenarios.Animportanttaskforthedesigneroftherobot,oranycyber-physicalsystem,istospecifythelimitsofadaptationforthecontroller.Thatis,byhowmuchcananygivenparameterchangebeforeacontrollerfailsandthesystemneedstoswitchtoadi˙erentmodeofexecutiondrivenbyadi˙erentcontroller?Inthepreviouschapter,weusedDEtoevolvetheparametersofanMFACinordertoincreasethecontroller'sadaptability.Speci˝cally,wedemonstratedthatexposingtheMFACtodi˙erentscenariosduringevolutionproducedamoreresilientcontroller,evencapableofSomeoftheresultsanddescriptionsinthischapterwerepublishedin[110]and[7].111adaptingtoconditionsbeyondthosetowhichthesystemwasexposedduringevolution.However,thefocusofthatworkwasonadaptability,andthescenariosweredesignedbyhand.Inaddition,forsomescenarios(suchasextensivedamagetothe˝n),thecontrollerwassimplyunabletoe˙ectivelyadapt.Inthischapter,weexploretheroleofECindiscoveringtheboundariesofadaptivecontrollers.Weagainemploydi˙erentialevolution,andcombineitwithanumericalsim-ulationinordertoevolveadaptivecontrollerparameters.Overthecourseofevolutionweexposecontrollerstodi˙erentscenariosnotonlytoenhancetheadaptivecapabilitiesofthecontroller(asinChapter6),butalsotoidentifyconditionsunderwhichthecontrollerwillfail.Thisinformationcanbeusedtode˝netheboundariesofanexecutionmode,enablingthesystemdesignertoimplementhigher-orderstrategiesforswitchingamongexecutionmodes,typicallyinvolvingaswitchincontrollers,whenthesystemdetectsthatachangeisnecessary[64].Butwhichmethodofgeneratingscenariosismoste˙ectiveinexploringtheybofthecontroller?Themaincontributionchapterisanapproachthataddressesuncertaintyforautonomousrobotsattwolevels.First,weevolveadaptivecontroltohandleincreasinglymoreadverse(anddiverse)environmentalconditions.Andsec-ond,weautomatetheprocessofdiscoveringtheboundaries/limitationsofadaptation,whichwillfacilitatethedevelopmentofdecisionlogicforswitchingbetweenadaptivecontrollersandtheircorrespondinglymodeofbehavior.7.1EvolvingaBaseMorphologyAsintheprecedingchapters,themorphologyofthecaudal˝nisdescribedbythreeparameters:the˛exibility,thedepth(orheight),andthelength(simulatedand3Dprintedrobotic˝shareshowninFigure7.1).Webeginbyevolvingthethree˝nparametersforasimpletask(maximumaveragespeed),whileusingasinusoidalsignalasthetroller.Theadaptivecontrollersevolvedinthenextsectionswillberequiredtorespondtochanges112inthismorphology.InChapters4and5,wevalidatedtheresultsofthisprocessby3D-printing˝nsandtestingthemonphysicalrobots.(a)(b)Figure7.1:(a)Renderingofthesimulatedrobotic˝sh.Individual˝nsegmentsappearindi˙erentcolors,andthe˝nappearstoextendabovethesurfaceofwaterforvisualizationpurposesonly.(b)Aprototype3D-printedrobotic˝shwitha˛exiblecaudal˝n(topcoverremovedforillustration).Weconducted30replicatedi˙erentialevolutionexperimentsinwhichthecaudal˝nmorphologyandthefrequencyandamplitudeofthesinusoidalcontrolsignalaresubjecttoevolution.Figure7.2(a)plotstheaverageevolutionarytrajectoryofthesereplicatesandshowsthattheyconvergedquickly(within10generations)toa˝nal˝tnessvalueofapproximately22cm/s.Thebox-plotsinFigure7.2(b)demonstratethateachofthe˝veevolvedparametersconverged(themaximumvaluefora˝n'sYoung'smodulusisdictatedbyourfabricationprocess[27]).Onlythe˝rstfourparametervalues(thethreemorphologicalparametersandcontrolamplitude)areusedtorepresentourbasesystem.Insubsequentexperiments,the˝fthparameter(frequencyofthecaudal˝n)willbegovernedbyanadaptivecontrollertaskedwithmatchingthespeedoftherobottoaninputreferencesignal,despitechangestothebasemorphology.7.2ModeDiscoveryAlgorithmWiththisbasemorphologyinhand,wecanapplyevolutionarysearchinordertodiscoverandexplicitlyde˝netheboundariesofadaptabilityforanMFACthatcontrolsthecaudal113(a)(b)Figure7.2:Evolutionofbasemorphology:(a)Meanvaluesforthebestperformersacrossall30replicateevolutionaryruns.Theshadedregionrepresentsthe95%con˝denceinterval.(b)Box-plotsofdistributionsofthe˝nalevolvedparameters,acrossthereplicateruns.Themedianvaluesarerepresentedbyahorizontalredline,blueboxesrepresentonestandarddeviationeithersideofthemean(theblueboxesaresometimescoveredbythemedianline),andblackcirclesdenoteoutliers.˝n.Notably,thisapproachshouldalsoproduceanMFACthatcanoperatee˙ectivelywithinthoseboundaries.Ingeneral,theenvironmentforarobotic˝shcontrollerincludesthemorphologyofthephysicalsystemandtheexternalaquaticenvironment(watereddies,turbulence,etc.).Here,wefocusonlyonchangestomorphology,whichtypicallyoccurduetouncertaintyandunpredictablecircumstancessuchasdamageorwhenthedevicebecomesentangled.OurbasicapproachistoexposetheMFACtomultiplescenariosduringtheevolutionaryprocess.Next,wede˝nepreciselywhatconstitutesascenarioandhowwedeterminewhetherascenarioisfeasible,followedbyadescriptionofthedevelopedalgorithm.7.2.1ScenarioParameters.Eachscenarioincludesthethreemorphologicalparametersdiscussedearlier:˝nlength,˝ndepth,and˝n˛exibility.BecausetheMFACwillberequiredtotrackavarietyofref-erencesignals(i.e.,desiredbehaviors),eachscenarioalsoincludesareferenceinputwhoseparametersaregeneratedrandomly.Forthisstudy,referencesdescribeonlydynamicsasso-ciatedwiththespeedandaccelerationsoftherobot,butthesameapproachcouldaddressothermorecomplexbehaviorsandmaneuvers.AsdepictedinFigure7.3,eachreferencesig-114nalinvolvesanintervalofaccelerationfromzerocm/suptoaconstantspeedS1,followedbyeitheranotheraccelerationoradecelerationtoasecondconstantspeedS2.Thedurationsofthefourintervalsarede˝nedbyparameterst1,t2andt3.Thisapproachenablesgenerationofnovelreferencesignalsthatcontainarichsetofdynamics(i.e.,di˙erentmaximumandminimumspeedsandaccelerations/decelerations).Figure7.3:Referencesignalscenarioparametersincludevaluestodescribefourtimeseg-ments.Inthe˝rstsegment(fromt=0tot=t1)speedrampsfrom0toS1,inthesecondsegmentthereferencespeedremainssteadyatS1.Inthethirdsegment(fromt=t2tot=t3)thespeedramps(upordown)tothe˝nalspeed,whichisheldsteadyduringthe˝naltimesegment.7.2.2DeterminingScenarioFeasibility.Ofcourse,somecombinationsofmorphologicalparametersmayproducearobotthatsimplycannotbecontrollede˙ectively.Forexample,ifthe˝nisseverelydamagedortoo˛exible,thenitmightbeimpossibletogeneratesu˚cientthrusttoreachaspeci˝edreferencespeed,muchlessmaintainit.Suchascenarioisdeemedinfeasible,orinvalid.Weimple-mentedanautomatedprocedureforidentifyingsuchscenarios,sothattheycanbeexcludedfromtheresultingexecutionmode(aswellasfromtheevolutionarysearchprocess).ThefeasibilityprocedurealsoconsidersthefactthattheMFACusedinthisstudycannotbeappliedtosystemsthatswitchbetweendirectandreverseacting.Direct(orreverse)acting115signi˝esthatthesystemoutput,speedinthiscase,willalwaysincrease(ordecrease)asthesysteminput,frequency,increases.Forexample,forvery˛exible˝ns,speedcanincreaseatlowfrequencies,butstarttodecreaseathigherfrequencies[27].ThesebehaviorsareshowninFigure9.Figure7.4:Threebehaviorsareshown.Theblueandgreenlinesdenotesystemsthataredirectandreverseacting,respectively.Theredlineshowsasystemthatswitchesactingmodesandisdeemedinfeasible.Beforeconsideringarandomlygeneratedscenarioforintegrationintheevolutionaryalgorithm,itis˝rsttestedforfeasibilityasfollows.Arobotwiththespeci˝ed˝nchar-acteristicsissimulatedfor15secondswith˝nfrequenciesof0.5,1.5,and2.5Hertz.Foreachfrequency,therobotis˝rstallowedtoreachasteady-statespeed,andthenanaveragespeedismeasuredoverthe˝nal5seconds.Iftheresultsshowthatthebehaviorchangesfromdirecttoreverseacting,oriftherobotfailstoreachaspeedof15cm/s,thenthescenarioisconsideredtobeinfeasible.Wechose15cm/sbecausethereferencesignalvaluesareallowedtoincludeamaximumspeedof20cm/s,whichisstilllowerthanthemaximumspeedofthedevice.Wenotethatnoevolution(orfeedbackcontrol)isinvolvedintheseevaluations;rather,theyareintendedonlytoevaluatethebehavioroftheparameterizedrobotatincreasingcontrolfrequencies.1167.2.3EvolvableParameters.AlthoughanMFACadaptsbyupdatingANNlinkweightsduringexecution,otherpa-rametersoftheMFACdetermineitsabilitytoadapt.Table7.1liststheninesuchparameterstargetedinthisstudy.Collectively,theseparametersde˝netheresponsivenessandsensitiv-ityofthecontroller,thestructureandprocessingcapabilitiesoftheANN,andtheupdateperiodsfortheANNweightsandthecontrolleroutput.Theseparametersarecon˝gurableatdesigntime,buttypicallydonotchangeafterdeployment.Determiningtheiroptimalvaluesischallenginganddependsontheapplicationdomain.Traditionally,MFACparametersareeitherchosenbasedonexpertknowledgeortunedusingproprietarysoftwarespeci˝ctotheapplication.Hereweevolvetheseparametersinordertoenhancetheadaptabilityofthecontroller.Table7.1:EvolvableMFACParametersKccontrollergaintoamplify/reduceoutputNInumberofANNinputnodesNHnumberofANNhiddennodesEBvalueusedtonormalizeMFACerrorinputslearningrateforANNedgeweightsToutupdateperiodforgeneratingMFACoutputTwtupdateperiodforMFACweightssensitivityparameter,reactivityofMFACsensitivityparameternumerator7.2.4Algorithm.Figure7.5showsa˛owchartofthealgorithmdevelopedforthisstudy,whichwerefertoastheModeDiscoveryAlgorithm.Webeginbyevolvingthebasemorphology.asdescribedearlier.Thebase˝nparametersarecombinedwithahand-designedreferencesignaltoproducethebasescenario,whichisplacedinthesetofscenariosSappliedinevolution.Thebasereferencesignalstartsat0,rampsupto15cm/s,andthenrampsdownto5cm/s.ThealgorithmthenalternatesbetweenevolvingtheMFACagainstscenariosin117Sfora˝xednumberofgenerations(bydefault,10)andgeneratinganewscenariotoaddtoS.Speci˝cally,weevaluatetherobot/MFACforeachscenarioinS(takingthemean-absolute-valueoftheerrorsignal),and˝tnessiscalculatedastheaverageoftheseresults.Thenumberofiterationsthroughthisbasicloopiscon˝gurable(also10bydefault).ThepopulationsizefortheDEalgorithmis90.EvolvinginitiallyagainstonlythebasescenarioisintendedtobootstrapMFACevolution,enablingtheoptimizationprocesstostartwithaneasierobjectivebeforeaddingapressureforadaptability.Weimplementedandevaluatedtwomethodsforgeneratingandselectingscenarios,termedboundaryselectionandvolumeselection.Thesemethods,alongwiththecorrespondingresults,aredescribedinthenextsection.Figure7.5:FlowchartoftheModeDiscoveryAlgorithmusedtodiscoverexecutionmodeboundariesandproduceaMFACparametervaluesforthatmode.1187.3ExperimentsandResultsInthissection,wedescriberesultsforthetwoscenarioselectionmethods,aswellasforaparametersweepexperimentthatprovidesaroundfortheexecutionmode.Bothapproachesaimtoenhanceadaptabilitywhileatthesametimeprovidinginformationregardingexecutionmodeboundaries.7.3.1BoundaryScenarioSelection.The˝rstmethodstartsbyidentifyingthelimitsofeachmorphologicalparameter(i.e.,˝nlength,depth,and˛exibility),usingthefeasibilitytestdescribedabove.Speci˝cally,westartatthebasevalueforeachparameterandincrease/deceasethevalueuntilthesystembecomesinfeasible.Duringthisprocess,weconsideroneparameteratatime,whiletheotherparametersare˝xedattheirbasevalues.Theincrement/decrementvalueswere1mmforlengthanddepthand0.5MPafor˛exibility.TheresultsofthisprocessaredepictedinFigure7.6.Wenotethatthemaximumvaluefor˛exibilityisequaltothebasevalueof3.0GPa;our3Dprintercannotproduce˝nsmorerigidthan3.0GPa,andthebasevalueevolvedfor˝n˛exibilityreachedthismaximum.Figure7.6:Boundaryscenariovaluesforthethree˝nparameters.Collectively,thesefeasibilitytestsproducethesixboundaryscenarioslistedinTable7.2.TheseareintegratedintotheModeDiscoveryAlgorithmasfollows.Duringthe˝rst10generations,theMFACevolvesusingonlythebasescenarioin˝tnessevaluation.Atthe119startofeachsubsequentiteration,oneoftheremainingscenariosisrandomlyselectedandaddedtosetS.Addingonescenarioatatimegraduallyincreasesthedi˚cultyofthetask.Atthestartofthe˝fthiteration(andthebeginningofthe60thgenerationoverall)allsixscenariosareinuse.Sincetheorderinwhichthescenariosareaddedisrandom,theorderingswilldi˙eracrossreplicateexperiments.Examinationof40replicateexperimentsshowsthattheorderhaslittleifanye˙ect.Table7.2:BoundaryScenarios(FinParameters)LengthDepthYMscenarioBase8.0cm2.6cm3.0GPascenariolengthmin6.0cm2.0cm3.0GPascenariolengthmax8.4cm2.6cm3.0GPascenariodepthmin8.0cm1.0cm3.0GPascenariodepthmax8.0cm2.7cm3.0GPascenarioYMmin8.0cm2.6cm2.5GPaConsistentwiththeresultsinChapter4,adaptivecontrollersresultingfromthispro-cessexhibitenhancedadaptabilitywhencomparedtothoseevolvedagainstonlythebasescenario.However,thisapproachhasalimitation:itdoesnotconsidertheinteractionsamongmorphologicalparameters.Inearlierexperiments[25],weobservedcombinationsofparametersthatwerefeasible,despitelyingoutsidethefeasibleregiondepictedinFigure7.6.Forexample,whenacaudal˝nisshorteneditwillgenerallycontinuetoworkwellifitissu˚ciently˛exible.Forexample,whenwetestthefeasibilityofa˝nwithalengthof6.4cmandaYoung'smodulusof2.1GPa(belowthe2.5GPathreshold)we˝ndthatitis,infact,feasible.Theseobservationsledustoexploretheexecutionmodeboundariesintwoadditionalways.The˝rstisabruteforceapproachthatchecksforfeasibilityasbefore,butconsidersallcombinationsofthethree˝nparameterssimultaneously.Thismethodenablesusto˝ndthefortheexecutionmode.Whileapplicabletothisrelativelysmallproblem,weemphasizethatsuchanapproachwouldlikelybeimpracticalonamorecomplexsystemdue120tocomputationalrequirements.Thesecondapproachistodetermineifwecan˝ndsimilarboundariesusingtheModeDiscoveryAlgorithm.Eachapproachisdiscussedinturn.7.3.2SimultaneousParameterSweep.Weperformedaparametersweepsimultaneouslyacrossallmorphologicalparameters,usingthesamegranularityasbefore.Resultsforthesesimulations,62,068intotal,areshowninFigure7.7.Eachbluedotrepresentsafeasiblescenario(andwhichtheoreticallyanMFACcanhandle),whileeachgraydotrepresentsinfeasiblescenario.Collectively,theregionsofbluedotsde˝neapossibleexecutionmodeforanadaptivecontroller.Theredboxesrepresenttheareasfoundbytheboundaryselectionmethod,andthediagonallineinFigure7.7isduetoaconstraintinthesimulationmodelofthe˛exible˝n,whichrequiresthatthelengthmustbeatleastthreetimesthedepth[129].(a)(b)(c)Figure7.7:Parametersweepplots:(a)lengthvs.depth;(b)lengthvs.˛exibility(c)depthvs.˛exibility.Theredboxesdenotethelimitationsofadaptabilityfoundbytheboundaryselectionmethod,andtheblacklinein(a)correspondstothelength-widthlimitationofthesimulationmodel.AsshowninFigure7.7,thesesimultaneousparametersweepsrevealmuchlargerandmorecomplexfeasibleregionsthantheboundaryexperiments.However,thisapproachiscomputationallyexpensiveandlikelyimpracticalforhigh-dimensionalscenarios.Moreover,itdoesnotproduceanadaptivecontrollerforthisexecutionmode.1217.3.3Volume-BasedScenarioSelection.Giventheseresults,wedevelopedasecondscenarioselectionmethodthattakesintoaccountinteractionsamongparameters.Sinceitiswovenintotheevolutionaryprocess,itproducesboththeexecutionmodeboundaries(atlowercostthansweeps)aswellasanadaptivecontrollerforthatmode.Table7.3describesthelimitsforeachscenarioparameter.Unliketheboundaryselectionmethod,wherethereferencesignalwasalwaysthesamebasepattern,herewerandomizethereferencesignalvaluesforeachscenario,whichaddsanadditionalchallengefortheadaptivecontrollers(i.e.,theymustadapttodi˙erentcontrolrequirementsinadditiontodi˙erentcaudal˝ndynamics).Eachevaluationtakes60sofsimulationtime.Thetimevalues(t1,t2,andt3)showninFigure7.3wereproducedbygeneratingthreerandomnumberssuchthatt1=trand1,t2=t1+trand2,andt3=t2+trand3.Eachofthe˝rstthreetimesegmentsisintherange[5,25]s,andift3islessthan60secondsthenthe˝nalsegmentincludesalltimeremaining.Table7.3:VolumeScenarioLimitsMinimumMaximumBaseFinLength2.0cm20.0cm8.0cmFinDepth0.5cm4.0cm2.6cmFinYM0.1GPa3.0GPa3.0GPaSi0.0cm/s20.0cm/s15.0cm/strandi5.0s25.0s15.0sAswiththeboundaryapproach,thismethodstartswiththebasescenarioandaddsonenewscenarioatthebeginningofeachiterationoftheModeDiscoveryAlgorithm.However,sincerandomlygeneratedscenariosarenotguaranteedtobefeasiblewegeneratemultiple(25)candidatescenariosperiteration.Fromthesecandidatescenarios,wechoosethefeasiblescenariothatproducesthelowest˝tnessscorewhenevaluatedwiththecurrentbest(highest˝tness)MFACcontroller.Essentially,weseekscenariosthatarethemostdi˚cultfortheadaptationprocess.Attheendoftheexperiment,thesetScontains11totalscenarios.However,forcomparisonpurposes,wecapthevolume-basedselectionmethodtohavea122comparablenumberofevaluationsastheboundaryselectionmethod.So,foreachiteration,ifjSjisgreaterthan˝ve,only˝vescenariosarerandomlyselectedforuseinevolution(eachreplicateexperimentwillhaveadi˙erentsetjSj).Byconsideringallfeasiblescenarios(includingacrossreplicateexperiments)wecande˝netheexecutionmodefortheevolvedcontroller.The40replicateexperiments,10iterations,andtestingof25randomscenariosperiteration,yields10,000testedscenariosintotal.Figure7.8showsthatwithfarfewersimulations(10Kvsapproximately62K),thevolumescenariomethodde˝nesthesameboundariesasthefullparametersweepsshowninFigure7.7.Theadvantageine˚ciencyisexpectedtoincreasewiththenumberofscenarioparameters,whichwillbethecaseformanycyber-physicalsystems.Infact,parametersweepsmaybecompletelyinfeasibleundersomecircumstances.However,generatingagoodspreadofscenariosinhigherdimensionalspacemaybeacomplexprocessashasbeenshownbytheevolutionarymultiobjectivecommunity[8].Morespeci˝cally,itisdesirabletohaveasetofscenariosthatbehaviorallydi˙erentfromoneanother,andthusplacinganappropriateevolutionarypressureonadaptivecontrollerstohandlethewidestrangeofpossibleconditions.(a)(b)(c)Figure7.8:Plotsformodediscoveryusingvolumeselection:(a)lengthvs.depth;(b)lengthvs.˛exibility;(c)depthvs.˛exibility.123ThereaderwillnoticethatinFigure7.8,somegraydots(representinginfeasiblescenar-ios)seemtofallwithinthediscoveredmodeboundaries.Thisphenomenonisparticularlyevidentinthedepthvs.YMplot.Wenotethatthisappearanceissimplyasidee˙ectofthepairwiseplottingofthethreeparameters.Speci˝callythosescenariosareinfeasiblebecauseofthevalueofthethirdparameter(lengthinthecaseofthedepthvs.YMplot).ThisrelationshipisalsopresentinthedataplottedinFigure7.7,butdoesnotappearinthoseplotsbecauseintheparametersweeps,thevaluesofparametersvaryatregularintervals,andbluedotscoverallthegrayones.Inadditiontode˝ningmodeboundaries,thevolumemethodgeneratede˙ectiveadap-tivecontrollersbysequentiallyintegratingscenariosintotheevolutionaryprocess.Figure7.9showsanexamplebehaviorofasimulatedrobotic˝shthatexperiencesdamagehalfwaythroughoperation.At60seconds,the˝nlengthisreducedfrom8.0to6.4cm,depthisreducedfrom2.6to2.1cm,and˛exibilityischangedfrom3.0to2.1GPa.Despitetheseratherseverechanges,thecontrollerisabletoquicklyadaptandre-establishtracking.Figure7.9:AnexampleofanevolvedMFACadaptingtosuddendamage.At60seconds,eachofthe˝nmorphologyparametersareabruptlychanged.Whencomparingthetwoscenarioselectionmethods,wefoundthatbothleadtosimilaralgorithmconvergenceratesandsimilarMFACbehaviors.Table7.4comparesthe˝tnessscores(theaveragemean-absolute-errorrecordedforeachscenariousedduring˝tnesseval-uation)fortheoverallbestsetofMFACparametersfromeachexperiment(lowervaluesare124Table7.4:MFACPerformanceComparisonBoundaryVolumeBase2.76%2.60%lengthmin9.30%7.63%lengthmax2.74%2.73%depthmin6.23%4.87%depthmax3.12%2.92%YMmin2.98%2.93%randboundary4.70%4.54%randvolume3.19%3.14%better).Therandboundaryandrandvolumerowsrepresentsthemean˝tnessscorefor100ran-domlygenerated,feasiblescenariosthatarewithinthemodesfoundbytheboundaryandvolumeselectionmethods,respectively.Thetableshowsthatthevolume-basedscenariogenerationapproachdoesanequalorbetterjobineverytestcase.7.4Self-ModelingTheModeDiscoveryAlgorithmdevelopedinthischapterwillautomaticallydiscoverthelimitationsofagivenadaptivecontroller.Withtheabilitytode˝neexecutionmodesforasetofcontrollers,asystemcancontinuetooperatee˙ectivelydespiteunexpectedchangesduringruntime.Aspartofthisprocess,however,thesystemmustdecidetowhichmodeitcurrentlybelongs.Thus,therelevantquestionis:canthesystemdiscoveranupdatedmodelforitself,atruntime,thataccountsforthedamage?Inthissection,wepresentanapproachtoachievingsuchruntimeadaptabilitybaseonamodi˝edversionoftheself-modelingprocessdevelopedbyBongardetal.[12],wherethesystemmaintainsaninternaltalofitself.Ifthesystemhasanaccuratesimulationmodelofitsownmorphology,includingsensorsandactuators,itcanderivecompensatorybehaviorsdynamicallyusingthemodelratherthanitsphysicalsystem.TheEstimation-ExplorationAlgorithm(EEA)[12],isageneralpurposeself-modelingalgorithmforreverseengineeringcomplex,nonlinearsystems,thathasbeenappliedtotheself-repairofterrestrialrobotsandotherapplications.125(a)(b)(c)(d)(e)Figure7.10:Modelingandfabricationofanaquaticrobot[92]:(a)simulationmodelinOpenDynamicsEngine(ODE);(b)correspondingSolidWorksmodelforfabricatingprototype;(c)3D-printedpassivecomponentsofprototype;(d)integrationofelectroniccomponentsandbatteryintotheprototype;(e)assembled,paintedandwaterproofedprototypeinanelliptical˛owtank.Themainbodyofthephysicalprototypeis13cmlongand8cmindiameterwithpectoral˛ippersthatare8cmlongand2cmwide.7.4.1TheAquaticRobot.ThedeviceusedinthissectionisshowninFigure7.10.Thisaquaticdevicecomprisesacapsule-shapedmainbody,twopectoral˛ippers,andasinglecaudal˝n.ThemainbodyofthephysicalrobotcontainsanArduinomicrocontrollerboard,twolithiumpolymerbatteries,threeservomotors,anda6-axisinertialmeasurementunit(IMU).TheIMUincludesanintegratedgyroscopeandaccelerometer,enablingtherobottocomputeitscurrentposition,orientationandvelocityinthree-dimensionalspace.Therobotisdesignedtobecon˝gurable,inthatplasticsleevesenable3D-printed˛ippersand˝nswithdi˙erentcharacteristics(size,shape,˛exibility)tobeswappedinandoutfordi˙erentexperiments.Theservomotorspoweringthe˛ippersareacontinuousrotationtypethato˙eronlyavariablespeedineitherdirectionwithnopositionfeedback,afeaturetypicalofservomotorsofthissize.Wechosethisplatforminsteadofarobotic˝shasithasseveralpropertiesthatmaketheself-modelingprocessmoredi˚cult.Speci˝cally,thelackofpositioninformationcomplicatestheself-modelingprocess,sincethebehavioroftherobotishighlycoupledtoitsinitialcon˝guration.SimilartotheworkpresentedinChapter3,weimplementedthesimulatedrobotusingtheOpenDynamicsEngine(ODE)[116].Eachsimulationisparameterizedtoallowforchangesintherobotmorphology,speci˝cally,thedimensionsofeachofthethree126˛ipperscanbealtered(wedonotconsider˝n˛exibilityinexperimentsdescribedinthissection).7.4.2Self-ModelingAlgorithm.AsdepictedinFigure7.11,theEEAisaco-evolutionaryprocessthatalternatesbetweengaininginformationaboutthetargetsystem(exploration)andintegratingthatinformationintoitsmodelhypothesis(estimation).Thecombinationofanexplorationphaseandanestimationphaseisreferredtoasaround.Forthisstudythesetwophasesareiterateduntiluntilacomputationtimelimithasbeenexceeded.Figure7.11:BasicoperationoftheEstimation-ExplorationAlgorithm[11].127TheExplorationPhaseisresponsibleforcollectingbehavioralinformationfromthetargetsystemforuseintheestimationphase.Speci˝cally,thealgorithmattemptstoidentifyanaction(inourcaseaparticularmovementoftherobot's˛ippers)thatprovidesmaximalinformationregardingthecurrenthypothesesoftherobotmorphology.Ageneticalgorithmisusedtosearchforsuchanaction;apopulationofindividuals,eachanencodingofacandidateaction,executesforapredeterminednumberofgenerations.The˝tnessofacandidateactionisbasedontheamountofdisagreementitgeneratesamongthecurrentmodelhypotheses,asdiscussedbelow.Theactionwiththehighest˝tnessinthe˝nalgenerationisselectedandappliedtothetargetsystem.Theresponsefromthetargetiscollectedfromthesystem'ssensorsandrecorded,alongwiththeactionthatgeneratedit(collectivelyknownasanexperiment),foruseintheestimationphase.TheEstimationPhaseisresponsibleforre˝ningthehypothesisoftherobot'smor-phology.Tothisend,itgeneratesasetofcandidatehypothesesthatbestexplaintheexperimentsperformedinpriorexplorationphases.Asintheexplorationphase,ageneticalgorithmisusedforthissearchprocess,withcandidatemodelsencodedasbinarystrings.Inthisstudy,eachmodelencodingcomprisestwosetsofpectoral˝ndimensionsandonesetofcaudal˝ndimensions.Foreachexperimentfoundduringtheexplorationphase,thecorrespondingactionisappliedtothecandidatemodel,andtheresponseiscomparedtothatofthetargetsystem.The˝tnessofacandidatemodelisbasedonhowwellitreproducestheresponseofthetarget.7.4.3InferenceTechnique.Themaindi˚cultyinapplyingtheabovemethodsdirectlyisthattheresponseoftherobottoagivensetofcommandsishighlydependentontheinitialpositionsofthe˛ippers.Withoutdirectknowledgeofthisinformation,weneedtoinferthepositionsaspartoftheevolutionaryprocess.Toperform˛ipperpositioninference,theEEAispausedevery35generations,andinitialcon˝gurationsareevolvedforuseduringthemodeldiscoveryprocess.128The˝tnessofcandidateinitialcon˝gurationsre˛ectstheirabilitytohelpthecandidatemodelsreproducethetargetbehavior.Thatis,asetofinitial˛ipperpositionsthatenablethecandidatemodelstoaccuratelyreproducethecurrentexperimentwillreceiveahigh˝tness.Forcomparison,weperformedthreeexperiments:(1)anidealbaselineexperimentfortheunmodi˝edEEAwithknowninitial˛ipper/˝npositions,(2)anuncertaintybaselineexperimentfortheunmodi˝edEEAinthepresenceofunknownstartingpositions,and(3)aninferenceexperimentinwhichweemploythedescribedinferencetechnique.7.4.4ExperimentalResults.Eachexperimentwasexecutedasasetof25replicateruns,eachofwhichwasseededwithadi˙erentrandomnumberseed.Theindividualrunswereexecutedasasinglethreadedprocessonlate-model,commodityhardwareandgiven48hoursofCPUtimetocomplete25roundsoftheEEAprocessbeforebeingterminated.Theorientationofthetargetrobotwasrandomizedbetweeneveryaction,re˛ectingwhatwouldlikelybethecaseinareal-worldsituation.Thegeneticalgorithmintheexplorationphaseusedapopulationof25actionsandexecutedfor25generations.Thegeneticalgorithmintheestimationphaseusedapopulationof25modelhypothesesandexecutedfor100generations.Bothgeneticalgorithmsusedtournamentselectionwithatournamentsizeoftwo,acrossoverrateof0.75andameanmutationrateofonegenepergenome.Fortheidealbaselinecon˝guration,thetargetaquaticrobothasrandom˛ipperori-entationsatthestartofeachEEAexperiment,butthoseinitial˛ipperpositionsarenotedandpassedbacktothemodelingprocessforusebycandidatemodels.Thiscaserepresentswhatthemodelingprocesscouldachieveifthe˛ipperpositionswereameasurablequantity(e.g.,viaamotorencoder)ratherthanacontextualuncertainty.Inthesecondbaseline,weremovedtheassumptionofknowninitial˛ipperpositions,introducinguncertaintyintothemodelingprocess.Thiscaserepresentstheperformanceoftheunmodi˝edEEAmodelingprocesswhenfacedwithuncertainty.Intheidealbaselineexperiment,theEEAalgorithmis129abletoachievemaximum˝tnessacrossallreplicateexperiments,inthatitproducesanaccu-ratemodelofthedamagedrobot.However,when˛ipperpositionsareunknownperformanceofthealgorithmdegradestoapproximately70%ofthemaximumscore.Withtheproposedinferencemethod˝tnessreachedapproximately90%oftheperfectscore.Theseexperimentsa˚rmourassumptionthatthebehaviorofmodels,andthereforethemodelingperformance,istightlycoupledtotheinitialcon˝gurationoftheaquaticrobot.InFigure7.12,weshowresultsforthecaseofadamagedpectoral˝n.Asshowninthe˝gure,theunmodi˝ed(b)andinference-basedEEA(c)approachesarrivedatverydi˙erentmorphologies,withtheinference-basedEEAresultcloselyresemblingthedamagedrobot(a).(a)(b)(c)Figure7.12:Simulatedrobotsfromthecasestudy:(a)damagedtargetrobotwithapectoral˝nhalfthelengthofanormalone;(b)morphologicalmodelproducedusingtheunmodi˝edEEAapproach;(c)morphologicalmodelproducedbytheInference-basedEEAapproach.ResultsoftheseexperimentsdemonstratetheimportanceoftheEEAapproachandtheneedtoreduceuncertaintyaspartoftheprocess.Whiletheunmodi˝edEEAapproachwasunabletoarriveatagoodinternalrepresentationtofacilitatetransferintothedamagedrobot,theInferenceapproachwasabletobuildafairinternalrepresentationtoproducee˙ectiveswimminginthedamagedindividual.1307.5DiscussionInthischapter,wedemonstratedanapproachtoautomaticallydiscovertheboundariesofadaptabilityforacyber-physicalsystem.Speci˝cally,thisapproachcharacterizestherangeofoperation(i.e.,mode)forarobotic˝shanditsadaptivecontrolsoftware.Fromthesoftware'spoint-of-view,themorphologicalpropertiesoftherobotareaspectsofanuncertainenvironment.Theapproachtodiscoveringmodeboundariesinvolvesgeneratingscenariosuniformlywithintherangeoftheconditionsthatthesystemisexpectedtoencounter,andthenevolvinganadaptivecontrollerforthosescenarios.Theresultinginformationcouldaidinthedesignofmode-switchinglogic,whereeachmodeincludesanappropriateadaptivecontroller.Throughaseriesofexperiments,wefoundthevolume-basedscenarioselectionmethodtobebothe˙ectiveandcomputationallye˚cient.Althoughtheapproachwasabletodiscovermodeboundaries,forhigherdimensionalspaces,morestrategicmethodsofgeneratingandselectingscenariosmightbewarranted.Additionally,oncetheboundariesofamodearediscovered,itwillbeusefulto˝ndcontrolstrategiesforadjacentmodes.Inthisway,thesystemcanhaveprede˝nedcontrollersthataree˙ectiveformultiplescenarios,includingfail-safeoperation(e.g.,whenthesystemisdamageditcanstillbecontrolledtoreturntoabasestation).Next,wepresentedresultsfromourself-modelingresearchwhereanaquaticrobotwasabletodiscoveranewmodelforitself.Withtheabilitytomapoutmodesandthendecidewhichmodebestmatchessensedconditions,autonomousroboticsystemswillbeabletobothcharacterizedamageandselect/generatenewbehaviors(controllers).131Chapter8ConclusionsTheoverarchinggoaloftheresearchpresentedinthisdissertationistodiscoveraholis-ticapproachforimprovingtheperformance,adaptability,andsurvivabilityofautonomousroboticsystems.Themethodsandtechniquesdevelopedaspartofthisresearchhaveusedaquaticrobots,primarilybiomimeticrobotic˝sh,asatestplatform.Thesedevicesprovidedseveraladvantagesforthislineofresearch.Speci˝cally,theyoperateindi˚cult,nonlinearenvironmentsandrequirefewactuatorsenabling,ustofocusondevelopinggeneraltech-niquesratherthanfocusingonrobotmanufacturingdetails.Althoughwehavediscussedonlyaquaticrobotsinthisdocument,thetechniquesdescribedareapplicabletoabroadrangeofcyber-physicalsystems.8.1O˜ineOptimizationofMorphologyandControl.Ourinitialworkfocusedonimprovingthemorphology(i.e.,physicaldesigns)andcontrolpatternsofroboticsystems.Oneadvantageofevolutionaryroboticsisthatiscanproduceacouplingbetweenbody(morphology)andbrain(control).Inthecaseofrobotic˝shwith˛exible˝ns,ourresultsshowthatitisessentialforcontrolpatternstomatch˝ncharacteristics.Sincetheelongated˝nsbehaveassprings,impropercontrolfrequenciescanactuallyworkthesystem,resultinginlowerspeeds.Infact,foragivencaudal132˝nmorphology(dimensionsand˛exibility)asingleoptimalcontrolfrequencycanbefound.Furthermore,wediscoveredthatpuresinusoidswereas(ifnotmore)e˙ectivethanotherpossiblecontrolpatterns.8.2MultiobjectiveOptimization.Inpracticalsettings,mostroboticsystemsmustbalancetwoormorecompetingobjec-tives.Forexample,theprimarymissionofmostautonomousrobotsistothecompletionofaspeci˝ctask.However,whileperformingtherequisiteactions,suchsystemsmustalsobewaryofdepletingtheirpowersource(i.e.,batteries).Thus,itisalsoimportantforthesedevicestoperforme˚ciently.Intuitively,thesetwoobjectives(quicklycompletingtasksandpowere˚ciency)areoftenatoddswithonespeedand/ormaneuverabilityin-creasee˚ciencyislikelytodecrease.Thisoptimizationprocessisfurthercomplicatedwhenincorporatingcomplexmaterialcharacteristics,suchasthe˛exibilityofarobotic˝shcaudal˝n.Inthisworkwedemonstratedthatevolutionarymultiobjectivealgorithmscanprovideadesignerwithadditionalinformationthatcanenhancetheoperationofarobotic˝sh.AsetofrulescanbederivedfromtheParetofrontthatgivesinsightintowhichparametersaremostimportantinbalancingobjectives,whileatthesametime˝ndingnovelsolutionswithregardstotheuseof˛exiblematerials.Withthisinformation,a˝nalmorphologycanbechosenthatisconsideredthebestcompromiseofallpossiblePareto-optimalchoices.Furthermore,byexaminingthecontrolparametersonly(whileignoringmorphologicalpa-rameters)the˝nalParetofront,wecanprovideinsightintohowobjectivescanbebalancedonlinebyadjustingthecontrolstrategy.8.3EnhancingAdaptiveControl.Oncedeployed,anautonomousrobotwilllikelyexperiencechangesduetotheun-avoidablewearanddegradationofelectromechanicalcomponents.Forexample,servomotor133performancewillchangeasthedeviceexperiencesdi˙erenttorques/loadsandasitages.Likewise,theperformancecharacteristicsof˛exiblematerialsarelikelytochangeduringtheirlifetimeduetothenatureofthefabricationprocess.Our3D-printedcaudal˝ns,forinstance,willexhibitdi˙erentcharacteristicsdependingonwaterabsorption,watertem-perature,andwhetherornottheyhavebeencoatedwithasiliconsealant.The˝eldofcontroltheoryhasdevelopedasetoftechniquesspeci˝callytargetedatalleviatingthesegradualchanges:adaptivecontrol.Inthisdissertation,wedemonstratedthatevolutionaryalgorithmscanimprovetheperformanceofmodel-freeadaptivecontrollerssuchthattheyarecapableofhandlingabroaderrangeofconditions.Speci˝cally,subjectingadaptivecon-trollerstoavarietyofconditionsduringevolutionproducedcontrollerscapableofadaptingtoconditionsevenbeyondthoseencounteredduringevolution.8.4ModeDiscovery.Ofcourse,evenanadaptivecontrollerhaslimits.Whenanadaptivecontrollerbeginstofailduetodrasticchangestothesystemoritsenvironment,aroboticdevicemaybecomeunabletocompleteitstask.However,thissituationdoesnotmeanthatthedeviceitselfcannotcontinuetooperate.Infact,itmaybepossibleforthesystemtoswitchtoanewadaptivecontroller,andcarryon,albeitinalower-performancemanner.Our˝nalareaofresearchaddressedthediscoveryofthelimitsonagivenadaptivecontroller.Theapproachtodiscoveringmodeboundariesinvolvesgeneratingscenariosuniformlyatrandomwithintherangeoftheconditionsthatthesystemisexpectedtoencounter,andthenevolvinganadaptivecontrollerforthosescenarios.Withthisknowledgeregardingcontrollerlimitations,wecande˝neexecutionmodes,whereeachmodeisassociatedwithadi˙erentadaptivecontroller.Thesystemcanthenswitchtoanappropriatecontrollerbasedonthediscoveredexecutionmodes.Theswitchingprocesscanbefacilitatedbyself-modelingtodeterminewhichexecutionmodeisappropriate.Morespeci˝cally,whenaroboticdevicesu˙ersdamage134itcanpreciselyupdateitsownself-modeltore˛ectthedamage,andthenswitchtothecorrectcontroller.Givenanewmodel,thesystemcandooneoftwothings:(1)selectanewcontrollerbasedonadatabaseofmodesandtheiraccompanyingcontrollers,or(2)generateanewbehaviorusinganon-boardevolutionaryalgorithm.Ineithercase,thesystemwillbeabletocontinueoperationdespitesigni˝cantdamage.8.5PotentialforFutureInvestigations.Theresultspresentedinthisdissertationprovideafoundationforseveralpotentialre-searchprojects.The˝rstareaistheon-boardapplicationofEMOParetofrontdata.Inourstudies(Chapters4and5),oncetheevolutionaryexperimentshavecompletedweselectasinglePareto-optimalsolutionanddiscardtheothers.However,itmightbehelpfultoretaintheadditionalinformationanddevelopanalgorithmthatoperatesonlinetoautomat-icallyadjusttherelativeimportanceofmultipleobjectivesdependingonthesituation.Forexample,undercertainconditionsitmaybemoreusefultofocusonspeed(suchaswhenarobotisneararechargingstation),andatothertimesitwillbeimportanttobemoreenergy-e˚cient.Second,wenotethattheMFACcontrollerspresentedinthisworkaredesignedtohandleasingle-input,single-output(SISO)system.However,morecomplexMFACcontrollershavebeendesignedtoaccommodatemultipleinputsandmultipleoutputs.Thus,apotentiallineofinvestigationistoextendtheapproachpresentedinthisdissertationtoincludetrackingmultiplereferencesignalssimultaneously.Forexample,ourcurrentworkcouldbeextendedsuchthatarobotic˝shmighttrackbothadesiredreferencespeedandadesiredreferenceheading.Resultingcontrollerswouldbecapableofmorecomplexbehaviorsinvolvingbothturningandforwardlocomotion.Increasingthenumberofcontrollerinputsandoutputsmakestheoptimizationprocesssigni˝cantlymorechallengingbecause,aswehaveshownin135Chapter3,thereexisttrade-o˙sbetweenachievingthemaximumpossiblespeedandhighmaneuverability.Anotherpotentialareaofinterest,istoinvestigatehowincreasinglycomplextasksa˙ecttheproposeddesignprocess.Currently,ourevaluationfunctionsconsideronlysimpletasks(i.e.,swimmingspeedande˚ciency).Amorecomplexbehaviormightincludeway-pointfollowingorstation-keeping.Weanticipatethatwithincreasedtaskcomplexitytheproposedmethodswillprovideananevengreatervaluewhencomparedtomoretraditionaloptimizationmethods.Indeed,evolutionaryalgorithmsareknownfordiscoveringnovel,andsometimeunintuitive,solutionstodi˚cultproblems.Finally,forthisdissertationwereliedsolelyonaquaticrobots.Itwillbeimportanttoapplythesimilarexperimentstodi˙erentcategoriesofautonomousrobots.Groundvehiclesandunmannedaerialvehiclessharemanydynamicswithaquaticrobots,however,theypresenttheirownchallenges.Forexample,groundvehiclesaremorelikelytoencounterobstacleswhilecarryingoutatask,andaerialvehiclesrequiregreatercarewithrespecttosafety.Thesenewdomainswillnecessitateextendingourworktoincorporatenewalgorithmsforcollisionavoidanceandnewcontrollerveri˝cationtechniquesinordertoensurethatevolvedcontrollersavoiddangerousfailures.Weanticipatethatthemethodsdescribedinthisdissertationwillbewidelyapplicabletosuchdevicesandmanyothercyber-physicalsystems.136BIBLIOGRAPHY137BIBLIOGRAPHY[1]Awardsforhuman-competitiveresultsproducedbygeneticandevolutionarycom-putation.CompetitionheldaspartoftheannualGeneticandEvolutionaryCom-putationConference(GECCO),sponsoredbyACMSIGEVO.Resultsavailableathttp://www.human-competitive.org.[2]DynamicsandControlofTurningManeuverforBiomimeticRoboticFish.IEEE,Oc-tober2006.[3]XuAidong,ZhengYangbo,SongYan,andLiuMingzhe.Animprovedmodelfreeadaptivecontrolalgorithm.InProceedingsoftheFifthInternationalConferenceonNaturalComputation(ICNC),volume1,pagesIEEE,2009.[4]AlessioAlessi,AngeloSudano,DinoAccoto,andEugenioGuglielmelli.Developmentofanautonomousrobotic˝sh.InProceedingsofthe20124thIEEERAS&EMBSInternationalConferenceonBiomedicalRoboticsandBiomechatronics(BioRob),pagesIEEE,2012.[5]P.V.AlvaradoandK.Youcef-Toumi.Designofmachineswithcompliantbodiesforbiomimeticlocomotioninliquidenvironments.JournalofDynamicSystems,Measure-ment,andControl,2006.[6]J.M.AndersonandN.K.Chhabra.Maneuveringandstabilityperformanceofarobotictuna.Integr.Comp.Biol.,2002.[7]JaredM.MooreBettyH.C.ChengAnthonyJ.Clark,ByronDeVriesandPhilipK.McKinley.(submitted)automateddiscoveryofadaptabilityboundariesthroughcom-putationalevolution.In10thIEEEInternationalConferenceonSelf-AdaptiveandSelf-OrganizingSystems,2016.[8]AnneAuger,JohannesBader,DimoBrockho˙,andEckartZitzler.Hypervolume-basedmultiobjectiveoptimization:Theoreticalfoundationsandpracticalimplications.TheoreticalComputerScience,2012.[9]AtˇlˇmGüne³Baydin.Evolutionofcentralpatterngeneratorsforthecontrolofa˝ve-linkbipedalwalkingmechanism.Paladyn,JournalofBehavioralRobotics,2012.[10]JoshBongardandHodLipson.Evolvedmachinesshedlightonrobustnessandre-silience.ProceedingsoftheIEEE,May2014.[11]JoshBongard,VictorZykov,andHodLipson.Automatedsynthesisofbodyschemausingmultiplesensormodalities.InInProceedingsoftheTenthInternationalCon-ferenceontheSimulationandSynthesisofLivingSystems(ALIFEX,pages2006.138[12]JoshBongard,VictorZykov,andHodLipson.Resilientmachinesthroughcontinuousself-modeling.Science,2006.[13]JoshCBongard.Evolutionaryrobotics.CommunicationsoftheACM,56(82013.[14]JoshCBongardandHodLipson.Oncemoreuntothebreach:Co-evolvingarobotanditssimulator.InProceedingsoftheNinthInternationalConferenceontheSimulationandSynthesisofLivingSystems(ALIFE9),pages2004.[15]D.J.Braun,F.Petit,F.Huber,S.Haddadin,P.vanderSmagt,A.Albu-Scha˙er,andS.Vijayakumar.Robotsdrivenbycompliantactuators:Optimalcontrolunderactuationconstraints.IEEETransactionsonRobotics,2013.[16]NicolasBredeche,EvertHaasdijk,andAEEiben.On-line,on-boardevolutionofrobotcontrollers.InArti˝calEvolution,pagesSpringer,2010.[17]RodneyA.Brooks.Arti˝ciallifeandrealrobots.InProceedingsoftheFirstEuropeanConferenceonArti˝cialLife,pagesMITPress,Cambridge,MA,1992.[18]Z.Chen,S.Shatara,andX.Tan.Modelingofbiomimeticrobotic˝shpropelledbyanionicpolymer-metalcompositecaudal˝n.IEEE/ASMETransactionsonMechatronics,2010.[19]NickCheney,RobertMacCurdy,Je˙Clune,andHodLipson.Unshacklingevolution:evolvingsoftrobotswithmultiplematerialsandapowerfulgenerativeencoding.InProceedingsofthe15thAnnualConferenceonGeneticandEvolutionaryComputation,pagesAmsterdam,TheNetherlands,2013.ACM.[20]GeorgeSCheng.Model-freeadaptive(MFA)control.ComputingandControlEngi-neering,2004.[21]HillelJ.ChielandRandallD.Beer.Thebrainhasabody:adaptivebehavioremergesfrominteractionsofnervoussystem,bodyandenvironment.TrendsinNeurosciences,20(12):553557,1997.[22]ProjectChrono.Chrono::Engine,http://www.chronoengine.info/,2014.[23]AnthonyJ.ClarkandPhilipK.McKinley.Evolutionaryoptimizationofrobotic˝shcontrolandmorphology.InProceedingsofthe15thAnnualConferenceCompaniononGeneticandEvolutionaryComputation,GECCO'13Companion,pagesNewYork,NY,USA,2013.ACM.[24]AnthonyJ.Clark,PhilipK.McKinley,andXiaoboTan.On-boardevolutionofamodel-freeadaptivecontrollerforarobotic˝sh.InProceedingsofEvolutionofPhysicalSystemsWorkshop,heldinconjunctionwiththe14thInternationalConferenceontheSynthesisandSimulationofLivingSystems(ALIFE14),NewYork,NewYork,July2014.139[25]AnthonyJ.Clark,PhilipK.McKinley,andXiaoboTan.Enhancingamodel-freeadaptivecontrollerthroughevolutionarycomputation.Technicalreport,MichiganStateUniversity,NewYork,NY,USA,July2015.[26]AnthonyJ.Clark,JaredM.Moore,JianxunWang,XiaoboTan,andPhilipK.McKin-ley.Evolutionarydesignandexperimentalvalidationofa˛exiblecaudal˝nforrobotic˝sh.InProceedingsoftheThirteenthInternationalConferenceontheSynthesisandSimulationofLivingSystems,pagesEastLansing,Michigan,USA,July2012.BestPaperAward.[27]AnthonyJ.Clark,XiaoboTan,andPhilipK.McKinley.Evolutionarymultiobjectivedesignofa˛exiblecaudal˝nforrobotic˝sh.Bioinspiration&Biomimetics,specialissueonBioinspiredSoftRobotics,10(6):065006,November2015.[28]AnthonyJ.Clark,JianxunWang,XiaoboTan,andPhilipK.McKinley.Balancingperformanceande˚ciencyinarobotic˝shwithevolutionarymultiobjectiveoptimiza-tion.InProceedingsoftheIEEEInternationalConferenceonEvolvableSystems,heldinconjunctionwiththe2014IEEESymposiumonComputationalIntelligence(SSCI),pagesOrlando,Florida,Dec2014.[29]Je˙Clune,BenjaminEBeckmann,CharlesOfria,andRobertTPennock.Evolvingcoordinatedquadrupedgaitswiththehyperneatgenerativeencoding.InProceedingsoftheIEEECongressonEvolutionaryComputation,pagesTrondheim,Norway,2009.IEEE.[30]L.S.Coelho,MarceloWictho˙Pessôa,RodrigoRodriguesSumar,andAntonioAu-gustoRodriguesCoelho.Model-freeadaptivecontroldesignusingevolutionary-neuralcompensator.ExpertSystemswithApplications,2010.[31]CarlosA.CoelloCoello,DavidA.VanVeldhuizen,andGaryB.Lamont.Evolutionaryalgorithmsforsolvingmulti-objectiveproblems,volume242.Springer,2002.[32]ErwinCoumans.BulletPhysicsLibrary,http://www.bulletphysics.org/,2014.[33]AlessandroCrespi,DaisyLachat,ArianePasquier,andAukeJanIjspeert.Controllingswimmingandcrawlingina˝shrobotusingacentralpatterngenerator.AutonomousRobots,December2007.[34]AntoineCully,Je˙Clune,DaneshTarapore,andJean-BaptisteMouret.Robotsthatcanadaptlikeanimals.Nature,2015.[35]KennethA.DeJong.EvolutionaryComputation:AUni˝edApproach.MITPress,2002.[36]KennethADeJong.Evolutionarycomputation:auni˝edapproach.MITpress,2006.[37]KalyanmoyDeb,AmritPratap,SameerAgarwal,andTAMTMeyarivan.Afastandelitistmultiobjectivegeneticalgorithm:Nsga-ii.IEEETransactionsonEvolutionaryComputation,2002.140[38]F.C.DyerandH.J.Brockmann.BiologyoftheUmwelt.InL.D.HouckandL.C.Drickamer,editors,FoundationsofAnimalBehavior,pages5UniversityofChicagoPress,1996.[39]MatthiasEhrgott.Adiscussionofscalarizationtechniquesformultipleobjectiveintegerprogramming.AnnalsofOperationsResearch,2006.[40]HadiElDaou,TaaviSalumäe,LilyDChambers,WilliamMMegill,andMaarjaKru-usmaa.Modellingofabiologicallyinspiredrobotic˝shdrivenbycompliantparts.Bioinspiration&Biomimetics,9(1):016010,2014.[41]M.Epstein,J.E.Colgate,andM.A.MacIver.Generatingthrustwithabiologically-inspiredroboticribbon˝n.InProceedingsofthe2006IEEE/RSJInternationalCon-ferenceonIntelligentRobotsandSystems,pagesBeijing,China,2006.[42]ChristopherJEsposito,JamesLTangorra,BrookeEFlammang,andGeorgeVLauder.Arobotic˝shcaudal˝n:e˙ectsofsti˙nessandmotorprogramonlocomotorperformance.TheJournalofexperimentalbiology,2012.[43]JolyonJ.Faria,JohnR.G.Dyer,RomainO.Clément,IainD.Couzin,NatalieHolt,AshleyJ.W.Ward,DeanWaters,andJensKrause.Anovelmethodforinvestigat-ingthecollectivebehaviourof˝sh:introducing`robo˝sh'.BehavioralEcologyandSociobiology,2010.[44]KaraLFeilichandGeorgeVLauder.Passivemechanicalmodelsof˝shcaudal˝ns:e˙ectsofshapeandsti˙nessonself-propulsion.Bioinspiration&Biomimetics,10(3):036002,2015.[45]DarioFloreano,PhilHusbands,andStefanoNol˝.EvolutionaryRobotics.InHandbookofRobotics.SpringerVerlag,Berlin,2008.[46]DarioFloreanoandFrancescoMondada.Evolutionaryneurocontrollersforautonomousmobilerobots.Neuralnetworks,1998.[47]DarioFloreanoandJosebaUrzelai.Evolutionofplasticcontrolnetworks.Autonomousrobots,2001.[48]StephanieForrest,ThanhVuNguyen,WestleyWeimer,andClaireLeGoues.Ageneticprogrammingapproachtoautomatedsoftwarerepair.InProceedingsofthe11thAnnualconferenceonGeneticandevolutionarycomputation,pagesACM,2009.[49]MinyueFuandB.Barmish.Adaptivestabilizationoflinearsystemsviaswitchingcontrol.IEEETransactionsonAutomaticControl,Dec1986.[50]DavidEdwardGoldbergetal.Geneticalgorithmsinsearch,optimization,andmachinelearning,volume412.Addison-wesleyReadingMenloPark,1989.[51]FaustinoJGomezandRistoMiikkulainen.Activeguidancefora˝nlessrocketus-ingneuroevolution.InGeneticandEvolutionary2003,pagesSanFrancisco,2003.Springer,MorganKaufmann.141[52]SimonHaykin.Neuralnetworksandlearningmachines.Prentice-Hall,2009.[53]JoãoHespanha,DanielLiberzon,A.StephenMorse,BrianD.O.Anderson,ThomasS.Brinsmead,andFrankyDeBruyne.Multiplemodeladaptivecontrol.part2:switching.InternationalJournalofRobustandNonlinearControl,2001.[54]JoãoP.Hespanha,DanielLiberzon,andA.StephenMorse.Overcomingthelimitationsofadaptivecontrolbymeansoflogic-basedswitching.Systems&ControlLetters,49(1):4965,2003.AdaptiveControl.[55]JoãoP.HespanhaandA.StephenMorse.Switchingbetweenstabilizingcontrollers.Automatica,38(11):19051917,2002.[56]JonathanD.HillerandHodLipson.Automaticdesignandmanufactureofsoftrobots.IEEETransactionsonRobotics,2012.[57]JohnHHolland.Adaptationinnaturalandarti˝cialsystems:Anintroductoryanalysiswithapplicationstobiology,control,andarti˝cialintelligence.UMichiganPress,1975.[58]GregorySHornby,JordanBPollack,etal.Body-brainco-evolutionusingl-systemsasagenerativeencoding.InProceedingsoftheGeneticandEvolutionaryComputationConference,pages2001.[59]Zhong-ShengHouandZhuoWang.Frommodel-basedcontroltodata-drivencontrol:Survey,classi˝cationandperspective.InformationSciences,235:335,2013.[60]ZhongshengHouandWenhuHuang.Themodel-freelearningadaptivecontrolofaclassofSISOnonlinearsystems.InProceedingsofthe1997AmericanControlConference,volume1,pages1997.[61]FranzS.Hover,RyanM.Eustice,AyoungKim,BrendanEnglot,HordurJohannsson,MichaelKaess,andJohnJ.Leonard.Advancedperception,navigationandplanningforautonomousin-watershiphullinspection.InternationalJournalofRoboticsResearch,2012.[62]QingsongHu,D.R.Hedgepeth,LihongXu,andXiaoboTan.Aframeworkformodelingsteadyturningofrobotic˝sh.InIEEEInternationalConferenceonRoboticsandAutomation(IRCA),pages2669may2009.[63]P.IoannouandB.Fidan.AdaptiveControlTutorial.AdvancesinDesignandControl.SocietyforIndustrialandAppliedMathematics,2006.[64]PetrosAIoannouandJingSun.RobustAdaptiveControl.DoverBooksonElectricalEngineering.DoverPublications,2012.[65]JulioJerezandAlainSuero.NewtonGameDynamics,http://newtondynamics.com/forum/newton.php,2012.142[66]S.Jiang,J.Zhang,Y.-S.Ong,A.N.Zhang,andP.S.Tan.Asimpleandfasthyper-volumeindicator-basedmultiobjectiveevolutionaryalgorithm.IEEETransactionsonCybernetics,2014.[67]RahulKala.Multi-robotpathplanningusingco-evolutionarygeneticprogramming.ExpertSystemswithApplications,2012.[68]IgorKaro«.Modi˝edmodel-freeadaptivecontrollerforanonlinearrotorsystem.InArti˝cialIntelligenceandSoftComputing,pagesSpringer,2012.[69]N.Kato.Controlperformanceinthehorizontalplaneofa˝shrobotwithmechanicalpectoral˝ns.IEEEJournalofOceanicEngineering,2000.[70]SangbaeKim,CeciliaLaschi,andBarryTrimmer.Softrobotics:abioinspiredevolu-tioninrobotics.TrendsinBiotechnology,2013.[71]EricKlavins,HaldunKomsuoglu,andJRobert.TheRoleofRe˛exesversusCentralPatternGeneratorsinDynamicalLeggedLocomotion.forbiomimeticrobots,2002.[72]AbdullahKonak,DavidWCoit,andAliceESmith.Multi-objectiveoptimizationusinggeneticalgorithms:Atutorial.ReliabilityEngineering&SystemSafety,91(9):9921007,2006.[73]SylvainKoos,AntoineCully,andJean-BaptisteMouret.Fastdamagerecoveryinroboticswiththet-resiliencealgorithm.TheInternationalJournalofRoboticsRe-search,2013.[74]SylvainKoos,JMouret,andStéphaneDoncieux.Automaticsystemidenti˝cationbasedoncoevolutionofmodelsandtests.InIEEECongressonEvolutionaryCompu-tation.,pagesIEEE,2009.[75]SylvainKoos,Jean-BaptisteMouret,andStéphaneDoncieux.Crossingtherealitygapinevolutionaryroboticsbypromotingtransferablecontrollers.InProceedingsofthe12thannualconferenceonGeneticandevolutionarycomputation,pagesACM,2010.[76]JohnR.Koza,MartinA.Keane,MatthewJ.Streeter,WilliamMydlowec,JessenYu,andGuidoLanza.GeneticProgrammingIV:RoutineHuman-CompetitiveMachineIntelligence.GeneticProgramming.Springer,1stedition,2005.[77]JohnR.KozaandJamesP.Rice.Automaticprogrammingofrobotsusinggeneticprogramming.InAAAI,pages1992.[78]G.V.Lauder,E.J.Anderson,J.Tangorra,andP.G.A.Madden.Fishbiorobotics:Kinematicsandhydrodynamicsofself-propulsion.JournalofExperimentalBiology,2007.[79]LjungLennart.Systemidenti˝cation:theoryfortheuser,1999.143[80]M.J.Lighthill.Large-AmplitudeElongated-BodyTheoryofFishLocomotion.Proceed-ingsoftheRoyalSocietyofLondon.SeriesB.BiologicalSciences,November1971.[81]HodLipson.Evolutionaryroboticsandopen-endeddesignautomation.InBarCohen,editor,Biomimetics,pagesCRCPress,2005.[82]HodLipson.Challengesandopportunitiesfordesign,simulation,andfabricationofsoftrobots.SoftRobotics,2014.[83]HodLipsonandJoshBongard.Anexploration-estimationalgorithmforsynthesisandanalysisofengineeringsystemsusingminimalphysicaltesting.InProceedingsofthe2004ASMEDesignEngineeringTechnicalConferencesandComputersandInforma-tioninEngineeringConference(IDETC/CIE2004),pages,SaltLakeCity,Utah,2004.[84]J.H.Long,Jr.,T.J.Koob,K.Irving,K.Combie,V.Engel,H.Livingston,A.Lammert,andJ.Schumacher.Biomimeticevolutionaryanalysis:Testingtheadaptivevalueofvertebratetailsti˙nessinautonomousswimmingrobots.JournalofExperimentalBiology,2006.[85]K.H.LowandC.W.Chong.Parametricstudyoftheswimmingperformanceofa˝shrobotpropelledbya˛exiblecaudal˝n.Bioinspiration&Biomimetics,5(4):046002,2010.[86]CarmelMajidi.Softrobotics:Aperspectivecurrenttrendsandprospectsforthefuture.SoftRobotics,July2013.[87]RichardMasonandJoelWBurdick.Experimentsincarangiformrobotic˝shlocomo-tion.InRoboticsandAutomation,2000.Proceedings.ICRA'00.IEEEInternationalConferenceon,volume1,pagesIEEE,2000.[88]KMatsuoka.Sustainedoscillationsgeneratedbymutuallyinhibitingneuronswithadaptation.BiologicalCybernetics,1985.[89]KiyotoshiMatsuoka.Analysisofaneuraloscillator.BiologicalCybernetics,104(4-May2011.[90]CraigMautnerandRichardBelew.Evolvingrobotmorphologyandcontrol.Arti˝cialLifeandRobotics,2000.[91]OrazioMiglino,HenrikHautopLund,andStefanoNol˝.Evolvingmobilerobotsinsimulatedandrealenvironments.Arti˝cialLife,1996.[92]JaredM.MooreandPhilipK.McKinley.Evolutionofanamphibiousrobotwithpassivejoints.InProceedingsofthe2013IEEECongressonEvolutionaryComputation,pagesCancun,Mexico,2013.144[93]JaredM.MooreandPhilipK.McKinley.Investigatingmodularcouplingofmor-phologyandcontrolwithdigitalmuscles.InProceedingsofthe14thInternationalConferenceontheSynthesisandSimulationofLivingSystems(ALIFE14),pagesNewYork,NewYork,July2014.[94]Jean-BaptisteMouretandPaulTonelli.Arti˝cialEvolutionofPlasticNeuralNetworks:AFewKeyConcepts,pages.SpringerBerlinHeidelberg,Berlin,Heidelberg,2014.[95]K.S.Narendra,J.Balakrishnan,andM.K.Ciliz.Adaptationandlearningusingmultiplemodels,switching,andtuning.IEEEControlSystems,Jun1995.[96]KumpatiSNarendra.Applicationsofadaptivecontrol.Elsevier,2012.[97]DuyNguyen-TuongandJanPeters.Modellearningforrobotcontrol:asurvey.Cog-nitiveprocessing,2011.[98]StefanoNol˝andDarioFloreano.Evolutionaryrobotics:TheBiology,Intelligence,andTechnologyofSelf-organizingMachines,volume150.MITpress,Cambridge,MA,USA,2000.[99]TakuyaOkada,ShuxiangGuo,NanXiao,FuQiang,andYasuhiroYamauchi.Controlofthewirelessmicrorobotwithmulti-dofslocomotionformedicalapplications.InProceedingsofthe2012InternationalConferenceonMechatronicsandAutomation(ICMA),pagesIEEE,2012.[100]NikhilPadhye,PiyushBhardawaj,andKalyanmoyDeb.Improvingdi˙erentialevolu-tionthroughauni˝edapproach.JournalofGlobalOptimization,pages2013.[101]DanielPaluskaandHughHerr.Thee˙ectofserieselasticityonactuatorpowerandworkoutput:Implicationsforroboticandprostheticjointdesign.RoboticsandAu-tonomousSystems,2006.[102]Y-JPark,UseokJeong,JeongsuLee,S-RKwon,H-YKim,andK-JCho.Kinematicconditionformaximizingthethrustofarobotic˝shusingacompliantcaudal˝n.IEEETransactionsonRobotics,2012.[103]RolfPfeifer,MaxLungarella,andFumiyaIida.Self-Organization,Embodiment,andBiologicallyInspiredRobotics.Science,2007.[104]RolfPfeifer,MaxLungarella,andFumiyaIida.Thechallengesaheadforbio-inspired'soft'robotics.CommunicationsoftheACM,2012.[105]P.Phamduy,G.Polverino,R.C.Fuller,andM.Por˝ri.Fishandrobotdancingtogether:blue˝nkilli˝shfemalesresponddi˙erentlytothecourtshipofarobotwithvaryingcolormorphs.Bioinspiration&Biomimetics,9(3):036021,2014.145[106]JordanBPollack,HodLipson,SevanFicici,PabloFunes,GregHornby,andRichardAWatson.Evolutionarytechniquesinphysicalrobotics.InEvolvableSystems:frombiologytohardware,pagesSpringer,2000.[107]TorstenReilandPhilHusbands.Evolutionofcentralpatterngeneratorsforbipedalwalkinginareal-timephysicsenvironment.IEEETransactionsonEvolutionaryCom-putation,2002.[108]CharlesRichterandHodLipson.Untetheredhovering˛apping˛ightofa3d-printedmechanicalinsect.Arti˝ciallife,2011.[109]JohnRie˙el,DavisKnox,SchuylerSmith,andBarryTrimmer.Growingandevolvingsoftrobots.Arti˝cialLife,2014.[110]MatthewJ.Rose,AnthonyJ.Clark,JaredM.Moore,andPhilipK.McKinley.Justkeepswimming:Accountingforuncertaintyinself-modelingaquaticrobots.InPro-ceedingsofthe6thInternationalWorkshoponEvolutionaryandReinforcementLearn-ingforAutonomousRobotSystems,Taormina,Italy,September2013.BestPaperAward.[111]WilsonJ.RughandJe˙S.Shamma.Researchongainscheduling.Automatica,36(10):14011425,2000.[112]HaithamSeadaandKalyanmoyDeb.U-NSGA-III:Auni˝edevolutionaryoptimizationprocedureforsingle,multiple,andmanyobjectivesproof-of-principleresults.InPro-ceedingsofEighthConferenceonEvolutionaryMulti-CriterionOptimization(EMO-2015),pagesSpringer,2015.[113]BuHyunShin,Kyung-MinLee,andSeung-YopLee.Aminiaturizedtadpolerobotusinganelectromagneticoscillatoryactuator.JournalofBionicEngineering,36,2015.[114]KarlSims.Evolving3Dmorphologyandbehaviorbycompetition.Arti˝cialLife,1994.[115]Simulink:DynamicsystemsimulationforMATLAB,User'sGuide.TheMathWorksInc.,Natick,Massachusetts,USA,1990-2013.[116]RussellSmith.OpenDynamicsEngine,http://www.ode.org/,2014.[117]LeeSpectorandJonKlein.Trivialgeographyingeneticprogramming.Geneticpro-grammingtheoryandpracticeIII,2006.[118]KennethOStanley,DavidBD'Ambrosio,andJasonGauci.Ahypercube-baseden-codingforevolvinglarge-scaleneuralnetworks.Arti˝ciallife,2009.[119]KennethOStanleyandRistoMiikkulainen.Evolvingneuralnetworksthroughaug-mentingtopologies.Evolutionarycomputation,2002.146[120]R.StornandK.Price.Di˙erentialevolution-asimpleande˚cientheuristicforglobaloptimizationovercontinuousspaces.JournalofGlobalOptimization,1997.[121]YogoTakada,RyosukeAraki,YukinobuNakanishi,MotohiroNonogaki,KazuakiEbita,andTomoyukiWakisaka.Developmentofsmall˝shrobotspoweredbysmallandultra-lightpassive-typepolymerelectrolytefuelcells.JournalofRoboticsandMechatronics,22(2):150,2010.[122]YogoTakada,KeisukeKoyama,andTakahiroUsami.Positionestimationofsmallrobotic˝shbasedoncamerainformationandgyrosensors.Robotics,2014.[123]XiaoboTan.Autonomousrobotic˝shasmobilesensorplatforms:Challengesandpotentialsolutions.MarineTechnologySocietyJournal,2011.[124]PaulTonelliandJean-BaptisteMouret.Ontherelationshipsbetweengenerativeencod-ings,regularity,andlearningabilitieswhenevolvingplasticarti˝cialneuralnetworks.PloSONE,8(11),2013.[125]MichaelS.TriantafyllouandGeorgeS.Triantafyllou.Ane˚cientswimmingmachine.Scienti˝cAmerican,272:64,1995.[126]R.VanHam,T.G.Sugar,B.Vanderborght,K.W.Hollander,andD.Lefeber.Compli-antactuatordesigns.IEEERobotics&AutomationMagazine,pagesSeptember2009.[127]BarthélémyvonHaller,AukeIjspeert,andDarioFloreano.Co-evolutionofstructuresandcontrollersforneubotunderwatermodularrobots.InAdvancesinArti˝cialLife,volume3630ofLectureNotesinComputerScience,pagesSpringerBerlinHeidelberg,2005.[128]J.Wang,P.K.McKinley,andX.Tan.Dynamicmodelingofrobotic˝shwitha˛exiblecaudal˝n.InProceedingsoftheASME20125thAnnualDynamicSystemsandControlConference,jointwiththeJSME201211thMotionandVibrationConference,Ft.Lauderdale,Florida,USA,Ft.Lauderdale,Florida,USA,October2012.PaperDSCC2012-MOVIC2012-8695.[129]JianxunWang,PhilipK.McKinley,andXiaoboTan.Dynamicmodelingofrobotic˝shwithabase-actuated˛exibletail.JournalofDynamicSystems,MeasurementandControl,137(1),August2014.[130]YuWang,RuiTan,GuoliangXing,JianxunWang,andXiaoboTan.Accuracy-awareaquaticdi˙usionprocesspro˝lingusingroboticsensornetworks.InProceedingsofthe11thACM/IEEEConferenceonInformationProcessinginSensorNetworks,pagesBeijing,China,2012.[131]X.Yao.Evolvingarti˝cialneuralnetworks.ProceedingsoftheIEEE,1999.147[132]JunzhiYu,RuiDing,QinghaiYang,MinTan,WeibingWang,andJianweiZhang.Onabio-inspiredamphibiousrobotcapableofmultimodalmotion.IEEE/ASMETransactionsonMechatronics,2011.[133]JunzhiYuandLizhongLiu.Dynamicmodelingandexperimentalvalidationofbiomimeticrobotic˝sh.InProceedingsofthe2006AmericanControlConference,pages2006.[134]JuanCristóbalZagalandJavierRuiz-Del-Solar.Combiningsimulationandrealityinevolutionaryrobotics.JournalofIntelligentandRoboticSystems,2007.[135]JiZhangandBettyH.C.Cheng.Model-baseddevelpmentofdynamicallyadap-tivesoftware.InIEEEInternationalConferenceonSoftwareEngineering(ICSE06),Shanghai,China,May2006.IEEE.DistinguishedPaperAward.[136]AiminZhou,Bo-YangQu,HuiLi,Shi-ZhengZhao,PonnuthuraiNagaratnamSug-anthan,andQingfuZhang.Multiobjectiveevolutionaryalgorithms:Asurveyofthestateoftheart.SwarmandEvolutionaryComputation,2011.[137]ChunlinZhou,C.W.Chong,YuZhong,andK.H.Low.Robustgaitcontrolforsteadyswimmingofacarangiform˝shrobot.InProceedingsoftheIEEE/ASMEInternationalConferenceonAdvancedIntelligentMechatronics,pagesJuly2009.[138]ChunlinZhouandK.H.Low.Locomotionplanningofbiomimeticrobotic˝shwithmulti-jointactuation.InProceedingsoftheIEEE/RSJInternationalConferenceonIntelligentRobotsandSystems,pages2009.[139]EckartZitzlerandSimonKünzli.Indicator-basedselectioninmultiobjectivesearch.InParallelProblemSolvingfromNature-PPSNVIII,pagesSpringer,2004.[140]EckartZitzler,MarcoLaumanns,andLotharThiele.SPEA2:ImprovingtheStrengthParetoEvolutionaryAlgorithm.TechnicalReport103,ComputerEngineeringandNetworksLaboratory(TIK),ETHZurich,Zurich,Switzerland,2001.148