SOFT PRESSURE SENSING SYSTEM WITH APPLICATION TO UNDERWATER SEA LAMPREY DETECTION By Hongyang Shi A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Electrical Engineering — Doctor of Philosophy 2022 ABSTRACT SOFT PRESSURE SENSING SYSTEM WITH APPLICATION TO UNDERWATER SEA LAMPREY DETECTION By Hongyang Shi Species-specific monitoring offers fundamental tools for natural resource management and conservation but requires techniques that target species-specific traits or markers. Sea lamprey, a destructive invasive species in the Great Lakes in North America and conservation target in Europe, is among very few fishes that rely on oral suction during migration and spawning. Yet attachment by suction has not been exploited for sea lamprey control or conservation. This dissertation is focused on advancing soft pressure sensing systems for underwater sea lamprey detection. First, a pressure sensing panel based on commercial vacuum sensors is developed to measure the suction dynamics of juvenile and adult sea lampreys, such as pressure amplitude, frequency and suction duration. Measurements from an array of sensors indicate that the suction pressure distribution is largely uniform across the mouths of lampreys, and the suction pressure does not differ between static and flowing water conditions when the water velocity is lower than 0.45 m/s. Such biological information could inform the design of new systems to monitor behavior, distribution and abundance of lampreys. Based on the measured biological information, two types of soft pressure sensors are proposed for underwater sea lamprey detection. First, a soft capacitive pressure sensor is developed, which is made using a low-cost screen-printing process and can reliably detect both positive and negative pressures. The sensor is made with a soft dielectric layer and stretchable conductive polymer electrodes. Air gaps are designed and incorporated into the dielectric layer to significantly enhance the sample deformation and the response to pressures especially negative pressure. This soft capacitive pressure sensor can successfully detect non- conductive objects like plastic blocks compressed against it or rubber suction cup attached to it; however, it does not work well underwater since water causes parasitic capacitance on the sensor that interferes with the detection. The second sensor we present is a low-cost and efficient piezoresistive pressure sensor, which consists of a layer of piezoresistive film patch matrix sandwiched between two layers of perpendicular copper tape electrodes. Here, the measured two-point resistance is not equal to the actual cell resistance for that pixel due to the cross-talk effect of the pixels. Sev- eral regularized least-squares algorithms are proposed to reconstruct the cell resistance map from the two-point resistance measurements. Experiments show that this pressure sensor is able to capture the pressure profiles during sea lamprey attachment. The performance and computational complexity of the reconstruction algorithms with different regularization functions are also compared. Finally, we design an automated sea lamprey detection system based on the piezoresistive pressure sensor array using machine learning. Three types of object detection algorithms are deployed to learn features of the mapping contours when effective attachment by “compres- sion” or “suction” is formed on the sensor array. Their validation performance and inference speeds are evaluated and compared in depth, and YOLOv5s proves to be the best detector. Furthermore, a detection approach based on the YOLOv5s model with a confidence filter unit, is proposed. In particular, different optimal detection thresholds are proposed for the compression and suction patterns, respectively, in order to reduce the false positive rate caused by the sensor’s memory effect. The efficacy of the proposed method is supported with experimental results on real-time underwater detection of sea lampreys. Copyright by HONGYANG SHI 2022 Dedicated to my beloved family for loving me unconditionally and supporting me in pursuit of my dreams and career aspirations. v ACKNOWLEDGMENTS It is a pleasure to thank many people who have made this thesis possible. First, I would like to express my warmest gratitude to my advisor, Professor Xiaobo Tan, for providing me with the opportunity to pursue my dreams. Without his support and advice throughout the years, I would never have completed the research work. I am happy and lucky that I have the chance to work with him. During my doctoral study, he has profoundly inspired me and influenced me in many ways with his knowledge, expertise, enthusiasm, and vision. I am grateful for his guidance and patience in the fascinating field of soft robotics. I would like to thank Prof. Vaibhav Srivastava, Prof. Wen Li, Prof. Chunqi Qian, and Prof. Tong (Tony) Gao for kindly serving on my guidance committee. They generously spent their time in offering me insightful comments and suggestions for my research, which made my work more solid and comprehensive. I want to express my appreciation of the co-advisors on the sea lamprey sensor project, Prof. Chuan Wang from Washington University in St. Louis, Prof. Nelson Sepúlveda at MSU, and Dr. Christopher Holbrook and Scott Miehls from the U.S. Geological Survey Hammond Bay Biological Station. Thank you all for your tremendous help and guidance in the experiments of sea lamprey detection and related sensor design and test. And for the controllable flexible surface project for underwater boundary layer modulation, which is not included in this dissertation, I want to thank Dr. Weilin (Will) Hou and Dr. Silvia Matt from the U. S. Naval Research Laboratory for their support and inspiration. It also means a lot to me to have been a member of the Smart Microsystems Lab, with so many nice and brilliant friends and colleagues: Dr. Montassar Sharif, Dr. Osama En-Nasr, Dr. Jason Greenberg, Dr. Pratap Bhanu Solanki, Dr. Thassyo Pinto, Dr. Maria Castano, vi Dr. Mohammed Al-Rubaiai, John Thon, Xinda Qi, Demetris Coleman, Yu Mei, Claudia Chen, and Jacob Blatt. Thank you for sharing your knowledge and ideas as well as valuable experimental experience in the past few years; it was a great time to work with you all in the lab. I cannot forget my dear friends at MSU or collaborating colleagues who went through hard times together, cheered me up, and celebrated every accomplishment: Dr. Lai Wei, Dr. Yunqi (Steven) Cao, Dr. Jinshui Miao, Dr. Le Cai, Dr. Tianyi He, Dr. Siyang Yang, Dr. Miaolei Zhou, Dr. Rui Xu, Ian González-Afanador, Li-Wei Lo, Yiheng Zhang, and many others. I wish to thank my girlfriend, Siwen Zhang, for always being beside me and giving me motivation to move forward. She has been my best friend and great companion, who loved, supported, encouraged, entertained, and helped me get through this distressed period in a positive way. And I would like to thank my little kitten, Bruno (Pidan), for his accompany during the past year. Lastly but most importantly, I wish to thank my mother, Licheng Qu, and my sisters, Hua Shi and Min Shi for their unconditional love and support, and timely encouragement and caring. I will always be indebted to them for their love and care despite the long distance between us. Words alone could not express my appreciation to them. vii TABLE OF CONTENTS LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii Chapter 1 Introduction and Motivation . . . . . . . . . . . . . . . . . . . . . 1 1.1 Sea Lamprey Detection via Attachment by Suction . . . . . . . . . . . . . . 2 1.1.1 State of the Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 Soft Pressure Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2.1 State of the Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.3 Soft Pressure Sensor-based Sea Lamprey Detection . . . . . . . . . . . . . . 11 1.3.1 The Need for Automated Sea Lamprey Detection . . . . . . . . . . . 11 1.3.2 State of the Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Chapter 2 Measurement of Suction Pressure Dynamics of Sea Lampreys, Petromyzon marinus . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.1.1 The Pressure Sensing Apparatuses . . . . . . . . . . . . . . . . . . . 17 2.1.2 Leakage Check of the Pressure Sensing System . . . . . . . . . . . . . 18 2.2 Measurement of Sea Lamprey Suction Dynamics . . . . . . . . . . . . . . . . 21 2.2.1 Experimental animals . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.2.2 Experiment 1: Recording of Adult Sea Lamprey Suction Dynamics with a Single Pressure Sensor in Static Water . . . . . . . . . . . . . 23 2.2.3 Experiment 2: Recording of Juvenile and Adult Sea Lamprey Suction Dynamics with a 9-port Pressure Sensing System in Static Water . . 25 2.2.4 Experiment 3: Recording of Adult Sea Lamprey Suction Dynamics with a 9-port Pressure Sensing System in Flowing Water . . . . . . . 25 2.2.5 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.3.1 Pressure Data for Leakage Check of the Apparatus . . . . . . . . . . 28 2.3.2 Suction Dynamics of Juvenile and Adult Sea Lampreys . . . . . . . . 29 2.3.3 Suction Pressure Distribution within the Mouth of Adult and Juvenile Lampreys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.3.4 Effects of flowing water on measured suction pressure . . . . . . . . . 36 2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Chapter 3 Screen-Printed Soft Capacitive Sensors for Spatial Mapping of Both Positive and Negative Pressures . . . . . . . . . . . . . . . 42 3.1 Fabrication of the Soft Capacitive Pressure Sensors with Air Gap Channels . 43 viii 3.2 Characterization of the Single-Pixel Sensors . . . . . . . . . . . . . . . . . . 45 3.3 Finite Element Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.4 Spatial Mapping of Pressure Distributions with the Sensor Array . . . . . . . 53 3.5 Challenges in Detecting Sea Lamprey Attachment Underwater . . . . . . . . 57 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Chapter 4 Soft Piezoresistive Pressure Sensor for Underwater Sea Lamprey Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.1 Sensor Design and Fabrication . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.1.1 Working Principle and Characterization . . . . . . . . . . . . . . . . 62 4.1.2 Sensor Structure and Fabrication Process . . . . . . . . . . . . . . . . 63 4.1.3 Sensor Matrix Structure and Fabrication Process . . . . . . . . . . . 67 4.2 Modeling of the Resistor Network . . . . . . . . . . . . . . . . . . . . . . . . 70 4.2.1 The 2D Resistor Network . . . . . . . . . . . . . . . . . . . . . . . . 70 4.2.2 Mapping Contours based on Measured Resistance . . . . . . . . . . . 71 4.2.3 Formulation of the Forward Problem . . . . . . . . . . . . . . . . . . 72 4.3 Cell Resistance Reconstruction via Least-Squares Regularization . . . . . . . 79 4.3.1 Least-Squares Minimization (LSM) . . . . . . . . . . . . . . . . . . . 80 4.3.2 Least-Squares Regularization on Cell Resistance (LSR-CR) . . . . . . 81 4.3.3 Least-Squares Regularization on Relative Change in Cell Resistance (LSR-∆CR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4.3.4 Least-Squares Regularization on Gradient of Cell Resistance (LSR-∇CR) 84 4.3.5 Least-Squares Regularization on Gradient of Relative Change in Cell Resistance (LSR-∇∆CR) . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 4.4.1 Experimental Animals . . . . . . . . . . . . . . . . . . . . . . . . . . 88 4.4.2 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 4.4.3 Experiment on Sea Lampreys with the Pressure Sensor . . . . . . . . 91 4.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.5.1 Comparison between Different Methods . . . . . . . . . . . . . . . . . 92 4.5.2 Mapping Contour Comparison between Sea Lampreys with Large and Small Mouth Diameters . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Chapter 5 Automated Soft Pressure Sensor Array-based Sea Lamprey Detection With Machine Learning . . . . . . . . . . . . . . . . . 99 5.1 Experimental Animals and Setup . . . . . . . . . . . . . . . . . . . . . . . . 100 5.1.1 Experimental Animals . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.1.2 Soft Pressure Sensor Array . . . . . . . . . . . . . . . . . . . . . . . . 101 5.1.3 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.2 Training Models on Sea Lamprey Datasets . . . . . . . . . . . . . . . . . . . 103 5.2.1 Mapping Contour Patterns . . . . . . . . . . . . . . . . . . . . . . . . 104 5.2.2 Image Annotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.2.3 Assessment of Three Object Detectors . . . . . . . . . . . . . . . . . 108 5.3 Filtered YOLOv5s For Mitigation of The Sensor Memory Effect . . . . . . . 111 ix 5.3.1 Feature Learning Backbone and Neck of YOLOv5 . . . . . . . . . . . 113 5.3.2 Postprocessing with Confidence Thresholds . . . . . . . . . . . . . . . 113 5.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Chapter 6 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . 122 6.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 x LIST OF TABLES Table 2.1: Summary of vacuum pressure tests of live sea lampreys, including months, instruments used (sensor type), flow conditions (water flow), and biological variables (life stage, sex (M = male; F = female; N/A = unavailable), number of individuals (N), body weight, body length, and mouth diameter). Data are presented in the type of mean ± standard deviation. Sex could not be determined for juvenile sea lampreys. Mouth diameter was not measured during tests conducted in Aug. 2019. . . . . . . . . . . . . . . . 22 Table 2.2: Statistics of the pressure measurement results corresponding to Table 2.1, including manual pull condition (pull), maximum suction pressure before disengagement (max vacuum), minimum suction pressure to remain at- tachment (min vacuum), and the duration period between two adjacent pressurizing processes (leakage time). . . . . . . . . . . . . . . . . . . . . 31 Table 3.1: Sensitivity of the single-pixel soft capacitive pressure sensors with different air gap configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Table 4.1: Performance comparison of different reconstruction methods. . . . . . . . 94 Table 5.1: Parameters for generating the mapping contour plots. . . . . . . . . . . . 107 Table 5.2: Hyperparameters for training the sea lamprey detection networks. . . . . 109 Table 5.3: Comparison of validation results of different algorithms for sea lamprey detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Table 5.4: Predicted information during an interval of the soft piezoresistive pressure sensor array’s memory effect. . . . . . . . . . . . . . . . . . . . . . . . . . 120 xi LIST OF FIGURES Figure 2.1: The single-port pressure sensing system used to measure oral suction pres- sures of sea lamprey, Petromyzon marinus. (a) The single-port sensing panel with a port on the acrylic plate encapsulated with polydimethyl- siloxane (PDMS) and connected to the vacuum sensor beside the Arduino Uno microcontroller board via soft tubing, and (b) an adult sea lamprey attached to the sensing panel at the sensing port in a water-filled tank. . 18 Figure 2.2: The 9-port circular pressure sensing system used to measure oral suction pressures of sea lamprey, Petromyzon marinus. (a) The full view of the setup, (b) enlarged view of an adult sea lamprey attached to the sensing panel with its mouth covering all the 9 sensing ports, and (c) a schematic of the 9 ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Figure 2.3: Schematic (not to scale) of experimental setups for checking leakage of the single- and 9-port pressure sensing systems used to measure oral suc- tion pressures of sea lamprey, Petromyzon marinus. (a) Apply vacuum from the vacuum chamber to the vacuum sensor through tubing B, sens- ing port and tubing C, (b) apply vacuum from the vacuum chamber to the vacuum sensor through tubing B, suction cup attached on the plate, sensing port and tubing C, and (c) test setup (b) under water by putting part of the sensing plate and the suction cup under water in a water tank. Abbreviations: PDMS = polydimethylsiloxane; PCB = printed circuit board. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Figure 2.4: Pictures of the suctorial mouths of the adult and juvenile sea lampreys.(a) Suctorial mouths of an adult sea lamprey, and (b) a juvenile sea lamprey showing that the two are morphologically similar. . . . . . . . . . . . . . 23 Figure 2.5: Relations between (a) natural logarithm of body weight (ln(weight)) and natural logarithm of length (ln(length)), and (b) ln(weight) and natural logarithm of mouth diameter (ln(mouth diameter)) for tested adult male, female, and juvenile sea lampreys. . . . . . . . . . . . . . . . . . . . . . . 24 Figure 2.6: The experimental setup for recording the suction dynamics of sea lampreys in flowing water. (a) Side view, and (b) top view. Abbreviations: PCB = printed circuit board. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 xii Figure 2.7: Vacuum pressures recorded during the first 1000 s for potential leakage tests of the single-port pressure sensing setups. Low (–10 kPa) and high (–20 kPa) vacuum pressures were applied with the vacuum chamber tubing attached to the sensor port via one of three methods: direct-fit potting, tested in air (Setup in Figure 2.3-a); suction cup interface in air (Setup in Figure 2.3-b), suction cup interface in water (Setup in Figure 2.3-c). Data were down-sampled for display. . . . . . . . . . . . . . . . . . . . . 29 Figure 2.8: Vacuum pressures recorded during the first 1000 s for potential leakage tests of the 9-port pressure sensing setups. (a) Mean and standard de- viation (shown as the error bar) of the pressure at port P4 at each time instant for three rounds of measurements, and (b) mean and standard de- viation of the mean pressure at all these 9 ports at each time instant for three rounds of measurements. . . . . . . . . . . . . . . . . . . . . . . . . 30 Figure 2.9: Relations between (a) maximum free suction pressure and weight, and (b) leakage time and maximum free suction pressure for tested adult male, female, and juvenile sea lampreys. . . . . . . . . . . . . . . . . . . . . . . 32 Figure 2.10: Suction dynamics of an adult female sea lamprey in Experiment 1. (a) Suction pressure when the adult female lamprey attached to the panel during 400 s to 1200 s after attachment, (b) suction pressure profile dur- ing 1080 s to 1140 s after attachment with shaded rectangles showing the time when the lamprey’s tongue was protracted or retracted, (c) suction pressure profile when the adult lamprey was pulled away from the panel with corresponding snapshot of the lamprey’s mouth disengaged from the panel, and (d) pictures of an adult lamprey’s mouth showing the protrac- tion and retraction states of its tongue. . . . . . . . . . . . . . . . . . . . 33 Figure 2.11: Suction dynamics of two adult sea lampreys in Experiment 2. (a) Suction pressure when an adult female lamprey was attaching by itself on the panel, (b) suction pressure when the lamprey in (a) was pulled away from the panel, (c) suction pressure when another adult female lamprey was attaching by itself on the panel, and (d) suction pressure when the lamprey in (c) was pulled away from the panel. . . . . . . . . . . . . . . . . . . . 35 Figure 2.12: Suction dynamics of a juvenile sea lamprey in Experiment 2. (a) Suction pressure when a juvenile sea lamprey was attaching by itself on the panel, (f) suction pressure when the sea lamprey in (a) was pulled away from the panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Figure 2.13: Suction dynamics of two adult sea lampreys showing characteristic re- sponses in Experiment 3. (a) Suction pressures of an adult male lamprey in the flowing water test, and (b) suction pressures of an adult female lamprey in the flowing water test. . . . . . . . . . . . . . . . . . . . . . . 37 xiii Figure 3.1: (a) Schematic illustration of the soft capacitive pressure sensor array with air gap channels and conductive polymer PEDOT:PSS electrodes. (b) Optical micrograph (top) and SEM image (bottom) of the screen-printed PEDOT:PSS film with a feature thickness of approximately 200 µm. The PEDOT:PSS contains 10:1 wt% Bis(trifluoromethane)sulfonimide lithium salt as stretchability and electrical conductivity enhancer. Scale bars: 20 µm and 2 µm, respectively. (c) Photograph of a 12×12 capacitive pressure sensor array. Scale bar: 1 cm. (d) Schematic illustrating the fabrication procedures of the soft capacitive pressure sensor array. . . . . . . . . . . 46 Figure 3.2: Characterization of a single-pixel soft capacitive pressure sensor. (a) Schematic diagram representing the multi-layer structure of the single capacitive pressure sensor. (b) Relative change in capacitance in response to applied pressure for a single-pixel capacitive pressure sensor with vari- ous configurations of air gap in the Ecoflex dielectric layer. (c) Cyclic test of the sensor response for 1000 cycles at different pressures. (d) Schematic illustration of the sensor in the bent state under normal force. (e) Pres- sure response of the sensor with air gap (H = 0.5 mm, W = 2.0 mm) under a sequence of normal forces, 2 N, 3 N and 2 N when the sensor is bent to curvature radii of 50, 30, and 17 mm, respectively. (f) Cyclic test of the sensor at a bending radius of 17 mm for 1000 cycles. (g) Cyclic stretching test of the sensor with air gap (H = 0.5 mm, W = 2.0 mm) at 10 % strain with the air gap direction parallel to the stretch direction. (h) Cyclic stretching test of the sensor with air gap (H = 0.5 mm, W = 2.0 mm) at 10 % strain with the air gap direction perpendicular to the stretch direction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Figure 3.3: FEA Simulation of the sensor with and without an air gap in the Ecoflex 00-30 dielectric layer under different pressures. . . . . . . . . . . . . . . . 54 Figure 3.4: Spatial mapping of pressure distributions with the sensor array under both positive and negative pressures. (a) Photograph of the 12 × 12 pixel sensor array with different shaped objects placed on top for pressure mapping testing. Scale bar: 2 cm. (b) The corresponding distributions of the normalized capacitance change with +10 kPa pressure applied. (c) Photograph of the 12×12 pixel sensor array with air gaps (H = 0.5 mm, W = 1.2 mm) and the corresponding contour plots of relative change in capacitance under a negative pressure of –10 kPa (left) and –20 kPa (right). Scale bar: 2 cm. (d) Photograph of the 12 × 12 pixel sensor array without an air gap and the corresponding contour plot of relative change in capacitance under a negative pressure of –10 kPa (left) and –20 kPa (right). Scale bar: 2 cm. . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 xiv Figure 3.5: Performance of the soft capacitive pressure sensor for underwater sea lam- prey detection. (a) Photograph of the 16×16 pixel sensor array placed ver- tically on the wall of a water container for underwater experiment with a PDMS suction cup under –20 kPa. (b) Front view of the soft pressure sensor array with the PDMS suction cup attaching on the left bottom corner. (c) Corresponding mapping contour plots of relative change in ca- pacitance of the sensor array under a –20 kPa from the PDMS suction cup. (d) Photograph of the soft pressure sensor array clamped vertically in water on the wall of a water tank with a sea lamprey attaching on it. (e) Back view of the soft pressure sensor array under suction of a sea lamprey. (f) Corresponding mapping contour plots of relative change in capacitance of the sensor array under the suction of the sea lamprey in (d). (g) Photograph of the soft pressure sensor array clamped horizon- tally out of water over the water tank with a sea lamprey attaching on it vertically. (h) Side view of the experiment setup in (g). (i) Corresponding mapping contour plots for experiment in (g). . . . . . . . . . . . . . . . . 59 Figure 4.1: Working mechanism of the soft pressure sensor and characterization of a single-pixel sensor device. (a) The soft pressure sensor in the initial relaxed state, which shows an initial resistance of R0 between the top and bottom conductors, and (b) the pressure sensor under compressive pressure, showing a new resistance RP . (c) The soft pressure sensor with a piezoresistive film between two perpendicular copper tape electrodes under a compressive load, (d) the sensor under a suction cup to test negative pressure response, and (e) the average relative change in resistance (along with the standard deviation) of the single-pixel sensor versus pressure when tested on a flat substrate. . . . . . . . . . . . . . . . . . . . . . . . 64 Figure 4.2: Performance of the soft pressure sensor device under different bending conditions. (a) The initial (unloaded) resistance of the single-pixel pres- sure sensor at different curvature radii. (b) The experimental setup for loading pressure on the sensor on curved surfaces. (c) Time-resolved mea- surements of the output signal for an applied pressure with three rounds of loading and unloading processes on a curved surface with radius 50 mm. (d) Pressure response comparison of the single-pixel sensor at different curvature radii with an applied pressure up to 40 kPa. . . . . . . . . . . 65 Figure 4.3: Schematic of a 4-by-4 piezoresistive pressure sensing matrix. (a) The bonding status of all layers, and (b) the exploded view. . . . . . . . . . . 68 xv Figure 4.4: Fabrication process of the 10-by-10 soft pressure sensing matrix. The pa- per liners of the double-sided tapes were not peeled off in the top left and bottom left pictures, but were peeled off in the following steps. Pic- ture of the final fabricated 10-by-10 pressure sensing panel with PDMS (polydimethylsiloxane) waterproof encapsulation. The red dashed lines show the edges of the mold formed by 3M VHB 4905 tapes, while the white bounding box titled “PDMS” shows the PDMS encapsulation layer between the inner and outer 3M VHB 4905 tape boundaries. . . . . . . . 69 Figure 4.5: Schematic of the M -by-N resistor network and the voltage-divider circuits for resistance measurement. . . . . . . . . . . . . . . . . . . . . . . . . . 71 Figure 4.6: Mapping contours of the soft pressure sensing matrix based on relative change in directly measured resistance with the following experimental conditions: (a) a ϕ40 mm (ϕ represents diameter), 680 g aluminum rod was loaded on the sensing matrix, (b) a ϕ27 mm × ϕ35 mm × 5mm 3D printed ring part under a 850 g aluminum rod was loaded on the sensing matrix, (c) –10 kPa and (d) –20 kPa, respectively, negative pressure was applied on the sensing matrix via a ϕ27 mm × ϕ35 mm PDMS suction cup in air, and (e) –10 kPa and (f) –20 kPa, respectively, negative pressure was applied on the sensing matrix via the same suction cup under water, where the top row of copper tape electrode of the soft pressure sensor was about 7 cm lower than the water level. . . . . . . . . . . . . . . . . . . . 73 Figure 4.7: Forward mapping algorithm for computing the two-point resistance using the cell resistance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Figure 4.8: Reconstruction algorithm for computing the cell resistance using the mea- sured two-point resistance based on least-squares minimization (LSM). . 81 Figure 4.9: Reconstruction algorithm for computing the cell resistance using the mea- sured two-point resistance based on least-squares regularization on cell resistance (LSR-CR). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Figure 4.10: Reconstruction algorithm for computing the cell resistance using the mea- sured two-point resistance based on least-squares regularization on relative change in cell resistance (LSR-∆CR). . . . . . . . . . . . . . . . . . . . . 85 Figure 4.11: Reconstruction algorithm for computing the cell resistance using the mea- sured two-point resistance based on least-squares regularization on gradi- ent of cell resistance (LSR-∇CR). . . . . . . . . . . . . . . . . . . . . . . 86 Figure 4.12: Algorithm for calculating the gradient. . . . . . . . . . . . . . . . . . . . 87 xvi Figure 4.13: Reconstruction algorithm for computing the cell resistance using the mea- sured two-point resistance based on least-squares regularization on gradi- ent of relative change in cell resistance (LSR-∇∆CR). . . . . . . . . . . . 89 Figure 4.14: Experimental setup of the pressure sensing system for sea lamprey detec- tion.(a) The soft piezoresistive pressure sensing panel and the hardware attached on the back side of the panel, (b) the back view of the pres- sure sensing panel with an adult sea lamprey attaching on it, and (c) the front view of the pressure sensing panel with another adult sea lamprey attaching on it. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Figure 4.15: Comparison between mapping contours from different methods. Map- ping contours of (a) relative change in directly measured two-point re- sistance, (b) relative change in cell resistance from least-squares mini- mization (LSM), (c) relative change in cell resistance from least-squares regularization on the cell resistance (LSR-CR) with λ=0.001, (d) relative change in cell resistance from least-squares regularization on the relative change in cell resistance (LSR-∆CR) with λ=10, (e) relative change in cell resistance from least-squares regularization on the gradient of cell resis- tance (LSR-∇CR) with λ=0.001, and (f) relative change in cell resistance from least-squares regularization on the gradient of relative change in cell resistance (LSR-∇∆R) with λ=10. . . . . . . . . . . . . . . . . . . . . . 93 Figure 4.16: Comparison of the mapping contours between sea lampreys with large and small mouth diameters. (a) An adult sea lamprey attaching on the sensing panel with a mouth diameter of about 35 mm, and (b) the mapping contour corresponding to the attachment condition in (a) using the LSR- ∇CR method. (c) Another smaller adult sea lamprey attaching on the sensing panel with a mouth diameter of about 25 mm, and (d) the mapping contour corresponding to the attachment condition in (c) using the LSR- ∇CR method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Figure 5.1: Hardware of the soft pressure sensor array-based sea lamprey detection system [1]. (a) Experimental setup, and (b) circuit model of the resistor network formed by the pressure sensor array. . . . . . . . . . . . . . . . . 102 xvii Figure 5.2: Typical mapping contour plots of the 10-by-10 soft pressure sensor array with attachment of an adult sea lamprey. Typical compression patterns: (a) with partial edges or points shown in blue reflecting compression of the lamprey’s suction disc, (b) with an intact circular pattern shown in blue, (c) with the compression area on the boundary of the sensing area, and (d) with a corrupted circular pattern connected to adjacent rows or columns due to circuit crosstalk and mechanical stress of the sensor array. Typical suction patterns: (e) with a triangular blob shown in red reflecting the suction area, (f) with an intact round (or octagon) blob shown in red, (g) with the suction area on the boundary of the sensing area, and (h) with a corrupted polygon pattern extended to adjacent rows or columns due to circuit crosstalk and mechanical stress of the sensor array. . . . . 105 Figure 5.3: Illustration of the ground truth bounding box annotation. (a) image of the 10-by-10 soft pressure sensor array under suction of an adult sea lamprey (as shown in the inset), with the bounding box shown in yellow dashed lines covering the attachment area, and with the row and column axis labelled for dataset annotation. (b) The corresponding mapping contour plot of relative change in measured two-point resistance, with the ground truth bounding box in red solid lines depicted on it. . . . . . . . . . . . 106 Figure 5.4: The plots of mean average precision (mAP) curves of different approaches in the training and validation process. (a) SSD, (b) RetinaNet, and (c) YOLOv5s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Figure 5.5: Diagram of the soft pressure sensor and YOLOv5s model-based sea lam- prey detection approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Figure 5.6: Postprocessing results on the sea lamprey testing dataset with the confi- dence threshold as the variable. (a) The F1-score curves, and (b) the false positive rate curves, for both compression and suction patterns. . . . . . 117 Figure 5.7: Illustration of the faulty detection problem due to the sensor’s memory effect. (a) Image sequence of the pressure sensor array with ground truth bounding boxes. (b) Corresponding mapping contour plots with bounding box predictions from the YOLOv5s detection model without postprocess- ing (which means the confidence thresholds are both 0.05 for compression and suction patterns). (c) The filtered prediction results with a confidence threshold of 0.131 for the compression pattern, and 0.344 for the suction pattern, respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 xviii Chapter 1 Introduction and Motivation The sea lamprey (Petromyzon marinus) in North America is a species of anadromous fish native along the Atlantic coast. It presumably invaded the Laurentian Great Lakes in the early 1900s, and contributed to the collapse of major fish stocks in the Great Lakes. In order to control the sea lamprey invasion and restore the fish community, real-time detection of adult sea lampreys within or near a fish passage is of much significance and practical merits, which could afford the opportunity to alter fishway operation to block the sea lampreys in response to the detected information. Attachment to surfaces by oral suction is a prominent characteristic of sea lampreys, which offers an opportunity for utilizing soft pressure sensors to detect the sea lamprey while it is sucking onto the sensor. In this chapter, the challenges of sea lamprey detection, the mechanism of sea lamprey’s suction, the problems of soft pressure sensors, and the feasibility to use soft pressure sensor array for underwater sea lamprey detection as well as the automated detection system are discussed. The state of the art in these areas is surveyed, followed by a summary of contributions we have made for each research topic. 1 1.1 Sea Lamprey Detection via Attachment by Suction 1.1.1 State of the Art Species-specific monitoring activities, such as animal detection [2], behavior observation [3], population assessment [4] and habitat use evaluation [5], represent fundamental tools for natural resource management and conservation. In aquatic environments, monitoring meth- ods have long relied on sonar imaging to characterize distributions and densities of groups and on capturing methods to identify individuals [6, 7]. Emerging technologies promise to overcome many of the inherent limitations of those methods. For example, autonomous un- derwater vehicles (AUV) can be used to collect multimedia data and detect animals through image processing [4]. However, automated species identification remains a great challenge and may not be practical in some environments [8]. Environmental DNA (eDNA) [9, 10] has also become popular for species-level monitoring, but may be limited to species-level occu- pancy and be subject to false positives caused by transport of tissues by currents and other processes in natural environments [11]. Therefore, more effective monitoring techniques that take advantage of species-specific characteristics are desired. Sea lampreys (Petromyzon marinus) are anadromous fish native to the Atlantic Ocean, that invaded the Laurentian Great Lakes in the early 1900s, contributed to the collapse of major fish stocks in the Great Lakes [12, 13, 14], and triggered formation of a bi-national, basin-wide population control program. Success of the control program has been attributed to the use of barriers, traps, sterilization, and lampricides [15, 16]. In Europe and North America, attempts to conserve and restore native lamprey populations have included dam removal [17] and artificial propagation [18]. In either case, strategies target individuals in life stages and habitats where they are most vulnerable to perturbations based on knowledge 2 of species ecology and life history. Sea lampreys are basal vertebrates with a life history comprised of distinct larval, juvenile, and adult stages [19, 20, 21]. Larval sea lampreys burrow into stream sediment and feed on micro-organisms for 3–5 years. They then undergo a drastic metamorphosis into the juvenile stage with a powerful suctorial mouth, migrate downstream into the Atlantic Ocean or a Laurentian Great Lake, and parasitize fish for about 1.5 years [22], killing 18 kg of host fishes on average during that time [23]. Next, adult sea lampreys migrate upstream in the spring, where they will spawn and die. Although adult sea lampreys do not feed, they rely on oral suction during migration and spawning, for station-holding, nest construction, competition for mates, and mating [24]. Although many fishes use oral suction to feed, the ability to attach to surfaces using oral suction is unique to lampreys among freshwater fishes in the northern hemisphere (others include suckermouth catfishes of the genus Hypostomus, which are native to South America). When a lamprey approaches the desired surface, its annular muscle contracts and the buccal funnel spreads over the surface. The tooth-studded oral disc conforms to the surface and completes a seal, and then the armed tongue retracts into the oral passage and seals off the buccal funnel from the pharyngeal cavity. With the expansion of the buccal cavity, a partial vacuum inside this cavity and a corresponding suction force are created, which maintains a suction attachment. Next, water is pushed out of the pharyngeal cavity to the velar-sealed branchial cavity through the compression of an inner sinus, which increases vacuum pressure within the pharyngeal cavity. Once the tongue protracts, the vacuum pressure spreads into the buccal cavity and thereby forms a stronger suction [25]. Over time, suction pressure decreases due to leakage and pressure must be re-applied. Therefore, sea lamprey suction dynamics are characterized as intermittent cycles of rapid “pumps” (application of suction) separated by periods of leakage (loss of suction). 3 Although oral suction is a prominent characteristic of lampreys, it has not been exploited for lamprey management or conservation. Knowledge of specific characteristics of sea lam- prey suction (e.g., amplitude, duration, and pattern of suction events; hereafter ‘suction dy- namics’) may be useful to develop devices that detect, record, and respond to the presence of sea lamprey at a given place and time or as feeding devices for aquaculture. Apart from the suction mechanism, few researchers have measured the suction pressures created by sea lam- preys. Gradwell [26] inserted cannulas into gill pouches and naris of four pre-spawning adult lampreys and measured the hydrostatic pressures of lampreys using a pressure transducer connected to each cannula. To obtain the pressure exerted by the spawning-run lampreys on an acrylic surface, Adams [25] connected an absolute pressure sensor to a port in the designated attachment area on the acrylic flange. However, the suction dynamics of adult lampreys and juvenile lampreys in flowing water have yet to be described. Additionally, it is not known if measurement of oral suction is sensitive to measurement location within the mouth (i.e., uniformity of vacuum pressure across the mouth). A pressure sensing panel which can measure the biological suction pressure, if deployed in tributaries or on dams, has the potential for sea lamprey detection, population assessment, and facilitation of lamprey passage or blockage. For instance, it could be deployed in a fishway to detect the sea lamprey according to the suction pressure change when it attaches, and to trigger action to block passage or repel the lamprey. It could also be deployed in streams to determine timing of stream entry and upstream migration, and to describe refuge habitat. Measurements of biological suction pressure dynamics, such as pressure amplitude, frequency, and suction duration, might also indicate condition, life stage, body size, or sex of sea lampreys, if those dynamics are related to those biological characteristics. Moreover, the understanding of sea lampreys’ suction dynamics can improve design of next generation 4 of soft pressure sensors, which are desired to be more bio-compatible and convenient for deployment. 1.1.2 Contribution Previous observations are limited to adult sea lampreys in static water. In this study, pressure sensing panels based on commercial vacuum sensors are constructed and used to measure oral suction pressures and describe suction dynamics of juvenile and adult sea lampreys at multiple locations within the mouth and in static and flowing water. Suction dynamics are largely consistent with previous descriptions, but more variation is observed. For adult sea lampreys, suction pressures range from –0.6 kPa to –26 kPa with 20 s to 200 s between pumps at rest, and increase to –8 kPa to –70 kPa when lampreys are manually disengaged. Measurements from an array of sensors indicate that suction pressure distribution is largely uniform across the mouths of both juvenile and adult lampreys; but some apparent variation is attributed to obstruction of sensing portal holes by teeth. Suction pressure dose not differ between static and flowing water when water velocity is lower than 0.45 m/s. Such information may inform the design of new systems to monitor behavior, distribution and abundance of lampreys. 1.2 Soft Pressure Sensor 1.2.1 State of the Art Soft pressure sensors are promising for various applications including wearable electronic skins [27, 28, 29, 30], soft robotics[31], environmental monitoring [32, 25, 33] and aerody- 5 namic control of vehicles [33]. Many of these applications require positive pressure sensing [29, 30, 31, 32] while others demand the capability of measuring negative pressure. Examples include the detection of suction events from lamprey’s mouth [25] or octopus’ suction cups [33] and measuring surface pressure distribution of cars during on-road driving [34]. Soft pressure sensors can be formed based on various transducing principles, such as piezoelectric sensing [35, 36, 37, 38, 39], capacitive sensing [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], piezore- sistive sensing [29, 50, 51, 52, 53, 54, 55, 56, 57], and transistor mechanism [46, 58, 59, 60]. Piezoelectric pressure sensing foil [35] shows hysteresis in response because of the character- istics of piezoelectric materials and the crosstalk in the sensor matrix is often pronounced. Moreover, to our best knowledge, no work has been reported so far on using piezoelectric sensors for negative pressure detection. Flexible pressure-sensitive organic thin film tran- sistors [46, 58] rely on the gate dielectric layer to convert the pressure input to change in the drain current; despite the high sensitivity, the multiple layers in the sensor films could delaminate under negative pressure and thus cause sensor failure. Capacitive pressure sensors can be made using simple parallel-plate capacitor [61] or crossbar capacitor structures [62], where the capacitance is proportional to the area and inversely proportional to the spacing between the two parallel electrodes. Ideally, such structures should be able to detect both positive and negative pressure because the spacing between the electrodes will decrease when the sensor is compressed (positive pressure) and increase when under partial vacuum (negative pressure). Nevertheless, soft capacitive pres- sure sensors are actually insensitive to negative pressure because of the viscos-elasticity of the materials in the substrate, resulting in very small thickness change of the dielectric layer under negative pressure. In order to improve the sensitivity of soft pressure sensors, a number of strategies have 6 been proposed including fabricating microstructures in the conducting materials or dielectric layers. Bao’s research group first introduced micro pyramid features into the dielectric layers of pressure sensors, gaining good sensitivity and short response time [30, 46, 58]. Hasan et al. [51] designed micro pillars in their pressure sensors to enhance the sensitivity under gentle touch, and the micropillar-based pressure sensors were also reported in Park [55] and Engel’s work [40]. In addition, guided mechanical cracks [56, 63], hierarchical structure [64], interlocked microdome structures [65, 66], and porous insulation layer [53, 54, 67, 49] have all been demonstrated to improve the performance of pressure sensors. All these microstructures would increase the number of conducting paths and conducting area in the materials, or augment the dielectric performance when the sensor films are compressed, which drastically enhances the conductivity and sensitivity of the pressure sensors. Nevertheless, most of these sensor designs would fail to work under negative pressure because the current flow would be cut off when the multilayers delaminate under negative pressures. Under negative pressure, the sensor needs to not only be responsive to negative pressure, but also form a good bonding between the layers in order to endure negative pressure and avoid delamination. A diaphragm structure that is well bonded with its substrate can easily deform under both negative and positive pressures, and the corresponding deformation can be converted to change in capacitance when the structure is constructed as a capacitor. Taking the above into consideration, a capacitive pressure sensor with a diaphragm structure [41, 44, 47] that deflects by external pressure has become a promising solution for measurement of both positive and negative pressures. The thin diaphragm instead of the entire sample deflects under external pressure, which enlarges the spacing change between the two electrodes and could be measured from the capacitance change. Lee et al. [44] fabricated a modular expandable tactile sensor in the diaphragm structure using PDMS substrate and copper strip 7 electrodes, and Nie et al. [47] reported a similar transparent iontronic film for capacitive pressure sensing using polyethylene terephthalate (PET) films. However, these sensors are not stretchable due to the copper electrodes and PET films used, and their responses to negative pressure have not been reported yet. To achieve soft pressure sensors that are stretchable, conductive nanomaterials such as graphene [56, 57], carbon nanotubes [50, 45], metal nanowires [29, 48, 68], conducting oxides [51, 41, 67, 46, 47, 49, 58], or conducting polymers [54, 69, 70] could be used. On the other hand, piezoresistive pressure sensors generally exhibit good sensitivity under compressive loads, but the layers of piezoresistive sensors tend to delaminate under negative pressure, which cuts off the current flow in the piezoresistive materials. Among the resistive pressure sensors, the low-cost pressure sensitive film called Velostat [71, 72, 73, 74, 75, 76, 77], comprised of carbon-impregnated polyolefin and exhibiting piezoresistive property, has been widely investigated in the applications such as finger gesture recognition [71], human grasp monitoring [72], foot pressure measurement [73], sitting posture monitoring [74], and prosthetic in-socket pressure sensing [75]. Another idea is to sandwich a matrix of individual Velostat film patches between two layers of perpendicular electrodes [77], where a resistor network forms in the circuit, which inevitably introduces the crosstalk issue between adjacent resistors – that is, the measured two-point resistance is influenced by all the other resistors in the network. The relation between the cell resistance at any pixel and the equivalent resistance between the two electrodes (i.e., corresponding column and row) is analytically derived in Tan’s work [78, 79], expressed as an explicit nonlinear forward function from the Laplacian matrix of the cell conductance to the equivalent two-point resistance matrix. Nevertheless, the inverse problem is intractable and ill-posed, and no analytical solution is available. Moreover, the crosstalk issue results in a resistor network dimension- 8 related problem, which is that the equivalent two-point resistance measured from the resistor networks of different row and column dimensions will have different amplitudes of changes (i.e., relative change in resistance) under the same pressure at the corresponding pixels. This characteristic of the resistor network will be problematic for pressure characterization of sensing panels of different dimensions. Therefore, a general method for reconstructing the cell resistance from the measured two-point resistance is needed for practical versatile applications. Some researchers studied the circuit hardware and proposed to place diodes as current barriers between adjacent pixels to avoid cross-talk [77], but this would make the fabrica- tion more complex and leave the sensing panel unsmooth for attachment. Other researchers analyzed the crosstalk error by circuit simulation [80, 81], and put forward a few numerical algorithms such as fixed-point formulation [82]. This algorithm was designed to calculate the cell conductance from measured conductance, but the algorithm unexpectedly generated negative conductance, which was directly replaced with zero in their work [82]. However, those results were not reasonable or adequate for obtaining all the estimated cell resistance values. A least-squares reconstruction method was also mentioned in [82]; however, the ill-posed nature of the inversion [83, 84] was not accounted for in their method since the minimization criterion was not regularized, a consequence of which is that a small amount of noise on the data could be sufficient to make the solution diverge. In the field of electrical impedance tomography (EIT) [84, 85], Tikhonov regularization technique is commonly uti- lized to cope with similar ill-posed inverse problems, but it has not been used for Velostat pressure sensing systems yet. 9 1.2.2 Contribution In this work, we have developed two types of soft pressure sensors. First, a soft capacitive sensor that is stretchable and responsive to both positive and negative pressures has been designed and prototyped. The sensor is comprised of a soft Ecoflex 00-30 dielectric layer sand- wiched in between and tightly bonded with screen-printed poly(3,4-ethylenedioxythiophene): poly (styrene-sulfonate) (PEDOT:PSS) electrodes and polydimethylsiloxane(PDMS) encap- sulation layers. More importantly, air gap channels are incorporated and uniformly dis- tributed in the Ecoflex 00-30 layer, which shape the diaphragm structures that greatly en- hance the dielectric layer deformation under an external pressure, resulting in significantly improved response, especially to negative pressure. Meanwhile, the PEDOT:PSS electrodes are encapuslated on the Ecoflex-030 substrate with PDMS layers, which successfully protect the electrodes and prevent the layers from delamination under negative pressure. The re- sponse of the single pixel sensor is systematically characterized and finite element method (FEM) simulation is used to study the influence of the air gap geometry on the sensor re- sponse. The sensor with optimized design exhibits good sensitivity from –60 kPa to –20 kPa and great repeatability under compressive loads, vacuum suction, and even bending or stretching conditions. Lastly, a 12×12-pixel sensor array that can measure the pressure distrubution for both positive and negative pressures with high fidelity is demonstrated. Second, we propose a pressure sensing system consisting of a matrix of individual Velostat cells which are sandwiched between two orthogonal layers of copper tape electrodes and are fixed separately by one layer of polyester tape and double-sided tapes with waterproof encapsulation around the sensing panel. This design is effective preventing the delamination between the sensor layers in the underwater attachment scenario. On the basis of the forward 10 problem formulation, we build a numerical algorithm to estimate the cell resistance for the resistor network using least-squares minimization criterion as the baseline. Moreover, we explore four novel compound minimization criteria to obtain satisfactory solutions, where an a priori term which penalizes the cell resistance, the relative change in cell resistance, the gradient of cell resistance, and the gradient of relative change in cell resistance, respectively, is added to the least-squares term to form the respective cost function. The performance of these new regularization methods is compared and discussed, which offers an insight into how sea lampreys can be detected according to the mapping contours corresponding to the suctorial mouth attachment. This soft piezoresistive pressure sensing system is also promising for other extensive applications in electronic skins and soft robotics. 1.3 Soft Pressure Sensor-based Sea Lamprey Detection 1.3.1 The Need for Automated Sea Lamprey Detection In order to control the sea lamprey invasion and restore the fish community, the Great Lakes Fishery Commission (GLFC) has recently developed a selective fish passage program [86, 87]. Such a fish passage is designed to allow native and desirable fishes to pass dams while preventing passage of sea lampreys. One of the key problems of implementing a selective fish passage is how to detect sea lampreys automatically when they approach. In addition to the operation of a selective fish passage, detection and monitoring of sea lampreys can facilitate the design of new trapping systems, support assessment of sea lamprey populations, and help understand their life history and ecology (e.g., refuge-seeking behavior and habitat characteristics). 11 1.3.2 State of the Art Object detection is a technology for identifying instances of objects of a certain class in im- ages, videos and other types of datasets. It has numerous applications, such as number plate recognition [88], product identification [89, 90], face detection and recognition [91], an- imal monitoring [92], and autonomous driving [93]. Video analysis based on object detection methods has been used to detect underwater adult Pacific lampreys (Entosphenus tridenta- tus) [94]; however, image quality may not be suitable under a wide range of light conditions. Instead, underwater sea lampreys could be detected by taking advantage of their oral suc- tion trait [95] in a more economical and efficient way using some proper pressure sensors [1, 39] or contact sensors [96]. This is feasible since the suction of a sea lamprey’s oral disc on the sensor would introduce distinct pressure patterns or related changes in the sensor’s characteristics. Nevertheless, the sensor system alone significantly relies on human interven- tion to recognize the pressure patterns or contact events from the sensor’s output signals, and to further decide whether, when, and where the sea lamprey is attaching on. In order to reduce burden from human decision, the sea lamprey detection system needs to be automated. In Chapter 5, we report the first automated sensing system for detecting sea lamprey attachment based on a soft pressure sensor array. Specifically, machine learning- based object detection algorithms are used to learn features from the measured data of a soft pressure sensor array and perform automatic detection of sea lamprey attachment on the generated mapping contours. Basically, the measured data from the soft pressure sensor array can be either used directly to learn features of the sensor’s response to sea lamprey’s attachment, or they can be first converted to mapping contour images, which are then used to learn features using 12 computer vision technologies. The data-based sea lamprey detection method can be realized by some basic neural network approaches such as the multilayer perceptron (MLP), which is a fully connected class of feedforward artificial neural network (ANN) that takes data as input, where each neuron in one layer is connected to all neurons in the next layer. On the other hand, the image-based sea lamprey detection method could be realized by some convolutional neural networks (CNNs) which analyze visual imagery using convolutional layers and sparsely connected or partially connected layers. Both the data-based method and the image-based method are promising for the soft pressure sensor array-based sea lamprey detection, but in Chapter 5, we only report the image-based method for sea lamprey detection as it is straightforward and convenient in visualization. The state-of-the-art object detection algorithms can be categorized into two main types: one-stage detectors and two-stage ones. Representative two-stage object detectors are region- based convolutional neural networks (R-CNN [97], fast R-CNN [98], faster R-CNN [99], and mask R-CNN [100]), which firstly use selective search algorithms to extract candidate region proposals from the image, and then classify each single object and estimate its size with a bounding box. They achieve higher detection accuracy but are typically slower than the one- stage detectors, which predict bounding boxes over the images without the region proposal step. Examples of the most popular one-stage detectors include YOLO (You Only Look Once) [101], SSD (Single-Shot Detector) [102], and RetinaNet [103]. YOLO reframes object detec- tion as a single regression problem from the image pixels to the bounding box coordinates and associated class probabilities. More advanced versions of YOLO have been released in the past few years, such as YOLOv3 [104], YOLOv4 [105], and YOLOv5 [106]. Different from all prior releases, YOLOv5 is implemented in PyTorch, which is well supported on ma- 13 jor platforms and is versatile for research prototyping. SSD uses a single deep neural network to output multi-scale convolutional bounding box, and predicts category scores and box off- sets for a set of default bounding boxes. RetinaNet applies a modulating factor to the cross entropy loss in order to address the extreme foreground-background class imbalance during training. However, RetinaNet still designs two separate sub-networks in the end: one for classifying anchor boxes, and the other for the regression of object boxes; which could be merged into one regression network in theory. 1.3.3 Contribution In this work, a comprehensive sea lamprey mapping contour dataset is first generated for the training model to learn features. These mappings typically show two different types of patterns under lamprey attachment: a high-pressure circular pattern corresponding to the mouth rim compressed against the sensor (“compression” pattern), and a low-pressure blob corresponding to the partial vacuum region of the sucking mouth (“suction” pattern). Three types of object detection algorithms are deployed for sea lamprey detection, including SSD, RetinaNet, and YOLOv5s (which is a small scale model of YOLOv5 that has fewer layers of convolutional neural networks for faster and simpler object detection tasks). Their validation performance and inference speeds are evaluated and compared in depth, and the results show that YOLOv5s achieves the highest mean average prevision (mAP @0.5 : 0.95 up to 69.77%), and the fastest inference speed (up to 8.4 ms per image) on the experimental GPU device. Finally, a detection approach based on the YOLOv5s model with a confidence filter unit, is proposed. In particular, different optimal detection thresholds are proposed for the compression and suction patterns, respectively, in order to reduce the false positive rate caused by the sensor’s memory effect. The efficacy of the proposed method is supported 14 with experimental results on real-time underwater detection of sea lampreys. 15 Chapter 2 Measurement of Suction Pressure Dynamics of Sea Lampreys, Petromyzon marinus In this chapter, an effective pressure sensing panel, comprised of arrays of commercial vacuum pressure sensors, is developed to characterize suction dynamics of sea lampreys underwater. Objectives of this study are: (1) to record and describe the range and distribution of suc- tion pressures exerted by individual lampreys at two life stages (juvenile and adult); (2) to describe the frequency of suction events (re-pressurizing “pumps”); (3) to determine if vac- uum pressure varies spatially across the sea lamprey mouth; and (4) to determine if suction dynamics (i.e., pressure ranges; event frequencies) differ when the sea lamprey is in static vs. flowing water. This chapter was previously published as [95], and has been reformatted to meet the requirements of this dissertation. This chapter is organized as follows. The pressure sensing apparatuses are first presented in Section 2.1 followed by the leakage checking method. Experimental animals and three experiments with the data analysis method are proposed in Section 2.2. Experimental results of both the sensing systems’ leakage performance and the sea lamprey’s suction dynamics are presented in Section 2.3. Finally, discussions are provided in Section 2.4. 16 2.1 Methods 2.1.1 The Pressure Sensing Apparatuses Commercial vacuum pressure sensors (Honeywell 40PC015V2A) were used to construct the sensing system for monitoring vacuum pressures exerted by sea lampreys on test surfaces. Each pressure sensor had an operating pressure range of 0 kPa to –103.4 kPa and response time of 1 ms maximum. Two types of pressure-sensing apparatuses were developed for comprehensive experiments, including a panel with a single sensing port (Figure 2.1), and a panel with a 9-port circular pressure sensing matrix Figure 2.2), respectively. The single port system was comprised of an elbow-shape tight-seal moisture-resistant barbed tube connector, which was the smallest we could find to build this pressure sensing system, glued into a 2.5- mm port in a smooth acrylic plate. A soft plastic tube connected the tube fitting to a vacuum pressure sensor 20 cm away. For encapsulation of the port, a 5-mm-thick layer of polydimethylsiloxane (PDMS, Sylgard-184, Dow Corning) was cast on the acrylic plate with a mixing ratio of 10:1 (PDMS base: curing agent, wt.%), and then cured at 70 ℃ for 2 hours. The acrylic plate was then placed vertically in water against the wall of a 200 L aquarium so that the side of the panel with barb fitting, tubing and PMDS was against the glass. During each test, a sea lamprey was gently held with its mouth centered over the port on the acrylic plate until it attached to the plate via oral suction. Pressure data from the sensor, measured in kPa, were acquired by an Arduino processor board (Arduino Uno for single port sensor; Arduino Mega 2560 for 9-port sensing panel) at a sampling frequency of 200 Hz and stored in a computer. The 9-port system was fabricated in a similar way, but contained a central port surrounded by eight ports arranged in a circular pattern with a radius of 8 mm (Figure 2.2-b) such that all ports would be covered by the sea lamprey’s oral disc. 17 Figure 2.1: The single-port pressure sensing system used to measure oral suction pressures of sea lamprey, Petromyzon marinus. (a) The single-port sensing panel with a port on the acrylic plate encapsulated with polydimethylsiloxane (PDMS) and connected to the vacuum sensor beside the Arduino Uno microcontroller board via soft tubing, and (b) an adult sea lamprey attached to the sensing panel at the sensing port in a water-filled tank. 2.1.2 Leakage Check of the Pressure Sensing System A three-part experiment was used to determine if sea lamprey suction measurements were influenced by air leakage (Objective 1; Figure 2.3-a,b,c). Specifically, for the single-port pressure sensing system, we used a vacuum pump and vacuum chamber to determine if leakage was related to the type of connection used at the port (direct-fit potted vs. simulated lamprey mouth suction cup), magnitude of suction applied (–10 vs. –20 kPa), or surrounding medium (air vs. water). In the direct-fit test, small tubing from the vacuum chamber was connected to the sensing port on the acrylic plate and PDMS was used to encapsulate the sensing port (Figure 2.3-a). The PDMS encapsulation was assumed robust enough that any observed leakage would be attributed to other parts of the system that were also used in 18 Figure 2.2: The 9-port circular pressure sensing system used to measure oral suction pressures of sea lamprey, Petromyzon marinus. (a) The full view of the setup, (b) enlarged view of an adult sea lamprey attached to the sensing panel with its mouth covering all the 9 sensing ports, and (c) a schematic of the 9 ports. measurements of live lampreys (i.e., Tubing B in Figure 2.3-a, the vacuum sensor, or the connection between them). A second setup used to simulate oral suction from a lamprey, was comprised of a suction cup (rim diameters: inner, 32 mm, outer, 40 mm; depth: 20 mm) made of PDMS via a molding and casting process, and placed on the sensing port with its inlet connected to the vacuum chamber (Figure 2.3-b). Leakage was evaluated as described for the first setup. Finally, to simulate the lamprey’s suction under water, the suction cup setup was immersed in water in a 45 cm × 30 cm × 30 cm water tank (Figure 2.3-c). During each test, the vacuum pump was turned on to create vacuum pressure in the vacuum chamber. When the pressure reached the set-point, a valve on the vacuum chamber was closed to block the inlet between the vacuum pump and the vacuum chamber, but equalize the air pressure between the vacuum chamber and the vacuum sensor. Under this state, the pressure sensor output was recorded for 1000 s and possible leakage was evaluated. For 19 each setup, three times of leakage checking tests, each one lasting for 1000 s, were conducted in order to show the reliability of the single-port pressure sensing system. Similarly, the 9-port pressure sensing system was also tested to check possible leakage under these three setups, with three experiments conducted for 1000 s for each setup. Figure 2.3-d shows the counterpart of the direct-fit potting setup for the 9-port sensing system, where a 1.5 mm thick layer of 3M VHB4905 double-sided tape was used to bond the bottom surface of suction cup and the acrylic plate around the nine ports, which had a good seal and any observed leakage would be attributed to other parts of the system. Figure 2.3: Schematic (not to scale) of experimental setups for checking leakage of the single- and 9-port pressure sensing systems used to measure oral suction pressures of sea lamprey, Petromyzon marinus. (a) Apply vacuum from the vacuum chamber to the vacuum sensor through tubing B, sensing port and tubing C, (b) apply vacuum from the vacuum chamber to the vacuum sensor through tubing B, suction cup attached on the plate, sensing port and tubing C, and (c) test setup (b) under water by putting part of the sensing plate and the suction cup under water in a water tank. Abbreviations: PDMS = polydimethylsiloxane; PCB = printed circuit board. 20 2.2 Measurement of Sea Lamprey Suction Dynamics 2.2.1 Experimental animals Three experiments were conducted in 2019 and 2020 using spawning phase adult and para- sitic juvenile sea lampreys (see Table 2.1). During Experiment 1, suction pressures of adult sea lampreys were measured using the single port pressure sensing system in a 200 L aquar- ium with static water. During Experiment 2, suction pressures of adult and juvenile sea lampreys were measured using single and 9-port pressure sensing systems in a 200 L aquar- ium with static water. During Experiment 3, suction pressures of adult sea lampreys were measured using a 9-port pressure sensing system in a 1000 L aquarium with static and flow- ing water. Adult sea lampreys used in this study were captured in traps during upstream spawning migration in the St. Marys River (Michigan, USA and Ontario, Canada) during July 2019 and 2020. Traps were operated by Canada Department of Fisheries and Oceans and the U. S. Fish and Wildlife Service. Juvenile sea lampreys were collected by commer- cial fishers in northern Lake Huron during September 2019. All lampreys were transported to the U. S. Geological Survey Great Lakes Science Center’s Hammond Bay Biological Sta- tion, Millersburg, Michigan, USA where they were held in aerated 1000 L tanks supplied continuously with Lake Huron water maintained at 8 ℃ until tests were conducted. Prior to tests, body weight, total body length, and mouth diameter were measured for most, but not all sea lampreys (Table 2.1). All sea lamprey experiments were performed in accordance with protocols and guidelines approved by Michigan State University’s Institutional Animal Care and Use Committee (IACUC, No. 02/18-028-00). After the suction pressure experi- ments in this study, the sea lampreys were housed for use in further research by Hammond Bay Biological Station. 21 Table 2.1: Summary of vacuum pressure tests of live sea lampreys, including months, instru- ments used (sensor type), flow conditions (water flow), and biological variables (life stage, sex (M = male; F = female; N/A = unavailable), number of individuals (N), body weight, body length, and mouth diameter). Data are presented in the type of mean ± standard de- viation. Sex could not be determined for juvenile sea lampreys. Mouth diameter was not measured during tests conducted in Aug. 2019. Body Body Mouth Sensor Water Life No. Month Sex N Weight Length Diameter Type Flow Stage (g) (mm) (mm) Aug. M 2 246.5± 25.5 490.0± 20.0 N/A 1 Single Static Adult 2019 F 2 233.0± 50.0 480.0± 30.0 32.0± 1.0 M 2 150.0± 31.0 416.0± 35.0 31.5± 1.5 Single Static Adult Oct. F 7 224.2± 24.8 469.8± 17.4 32.1± 0.9 2 2019 Adult F 5 198.0± 30.2 459.6± 20.8 30.6± 2.0 9-port Static Juvenile N/A 6 109.0± 7.4 422.3± 11.8 30.5± 1.1 Aug. M 3 187.0± 34.2 433.0± 31.8 31.3± 1.9 9-port Flow Adult 2020 F 4 212.3± 26.0 447.5± 18.4 27.8± 1.4 3 Sept. M 4 147.5± 17.6 395.0± 16.6 28.2± 1.5 9-port Flow Adult 2020 F 5 163.4± 16.2 420.0± 25.7 25.1± 1.6 Juvenile lampreys were shorter and lighter than the adult lampreys on average, less than half of the total weight of adults, but the lampreys of both these two stages had suctorial mouths in similar size and anatomy (e.g., number and location of teeth; Figure 2.4). The natural logarithm (ln) of body weight was positively correlated with the natural logarithms of length of adult male, female and juvenile lampreys (Figure 2.5-a). The length-weight relation was similar between male (linear regression: slope = 2.449; R2 = 0.90; p-value = 0.000) and female (slope = 2.492; R2 = 0.83; p-value = 0.000) adult lampreys, but juveniles (slope = 2.205; R2 = 0.76; p-value = 0.023) weighed less than similar-body-length adults, possibly because their reproductive organs were not yet developed. Smaller slope for juveniles than adults may be a consequence of physiological differences between life stages — juveniles are growing while non-feeding adults are senescing (and shrinking). Male (Figure 2.5-b; linear regression: slope = 1.219; R2 = 0.22; p-value = 0.202) and female (slope = 1.097; R2 = 22 0.29; p-value = 0.012) adult lampreys weighed more than juveniles with similar-sized mouths (slope = –0.085; R2 = 0.00; p-value = 0.936), but differences were not large enough to allow sex or life stage determination based on mouth diameter. However, we also reiterate that the juveniles used in this study were in the parasitic form, so it is unlikely that these two life stages would co-occur (e.g., in a stream at the same time). Figure 2.4: Pictures of the suctorial mouths of the adult and juvenile sea lampreys.(a) Suctorial mouths of an adult sea lamprey, and (b) a juvenile sea lamprey showing that the two are morphologically similar. 2.2.2 Experiment 1: Recording of Adult Sea Lamprey Suction Dynamics with a Single Pressure Sensor in Static Water First, the single-port panel was used to measure suction pressures of two male and two female adult sea lampreys in August 2019, and another two male and seven female adult sea lampreys in October 2019 in a 200 L rectangular aquarium tank supplied with Lake Huron water but with no noticeable flow (i.e., static flow condition; Table 2.1). Each sea 23 Figure 2.5: Relations between (a) natural logarithm of body weight (ln(weight)) and natural logarithm of length (ln(length)), and (b) ln(weight) and natural logarithm of mouth diameter (ln(mouth diameter)) for tested adult male, female, and juvenile sea lampreys. lamprey was measured individually. In the experimental water tank, the sensing panel was placed vertically along a glass wall of the tank so that the side with the elbow fitting and tubing was against the wall and the smooth side with sensing port was available to the sea lamprey (Figure 2.1-b). The water level in the tank was about 6 cm higher than the sensing port, submerging all the sensing area around the sensing port. An adult sea lamprey was transferred to the tank and allowed to explore the tank until it attached to the tank surface via oral suction. If the lamprey did not attach onto the sensing area, it would be gently re-positioned and held with its mouth over the sensing port until it attached. Pressure measurements were recorded until the lamprey voluntarily detached from the panel or first 20 minutes of attachment elapsed. If a lamprey voluntarily detached or was manually disengaged from the panel after 20 minutes of attachment, it was allowed to rest for 10 min before it was manually reattached to the panel for the pull test. After reattaching to the panel for 5 minutes, the lamprey was gradually detached from the panel by gently pulling the lamprey away from the panel until disengagement. Suction dynamics were summarized from 24 recorded data. If the lamprey detached voluntarily before 5 minutes of static attachment achieved, the lamprey was removed from the tank and manual pull test was not conducted. 2.2.3 Experiment 2: Recording of Juvenile and Adult Sea Lam- prey Suction Dynamics with a 9-port Pressure Sensing Sys- tem in Static Water A second set of tests were conducted in October 2019 using the 9-port pressure sensing system to determine if suction pressures varied spatially across a sea lamprey mouth (Objective 3). Data were collected from 5 adult female lampreys and 6 juvenile lampreys using the 9-port panel in rectangular aquarium tanks as described above (Table 2.1). No male sea lampreys were used for those tests because they were not available. Adult female lampreys were smaller during October 2019 tests than those measured during August 2019 tests (Table 2.1). Loss of weight between August and October was expected because adult sea lampreys permanently cease feeding prior to commencement of spawning migration and thus lose energy and mass until death occurs. Body sizes of juvenile lampreys were also smaller than adults in October 2019, but suctorial mouths of adults and juvenile were similar in anatomy (e.g., number and location of teeth) and size (Figure 2.4). 2.2.4 Experiment 3: Recording of Adult Sea Lamprey Suction Dynamics with a 9-port Pressure Sensing System in Flowing Water A third set of tests were conducted in August and September 2020 to determine if suction dynamics differed when the sea lamprey was in static vs. flowing water (Objective 4). In 25 natural aquatic environments like rivers and streams, sea lampreys attach onto rocks or other surfaces for resting; it is of interest to characterize the suction pressure in the presence of water currents. Data were collected from 16 adult lampreys (Table 2.1) using the 9-port panel in a 1000 L circular water tank (1.8 m in diameter) fed continuously with fresh lake water from a spray bar near the surface to create consistent annular water flow around a central stand-pipe drain. The 9-port sensing panel was affixed to the wall of the tank near the surface (Figure 2.6) and water velocity was measured at the sensor panel using a portable flow meter (Marsh McBirney Flo-mate 2000). At the start of each test, each sea lamprey was manually placed on the sensing panel with its mouth over the 9 sensing ports. Each lamprey was tested at four water velocities during each test. The test began with static water conditions for 30 s after which time the water velocity was increased to 0.15 m/s, 0.30 m/s, and 0.45 m/s for 20 s at each velocity. Water flow was then returned to static conditions and the sea lamprey was gently removed. Water velocities tested were within typical ranges in sea lamprey spawning habitats [107, 108]. 2.2.5 Data Analysis Objectives 1 and 2 were addressed using suction pressure measurements from all experiments. For each individual sea lamprey, maximum observed suction pressure, minimum observed suction pressure, and median leakage time were calculated during each test. Non-parametric Mann-Whitney U Tests were used to test the null hypothesis that maximum free suction pressure (in static or flowing water without being manually pulled) and median leakage time did not differ between life stages (juvenile vs. adult; sexes combined for adults) or between sexes (male vs. female; adult life stage only). Test statistics and p-values were calculated according to [109]. Simple linear regressions were used to determine if maximum 26 Figure 2.6: The experimental setup for recording the suction dynamics of sea lampreys in flowing water. (a) Side view, and (b) top view. Abbreviations: PCB = printed circuit board. suction pressure were related to body weight within each group (i.e., adult male, adult female, juvenile). Linear regressions were also used to determine if leakage time was related to maximum suction pressure within each group. Coefficient of determination (R2 ) was used as an indicator of strength of association between independent (e.g., body weight) and response variables (e.g., max. suction pressure). Two-sided T tests were used to test the null hypothesis that the slope between independent and dependent variables was not equal to zero. Regression analysis were conducted using the Regression Analysis Tool in Microsoft Excel 2008. Objective 3 and 4 were addressed using visual assessment of measured pressure curves for each individual sea lamprey during each test. 27 2.3 Results 2.3.1 Pressure Data for Leakage Check of the Apparatus For the single-port sensing system, when Tubing B was potted directly to the sensing port (Figure 2.3-a), initial pressure was maintained for the entire period recorded (1000 s) when initial pressures were –10 kPa and –20 kPa (Figure 2.7), which implies that the system was well sealed. When Tubing B was connected to the sensing port via PDMS suction cup in air (Figure 2.3-b), measured vacuum pressure decreased over time, reaching 0 kPa after 500 s with an initial pressure of –10 kPa and reaching about –8 kPa after 1000 s when initial pressure was –20 kPa (Figure 2.7). Slower leakage at higher pressures were attributed to the larger compression of rim of the suction cup (and thus better seal at the interface) under the higher pressure. Finally, the suction cup demonstrated better seal properties when submerged in water. When the suction cup interface was submerged in water, measured pressure decreased to –8 kPa after 1000 s when initial pressure was –10 kPa, and remained near –20 kPa after 1000 s when initial pressure was –20 kPa. Similar results were observed from the 9-port pressure sensing system (Figure 2.8). Measurements from the center port (P4, Figure 2.2-c) of the 9-port sensor were very similar to single-port sensor measurements for each connection method and initial pressure (Figure 2.8-a). Measurements were also consistent among ports and similar to single port measurements for each connection method and initial pressure (Figure 2.8-b). These results indicate that the single-port and 9-port pressure sensing systems (in particular, the connection from the sensing port on the plate to the sensor itself) were well sealed, the reading from the pressure sensor indeed reflected the pressure at the suction point, and that apparent leakage with measurements of live lampreys likely occurred through the interface between the mouth and plate or elsewhere in 28 the lamprey body. Figure 2.7: Vacuum pressures recorded during the first 1000 s for potential leakage tests of the single-port pressure sensing setups. Low (–10 kPa) and high (–20 kPa) vacuum pressures were applied with the vacuum chamber tubing attached to the sensor port via one of three methods: direct-fit potting, tested in air (Setup in Figure 2.3-a); suction cup interface in air (Setup in Figure 2.3-b), suction cup interface in water (Setup in Figure 2.3-c). Data were down-sampled for display. 2.3.2 Suction Dynamics of Juvenile and Adult Sea Lampreys Among all tests using the single-port sensing panel with live adult sea lampreys at rest (excluding pull tests), free suction pressures ranged from –3.3 ± 0.9 kPa to –13.8 ± 3.2 kPa with a leakage time (period between re-pressurizing pumps) of 319.5 ± 187.0 s (Table 2.2). During pull tests, maximum suction pressures ranged from –8 kPa to –70 kPa. Owing to small sample sizes and large variability (Figure 2.9), we did not detect statistically significant 29 Figure 2.8: Vacuum pressures recorded during the first 1000 s for potential leakage tests of the 9-port pressure sensing setups. (a) Mean and standard deviation (shown as the error bar) of the pressure at port P4 at each time instant for three rounds of measurements, and (b) mean and standard deviation of the mean pressure at all these 9 ports at each time instant for three rounds of measurements. differences in maximum free suction pressures between male and female adult sea lampreys (Mann-Whitney U = 77.5, z = –1.342, p-value = 0.180) or between adult and juvenile sea lampreys (U = 80.5, z = –0.515, p-value = 0.607). Similarly, we did not detect differences in median leakage time between male and female adult sea lampreys (U = 104, z = –0.649, p-value = 0.516). However, median leakage time was significantly greater for juvenile than adult sea lampreys (U = 27, z = –2.803, p-value = 0.005). among groups (i.e., adult male, adult female, juvenile). Maximum vacuum pressure was weakly associated with body weight in juvenile sea lampreys (Figure 2.9-a; linear regression: slope = –0.376; R2 = 0.60; p-value = 0.071) but not adult male (slope = 0.052; R2 = 0.04; p-value = 0.537) or female (slope = 0.023; R2 = 0.01; p-value = 0.639) sea lampreys. Leakage time was weakly associated with maximum vacuum pressure in juvenile sea lampreys (Figure 2.9-b; slope = –32.902; R2 = 0.68; p-value = 0.042) but not adult male (slope = 3.488; R2 = 0.04; p-value = 0.534) or 30 Table 2.2: Statistics of the pressure measurement results corresponding to Table 2.1, in- cluding manual pull condition (pull), maximum suction pressure before disengagement (max vacuum), minimum suction pressure to remain attachment (min vacuum), and the duration period between two adjacent pressurizing processes (leakage time). Max Min Leakage Sensor Water Life No. Sex Pull Vacuum Vacuum Time Type Flow Stage (kPa) (kPa) (s) No -13.8± 3.2 -3.3± 0.9 319.5± 187.0 M Yes -39.0± 31.0 0 19.0± 16.0 1,2 Single Static Adult No -13.2± 2.9 -2.5± 1.1 379.1± 134.8 F Yes -37.2± 4.0 0 16.0± 4.1 No -10.7± 2.4 -1.1± 0.3 304.0± 131.3 Adult F Yes -32.8± 7.6 0 10.5± 2.3 2 9-port Static No -18.8± 3.6 -5.4± 2.4 608.3± 142.8 Juvenile N/A Yes -35.0± 1.2 0 7.1± 1.5 M No -34.0± 3.9 -17.4± 4.5 161.4± 17.7 3 9-port Flowing Adult F No -27.7± 2.9 -8.2± 1.7 182.2± 18.6 female (slope = –3.632; R2 = 0.02; p-value = 0.520) sea lampreys. Observed suction dynamics were largely consistent with previous descriptions but high measurement rate coupled with experimental video provided detailed insights. For example, during the 20 min experiment of suction on the sensing panel without change of attaching position, the suction pressure of an adult female lamprey frequently fluctuated between –6 kPa and –1 kPa (Figure 2.10-a). It rose to –6 kPa suddenly in about 0.3 s, and then slowly returned to –1 kPa in about 30 s. For some cycles, the recovering time lasted for 50 100 s. During static attachment, the lamprey’s tongue was observed to protract (Figure 2.10-b), i.e., the buccal cavity and the pharyngeal cavity were connected due to the protraction of the tongue, and when the suction pressure was too low to maintain attachment, the tongue retracted quickly to enclose the buccal funnel, and then, protracted immediately again to open the oral passage (the timing of these tongue movements is indicated by the shaded rectangles in Figure 2.10-b), which was consistent with the suction mechanism described in 31 Figure 2.9: Relations between (a) maximum free suction pressure and weight, and (b) leakage time and maximum free suction pressure for tested adult male, female, and juvenile sea lampreys. [25]. With these actions, the suction pressure increased rapidly, maintaining attachment on the panel. Suction pressure leakage was common, and leaking and pressurizing processes alternated until disengagement from the panel. During the pull test of the same adult lamprey suction pressure rose slowly from –5 kPa to –50 kPa in about 30 s, and then returned to 0 kPa in only one second (Figure 2.10- c). During this test, the suction pressure increased to 10 times of its initial level, and the sea lamprey’s tongue retracted and protracted more frequently and with larger amplitude (Figure 2.10-d). To resist the drag force, the annular muscle around the mouth contracted more vigorously until disengagement. 32 Figure 2.10: Suction dynamics of an adult female sea lamprey in Experiment 1. (a) Suc- tion pressure when the adult female lamprey attached to the panel during 400 s to 1200 s after attachment, (b) suction pressure profile during 1080 s to 1140 s after attachment with shaded rectangles showing the time when the lamprey’s tongue was protracted or retracted, (c) suction pressure profile when the adult lamprey was pulled away from the panel with corresponding snapshot of the lamprey’s mouth disengaged from the panel, and (d) pictures of an adult lamprey’s mouth showing the protraction and retraction states of its tongue. 2.3.3 Suction Pressure Distribution within the Mouth of Adult and Juvenile Lampreys Among all tests using the 9-port sensing panel with live adult and juvenile sea lampreys at rest (excluding pull tests), free suction pressures ranged from –0.5 kPa to –34 kPa (Table 2.2). During pull tests, maximum suction pressures ranged from –16 kPa to –47 kPa. The leakage time varied within and among lampreys, but ranged from 80 s to 800 s for adult lampreys and more than 800 s for a half of the tested juvenile lampreys. Suctions pressure 33 measurements with the 9-port sensing system suggested that pressure distributions were uniform throughout the mouth in some cases but not all. For example, suction pressures from an adult female sea lamprey were very similar among all nine sensor ports during resting state and pull tests, despite rapid changes in suction pressure over time (Figure 2.11-a,b). For that individual, the median leakage time was about 150 s, and the maximum pressure in static suction was about –7 kPa, while the maximum in the pull test was about –22.5 kPa. However, suction pressure from another adult female lamprey varied considerably among some sensor ports with no apparent pattern over time or among tests (Figure 2.11-c,d). For this individual, the median leakage time was about 160 s, and the maximum pressure in static suction was about –10 kPa, while the maximum in the pull test was about –45 kPa. Specifically, the curves representing ports P0, P3 and P6 (labeled in Figure 2.2-c), which were located at the left column of the ports, remained at some pressure levels for a few minutes while pressures fluctuated in synchrony among other ports. However, as seen in the picture in Figure 2.2-b (which corresponds to this case), these three ports were under the coverage of the oral disc. Hence, this pressure difference might be related to the blocking of the ports by some teeth on the oral disc. It is hypothesized that the oral disc covered the port matrix and formed vacuum pressure in that area, but then some teeth might have fallen into the ports and blocked the passage between the buccal cavity of the mouth and the tubing connected to the pressure sensor, causing the pressure stagnation. As a final example, suction pressures from one juvenile lamprey were very consistent among all the sensor ports during static suction and pull tests (Figure 2.12-a,b). The juvenile lamprey created a maximum suction pressure of –22 kPa and hence an effective –8 kPa in the static states, and maintained attachment without pressurization for about 850 s. From the pull test, it was recorded that the maximum suction pressure reached –37 kPa. 34 Figure 2.11: Suction dynamics of two adult sea lampreys in Experiment 2. (a) Suction pressure when an adult female lamprey was attaching by itself on the panel, (b) suction pressure when the lamprey in (a) was pulled away from the panel, (c) suction pressure when another adult female lamprey was attaching by itself on the panel, and (d) suction pressure when the lamprey in (c) was pulled away from the panel. And the suction pressure profiles of other juvenile sea lampreys tested in this experiment showed similar pressure range and duration time of attachment to this example case both at rest and in pull tests. Besides, compared to the pressure profiles of those adult female lampreys (Figure 2.11-a,b,c,d), the juvenile lampreys appeared to be physically stronger as they created higher maximum suction pressure at rest, and each period of attachment without pressurization lasted longer than the adult lampreys. 35 Figure 2.12: Suction dynamics of a juvenile sea lamprey in Experiment 2. (a) Suction pressure when a juvenile sea lamprey was attaching by itself on the panel, (f) suction pressure when the sea lamprey in (a) was pulled away from the panel. 2.3.4 Effects of flowing water on measured suction pressure Among all tests in August and September 2020 using the 9-port sensing panel with live adult lampreys in both static and flowing water (excluding pull tests), suction pressures ranged from –2.5 kPa to –40 kPa (Table 2.2), the maximum of which was much higher than that of the static water tests in August and October 2019. The median leakage time varied within and among lampreys, but were all longer than 180 s, which implied the pump frequencies were lower than those observed in the tests in 2019. More importantly, as the water velocity increased step by step, the suction pressure of the adult lampreys did not appear to increase accordingly, nor was a new suction event (re-pressurization) detected from the pressure data. In contrast, the suction pressure of adult lampreys seemed to be insensitive to water flows (velocity ⩽ 0.45 m/s), and decreased slowly due to water leakage. For example, suction pressure of an adult male sea lamprey gradually decreased from initial pressure of –23 kPa to –18 kPa while water velocity increased from 0 m/s to 0.45 m/s (190 s) and then was 36 abruptly reduced to 0 m/s where it remained until disengagement at the end of the test (225 s; Figure 2.13-a). Pressure curves were similar among all nine sensor ports during the test. Similarly, suction pressures of an adult female showed no response to changing water velocities during a flowing water test (Figure 2.13-b). The rate of leakage, however, did vary among sensor ports during that test, perhaps because three ports were blocked by teeth. Regardless, suction pressure did not increase as water flow increased (⩽ 0.45 m/s). Figure 2.13: Suction dynamics of two adult sea lampreys showing characteristic responses in Experiment 3. (a) Suction pressures of an adult male lamprey in the flowing water test, and (b) suction pressures of an adult female lamprey in the flowing water test. 2.4 Discussion We successfully recorded the suction dynamics for two life stages of sea lampreys in static and flowing water. Observed suction behavior was consistent with previously described suction mechanisms [25] and qualitatively similar to previous suction pressure measurements [25, 26], and additional insight obtained may be important for future sensor design and practical deployment. To our knowledge, our measurements are the first to describe suction of adult 37 sea lampreys in flowing water, and to describe variation across the mouth opening. Our 9-port sensor revealed that the pressure distributions were uniform across the lamprey’s oral disc, but that obstruction of sensor ports by teeth may impose measurement error for certain devices. Finally, coupling of video recordings with high-sampling-frequency (200 Hz) pressure measurements revealed the lamprey’s tongue movement and the oral disc contraction during the pull test, which supported previous descriptions of the suction mechanism. Observed minimum suction pressures showed that lampreys are capable of maintaining attachment to smooth surfaces with relatively little suction, even in flowing water. As shown in Figure 2.10-a and Figure 2.11-a, the suction pressure levels at each moment of the re-pressurization processes were less than about –0.6 kPa, which implied a threshold of pressure level that triggers re-pressurization. When the suction pressure was higher than this threshold, the lamprey did not take action but remained attached; whereas if the leakage reduced the pressure below the threshold, a pressurization process would be taken quickly. The pressure threshold likely depends on a number of variables, including the area and texture of the attachment substrate, size and health condition of the lamprey, the attachment orientation during suction, and the ambient current flow or any other external force causing disengagement. Although sea lampreys may be capable of attaching to certain surfaces with relatively small suction pressures, leakage at the mouth-substrate interface may necessitate the need for lampreys to apply relatively large suction pressures (e.g., two orders of magnitude greater) at the start of each pump cycle. A trade-off may exist between pump magnitude (pressure applied) and frequency, wherein greater pressure applied to each pump results in fewer pumps overall due to longer time until low-pressure threshold is reached, though we did not detect any association between maximum pressure and leakage time in this study. Results from our 38 leak test further suggest that higher pressure may improve the seal between the oral disk and the substrate, thereby further reducing the leak rate and the number of pumps required per unit time. Indeed, the rate of leakage slowed as pressure decreased in each of our tests. Unlike minimum pressures, however, maximum suction pressures were highly variable among individual lampreys, although it remains unknown if pressure applied is context-dependent, if lampreys can sense the amount of pressure applied, or if pressure applied is simply related to anatomical proportions (e.g., volume of buccal cavity). Clearly, any device used to detect sea lamprey suction and differentiate it from other sources needs to detect the range of negative pressures exerted by lampreys (described above). To our knowledge, suction dynamics of other organisms and elements of the environment have not been characterized in sea lamprey habitats, though we speculate that that the risk of false-positive measurements from non-lamprey sources, such as water currents, are greater at lower pressures. Thus, a threshold pressure for reliable sea lamprey identification may need to be greater than the minimum pressure observed in sea lampreys. Fortunately, our results imply that negative pressure is uniform throughout the mouth, so a single sensor should be adequate, although improved designs may be needed to prevent obstruction of port holes by teeth. For example, pressure curves in Figure 2.11-c where all the nine ports were covered by the oral disc, six pressure curves coincided with each other during the 20 min test, but the other three pressure curves remained at different levels. It was inferred that after the seal of the suctorial disc on the sensing plate, three ports on the left column of the 9-port sensing plate might have been blocked by teeth, which cut off the pressure transduction between the ports and the vacuum sensors. Therefore, the corresponding pressure curves at these three ports remained at some levels for a long period without decreasing, while those of the other six ports gradually leaked until a sudden re-pressurization process was induced. 39 Similarly, the uniformity of measured pressures from juvenile lampreys with the 9-port sensor may have been due to characteristics of their teeth, which were smaller and shallower (less than 1 mm long) than the adult lampreys (about 2 mm long in a cone-hook shape). Hence, the juveniles’ teeth would have a lesser chance to block the ports on the panel, and all the ports below the oral disc should be connected to each other and shared the same pressure. Future sensors of the design used in our study may benefit from modifying the port hole size to minimize occlusion by teeth, or compensating the influence by connecting port hole matrix to the same pressure transduction channel. We also compared the suction pressures of juvenile lampreys (Figure 2.12-a) with those of adult lampreys (Figure 2.11-a,c). Observed differences in suction pressures and leakage times between juvenile and adult sea lampreys may have been related to different energetic states of two life stages. We hypothesize that some juveniles were in general physically stronger than adults at the end phase of the lifecycle, demonstrating higher suction pressures and longer seal, so the juvenile lampreys did not have to re-pressurize as frequently. This might be attributed to the biological activity of the lampreys in different life stages: the juvenile lampreys were in the parasitic stage, and thus were more active and energetic, with stronger attachment for preparation of parasitizing and feeding; the adult lampreys, on the other hand, were presumably much weaker after a long period of non-feeding during the spawning stage, resulting in the suction pressure fluctuating frequently due to leakage. We also analyzed the suction dynamics of adult lampreys in both static and flowing water (Figure 2.13). By increasing the water velocity from 0 to 0.45 m/s step by step while the lamprey was attaching to the sensing panel, we found that the influence of water flow on the suction dynamics of adult sea lampreys was insignificant. Apparently, suction pressure was adequate for maintaining attachment in flowing currents with water velocity below 0.45 m/s. 40 Oral suction is a multi-function tool that sea lampreys use for feeding, transportation, resting, nest building, and mating, yet has not been utilized for species-specific detection to inform conservation or management. Observations from our new pressure-sensing panels have confirmed previous descriptions of suction dynamics of adult sea lampreys on smooth surfaces and provided new insights into the suction parameters of sea lampreys at the juvenile life stage and in flowing water. Our 9-port sensor array allowed us to investigate the suction dynamics across the lamprey’s oral disc. These results are expected to inform development of the next generation of lamprey assessment gears and may inspire similar efforts to develop detection systems for other taxa with unique characteristics. Finally, sea lampreys often attach onto other living organisms and rocks in the water, but it is still unclear how different attachment surfaces engineered with different materials and roughness patterns will affect the suction behavior. This will be part of our future work when designing deployable sensing panels for operation in natural environments. 41 Chapter 3 Screen-Printed Soft Capacitive Sensors for Spatial Mapping of Both Positive and Negative Pressures In this chapter, we present a convenient and low-cost process for fabricating a soft capaci- tive sensor that is stretchable and responsive to both positive and negative pressures. The sensor is comprised of a soft Ecoflex 00-30 dielectric layer sandwiched in between and tightly bonded with screen-printed poly(3,4-ethylenedioxythiophene): poly(styrene-sulfonate) (PE- DOT:PSS) electrodes and polydimethylsiloxane(PDMS) encapsulation layers. More impor- tantly, air gap channels are incorporated and uniformly distributed in the Ecoflex 00-30 layer, which shape the diaphragm structures that greatly enhance the dielectric layer deformation under an external pressure, resulting in significantly improved response, especially to nega- tive pressure. Meanwhile, the PEDOT:PSS electrodes are encapuslated on the Ecoflex-030 substrate with PDMS layers, which successfully protect the electrodes and prevent the layers from delamination under negative pressure. The response of the single pixel sensor is sys- tematically characterized and finite element method (FEM) simulation is used to study the influence of the air gap geometry on the sensor response. The sensor with optimized design exhibits good sensitivity from –60 kPa to 20 kPa and great repeatability under compressive 42 loads, vacuum suction, and even bending or stretching conditions. Lastly, a 12×12-pixel sensor array that can automatically measure the pressure distribution for both positive and negative pressures with high fidelity is demonstrated. However, from our experimental stud- ies, it is found that there are some challenges for the soft capacitive pressure sensor for detecting the attachment of sea lampreys underwater. When the soft capacitive pressure sensor is deployed in water and attached by a sea lamprey, since both water and the sea lam- prey are conductive, they will introduce parasitic capacitance to the pixels of the capacitive sensor, and cause significant signal interference to the capacitance measurement, which re- sults in failure in the sea lamprey detection. To address this challenge, additional effort is required in the future. Materials in this chapter are an extension of the work reported in [110]. The remainder of the chapter is organized as follows. The fabrication of the soft ca- pacitive pressure sensors with air gap channels are first presented in Section 3.1. Then the characterization of the single-pixel sensors is discussed in Section 3.2. Finite Element Simulation results are described in Section 3.3, followed by spatial mapping results of pres- sure distribution with the sensor array in Section 3.4. Further challenges of the capacitive pressure sensor in detecting sea lamprey attachment are discussed in Section 3.5. Finally, concluding remarks are provided in Section 3.6. 3.1 Fabrication of the Soft Capacitive Pressure Sensors with Air Gap Channels A crossbar array of soft capacitive pressure sensors is designed and fabricated and its schematic is illustrated in Figure 3.1-a. The soft capacitive pressure sensor is comprised 43 of an Ecoflex 00-30 dielectric layer sandwiched in between conductive polymer PEDOT:PSS electrodes and PDMS encapsulation layers. The Ecoflex 00-30 dielectric layer is 1.4 mm thick with multiple air gap channels (height: 0.5 mm, width: 1.2 mm, spacing: 1.5 mm) uniformly distributed in the center. As will be discussed later, these air gap channels play a critical role in the sensing of both positive and negative pressure by significantly enhanc- ing the deformation of the dielectric layer under pressure, thereby increasing the sensitivity. PEDOT:PSS polymer is used as the electrode due to its high conductivity (sheet resistance of ∼ 34 Ω/sq) and ionic additive Bis(trifluoromethane)sulfonimide lithium salt is incorpo- rated into the PEDOT:PSS to further improve its stretchability [70]. Figure 3.1-b shows the bright field optical micrograph and scanning electron microscopy (SEM) image of the cured PEDOT:PSS film with ionic additive transferred on a Si wafer with very good uniformity. The picture of a completed 12×12 soft capacitive pressure sensor array is shown in Figure 3.1-c and its fabrication process is illustrated in Figure 3.1-d. First, an Ecoflex 00-30 dielectric layer is cast in a mold on a treated glass plate with uniformly distributed 0.5 mm thick and 1.2 mm wide polycarbonate strips. After the Ecoflex 00-30 is cured, the polycarbonate strips are removed, resulting in the Ecoflex 00-30 film with built-in air gap channels. A 0.5 mm thick PEDOT:PSS film is then screen-printed onto one side of the Ecoflex 00-30 dielectric layer with the PEDOT:PSS electrodes patterned in parallel but perpendicular to the air gap channels. After curing by heating up at 70 °for 1 h, the PEDOT:PSS electrodes were encapsulated by a 0.7 mm thick PDMS layer (PDMS base: curing agent = 10:1 wt%). This encapsulation layer effectively protects the PEDOT:PSS electrodes from cracking under external force and also forms a very strong bonding between PDMS and Ecoflex 00-30 substrate, preventing the sandwiched layers from delamination when negative pressure is applied. Next, the device is peeled off from the glass plate and placed upside down. The 44 PEDOT:PSS screen-printing and PDMS encapsulation process are then repeated on the other side of the Ecoflex 00-30 dielectric with the newly patterned PEDOT:PSS electrodes orthogonal to the previous electrodes in order to form a crossbar capacitor array. Also, this layer of patterned PEDOT:PSS electrodes should be aligned with the air gap channels in the substrate so that the electrodes could conform with the deformation of the diaphragms in case of cracking. 3.2 Characterization of the Single-Pixel Sensors The effect of the air gap channel geometry on the performance of the single-pixel sensors is investigated. The devices are characterized by measuring the relative change in capacitance as a function of pressure (∆C/C0 vs. P) and the results are shown in Figure 3.2. Figure 3.2-a presents the structure of the single-pixel sensor with height (H), width (W) and the number (N) of the air gap channels labeled in the figure. Six groups of air gap parame- ters have been selected, three devices have been fabricated for each configuration and three rounds of measurements have been taken for each device to obtain the average response per- formance. The error bar in Figure 3.2-a represents the standard error of the mean for each group of testing points. As shown in Figure 3.2-b, all five configurations respond similarly to positive pressure and exhibit a monotonic increase in relative change in capacitance with in- creasing pressure. The pressure response also increases with increasing air gap size, reaching a maximum ∆C/C0 value of 4.01 %, 6.15 %, 7.16 %, 8.10 %, 9.48 % at a positive pressure of 20 kPa, for the sensors without air gap, and with air gap of dimensions H = 0.3 mm and W = 1.2 mm, H = 0.5 mm and W = 1.2 mm, H = 0.5 mm and W = 1.6 mm, and H = 0.5 mm and W = 2.0 mm, respectively. 45 Figure 3.1: (a) Schematic illustration of the soft capacitive pressure sensor array with air gap channels and conductive polymer PEDOT:PSS electrodes. (b) Optical micrograph (top) and SEM image (bottom) of the screen-printed PEDOT:PSS film with a feature thickness of ap- proximately 200 µm. The PEDOT:PSS contains 10:1 wt% Bis(trifluoromethane)sulfonimide lithium salt as stretchability and electrical conductivity enhancer. Scale bars: 20 µm and 2 µm, respectively. (c) Photograph of a 12×12 capacitive pressure sensor array. Scale bar: 1 cm. (d) Schematic illustrating the fabrication procedures of the soft capacitive pressure sensor array. 46 Table 3.1: Sensitivity of the single-pixel soft capacitive pressure sensors with different air gap configurations. Sensitivity –60 ∼ –20 –20 ∼ 0 0 ∼ +10 +10 ∼ +20 (% / kPa) (kPa) (kPa) (kPa) (kPa) Without Air Gap — — 0.18 0.13 H = 0.3 mm, W = 1.2 mm 0.34 0.30 0.38 0.15 H = 0.5 mm, W = 1.2 mm 0.34 0.34 0.56 0.13 H = 0.5 mm, W = 1.6 mm 0.35 0.40 0.62 0.12 H = 0.5 mm, W = 2.0 mm 0.30 0.56 0.77 0.15 For sensing a negative pressure, the importance of having an air gap channel in the dielectric layer becomes evident as the sensor without an air gap channel fails to respond properly and exhibits negligible capacitance change as the pressure changes between 0 ∼ –30 kPa. The slight increase of ∆C/C0 between –30 and –60 kPa can be attributed to the interior of the suction cup pressing against the surface of the sensor under vacuum. In contrast, for the sensors with air gap channels, ∆C/C0 decreases monotonically as the pressure varies from 0 to –60 kPa, reaching –21.91 %, –22.90 %, –23.46 %, and –24.64 % for the sensor with air gap H = 0.3 mm and W = 1.2 mm, H = 0.5 mm and W = 1.2 mm, H = 0.5 mm and W = 1.6 mm, and H = 0.5 mm and W = 2.0 mm, respectively. Here, the sensor with air gap H = 0.5 mm and W = 2.0 mm achieves the best sensing performance among all the air gap configurations. Table 3.1 summaries the calculated sensitivity (defined as the slope of the relative change in capacitance to pressure response curve, S = d(∆C/C0 ) / dP of the five types of sensors above for different pressure ranges. Basically, a larger air gap would result in sensors with higher sensitive to both negative and positive pressure. The repeatability of the device is also evaluated on the sensor with air gap H = 0.5 mm and W = 2.0 mm by cyclic test with 1,000 repetitions for four pressure levels: 5 kPa, 10 kPa, –10 kPa, and –20 kPa. As illustrated in Figure 3.2-c, the device can preserve stable response throughout the 1,000 cycles for all pressure levels. 47 Figure 3.2: Characterization of a single-pixel soft capacitive pressure sensor. (a) Schematic diagram representing the multi-layer structure of the single capacitive pressure sensor. (b) Relative change in capacitance in response to applied pressure for a single-pixel capacitive pressure sensor with various configurations of air gap in the Ecoflex dielectric layer. (c) Cyclic test of the sensor response for 1000 cycles at different pressures. (d) Schematic illustration of the sensor in the bent state under normal force. (e) Pressure response of the sensor with air gap (H = 0.5 mm, W = 2.0 mm) under a sequence of normal forces, 2 N, 3 N and 2 N when the sensor is bent to curvature radii of 50, 30, and 17 mm, respectively. (f) Cyclic test of the sensor at a bending radius of 17 mm for 1000 cycles. (g) Cyclic stretching test of the sensor with air gap (H = 0.5 mm, W = 2.0 mm) at 10 % strain with the air gap direction parallel to the stretch direction. (h) Cyclic stretching test of the sensor with air gap (H = 0.5 mm, W = 2.0 mm) at 10 % strain with the air gap direction perpendicular to the stretch direction. 48 In order to demonstrate the capability of the soft capacitive pressure sensor working on curved surfaces, we examine the device’s response to pressure when attached onto a pipe, as illustrated in Figure 3.2-d. Figure 3.2-e presents the response of the sensor with air gap H = 0.5 mm and W = 2.0 mm to pressure when mounted on cylindrical surfaces with various radii (Rb = 50 mm, 30 mm, and 17 mm). Again, three rounds of measurements are taken on each cylindrical surface when the sensor sample is tested under a sequence of force 2 N, 3 N and 2 N for about 20 s at each stage. The response curves reveal that the sensor is more sensitive to compressive load when it is bent on a curved substrate compared to a non- curved substrate. What’s more, the smaller the bending radius is, the higher the relative change in capacitance would be. This is because when the sensor is in a bent state, the compressive load will have a smaller contact surface area with the sensor device, thereby the effective pressure applied on the electrodes and air gap channel would be much higher and the corresponding larger deformation will cause more change in capacitance. The sensor also exhibits good repeatability in pressure response even when bent with a radius of 17 mm as shown in Figure 3.2-f. The stretchability of the single-pixel sensor is tested. Tensile strain is applied along two directions, parallel, and perpendicular to the air gap channel. As illustrated in Figure 3.2-g, when the strain is set to 10 % with a loading-unloading period of 12 s, the response (∆C/C0 ) would increase by about 0.5 % every time when stretching. For the stretching direction perpendicular to the air gap channel as shown in the inset in Figure 3.2-h, in each period the response would increase and decrease by about 1.8 %. This change in capacitance is reasonable according to the Poisson effect, which is that a material tends to compress in directions perpendicular to the direction of expansion. And for the case in Figure 3.2-h, since the air gap is perpendicular to the stretching direction, the sensor substrate would 49 more easily deform. On the other hand, for both stretching directions, the overall response would increase slightly by about 0.1 % after 1000 cycles. This increase of about 0.1 % might be attributed to be the results of the sliding of the sensor substrate from the clamps in the experiment setup. We also try to increase the strain to 11 % and test for 450 cycles during the experiment. The increased strain leads to a crack in PEDOT:PSS electrode at the intersection point between PEDOT:PSS and the silver paint, which is used for connecting and fixing the copper wire to the PEDOT:PSS electrode. For better stretchability, the silver paint can be replaced with some other stretchable conductive glue or liquid metal. It is worth noting that this section focuses on the characterization of the single pixel sensor with a single air gap channel in the Ecoflex 00-30 dielectric layer located right underneath the electrode. For the single pixel sensor where there are multiple air gap channels distributed evenly in the dielectric layer, the sensitivity of this single pixel sensor under both positive and negative pressure would be much higher. As demonstrated in Figure 3.2-i, the single pixel sensor with 9 air gap channels (H = 0.5 mm, W = 2.0 mm) can achieve relative change in capacitance of about +15 % at +20 kPa and –32 % at –60 kPa, which are more sensitive than the sensor with only one air gap channel as characterized in Figure 3.2-b. This is also true for the sensor array with multiple air gap channels, which has higher sensitivity to both positive and negative pressures than a single pixel sensor with only one air gap channel, as will be explained in Section 2.4. 3.3 Finite Element Simulation As discussed in Section 2.2, the geometry of the air gap channel has significant influence on the sensitivity of the sensor responses. Therefore, studying the mechanical properties of 50 Ecoflex 00-30 rubber, PDMS silicone and PEDOT:PSS conductive polymers is essential for understanding the pressure-induced deformation of the sensor and the capacitive response under external pressures. In this section, we use these material properties to set up the finite element analysis models of the soft capacitive pressure sensor under both positive and negative pressures and examine the distance change in between the two electrodes as well as the vertical deformation of the whole sensor. The finite element analysis modeling is conducted in ANSYS Workbench and one static structural module is used to build up four separate analysis models: a sensor without air gap channel under +20 kPa and –30 kPa pressure, and a sensor with air gap channel under +20 kPa and 30 kPa pressure. The positive pressure is applied onto a glass slide (20 mm × 18 mm × 1 mm) which is placed on the sensor (60 mm × 60 mm × 2.8 mm) in the center, while the negative pressure is applied on the surface of the sensor in the region beneath the suction cup (ϕ20.5 mm × ϕ14.5 mm × 15 mm in depth) with Dragon Skin 30 as the assigned material, and the air gap channel in the dielectric layer designed with dimensions H = 0.5 mm and W = 2.0 mm. Four materials of interest are the silicone materials (Ecoflex 00-30, PDMS 10:1, Dragon Skin 30), and conductive polymer PEDOT:PSS. To characterize the material properties of the silicones, a uniaxial tensile test is performed following the D412-15a standard [111]. The dumbbell test samples are stretched at a speed of 500 mm/min. Five samples of each material are stretched using a tensile tester (Universal Testing Machine 3345, Instron) and the average stress strain data obtained from the five samples are used in the finite element analysis by data fitting to obtain the most fitted hyperelastic models. The Mooney-Rivlin 3 parameter model[112, 113] proves to be the best constitutive model for Ecoflex 00-30 with parameters C10 = 90.523 Pa, C01 = 10137 Pa, and C11 =0.0504 Pa; the Yeoh 3rd order 51 model [114] proves to be the best model for PDMS with parameters C10 = 2474.5 Pa, C20 = 0.00297 Pa, and C30 = 1.498E-7 Pa; and the Ogden 1st order model[115] turns out to be the best one for Dragon Skin 30 with parameters α1 = 2.717, µ1 = 0.158 MPa. For PEDOT:PSS (thickness range 150 to 200 µm) with 10 wt% of Bis(trifluoromethane)-sulfonimide lithium salt, an isotropic elastic model with Young’s modulus 55 MPa is used according to the report [70]. Under a positive pressure, the sensor is compressed and the air gap channel in the di- electric layer will be squeezed. Note that the top and bottom surface of the air gap channel cannot penetrate each other in reality, hence in the simulation, we define a frictional contact between these two surfaces with a friction coefficient of 1.5, and this constraint condition will work once these surfaces contact each other. When a negative pressure is applied, the top surface of the sensor beneath the suction cup is pulled up, and the air gap channel will be enlarged. Figure 3.3 displays the simulation results of all four analysis models, where the top subfigures are the sectional views of the sensors under a glass slide or a suction cup, and the bottom subfigures present the Z-directional deformation of the crossbar electrodes. Fig- ure 3.3-a shows the deformation in a soft pressure sensor without an air gap under +20 kPa pressure, in which the distance between the top and bottom electrodes decreases by –0.20 mm. In contrast, as shown in Figure 3.3-b, the total deformation in a sensor with an air gap results in the electrode spacing decreasing by –0.51 mm. Similarly, the data in Figure 3.3-c and d show that under a negative pressure of –30 kPa, the sensor with an air gap channel exhibits a much larger increase in electrode spacing (2.87 mm) compared to the sensor with- out an air gap channel (0.59 mm). In both cases, the larger deformation in the device with an air gap will result in larger capacitance change and better sensitivity to pressure. These four simulation results further validate the necessity of the air gap channel designed in the 52 dielectric layer. In addition, finite element models are also used to investigate the influence of dimensions (height H and width W) of the air gap channel on the deformation of the sensor device. Five configurations of the air gap channel are studied in the simulation corresponding to the sensors’ configurations in the experiments of Section 2.2. Since the deformation is not uniform along the electrodes, we focus on the center distance change between the crossbar electrodes at the center of the sensor, which is defined as d0 as illustrated in Figure 3.3-e. The distance d0 between the crossbar electrodes is 1.4 mm initially and becomes smaller under positive pressure or larger under negative pressure. The simulation results of the central distance change for these five groups of air gap configurations are plotted in Figure 3.3-f, where the sensor with larger air gap channel in width or height generates a larger change in central distance of the crossbar electrodes, and the sensor with air gap (H = 0.5 mm, W = 2.0 mm) achieves the largest change in distance under both positive and negative pressures, which is consistent with the experimental results present in Figure 3.2-b. From the simulation results and according to the formula of crossbar capacitance, which is C 1 inversely proportional to the spacing between the two parallel electrodes ∝ , we can ϵ d0 conclude that the sensor with larger air gap channel has higher sensitivity for both positive and negative pressures, which is consistent with the experimental results in Section 2.2. 3.4 Spatial Mapping of Pressure Distributions with the Sensor Array Based on the characterization of the single-pixel sensor with air gap channel, we extend our work to a 12 × 12 sensor array with air gap channels uniformly distributed underneath the 53 Figure 3.3: FEA Simulation of the sensor with and without an air gap in the Ecoflex 00-30 dielectric layer under different pressures. 54 top layer electrodes. A channel-selection circuit that uses NI LabVIEW program to control the communication between an Arduino Uno microcontroller board and AD7746 capacitive- to-digital converter is designed to automatically scan through all pixels with a period of 28.6 ms to measure the ∆C/C0 before and after the pressure is applied in order to determine the pressure distribution. Taking the special mapping resolution into consideration, a 12 × 12-pixel sensor array with 12 air gap channels (H = 0.5 mm, W = 1.2 mm) is fabricated (the one with W = 2.0 is not considered since the wider the air gap, the sparser the electrodes would be). Figure 3.4-a displays the photographs of the 12 × 12-pixel sensor array with air gap channels with 3D-printed M-, S-, U-, and O-shaped letter block placed on top. A loading pressure of +20 kPa is applied using a syringe pump and the pressure is set according to the contacting surface area of the letter block and the force between the loading part of the syringe pump and the letter block measured by a force sensing resistor. Figure 3.4-b depicts the mapping contours of relative change in capacitance with the sensor array under corresponding letter block, which are consistent with the profiles of the letter blocks used. Ideally, the relative change in capacitance could be mapped into pressure values based on the characterization of the single-pixel response. However, the single-pixel sensor reported in Section 2.2 only has a single air gap channel in the dielectric layer, whereas the 12 × 12 sensor array contains 12 air gap channels uniformly distributed in the dielectric layer underneath each column. The extra air gap channels in the sensor array increases the deformation of the sensor, which is why the sensor array exhibits ∆C/C0 of more than 14 % under +20 kPa but the single-pixel sensor in Figure 3.2 only exhibits approximately 5 % change at the same pressure. For the reason above, the relative change in capacitance in the mapping contour cannot be simply converted into pressure values according to the characterization in Section 2.2. 55 Figure 3.4: Spatial mapping of pressure distributions with the sensor array under both positive and negative pressures. (a) Photograph of the 12 × 12 pixel sensor array with different shaped objects placed on top for pressure mapping testing. Scale bar: 2 cm. (b) The corresponding distributions of the normalized capacitance change with +10 kPa pressure applied. (c) Photograph of the 12×12 pixel sensor array with air gaps (H = 0.5 mm, W = 1.2 mm) and the corresponding contour plots of relative change in capacitance under a negative pressure of –10 kPa (left) and –20 kPa (right). Scale bar: 2 cm. (d) Photograph of the 12 × 12 pixel sensor array without an air gap and the corresponding contour plot of relative change in capacitance under a negative pressure of –10 kPa (left) and –20 kPa (right). Scale bar: 2 cm. 56 For the mapping of negative pressure, the sensor arrays with and without air gap channels (H = 0.5 mm, W = 1.2 mm, N = 12) exhibit drastically different results. Figure 3.4-c presents the pressure mapping results measured by the sensor array with air gap channels under –10 kPa and –20 kPa of pressure, where the maximum capacitance change reaches –4.1 % and –9.7 %, respectively. The contours also clearly show the profile of the rim of the suction cup, which is under positive pressure, and the area inside the suction cup, which is under negative pressure. In contrast, for the data collected from the sensor array without air gap (Figure 3.4-d), the negative pressure region is indiscernible. The results highlight the significance of incorporating air gap channels in the dielectric layer of the sensor for negative pressure sensing applications. 3.5 Challenges in Detecting Sea Lamprey Attachment Underwater The 12×12 pixel sensor array with air gaps has demonstrated its capability to measuring both positive and negative pressures when tested in air with those plastic letter blocks and PDMS elastomer suction cup. And we have extended this sensor array to 12×12 pixels with all the air gaps connected to the atmosphere through those tiny soft tubes. However, there are some challenges for the soft capacitive pressure sensor for detecting the attachment of sea lampreys underwater. From our experimental studies, it is found that capacive sensors are not only sensitive to pressures in contact mode, but also susceptible to conductive objects in non-contact mode such as proximity by a finger or a hand, which could change the electric field at the adjacent pixels and introduce unexpected parasitic capacitance. Also, once the soft capacitive 57 pressure sensor is deployed in water, the pixels will be surrounded all over by water, which is also conductive and will increase the capacitance measurments from the capacitive sensor. Finally, the desired relative capacitance change caused by actual contact/suction pressures of the sea lamprey might be attenuated by the combined influence of both water and the sea lamprey’s conductivity. As shown in Figure 3.5, three groups of experiments were conducted. In Figure 3.5-a,b, the soft pressure sensor was placed underwater on the wall of a water container, and the PDMS suction cup was attached onto the left bottom corner of the soft sensor array under –20 kPa negative pressure. We can get obvious mapping contour plot of relative change in capacitance in Figure 3.5-c which shows the correct suction area. But this setup did not work well for underwater sea lamprey detection when the sensor array was submerged underwater and clamped vertically against the wall of the water tank. As seen in Figure 3.5-f, no distinguishable pattern could be observed from this mapping contour. This experiment was conducted with several different sea lampreys, but the results were similar, with no apparent mapping contour obtained. It was conjectured that water and the animal introduced parasitic capacitance to each capacitor pixel and thus caused signal interference in the setup. On the other hand, we attempted to reduce the influence of the water for the soft capaci- tive sensor aray by fixing it above the water surface, but still made the sea lamprey attached onto it vertically, as shown in Figure 3.5-g,h. The capacitance were measurement before and after the lamprey’s suction in order to obtain the relative change in capacitance for this condition, and it is surprising to achieve the expected contour patterns under this setup. In summary, we have identified the challenges for soft capacitive pressure sensors for underwater sea lamprey detection, which require additional effort to address in the future. 58 Figure 3.5: Performance of the soft capacitive pressure sensor for underwater sea lamprey detection. (a) Photograph of the 16×16 pixel sensor array placed vertically on the wall of a water container for underwater experiment with a PDMS suction cup under –20 kPa. (b) Front view of the soft pressure sensor array with the PDMS suction cup attaching on the left bottom corner. (c) Corresponding mapping contour plots of relative change in capacitance of the sensor array under a –20 kPa from the PDMS suction cup. (d) Photograph of the soft pressure sensor array clamped vertically in water on the wall of a water tank with a sea lamprey attaching on it. (e) Back view of the soft pressure sensor array under suction of a sea lamprey. (f) Corresponding mapping contour plots of relative change in capacitance of the sensor array under the suction of the sea lamprey in (d). (g) Photograph of the soft pressure sensor array clamped horizontally out of water over the water tank with a sea lamprey attaching on it vertically. (h) Side view of the experiment setup in (g). (i) Corresponding mapping contour plots for experiment in (g). 59 3.6 Conclusion In this chapter, we have reported the design of a soft capacitive sensor that is capable of measuring both positive and negative pressures. The ability to detect negative pressure, which is rarely reported in the literature, is achieved in our work by designing air gap channels in the dielectric layer between the crossbar electrodes. The air gap channels enhance the deformation of the sensor and lead to significantly improved sensitivity especially for negative pressure. The influence of the air gap geometry on the sensitivity is also systematically studied through both single-pixel measurements and finite element simulation. Based on the experimental and simulation analysis of single pixel sensors, a 12 × 12 sensor array for spatial mapping of both positive and negative pressures is also demonstrated. With its convenient and low-cost fabrication process and repeatable response even when bent or stretched, our device may find a wide range of applications in soft robotics or wearable devices. On the other hand, there were some challenges for the soft capacitive pressure sensor for detecting the attachment of sea lampreys underwater, which were caused by the electromag- netic interference (EMI) of the water. Effective EMI shielding methods should be applied to the soft capacitive sensors before they can be deployed underwater for pressure sensing. 60 Chapter 4 Soft Piezoresistive Pressure Sensor for Underwater Sea Lamprey Detection In this chapter, an economical and effective soft pressure sensor is proposed, which consists of a layer of piezoresistive film matrix sandwiched between two layers of perpendicular copper tape electrodes. With multiplexers, the apparent resistance corresponding to each pixel of the sensing matrix can be measured directly, where the pixel is identified with the row and the column of the respective electrodes. However, this measured two-point resistance is not equal to the actual cell resistance for that pixel due to the crosstalk effect of the pixels. Since the cell resistance reflects directly the pressure applied on each pixel, the relationship between the cell resistance and the measured two-point resistance is analyzed, and more importantly, several regularized least-squares algorithms are put forward to reconstruct the cell resistance map from the two-point resistance measurements. The proposed pressure sensor is applied to detect the suction attachment of sea lampreys, a devastating invasive species in the Great Lakes region. Experimental results demonstrate that the pressure sensor can successfully capture the rim profile of the lamprey’s sucking mouth. Moreover, the performance and computational complexity of the reconstruction algorithms with different regularization functions are compared. The contents of this chapter first appeared as [1]. The remainder of the chapter is organized as follows. The fabrication and characteriza- 61 tion of the soft piezoresistive pressure sensors are first presented in Section 4.1. Then the modeling of the resistor network is discussed in Section 4.2. Several algorithms of the cell re- sistance reconstruction via least-squares regularization are proposed in Section 4.3, followed by experimental animals and setup introduced in Section 4.4. Experimental results and com- parison between different methods are discussed in Section 4.5. Finally, concluding remarks are provided in Section 4.6. 4.1 Sensor Design and Fabrication 4.1.1 Working Principle and Characterization To fabricate piezoresistive pressure sensor devices, the force-sensitive conductive film 1700 series (SCS company) is used in this work. This film is opaque, volume-conductive carbon- impregnated polyolefin, which has a thickness of 102 µm and a volume resistivity of less than 500 ohm·cm. Since the conductive carbon nanoparticles are embedded in the non- conductive polyolefin polymers, as shown in Figure 4.1-a, the film exhibits a high resistance in the initial state. When the film is under external compressive force or pressure (Figure 4.1-b), the carbon nanoparticles will get closer, which results in a lower resistance. The resistance change directly reflects the magnitude of the external compressive pressure, and this property can be used for piezoresistive pressure sensing. To start from a single-pixel soft pressure sensor, a 6 mm×6 mm piezoresistive film is between two cross-bar copper tape electrodes (100 mm×3 mm×0.04 mm) with polyester tape and double-sided tapes for adhesion. Figure 4.1-c shows the soft sensor under a compressive pressure, while Figure 4.1-d shows the sensor under suction pressure via a suction cup. Two single-pixel pressure sensors were characterized with different loads and suction pressures, 62 with each pressure tested for three rounds individually. The response results were averaged, and the characterization curve of relative change in measured resistance ∆R/R0 vs. pressure P (–10∼235 kPa) is shown in Figure 4.1-e. When the compressive load reaches 235 kPa, the resistance decreased by 98%. On the other hand, when the suction pressure was set to –10 kPa, the resistance increased by about 654%, likely due to local delamination upon suction, although the delamination has been greatly mitigated by this structure design and fabrication method. 4.1.2 Sensor Structure and Fabrication Process The structure of the proposed conductive film-based pressure sensor (4-by-4 matrix for schematic illustration) is shown in Figure 4.2-a,b, where individual conductive film patches were distributed uniformly and sandwiched between two layers of copper tape electrodes. Note that in order to have reliable resistance measurement, a good and stable contact be- tween the surfaces of the conductive film patches and the copper tapes need to be guaranteed. In this work, we used double-sided acrylic tapes and one-sided polyester tapes to bond the layers. With more individual piezoresistive film patches embedded into the matrix, a larger resistor network with M rows and N columns of pixels will be formed, as shown in Figure 4.2-c, which will be discussed in the modeling section. It is observed that ∆R/R0 decreases linearly with the applied pressure in the low pressure region. The pressure sensitivity, S = δ(∆R/R0 )/δP , indicates the local slope in the response curve. The inset of Figure 4.1-e shows the variation of the sensitivity depending on the applied pressure: an S value of –0.192 kPa−1 between 0 and 2.5 kPa, which reduces to about –0.016 kPa−1 for pressure between 2.5 and 28 kPa. When the pressure is above 28 kPa, the relative change in resistance seems to be largely saturated and not to decrease 63 Figure 4.1: Working mechanism of the soft pressure sensor and characterization of a single- pixel sensor device. (a) The soft pressure sensor in the initial relaxed state, which shows an initial resistance of R0 between the top and bottom conductors, and (b) the pressure sensor under compressive pressure, showing a new resistance RP . (c) The soft pressure sensor with a piezoresistive film between two perpendicular copper tape electrodes under a compressive load, (d) the sensor under a suction cup to test negative pressure response, and (e) the average relative change in resistance (along with the standard deviation) of the single-pixel sensor versus pressure when tested on a flat substrate. 64 Figure 4.2: Performance of the soft pressure sensor device under different bending conditions. (a) The initial (unloaded) resistance of the single-pixel pressure sensor at different curvature radii. (b) The experimental setup for loading pressure on the sensor on curved surfaces. (c) Time-resolved measurements of the output signal for an applied pressure with three rounds of loading and unloading processes on a curved surface with radius 50 mm. (d) Pressure response comparison of the single-pixel sensor at different curvature radii with an applied pressure up to 40 kPa. 65 appreciably with pressure. To investigate the mechanical flexibility such as bending deformation of this soft sensor, we examined a single-pixel sensor’s resistance when the sensor device was bent. Figure 4.2-a shows the resistance of the sensor when it was bent and attached onto curvy surfaces. The initial (unloaded) resistance was maximum at on a flat surface (zero curvature with a value about 3.05 kΩ), and then decreased to about 1.03 kΩ, 560 Ω, and 350 Ω at a curvature of 20 m−1 , 30.3 m−1 , and 58.8 m−1 , respectively, which demonstrates the significant dependence of the initial resistance on the curvature. The reason for this change in the initial resistance is that larger curvature implies higher bending stress in the sensor device, which leads to greater compression between the electrodes and causes a drop in resistance. To shed light on the pressure response of the sensor device on curvy surfaces, time-resolved measurements were further conducted. As shown in Figure 4.2-b, a program-customized syringe pump (Legato 110, KD Scientific, Inc.) was used to apply an external pressure of up to about 40 kPa onto the bending sensor (effective pressure contact area of 3 mm×3 mm from the copper electrodes) attached on a pipe, where the pressure was calculated based on the measured contact force through a load cell (GS0-100, Transducer Techniques, LLC). Three cycles of loading and unloading processes were repeated with a period of approximately 18 s. Figure 4.2-c shows the relative change in the resistance, ∆R/R0 , of the sensor for the case with curvature radius of 50 mm, where, ∆R = R − R0 , R0 is the initial resistance at the bending status, and R is the new resistance under the external pressure. During these three rounds of tests, the sensor was repeatable and robust. Furthermore, for different curvature radii (50 mm, 33 mm, and 17 mm), the pressure response curves of the same sensor device are plotted in Figure 4.2-d for comparison. Clearly, the relative change in resistance exhibits maximal values at 40 kPa, achieving –94% when the sensor device is on the flat substrate, then it 66 reduces to –82%, –69%, and –81% on the curvy surface with a curvature radius of 50 mm, 33 mm and 17 mm, respectively. The maximum (absolute) change in the resistance output for these curvy cases drops since the initial resistance of the sensor under bending on the curvy surfaces is much smaller than that on the flat substrate. 4.1.3 Sensor Matrix Structure and Fabrication Process The structure of the proposed conductive film-based pressure sensor (4-by-4 matrix for schematic illustration) is shown in Figure 4.3, where individual conductive film patches were distributed uniformly and encapsulated between two layers of copper tape electrodes. Note that in order to have reliable resistance measurement, a good and stable contact between the surfaces of the conductive film patches and the copper tapes needs to be guaranteed. In this work, we used double-sided acrylic tapes and one-sided polyester tape to bond the layers. With more individual piezoresistive film patches embedded into the matrix, a larger resis- tor network with M rows and N columns of pixels will be formed, which will be discussed in the modeling section. Figure 4.4 shows the fabrication process for a 10×10 pressure-sensing matrix with a sens- ing area of 10×10 cm2 . First, 10 pieces of 15 cm×3 mm×0.04 mm (length×width×thickness) copper foil tapes and 11 pieces of 15 cm×6.3 mm×0.04 mm (length×width×thickness) double-sided acrylic tapes were adhered side by side in an alternating manner onto a 300 mm×300 mm×3 mm acrylic plate; each copper tape has two double-sided tapes bordering on both sides. Then the conductive piezoresistive film was cut into one hundred pieces of square patches (each measuring 6 mm×6 mm), which were placed uniformly on the copper tapes as individual piezoresistive sensors. Here, the copper tapes would work as the col- umn electrodes with the double-sided acrylic tapes serving two purposes: filling the space 67 Figure 4.3: Schematic of a 4-by-4 piezoresistive pressure sensing matrix. (a) The bonding status of all layers, and (b) the exploded view. between the copper tapes (thus making the entire bottom layer flat) and fixing the edges of the conductive film patches (which was why the patch was wider than the copper tape). The shiny and non-adhesive surfaces of all the copper tapes were exposed outwards in order to contact the conductive film patches since the adhesive side of the copper tape was not prominently conductive. Similarly, another 10 pieces of copper foil tapes and 11 pieces of double-sided acrylic tapes were attached onto the adhesive side of a 10 cm×20 cm polyester tape, which would work as the top layer of the pressure sensing panel. Then the top layer was rotated by 90◦ C and put upside down to attach onto the bottom layer, with the conductive film patches between the top and bottom layers of copper tape electrodes. These two layers of copper electrodes would serve as the address lines of the sensing panel. The panel was then pressed with caution in order to form a stable bonding around each pixel between the adhesive layers. After that, each copper tape was connected with a jumper wire by soldering as the circuit extension for 68 Figure 4.4: Fabrication process of the 10-by-10 soft pressure sensing matrix. The paper liners of the double-sided tapes were not peeled off in the top left and bottom left pictures, but were peeled off in the following steps. Picture of the final fabricated 10-by-10 pressure sensing panel with PDMS (polydimethylsiloxane) waterproof encapsulation. The red dashed lines show the edges of the mold formed by 3M VHB 4905 tapes, while the white bounding box titled “PDMS” shows the PDMS encapsulation layer between the inner and outer 3M VHB 4905 tape boundaries. 69 measurements. Finally, in order to be able to deploy the pressure sensing panel underwater, waterproof encapsulation by polydimethylsiloxane (PDMS, with a 10:1 wt.% mixing ratio of PDMS base: curing agent) was achieved around the sensing panel, where the red dash lines show the edges of the 3M VHB 4905 double-sided tapes (3 layers bonded together, with a thickness of 1 mm for each layer) attached on the panel which were used to form a mold for the PDMS liquid before curing. 4.2 Modeling of the Resistor Network 4.2.1 The 2D Resistor Network For the M -by-N 2D resistor network shown in Figure 4.5, two multiplexers are used to select the column and the row to form the circuit for a given “pixel”. By using a voltage divider with a reference resistor Rref , the resistance measurement Rjk between the selected j th row and k th column can be calculated as: Vout Rjk = R (4.1) Vcc − Vout ref Note, however, that the measured two-point resistance Rjk is not equal to the cell resis- tance rjk at that pixel (j, k) due to crosstalk; in particular, Rjk is theoretically smaller than rjk since it is a parallel connection between rjk and a network of resistors between row j and column k. For instance, if row 1 and column 1 are selected by the multiplexers, the current would be injected from node V 1 to V1 through cell resistor r11 and other branches; for ex- ample, the current could flow from node V 1 to V2 through r12 , then to V 2 through r22 , and finally back to V1 through r21 . With larger dimensions of the network, there will be more 70 Figure 4.5: Schematic of the M -by-N resistor network and the voltage-divider circuits for resistance measurement. circuit loops involved between the selected row and column. 4.2.2 Mapping Contours based on Measured Resistance With the fabricated 10×10 soft pressure sensor array, using two 16-channel multiplexers (SparkFun CD74HC4067) and a 1k ohm reference resistor, the two-point resistance between each row and each column could be measured directly through the voltage divider circuit given in Figure 4.5. A series of experiments were conducted on the 10×10 soft pressure sensor array, such as the loading of an aluminum rod (Figure 4.6-a), the loading of weight through a 3D-printed ring part (Figure 4.6-b), the suction and attachment of a suction cup under different negative pressures in air (Figure 4.6-c,d), and also the suction cup experiments 71 with the soft pressure sensor matrix under water in a tank (Figure 4.6-e,f). All the mapping contours of relative change in directly measured resistance are shown side-by-side with the corresponding experimental picture, which demonstrates that this soft pressure sensor can successfully detect multiple kinds of pressure patterns. 4.2.3 Formulation of the Forward Problem It is of interest to find the relation between the cell resistance values {rjk } and the measured resistance values {Rjk }, which is needed in the reconstruction algorithms. To derive this relationship, nodal analysis or the branch current method is used in this work. In nodal analysis one equation is given at each node, requiring that the branch currents incident at a node must sum to zero based on the Kirchhoff’s current law (KCL). Once the branch currents are expressed in terms of the circuit node voltages, the conductance between any two nodes could be discovered. In general, for the M × N resistor network in Figure 4.5, if the voltage source is replaced with a current source, M voltage nodes for the rows and N voltage nodes for the columns can be studied; correspondingly, (M + N ) current sources (including possibly zero current) would be present at these (M + N ) nodes. According to KCL, the node-voltage equations can be written in a matrix form as: LV = I (4.2) 72 Figure 4.6: Mapping contours of the soft pressure sensing matrix based on relative change in directly measured resistance with the following experimental conditions: (a) a ϕ40 mm (ϕ represents diameter), 680 g aluminum rod was loaded on the sensing matrix, (b) a ϕ27 mm × ϕ35 mm × 5mm 3D printed ring part under a 850 g aluminum rod was loaded on the sensing matrix, (c) –10 kPa and (d) –20 kPa, respectively, negative pressure was applied on the sensing matrix via a ϕ27 mm × ϕ35 mm PDMS suction cup in air, and (e) –10 kPa and (f) –20 kPa, respectively, negative pressure was applied on the sensing matrix via the same suction cup under water, where the top row of copper tape electrode of the soft pressure sensor was about 7 cm lower than the water level. 73    C1,1 ··· C1,M C11 ··· C1N    C21 N    C2,1  ··· C2,M ··· C2    .. .. .. .. .. ..     . . . . . .      · · · CM,M 1 ··· N CM,1 CM CM    L=    (4.3)  C1 ··· CM 1 C 1,1 ··· C 1,N   1       C2 ··· CM 2 C 2,1 ··· C 2,N   1     .. .. .. .. .. ..   .  . . . . .     C1N ··· CM N C N,1 · · · C N,N      V1   I1           V2   I2       ..   ..       .   .           VM  I      and V =  ,I =  M (4.4)     V1  I1              V2  I2           ..   ..   .   .          V N IN where, L(M +N )(M +N ) is the Laplacian matrix of the M × N resistor network, V is the voltage pattern, and I is the current pattern. Cj,j is the sum of the conductance between the row node Vj and any other node; C k,k is the sum of the conductance between the column node V k and any other node; Cjk is the negative of the sum of the conductance between the 74 row node Vj and the column node V k ; Cj,h = 0, where 1 ≤ j ̸= h ≤ M , is the conductance between row j and row h; and C k,l = 0, where 1 ≤ k ̸= l ≤ N , is the conductance between column k and column l, since the rows are not connected directly with each other and neither are the columns. L is singular since the sum of all rows of L is equal to 0, which means these (M +N ) equations are not independent [78]. To remove the redundant equation, the first row node can be chosen as the ground (zero voltage reference), V1 = 0, and the first equation in Equation (4.2) can be eliminated. Then a new cofactor matrix with a reduced dimension of (M + N − 1) × (M + N − 1) along with (M + N − 1) independent equations can be obtained from the Laplacian matrix, and Equation (4.2) is reduced to CV = I (4.5) 75 where,    C2,2 · · · C2,M C21 ··· C2N      C3,2 · · · C3,M C31 ··· C3  N     .. ... .. .. ... ..     . . . .      1 N  CM,2 · · · CM,M CM ··· CM   C=    C1 · · · C1 C 1,1 ··· C 1,N   2 M       C2 · · · C2 C 2,1 ··· C 2,N   2 M     .. .. .. .. .. ..   .  . . . . .    C2N · · · CM N C N,1 · · · C N,N (4.6)   N j –g21 –g2N  P  g2 0 0 ··· j=1      0 ... .. ... ..   0 . .    N   P j 1 N   0 0 gM –gM · · · –gM  j=1   =   M   –g21 · · · –gM 1 1 P gi 0 0      i=1    . ... .. ...  ..   . 0 0    M   N –g2 · · · –gM N 0 0 P N gi  i=1 76      V2   I2           V3   I3       ..   ..       .   .           VM  I      and V =  ,I =  M (4.7)     V1  I1              V 2  I2           ..   ..   .   .          VN IN 1 Here, C is non-singular, and gjk = k is the conductance of the cell resistor rjk . One can then rj obtain V = C−1 I (4.8) If all cell resistances {rjk } are known, the co-factor matrix C is available and so is its in- verse. The current pattern I can be specified in this way: for the current loop between the studied row node Vj and the column node V k , since the current source noted as i is in- jected into the column node V k , the corresponding current element I k = i; and since the current is withdrawn from the row node Vj to the ground, the corresponding current el- ement Ij = −i; and all the other row and column nodes have zero current sources. For instance, if the column node V 1 (flow in) and the row node V2 (flow out) are the two points  T 1 to measure the resistance, the current pattern I = I2 I3 · · · IM I I · · · I 2 N =  T 1 −i 0 · · · 0 i 0 · · · 0 . If V (flow in) and V1 (flow out) are the two points to 77  T measure the resistance, then the current pattern I = 0 0 · · · 0 i 0 · · · 0 . Based on Equation (4.8), the voltages at all the nodes can be expressed in terms of the current i, and thus according to Ohm’s Law, the two-point resistance Rjk between the studied row Vj and column V k can be solved as: V k − Vj Rjk = (4.9) i With Equations (4.5)-(4.8), there exists an implicit function f (·) mapping from the cell     resistance matrix r = rk to the measured two-point resistance matrix R = Rk : j j R = f (r) (4.10) The algorithm for computing R using r is given in Figure 4.7. Note that in reality the measured two-point resistance matrix, Rm , is not exactly equal to R as calculated in Equation (4.10), due to modeling errors and measurement noises. Although Equation (4.5) is linear in the cell conductance, the mapping from the cell conductance to the cell resistance is reciprocal and nonlinear. And since Cj,j and C k,k are the sums of the conductance connected to the same row or column node, respectively, Equation (4.10) for the forward problem is nonlinear and implicit. In the next section we present algorithms for solving the inverse problem. 78 Figure 4.7: Forward mapping algorithm for computing the two-point resistance using the cell resistance. 4.3 Cell Resistance Reconstruction via Least-Squares Regularization As discussed in the previous section, the forward problem from the cell resistance matrix r to the measured resistance matrix R is relatively straightforward. However, the inverse problem, which is reconstructing the cell resistance r based on the measured two-point resistance Rm , is much harder and does not admit an analytical solution. Consequently, numerical methods have to be used. We first present the basic least-squares algorithm, and then describe four regularized least-squares algorithms with different regularization functions that aim to enhance the robustness of the reconstruction in the presence of measurement 79 noises and modeling errors. 4.3.1 Least-Squares Minimization (LSM) The inverse problem for the resistive network can be formulated as an optimization problem where the cost function to be minimized is the sum of squared residuals between the mea- sured two-point resistances Rm and the calculated R based on Equation (4.10), with the requirement that the cell resistance is larger than or equal to the measured resistance: M,N 2 f (r)kj − (Rm )kj X r̂ = arg min (4.11) r j=1,k=1 s.t. rjk ≥ (Rm )kj for all j and k (4.12) where rjk is the cell resistance element at the pixel (j, k) while (Rm )kj is the corresponding measured two-point resistance. This least-squares problem is solved in MATLAB via the nonlinear least-square solver “lsqnonlin”, which starts at an initial guess r0 ≥ Rm (where “≥” holds true element- wise). The default algorithm for this solver is the trust-region-reflective algorithm based on the interior-reflective Newton method described in [116], which approximates the objective function by the first two terms of the Taylor-series approximation, restricts the trust-region subproblem to a two-dimensional subspace, and chooses the solver step to force global con- vergence via the gradient descent while achieving fast local convergence via the Newton step if it exists. The complete algorithm for this reconstruction method is given in Figure 4.8. 80 Figure 4.8: Reconstruction algorithm for computing the cell resistance using the measured two-point resistance based on least-squares minimization (LSM). 4.3.2 Least-Squares Regularization on Cell Resistance (LSR-CR) The resistor network inverse problem suffers from its ill-posed nature; in particular, the numerical inverse solution depends sensitively on the input data and thus its performance is susceptible to measurement noises and modeling uncertainties. In order to reconstruct the cell resistance robustly and to give preference to particular solutions with desirable properties, the Tikhonov regularization technique is exploited, where a regularization term is included in the least squares minimization. One of the typical a priori regularization terms is the L2 regularization, λ∥r∥22 , which is the sum of the squares of all elements from the inverse solution with a penalty weight λ that penalizes large cell resistance values: M,N 2 { f (r)kj − (Rm )kj + λ∥rjk ∥2 } X r̂ = arg min (4.13) r j=1,k=1 s.t. rjk ≥ (Rm )kj for all j and k. (4.14) 81 Figure 4.9: Reconstruction algorithm for computing the cell resistance using the measured two-point resistance based on least-squares regularization on cell resistance (LSR-CR). where λ ≥ 0 is the regularization (or penalty) parameter, which determines the trade-off between the modeling discrepancy term and the regularization term. The regularization method in Equation (4.13) accommodates simultaneously the norm of the residual [f (r) − Rm ] and the norm of the approximate solution r, enforcing the a priori knowledge on solving the cell resistance, and improving the smoothness of the solution. The complete algorithm for this reconstruction method is given in Figure 4.9. 4.3.3 Least-Squares Regularization on Relative Change in Cell Resistance (LSR-∆CR) Different sensor pixels might have quite different cell resistances in the initial relaxed state before a pressure is applied, due to, for example, imperfect fabrication processes. So, an alternative regularization function would be the relative change in the cell resistance values, 82 instead of these values themselves: M,N 2 { f (r0 )kj − (Rm0 )kj X [r̂0 r̂1 ] = arg min + r0 r1 j=1,k=1 (4.15) 2 2 (r1 )kj − (r0 )kj f (r1 )kj − (Rm1 )kj +λ × 100 } (r0 )kj s.t. (r0 )kj ≥ (Rm0 )kj and (r1 )kj ≥ (Rm1 )kj for all j and k. (4.16) where (r0 )kj and (Rm0 )kj are the cell resistance and the measured two-point resistance cor- responding to the first group of measurements (e.g., prior to the application of the external pressure), while (r1 )kj and (Rm1 )kj are those corresponding to the second group of mea- surements (e.g., after the pressure is applied). The data 100 in the equation denotes the percentage calculation in order to get the relative change in cell resistance. The relative change in cell resistance is evaluated based on two consecutive cell resistance matrices. For the initialization step of this regularization, in order to calculate the relative change in cell resistance (in percentage), two groups of measured resistance Rm0 and Rm1 are required to be fed into Equation (4.15) at the beginning. Once the first two sets of cell resistance solutions r0 and r1 are solved jointly, r0 , Rm0 , and Rm1 are not used any more, while r1 is taken as the known new r0′ . The next set of measured resistance Rm2 will be used as the new Rm1 ′ , and Equation (4.15) will be replaced with a new regularization in order to find the corresponding solution r1′ for the new measurements: M,N 2 r̂1′ { f (r1′ )kj − (Rm1 ′ )k X = arg min j + ′ r1 j=1,k=1 (4.17) 2 (r1′ )kj − (r0′ )kj λ × 100 } (r0′ )kj 83 s.t. ′ )k for all j and k. (r1′ )kj ≥ (Rm1 (4.18) j The reconstruction will be initialized first and then be updated iteratively for the following steps. The complete algorithm for this method is given in Figure 4.10. 4.3.4 Least-Squares Regularization on Gradient of Cell Resistance (LSR-∇CR) We also consider using the cell resistance gradient as the regularization term to minimize spikes in the mapping contours. This method is captured as below: M,N 2 { f (r)kj − (Rm )kj + λ∥∇rjk ∥2 } X r̂ = arg min (4.19) r j=1,k=1 s.t. rjk ≥ (Rm )kj for all j and k. (4.20) The complete algorithm for this reconstruction method is given in Figure 4.11. The gradient can be calculated differently according to the location of the pixel. If the pixel is in the interior of the sensing matrix, the gradient components are approximated by the central difference between the neighboring pixels. If the pixel is on the boundary, the appropriate gradient components are calculated with single-sided differences. The gradient calculation steps are summarized in Figure 4.12. 84 Figure 4.10: Reconstruction algorithm for computing the cell resistance using the measured two-point resistance based on least-squares regularization on relative change in cell resistance (LSR-∆CR). 85 Figure 4.11: Reconstruction algorithm for computing the cell resistance using the measured two-point resistance based on least-squares regularization on gradient of cell resistance (LSR- ∇CR). 4.3.5 Least-Squares Regularization on Gradient of Relative Change in Cell Resistance (LSR-∇∆CR) Finally, we consider regularization based on the gradient of the relative change in cell resis- tance M,N 2 { f (r0 )kj − (Rm0 )kj X [r̂0 r̂1 ] = arg min + r r 0 1 j=1,k=1 (4.21) 2 2 (r1 )kj − (r0 )kj f (r1 )kj − (Rm1 )kj + λ ∇( × 100) } (r0 )kj s.t. (r0 )kj ≥ (Rm0 )kj and (r1 )kj ≥ (Rm1 )kj for all j and k. (4.22) where two consecutive sets of measured resistances Rm0 and Rm1 are required for initial- ization at the beginning, and the gradient of the relative change in cell resistance can be 86 Figure 4.12: Algorithm for calculating the gradient. calculated accordingly. The updating rule of this algorithm is similar to that in the re- construction method LSR-∆CR: first, solve r0 and r1 jointly; then, take r1 as the known new r0′ ; and next, take a third set of resistance measurement as the new Rm1 ′ , and the corresponding new cell resistance r1′ could be generated from the following regularization: M,N 2 r̂1′ { f (r1′ )kj − (Rm1 ′ )k X = arg min j + ′ r1 j=1,k=1 (4.23) 2 (r1′ )kj − (r0′ )kj λ ∇( × 100) } (r0′ )kj s.t. (r1′ )kj ≥ (Rm1 ′ )k for all j and k. j (4.24) The reconstruction will be updated iteratively with the new measurements coming in, 87 using the latest measurement as (Rm1 ) and using the previous solution as r0 in order to guarantee the solving process to be consecutive and consistent. The complete algorithm is given in Figure 4.13, which is similar with the algorithm in Figure 4.10 with both the initialization step and the following steps, the only difference is that the regularization terms are now the gradient of the relative change in cell resistance. 4.4 Experiments 4.4.1 Experimental Animals In August 2020, thirty spawning phase adult sea lampreys were tested on the 10-by-10 pressure sensing panel. These sea lampreys were captured in traps during upstream spawning migration in the St. Marys River (Michigan, USA and Ontario, Canada) during May-July 2020. Traps were operated by Canada Department of Fisheries and Oceans and the U. S. Fish and Wildlife Service. Lampreys were transported to the U. S. Geological Survey Great Lakes Science Center’s Hammond Bay Biological Station, Millersburg, Michigan, USA where they were held in aerated 1000 L tanks supplied continuously with Lake Huron water (salinity: 0 ppt, pH: 7-8) maintained at 8-12 ◦ C with a dissolved oxygen saturation of over 90 % in the sea lampreys until tests were conducted. Prior to tests, body weight, total body length, and mouth diameter were measured. All sea lamprey experiments were performed in accordance with protocols and guidelines approved by Michigan State University’s Institutional Animal Care and Use Committee (IACUC, No. 02/18-028-00). After the suction pressure experiments in this study, the sea lampreys were housed for use in further research by Hammond Bay Biological Station staff. 88 Figure 4.13: Reconstruction algorithm for computing the cell resistance using the measured two-point resistance based on least-squares regularization on gradient of relative change in cell resistance (LSR-∇∆CR). 89 Figure 4.14: Experimental setup of the pressure sensing system for sea lamprey detection.(a) The soft piezoresistive pressure sensing panel and the hardware attached on the back side of the panel, (b) the back view of the pressure sensing panel with an adult sea lamprey attaching on it, and (c) the front view of the pressure sensing panel with another adult sea lamprey attaching on it. 4.4.2 Experimental Setup As shown in Figure 4.14, the resistance of the pressure sensors at each pixel was measured by a voltage divider with a 1k ohm reference resistor. An Arduino Mega 2560 microcon- troller board provided a 5 V voltage supply for the pressure sensing circuits, and generated digital output signals for channel selection. Two analog/digital multiplexer breakout boards (SparkFun CD74HC4067, 16 channels) were used to choose the circuits between one column and one row of the perpendicular address lines. The output voltage on the reference resis- tor could be measured by a 10-bit Analog-to-Digital Converter (ADC) through the analog input. The experimental setup is shown in Figure 4.14-a, in the experimental water tank (200 L). The pressure sensing panel was placed vertically on the acrylic hanger along a glass wall of the water tank, while the Arduino Mega board and the voltage divider on the breadboard 90 were adhered on the other side of the hanger. The acrylic hanger was clamped on the water tank wall via two clamps. The water level in the tank was about 5 cm higher than the top row electrode of the 10-by-10 pressure sensing panel, submerging all the sensing area. 4.4.3 Experiment on Sea Lampreys with the Pressure Sensor In each round of measurement, the pressure sensing system scanned the pressure sensors from the top left corner (X=1, Y=1) to the bottom right corner (X=10, Y=10) by selecting the channels of the multiplexers. Resistance was measured consecutively for 20 times at each pressure sensor, and then the average was taken as the measured two-point resistance at that pixel for that time instance. The Arduino program would repeat the scanning and measurement process every one second (overall sampling rate: 1 Hz) in loops by means of timer interrupt. The resistance measurement data would be stored in the computer hard drive once the program was closed. Once the Arduino program started to run and measure the resistance periodically, an adult sea lamprey was transferred to the tank and allowed to explore the tank until it attached to the tank surface via oral suction. If the lamprey did not attach onto the sensing area, it would be gently repositioned and held with its mouth over the sensing area until it attached. The top surface of the sensing area was relatively smooth, and experiments showed that most of the tested sea lampreys were able to attach to this sensor for a certain time (e.g., > 20 s) after a few trials. As demonstrated in Figure 4.14-b,c, a sea lamprey was attached onto the central area of the sensing panel, with a region spanning almost 4 rows and 4 columns of copper tapes covered by the sea lamprey’s oral disc. Resistance measurement lasted until the lamprey volitionally detached from the panel or until the first 2 minutes of attachment elapsed. The measurement data would be processed to plot the 91 mapping contours of relative change in the measured resistance directly, or would be used to reconstruct the cell resistance first using one of the reconstruction methods proposed in Section IV, and then to plot the mapping contours of the relative change in cell resistance. 4.5 Results 4.5.1 Comparison between Different Methods To have a better understanding of all the methods explored above, mapping contours from these methods are displayed in the same panel in Figure 4.15. For each regularization method, a mapping contour with “best” choice of λ is selected (by “best”, we mean visually perceived best tradeoff between data matching and smoothing). Figure 4.15-a shows the mapping contour of the relative change in the measured resistance (between –82.6% and –1.8%), which is a baseline for all the other results. Figure 4.15-b is the result from least- squares minimization (LSM algorithm without regularization) with the relative change in cell resistance between –99.5% and 11873.9%, and the following four mapping contours are the results of relative change in cell resistance based on regularization on the cell resistance (LSR-CR algorithm, Figure 4.15-c, λ=0.001, between –94% and 71%), regularization on the relative change in cell resistance (LSR-∆CR algorithm, Figure 4.15-d, λ=10, between –99.9% and 185%), regularization on the gradient of cell resistance (LSR-∇CR algorithm, Figure 4.15-e, λ=0.001, between –97% and 59%), and lastly regularization on the gradient of relative change in cell resistance (LSR-∇∆CR algorithm, Figure 4.15-f, λ=10, between –99.9% and 155%), respectively. As observed above, with the same color bar range, (1) directly measured resistance change (Figure 4.15-a) is “blurry” as the measured resistance is related to the cell resistance through 92 Figure 4.15: Comparison between mapping contours from different methods. Mapping con- tours of (a) relative change in directly measured two-point resistance, (b) relative change in cell resistance from least-squares minimization (LSM), (c) relative change in cell resistance from least-squares regularization on the cell resistance (LSR-CR) with λ=0.001, (d) rela- tive change in cell resistance from least-squares regularization on the relative change in cell resistance (LSR-∆CR) with λ=10, (e) relative change in cell resistance from least-squares regularization on the gradient of cell resistance (LSR-∇CR) with λ=0.001, and (f) relative change in cell resistance from least-squares regularization on the gradient of relative change in cell resistance (LSR-∇∆R) with λ=10. 93 Table 4.1: Performance comparison of different reconstruction methods. Absolute Computation Relative Method Specifications Time Error (s) (%) LSM Least Squares Minimization 16.45 ± 0.40 1.33 ± 1.29 LSR-CR Regularization on Cell Resistance 10.20 ± 2.04 7.36 ± 7.85 Regularization on Relative Change in 58.63 1.19 ± 1.18 LSR-∆CR Cell Resistance Following Steps after Initialization 8.05 ± 0.88 1.48 ± 1.51 LSR-∇CR Regularization on Gradient of Cell Re- 11.46 ± 1.71 5.67 ± 5.28 sistance Regularization on Gradient of Relative 60.85 1.13 ± 1.13 LSR-∆∇CR Change in Cell Resistance Following Steps after Initialization 11.01 ± 0.96 1.48 ± 1.48 (Data are presented in the type of mean ± standard deviation.) a nonlinear filter. (2) Plain LSM (Figure 4.15-b) produces large spikes at some pixels out- side of the actual suction area, since this reconstruction method is susceptible to the effect of measurement noises and modeling errors. (3) LSR-CR (Figure 4.15-c) and LSR-∇CR (Fig- ure 4.15-e) produce more distinct patterns than directly measured resistance changes while showing pronounced smoothing effect. And (4) LSR-∆CR (Figure 4.15-d) and LSR-∇∆CR (Figure 4.15-f) produce the most distinct suction patterns with cell resistance decreased along the rim of the oral disc and with cell resistance increased within oral disc. In order to further compare the performance of different reconstruction methods, 21 consecutive sets of measured 10-by-10 two-point resistance matrices obtained during the sea lamprey test were used for running these algorithms in MATLAB R2020b on the laptop with a CPU of Intel i7-6700HQ (2.60 GHz) and a 16.0 GB RAM. The computation time and absolute relative error (in percentage) between the derived two-point resistance and the measured two-point resistance were calculated in the form of “mean ± standard deviation” and are listed in Table 4.1. 94 For the regularization methods LSR-∆CR and LSR-∇∆CR, the initialization step took 58.63 s and 60.85 s, respectively, while the following steps took only 8.05 ± 0.88 s and 11.01 ± 0.96. The reason for significantly longer computation time in the initialization step is because these two methods need to solve for both matrices r0 and r1 jointly. But for the steps after, the computation time dropped greatly while the absolute relative errors remained within a desirable range. On the other hand, the computation time for the method LSM was 16.45 ± 0.40 s, which was larger than the other methods like LSR-CR and LSR- ∇CR. Although it had a smaller absolute relative error, the mapping contour did not reflect a perfect visualization result given the noise and the displayed shape. The final decision of reconstruction methods will be a trade-off between the computational complexity, the relative error in data matching, and the smoothing effect. Note that the mapping contours of relative change in measured two-point resistance could still be used instantaneously in real-time lamprey attachment detection, which takes about 0.31 s computation time to plot the mapping contour for each round of new measurements in MATLAB using the surf (·) function. The reconstruction methods require some time to compute the cell resistance change and will be best for post-processing to gain further information about the detected animal. 4.5.2 Mapping Contour Comparison between Sea Lampreys with Large and Small Mouth Diameters For demonstration, the least-squares regularization method on the gradient of cell resistance (LSR-∇CR) with λ = 0.001 is chosen to further show the capability of the proposed sensor panel in capturing the demographic information of the detected lampreys. The mapping 95 contours of the 10-by-10 pressure sensing panel under suction and attachment of two different adult sea lampreys are shown in Figure 4.16. The first adult male sea lamprey had a mouth diameter of 35 mm, while the other adult male had a mouth diameter of 25 mm. From the figures, we can observe that the blue mapping contour for the larger mouth was covering a 4-by-4 grid area (6a-b), while the smaller one was covering a 3-by-3 grid area (6c-d), indicating the ability to successfully measure the size of the sea lamprey’s mouth attaching on the sensing panel. 4.6 Conclusion An effective sensing technique to autonomously detect and monitor sea lampreys will be of significant interest to the sea lamprey control effort in the Great Lakes and potentially to programs that seek to conserve or restore lampreys elsewhere throughout their native ranges. Motivated by this practical application, we developed a low-cost and efficient piezoresistive pressure sensor based on a passive resistor network and proposed new algorithms for properly processing the measured data to reconstruct the pressure pattern. In particular, in order to recover the cell resistance from the measured two-point resistance, we derived the general inverse mapping relationship based on basic Kirchhoff’s current law, and introduced several inverse algorithms based on the least-squares minimization and Tikhonov regularization. These approaches are novel and distinct from previous reports as our methods are general and applicable to a passive resistor network of any size, with the measurement noises and modelling uncertainties taken into consideration. The approaches were validated with results from experiments with live sea lampreys. The pros and cons of the different reconstruction methods were discussed in depth. While the sensing system was motivated by the sea 96 Figure 4.16: Comparison of the mapping contours between sea lampreys with large and small mouth diameters. (a) An adult sea lamprey attaching on the sensing panel with a mouth diameter of about 35 mm, and (b) the mapping contour corresponding to the attachment condition in (a) using the LSR-∇CR method. (c) Another smaller adult sea lamprey attaching on the sensing panel with a mouth diameter of about 25 mm, and (d) the mapping contour corresponding to the attachment condition in (c) using the LSR-∇CR method. 97 lamprey detection problem, it is applicable to other applications in soft robotics, wearable electronics, biomonitoring, and human-machine interfaces. The choice of the value of the regularization parameter λ in this paper was determined by trying a few values in different orders of magnitude. While more principled methods of choosing the λ value are available in the literature, such as the Morozov discrepancy principle [83], and the ordinary cross-validation criteria [84], these methods are mostly applicable for linear models. Developing a more systematic approach to choosing the regularization parameter remains a direction for our future work. In addition, we will explore the refinement of the fabrication methods to improve both spatial resolution and scalability. Scalability is important for practical deployment of the sensing panel in detecting sea lampreys in fish passages or other natural environments. For that purpose, we will investigate approaches to integration of modular, elementary panels into larger panels (up to the size of 1 m × 1 m). We will also examine data analytics algorithms for automated recognition of suction patterns (instead of relying on human recognition). Finally, the developed pressure sensor is encapsulated and waterproofed; as such, envi- ronmental factors such as the pH value, oxygen saturation, and conductivity of the water are not expected to affect the outputs of the sensor. However, some other factors, such as the water temperature and the depth-induced hydrostatic pressure could have an impact on the sensor outputs. We will conduct further animal experiments to characterize the potential dependence of the sensor outputs on water temperatures and sensor deployment depths, and if needed, we will develop corresponding compensation algorithms to counter the influence of these environmental variables. 98 Chapter 5 Automated Soft Pressure Sensor Array-based Sea Lamprey Detection With Machine Learning As discussed in the previous chapter, attachment by suction has been studied in soft pressure sensors for sea lamprey detection. However, human decision is still required for recognition of patterns in the measured signals. In this chapter, an automated soft pressure sensor array-based sea lamprey detection approach is proposed using object detection convolutional neural networks (CNNs). First, the two-point resistance measurements of the 10-by-10 pressure sensor array are converted to mapping contour plots of relative change in resistance. Then, a comprehensive sea lamprey dataset of mapping contours from both compression and suction patterns is collected for machine learning. Three types of object detection algorithms are applied to the sea lamprey dataset, and comparison of their performance shows that YOLOv5s model achieves the highest mean average precision and the fastest inference speed. Furthermore, to improve the accuracy of the prediction model and reduce false positive rate due to the sensor’s memory effect, a postprocessing unit with two designed confidence thresholds for the compression pattern and suction pattern, respectively, is added to the original machine learning algorithm. The trained model is validated and used to 99 automatically detect sea lamprey attachment and locate the suction area on the sensor in real time. The remainder of this chapter is structured as follows. Section 5.1 introduces the exper- imental animals and the setup with the soft pressure sensor array. Section 5.2 presents the sea lamprey dataset with its image annotation formats, and shows the assessment results of three object detectors in order to find the best one. Then the sea lamprey detection approach with a postprocessing unit is proposed in Section 5.3. Experimental results for evaluating the postprocessing performance are presented in Section 5.4. Concluding remarks are provided in Section 5.5. 5.1 Experimental Animals and Setup 5.1.1 Experimental Animals In September 2021 and June 2022, 140 spawning phase adult sea lampreys were tested on the 10-by-10 pressure sensing panel. These sea lampreys were captured in traps during upstream spawning migration in the St. Marys River (Michigan, USA and Ontario, Canada). Lampreys were transported to Hammond Bay Biological Station of the U. S. Geological Survey Great Lakes Science Center at Millersburg, Michigan, where they were held in aer- ated 1000 L tanks supplied continuously with Lake Huron water maintained at 8-12 ◦ C until tests were conducted. All sea lamprey experiments were performed in accordance with pro- tocols and guidelines approved by Michigan State University’s Institutional Animal Care and Use Committee (IACUC, No. 02/18-028-00, and AMEND202200009 / PROTO202100177). After the experiments in this study, the sea lampreys were housed for use in further research by Hammond Bay Biological Station staff. 100 5.1.2 Soft Pressure Sensor Array As shown in Figure 5.1-a, this work uses a 10-by-10 soft pressure sensor array (with a sensing area of 10×10 cm2 ) that is made of piezoresistive films sandwiched between two layers of perpendicular copper tape electrodes, with polyester tape encapsulated on an acrylic plate. The detailed fabrication process is introduced in Chapter 4. The sensor array forms a resistor network, with its circuits illustrated in Figure 5.1-b. At each sensor pixel, when a compressive pressure load is applied (e.g., under the compres- sion of the lamprey mouth rim), the resistance at that pixel reduces, resulting in a reduction in the corresponding measured resistance via the coupling of the resistor network. Similarly, when a partial vacuum pressure (e.g., under the suction of the lamprey mouth) is applied on a sensor pixel, there will be a rise in the resistance measurement. It is observed that, likely due to the viscoelasticity of the films and their bonding, the resistance measurements do not immediately return to the at-rest values following the removal of the attachment. This memory effect, which would cause false positives in the detection, is explicitly addressed in the detection algorithm design. 5.1.3 Experimental Setup As shown in Figure 5.1-a, a voltage divider with a 1k ohm reference resistor (Rref ) was used to measure the resistance of the pressure sensors at each pixel. An Arduino Mega 2560 micro-controller board provided a 5 V voltage supply (Vcc ) for the pressure sensing circuits, and generated digital output signals for channel selection. Two analog/digital multiplexer breakout boards (SparkFun CD74HC4067, 16 channels) were used to choose the circuits between one column and one row of the perpendicular address lines. The output voltage 101 Figure 5.1: Hardware of the soft pressure sensor array-based sea lamprey detection system [1]. (a) Experimental setup, and (b) circuit model of the resistor network formed by the pressure sensor array. (Vout ) on the selected resistor network circuits was measured by a 10-bit Analog-to-Digital Converter (ADC) through the analog input. The two-point resistance measurement Rjk between the selected j-th row and k-th column can be calculated as: Vout Rjk = R (5.1) Vcc − Vout ref The pressure sensing panel was placed vertically along a glass wall of a 200 L water tank, while the micro-controller board and all other circuits were outside of the tank. The water level in the tank was about 5 cm higher than the top row electrode of the pressure sensing panel, submerging all the sensing area. 102 In each round of measurement, the pressure sensing system scanned the sensor array from the top left corner to the bottom right corner. Resistance was measured consecutively for 20 times at each pressure sensor, and then the average was taken as the measured two-point resistance at that pixel for that sampling cycle. The Arduino program repeated the scanning and measurement process every one second (1 Hz) in loops by means of timer interrupt. The resistance measurement data were transferred to a Python program on a computer via serial communication, and then the data would be stored as matrices in a spreadsheet file in the hard drive. Meanwhile, the relative change (in %) in the resistance matrix between the current sampling time and the initial value were calculated and converted to a mapping contour plot, which was also stored in the hard drive. Once the Arduino program started to run and measure the resistance periodically, an adult sea lamprey was transferred to the tank and introduced to attach onto the sensing area for a certain time (e.g., > 20 s). Resistance measurement lasted until the lamprey detached from the panel by itself or until the first 2 minutes of attachment elapsed. 5.2 Training Models on Sea Lamprey Datasets This section first introduces the dataset collected from the sea lamprey experiments on the soft pressure sensor array, which are mapping contour plots converted from the resistance measurements. They can be categorized into either a “compression” pattern or a “suction” pattern. We present the image annotation formats for three object detection models: SSD, RetinaNet and YOLOv5, and further implement the training and validation processes on each machine learning model in order to find the best sea lamprey detector. 103 5.2.1 Mapping Contour Patterns In this work, a total of 3,094 colored mapping contour plots generated during the sea lamprey attachment periods were collected from 120 groups of sea lamprey experiments, which were annotated with bounding box labels for training and validating the neural networks. Each of these selected mapping contours had a resolution of 640×640 pixels, and could be categorized into either “compression” pattern or “suction” pattern based on its overall appearance and contour levels. There were 623 compression plots and 2,471 suction plots, and eight typical mapping contour plots are shown in Figure 5.2, including four compression patterns (Figure 5.2-a-d) and four suction patterns (Figure 5.2-e-h). For instance, the compression pattern can be partial edges or discrete points in blue (Figure 5.2-a) reflecting non-uniform compression of the lamprey’s suction disc on the sensor array, a full circular pattern in blue (Figure 5.2-b), an arc in blue on the boundary (Figure 5.2-c), or a corrupted circular pattern connected to adjacent rows or columns (Figure 5.2-d) due to crosstalk of the sensor array. Similarly, the suction patterns are typically complementary to the compression patterns, which appear in red or orange blobs. Note that, when a mapping contour plot displayed both a compression pattern and a suction pattern, such as Figure 5.2-a, it would still be categorized into only one pattern with the higher magnitude in absolute relative change in resistance. The annotated mapping contour dataset was then split into training and validation subsets with a ratio of 8 : 2. On the other hand, a total of 3,875 mapping contours obtained from the remaining 20 groups out of the whole 140 sea lamprey experiments were used to test the trained model with a postprocessing filter in order to decide the optimal confidence thresholds for the compression pattern and suction pattern, respectively. 104 Figure 5.2: Typical mapping contour plots of the 10-by-10 soft pressure sensor array with attachment of an adult sea lamprey. Typical compression patterns: (a) with partial edges or points shown in blue reflecting compression of the lamprey’s suction disc, (b) with an intact circular pattern shown in blue, (c) with the compression area on the boundary of the sensing area, and (d) with a corrupted circular pattern connected to adjacent rows or columns due to circuit crosstalk and mechanical stress of the sensor array. Typical suction patterns: (e) with a triangular blob shown in red reflecting the suction area, (f) with an intact round (or octagon) blob shown in red, (g) with the suction area on the boundary of the sensing area, and (h) with a corrupted polygon pattern extended to adjacent rows or columns due to circuit crosstalk and mechanical stress of the sensor array. 5.2.2 Image Annotation Figure 5.3 shows an example of the annotation of the ground truth bounding box on a suction pattern mapping contour. The coordinates of the ground truth bounding box was obtained from the experimental videos synchronized with the pressure sensor measurements as follows. During the experiments, a cellphone camera was used to record activities on the whole sensor array. The mapping contour plots in a time sequence from a lamprey experiment were converted to an animation video. The animation contour video was then synchronized with the recorded experimental video. The video frames were extracted from 105 Figure 5.3: Illustration of the ground truth bounding box annotation. (a) image of the 10-by-10 soft pressure sensor array under suction of an adult sea lamprey (as shown in the inset), with the bounding box shown in yellow dashed lines covering the attachment area, and with the row and column axis labelled for dataset annotation. (b) The corresponding mapping contour plot of relative change in measured two-point resistance, with the ground truth bounding box in red solid lines depicted on it. the synchronized experimental video every one second, same frame rate as that for the mapping contour animation video. Finally, the coordinates of the top left vertex (Colmin , Rowmin ) and the bottom right vertex (Colmax , Rowmax ) of the ground truth bounding box were estimated with one decimal point between the boundary limits of 1.0 and 10.0. Different object detectors may accept different formats of bounding box labels. The RetinaNet framework uses (class, xmin , ymin , xmax , ymax ) as its label format, where class is either 0 or 1, which represents “compression” or “suction” pattern, respectively; (xmin , ymin ) denotes the pixel coordinates of the top left vertex, and (xmax , ymax ) denotes those of the bottom right vertex, which can be obtained from the row and column coordinates: 106 Table 5.1: Parameters for generating the mapping contour plots. Name Variable Value Figure width F igw 640 Figure height F igh 640 Ratio of contour width to figure width rw 0.9 Ratio of contour height to figure height rh 0.9 Ratio of contour left margin to figure width rlm 0.05 Ratio of contour top margin to figure height rtm 0.05 Colormap style cmap ‘jet’ Number of contour levels Nlevel 100 Colorbar min limit vmin -100 Colorbar max limit vmax 100 Colmin − 1 xmin = ( · rw + rlm ) · F igw (5.2) 10 − 1 Colmax − 1 xmax = ( · rw + rlm ) · F igw (5.3) 10 − 1 Rowmin − 1 ymin = ( · rh + rtm ) · F igh (5.4) 10 − 1 Rowmax − 1 ymax = ( · rh + rtm ) · F igh (5.5) 10 − 1 where the meanings of the parameters can be found in Table 5.1. On the other hand, in addition to the class label, the SSD and YOLOv5 object detection models take the normalized coordinates of the bounding box center (xcenter , ycenter ), and the normalized width wbbox and height hbbox of the bounding box as accepted labels, and the formulas are given below: 107 Colmin + Colmax −1 xcenter = ( 2 · rw + rlm ) (5.6) 10 − 1 Rowmin + Rowmax −1 ycenter = ( 2 · rh + rtm ) (5.7) 10 − 1 Colmax − Colmin − 1 wbbox = · rw (5.8) 10 − 1 Rowmax − Rowmin − 1 hbbox = · rh (5.9) 10 − 1 5.2.3 Assessment of Three Object Detectors The object detection frameworks, SSD, RetinaNet, and YOLOv5s, were evaluated for sea lamprey attachment pattern detection using the same training and validation datasets. The network hyperparameters for the training process, including the initial learning rate, momen- tum, weight decay, a minimum score threshold, and the non-maximum suppression (NMS) [117] threshold, are listed in Table 5.2. Here, the score threshold is a first filtering step to remove the very unlikely bounding boxes, while the NMS threshold is an evaluation metric to compare one candidate bounding box with multiple other bounding box candidates. The NMS is realized in this way: if they mutually share an IoU larger than the NMS threshold, then these bounding boxes could be merged into only one box with the maximum confidence score. The training and validation process was implemented on the desktop PC with a GPU of NVIDIA GeForce RTX 3060 Ti (1.69 GHz boost clock) and a 32.0 GB RAM. The input 108 Table 5.2: Hyperparameters for training the sea lamprey detection networks. Parameter Value Learning rate 0.005 Momentum 0.9 Weight decay 0.0005 Score threshold 0.05 NMS threshold 0.5 image of the mapping contours has a size of 640×640, without colorbar plotted in it. To quantitatively evaluate the validation performance, mean average precision (mAP ) is used, which is related to other performance metrics such as true positive (TP), false positive (FP), true negative (TN), false negative (FN), precision (P ), recall (R), and average precision (AP ). To decide whether a prediction is a TP or FP, the IoU between the predicted and g−t g−t ground-truth bounding boxes, IoUpred , was calculated. If IoUpred ⩾ IoU , then it is a TP, which means that the prediction as positive is correct; otherwise, it is regarded as an FP, meaning there was no object at that predicted place. Besides, FN means failing to predict an object that was actually there, and TN means the prediction as negative was true and there was indeed no object there. The corresponding true positive rate, false positive rate, true negative rate, and false negative rate are denoted as T P R, F P R, T N R, and F N R, respectively. The precision P represents the accuracy of the true positive prediction among all the positive predictions, while the recall R depicts the percentage of true positive prediction over all actual positives. They are calculated from the following formulas: TPR P = (5.10) TPR + FPR TPR R= (5.11) T P R + F NR 109 Once the confidence scores of all predicted bounding boxes were obtained, the predictions were sorted in a descent order according to the confidence value. A few additional rounds of performance evaluation were conducted. Specifically, each score was iteratively assigned as the IoU threshold to decide whether each prediction was a TP, FP, TN, or FN. And each new confidence-assigned IoU threshold resulted in a new precision value and a recall value, P and R, which could be used to plot the precision-recall curve. The average precision (AP ) was then calculated as the area underneath the precision-recall curve. Z 1 AP = P (R)dR, (5.12) 0 And the mean average precision (mAP ) would be calculated by taking the mean AP over all classes and/or overall IoU thresholds. For instance, mAP @0.5 represents the mean average precision over all classes with an IoU threshold of 0.5 in the first round of deciding the prediction’s performance, while mAP @0.5 : 0.95 has a similar meaning but it is further averaged over 10 consecutive IoU thresholds from 0.5 to 0.95 with a step of 0.05. The mAP curves are shown in Figure 5.4 for SSD and RetinaNet after training for 100 epochs, and for YOLOv5s models after 200 epochs. The epoch numbers were selected differently in order to achieve the best and stable performance for each model. Particularly, the maximum values of mAP @0.5 and mAP @0.5 : 0.95 as well as the averaged GPU speed for image inference, are listed in Table 5.3. As can be seen, RetinaNet achieved the highest mAP @0.5 among all three models, 93.68 %, compared to SSD’s 90.79 % and YOLOv5s’ 92.11 %. However, its maximum mAP @0.5 : 0.95 value was 66.63 %, which was smaller than that of YOLOv5s, 69.77 %. Basically, mAP @0.5 : 0.95 is a more comprehensive evaluation metric for object detection algorithms, as it takes multiple scales of IoU into consideration, which 110 Figure 5.4: The plots of mean average precision (mAP) curves of different approaches in the training and validation process. (a) SSD, (b) RetinaNet, and (c) YOLOv5s. Table 5.3: Comparison of validation results of different algorithms for sea lamprey detection. Framework mAP(val)@0.5 mAP(val)@0.5:0.95 GPU Speed [ms / img] SSD 90.79 % 56.81 % 11.4 RetinaNet 93.68 % 66.63 % 55.0 YOLOv5s 92.11 % 69.77 % 8.4 usually generates a more precise prediction. On the other hand, YOLOv5s cost the least inference time for each image, 8.4 ms on this GPU, which is qualified for future real-time sea lamprey detection applications. Therefore, YOLOv5s would be the best detector for this study. 5.3 Filtered YOLOv5s For Mitigation of The Sensor Memory Effect This section presents a real-time automated sea lamprey detection approach using an object detection method. As shown in Figure 5.5, the proposed YOLOv5s model-based sea lam- 111 Figure 5.5: Diagram of the soft pressure sensor and YOLOv5s model-based sea lamprey detection approach. prey detection neural networks consists of three parts: a deep convolutional neural network backbone extracting feature maps from the input mapping contour image, a top-down archi- tecture network neck constructing multi-scale feature maps, and a confidence score filter end. The backbone and the neck can directly learn features from the measurements of a soft pres- sure sensor array and then predict bounding box, class, and confidence of the input contour image. Meanwhile, due to the soft pressure sensor’s memory effect, the detection network will view the leftover patterns following the detachment as a normal compression or suction pattern, which could cause false positives in prediction. In order to mitigate such mem- ory effect-induced faulty detection, a postprocessing head that filters the confidence of the compression pattern and suction pattern separately is added to the sea lamprey detection network. Each of three aforementioned elements is elaborated next. 112 5.3.1 Feature Learning Backbone and Neck of YOLOv5 As shown in the detailed diagram in Figure 5.5, the feature learning networks of YOLOv5s mainly use three Bottleneck Cross Stage Partial (BottleneckCSP) Networks [118] as its back- bone. The backbone firstly adopts a Focus layer [106] to slice the input images and reshape the dimensions, then four ConvBNLeaky modules are deployed interdigitatedly between the BottleneckCSP modules, each of which contains a convolution layer that is connected with a batch normalization (BN) layer and a LeakyReLU activation layer. After the last ConvBN- Leaky layer, an Spatial Pyramid Pooling (SPP) [119] module is used to remove the fixed-size constraint of the networks. The feature maps extracted from three levels of the backbone will be merged to the following neck part at three corresponding levels. The feature fusion neck of YOLOv5s is constructed in a top-down Feature Pyramid Network (FPN) [120] for building high-level semantic feature maps at all scales. These features are then enhanced with the features from the previous bottom-up pathway via lateral connections by concatenation, and the fused feature maps will be transferred to a ConvBNLeaky layer followed by another BottleneckCSP network and a basic 2D convolution layer. The inference output will be sent to a sigmoid activation layer to regress the normalized bounding box center coordinates and the normalized widths and heights. Finally, a non- maximum suppression (NMS) technique is applied to select the best bounding boxes from multiple candidates. 5.3.2 Postprocessing with Confidence Thresholds After the feature fusion block, bounding box candidates of predicted sea lamprey attachment are obtained. Each of the valid candidate contains a pair of normalized center coordinates, 113 a pair of normalized width and height, a class label, and a final confidence score. The confidence score is a probability that an object belongs to one class, which means the product of the object confidence Confobj and the class confidence Confcls . The object confidence is calculated from the intersection over union (IoU) between the predicted bounding box and the ground-truth bounding box. g−t Area of Intersection IoUpred = (5.13) Area of Union  g−t   0, if IoUpred  =0 P robj = (5.14)   1, otherwise  g−t Confobj = P robj · IoUpred (5.15) The class confidence is a conditional probability of the class when there is an object being predicted at that cell: Confcls = P rcls|obj (5.16) So the final confidence score can be written as g−t Conf = Confcls · Confobj = P rcls|obj · P robj · IoUpred (5.17) The trained YOLOv5s model achieved a good performance for the sea lamprey compres- sion or suction pattern detection. Nevertheless, faulty prediction of sea lamprey attachment was found on many lamprey experiments in the testing dataset. As discussed in Section 114 III.B, the soft pressure sensor had some inherent memory effect when the compression was removed or when the suction pressure was released. Such a memory effect often lasted for more than 10 seconds after the lamprey detached from the sensor array. The overall memory effect showed a relatively low confidence score, thus it is promising to mitigate the false pre- diction by setting an additional postprocessing module with a higher threshold. Note that in most cases, the memory effect was more pronounced when the suction was removed than when the compression was removed from the sensor, which inspired us to set two different confidence thresholds for the compression pattern and the suction pattern, respectively. The final confidence scores are fed into a confidence filter to remove all the bound box predictions with a confidence score less than a designed threshold. This filtering process proves to be effective for suppressing the sensor’s memory effect as it only outputs the bounding box information in the beginning of the hardware’s memory stage, and prevents false detection in the remaining time. Two separate confidence thresholds (θC and θS ) for the compression pattern and the suction pattern, respectively, are optimally selected, which will be discussed in the next section. The output will be given according to the confidence value and the confidence threshold of that class:        BBoxcompression , if class = 0 and Conf ⩾ θC    Output = BBox (5.18)  suction , if class = 1 and Conf ⩾ θS      None, otherwise   115 5.4 Results and Discussion The testing dataset from the remaining 20 groups of sea lamprey experiments was used for testing the trained YOLOv5s model and getting class and confidence scores. Then the results with the ground-truth labels were investigated in depth to find the optimal confi- dence thresholds that could not only improve the positive predictions but also suppress false positive predictions. We first split the testing output dataset into four groups: the true com- pression subset, the false compression subset, the true suction subset, and the false suction subset. For the compression subsets, a confidence threshold (θC ) was set as a variable, chang- ing from 0.05 to 1.0. According to this compression confidence threshold, the compression prediction dataset could be divided into four categories: true positive compression (TPC), false positive compression (FPC), true negative compression (TNC), and false negative com- pression (FNC). The corresponding true positive rate, false positive rate, true negative rate, and false negative rate for the compression pattern are noted as T P RC, F P RC, T N RC, F N RC, respectively. In this way, the precision (PC ), recall (RC ), and the F-1 Score (F 1C ) of the compression pattern could be evaluated as follows. T P RC(θC ) PC (θC ) = (5.19) T P RC(θC ) + F P RC(θC ) T P RC(θC ) RC (θC ) = (5.20) T P RC(θC ) + F N RC(θC ) 2 · PC (θC ) · RC (θC ) F 1C (θC ) = (5.21) PC (θC ) + RC (θC ) Here F-1 score is a metric that balances the precision and the recall using their harmonic 116 mean. The performance evaluation metrics for the suction pattern can be obtained similarly from the suction dataset. Then, the F-1 score curves of both compression and suction patterns can be drawn, as shown in Figure 5.6-a. The maximum F-1 score is achieved as 0.88359 and 0.51842, when the confidence threshold is 0.1309 for the compression pattern, and 0.344 for the suction pattern, respectively. Figure 5.6: Postprocessing results on the sea lamprey testing dataset with the confidence threshold as the variable. (a) The F1-score curves, and (b) the false positive rate curves, for both compression and suction patterns. In the meantime, the corresponding false positive rate curves are shown in Figure 5.6-b, which are directly related to the faulty detection due to the memory effect. As depicted in the figure, when the maximum F-1 score is achieved for the compression pattern and the suction pattern, respectively, the corresponding false positive rate reaches 0.51923 and 0.65431, separately. Moreover, the higher the confidence threshold is, the lower the false positive rate for both compression and suction patterns. However, this affects the F-1 score 117 as well, and would possibly reduce it when the threshold is too high. Therefore, we propose to take both the F-1 score and the FPR into consideration in order to determine a “trade-off” between high positive prediction and low false prediction. This was realized by introducing a regularization co-efficient to the following cost function: LC (θC ) = F 1C (θC ) − λ · F P RC(θC ) (5.22) where λ ⩾ 0 is the regularization (or penalty) parameter, which controls the relative im- portance of the F-1 score with regard to the regularization FPR term, and the subtract operation is used since higher F-1 score and lower FPR are desirable. The choice of the value of the regularization parameter λ can be determined by the specific purpose or focus of that application. And the optimal confidence threshold θ̂C for the compression pattern was selected in order to maximize this cost function: θ̂C = arg max LC (θC ) (5.23) θC The cost function and the optimal confidence threshold for the suction pattern can be achieved similarly. As an illustration, Figure 5.7-a shows a sequence of images from the recorded experimen- tal video with ground-truth bounding box annotation, Figure 5.7-b shows the corresponding mapping contours with the original predictions from the YOLOv5s model, and Figure 5.7-c shows the corresponding mapping contours with the filtered predictions from the YOLOv5s model with two designed confidence thresholds. The corresponding predicted information for this time period is also listed in Table 5.4. At the first second (t = 231 s), a sea lamprey’s 118 Figure 5.7: Illustration of the faulty detection problem due to the sensor’s memory effect. (a) Image sequence of the pressure sensor array with ground truth bounding boxes. (b) Corresponding mapping contour plots with bounding box predictions from the YOLOv5s detection model without postprocessing (which means the confidence thresholds are both 0.05 for compression and suction patterns). (c) The filtered prediction results with a confidence threshold of 0.131 for the compression pattern, and 0.344 for the suction pattern, respectively. 119 Table 5.4: Predicted information during an interval of the soft piezoresistive pressure sensor array’s memory effect. Time (s) Class Confidence True or False 231 Compression 0.80 True 232 Suction 0.46 True 233 Suction 0.18 False 234 Suction 0.27 False 235 Suction 0.33 False 236 Suction 0.16 False 237 Suction 0.27 False 238 Suction 0.31 False mouth was attached on the right side of the sensor array while the original prediction showed a correct compression pattern in Figure 5.7-b. Then in the next second, the sea lamprey slid to the right boundary and disengaged from the sensor array. The original predicted bound- ing box in Figure 5.7-b at this time instant did not track the sliding of the suction pattern to the right side. Moreover, in the following time interval (231–238 s), the lamprey had been completely away from the sensor array, but there were still suction patterns shown in the following mapping contours, and consequently, the original YOLOv5s detection network still plotted predicted bounding boxes on each contour in Figure 5.7-b. On the contrary, when the confidence threshold was set as 0.131 for the compression pattern, and 0.344 for the suc- tion pattern in the postprocessing unit (in this case, λ = 0), the filtered predictions turned to be correct with most of the false prediction bounding boxes not displayed on the results. This filtering process with a pair of selected confidence thresholds proves to be simple but effective to the soft pressure sensor array-based sea lamprey detection. 120 5.5 Conclusion This work introduced an automated soft pressure sensor array-based sea lamprey detection approach using object detection neural networks, with a designed confidence threshold to mitigate the sensor’s memory effect before final prediction outputs. We first collected a comprehensive sea lamprey dataset of attachment mapping contours with two major pat- terns: “compression” and “suction” patterns, and annotated the dataset with ground-truth bounding box and class estimated from the synchronized experimental videos. Then three different object detection models were trained and validated on this sea lamprey dataset. By evaluating their overall performance, the YOLOv5s model was selected as our sea lamprey detection approach. More importantly, in order to achieve the best precision and suppress false prediction due to the sensor’s memory effect, a postprocessing unit was added to the YOLOv5s model with two different confidence thresholds for the two categories of patterns. The trade-off between higher precision and lower false positive rate could be achieved by a regularization method. 121 Chapter 6 Conclusion and Future Work 6.1 Conclusion This work aims to explore soft pressure sensing system for sea lamprey detection within the bigger context of controlling the sea lamprey invasion and restoring the fish community in the Great Lakes. It tackles the challenges that most soft sensing research facing. The novel and compact solutions explored in this work accompanied by the case studies provide new ways to build smart underwater sea lamprey detection systems. First, we developed an effective pressure sensing panel, comprised of arrays of commercial vacuum pressure sensors, to characterize suction dynamics of sea lampreys in static and flowing water. For adult sea lampreys, suction pressures ranged from –0.6 kPa to –26 kPa with 20 s to 200 s between pumps at rest, and increased to –8 kPa to –70 kPa when lampreys were manually disengaged. An array of sensors indicated that suction pressure distribution was largely uniform across the mouths of both juvenile and adult lampreys. Suction pressure did not differ between static and flowing water when water velocity was lower than 0.45 m/s. Such information benefits design of new systems to monitor behavior, distribution and abundance of lampreys. Then we reported the design of a soft capacitive sensor that is capable of measuring both positive and negative pressures. The ability to detect negative pressure, which is 122 rarely reported in the literature, was achieved in our work by designing air gap channels in the dielectric layer between the crossbar electrodes. The air gap channels enhance the deformation of the sensor and lead to significantly improved sensitivity especially for negative pressure. The influence of the air gap geometry on the sensitivity was also systematically studied through both single-pixel measurements and finite element simulation. Based on the experimental and simulation analysis of single pixel sensors, a 12-by-12 sensor array for spatial mapping of both positive and negative pressures was also demonstrated. However, this soft capacitive pressure sensor could not work well underwater unless we find a good way to shield the interference of the water from the sensor itself. We further developed a low-cost and efficient discrete piezoresistive pressure sensor array and new algorithms for properly processing the measured data to reconstruct the pressure pattern. In particular, in order to recover the cell resistance from the measured two-point resistance, we introduced several inverse algorithms based on the least-squares minimization and Tikhonov regularization. The approaches were validated with results from experiments with live sea lampreys underwater. The pros and cons of the different reconstruction methods were discussed in depth. Finally, we developed an automated soft pressure sensor array-based sea lamprey de- tection approach using object detection neural networks, with mapping contour images con- verted from the sensor array’s measurements as input, and with a designed confidence thresh- old to mitigate the sensor’s memory effect before final prediction outputs. Three different object detection models, including SSD, RetinaNet, and YOLOv5s, were trained and vali- dated on a comprehensive sea lamprey dataset of “compression” and “suction”” mapping contours, and the YOLOv5s model was selected as our sea lamprey detection approach due to its best mean average precision and fast inference speed. More importantly, a postprocess- 123 ing unit was added to the YOLOv5s model with two designed confidence thresholds for the compression and suction patterns, respectively, and the trade-off between higher precision and lower false positive rate could be achieved by a regularization method. 6.2 Future Work For the soft capacitive pressure sensors, as discussed in Chapter 3.5, there are some challenges in detecting sea lampreys underwater due to the electromagnetic interference (EMI) of water and sea lampreys, thus more investigation will be conducted with regard to EMI shielding. Thin conductive silicone layers or conductive tapes or fabrics could be tested for shielding the EMI interference from the soft capacitive pressure sensor device. For the automated sea lamprey detection system proposed in Chapter 5, it is worth noting that, although the image-based strategy which converts measurement data to mapping contour images as the input is a novelty for this work and has advantage in visualization and interpretation, the data-based strategy is still promising and of merit, and it can be achieved by developing some multilayer perceptron (MLP) neural networks to learn features directly from the data. Moreover, both the data-based strategy and the image-based strategy are actually processing the same set of original raw data, the difference is that the image- based method generates more data into the 2D colored images by adding RGB channels and interpolating data between the original raw data with multiple contour levels. Such data- to-image conversion should maintain the features of the original data, but it may become slower for the image-based neural networks to detect patterns as there are more data to be processed in the input image. In order to have a better understanding of both methods, the data-based method will be implemented and compared with the image-based method in our 124 future work. In addition, for the image-based method proposed in Chapter 5.3, although the filter unit added to the sea lamprey detection system has been proven to be simple but efficient for mitigation of the sensor’s memory effect, the mapping contour images are processed frame by frame individually according to the output confidence, while the suction dynamics of the sea lamprey’s attachment and the evolution of the historical data have not been taken into consideration at all. Therefore, an alternative way to mitigate the sensor’s memory effect might be looking into the time sequential data of the output class and confidence or even the bounding box information. By learning from these time sequential data, the sea lamprey’s suction dynamics such as the changing trend of the mapping contour patterns between the time of attachment and that of detachment is possible to be captured, thus a more robust detection might be achieved. For other future work, it will be of interest to develop soft pressure sensor arrays with larger sensing area and higher spatial resolution on soft and non-flat substrates, and test our automated sea lamprey detection approach on these sensors. The developed automated lamprey detection system needs to be experimentally tested in the field environment, to evaluate its performance and robustness in the presence of practical challenges such as flow disturbances and variations in ambient temperatures. Finally, the proposed smart sensing system could be used in a number of ways, such as selective fishway operation, where the sensing panel could be integrated with an electric field-based deterrence mechanism. In particular, once the smart panel detects an attachment of a sea lamprey, it will locally activate high electric fields to stun or deter the animal. While the smart pressure sensing system was motivated by the sea lamprey detection problem, it is applicable to other applications in soft robotics, wearable electronics, bio- 125 monitoring, human-robot interaction, underwater exploration and virtual reality. 126 BIBLIOGRAPHY 127 BIBLIOGRAPHY [1] H. Shi, I. González-Afanador, C. Holbrook, N. Sepúlveda, and X. Tan. Soft pressure sensor for underwater sea lamprey detection. IEEE Sensors Journal, 22(10):9932–9944, 2022. [2] A. G. Villa, A. Salazar, and F. Vargas. Towards automatic wild animal monitoring: Identification of animal species in camera-trap images using very deep convolutional neural networks. Ecological Informatics, 41:24–32, 2017. [3] A. Kaidarova, M. A. Khan, S. Amara, N. R. Geraldi, M. A. Karimi, and A. Shamim. Tunable, flexible composite magnets for marine monitoring applications. Adv. Eng. Mater., 20(1800229), 2018. [4] U. K. Verfuss, A. S. Aniceto, D. V. Harris, D. Gillespie, S. Fielding, and G. Jiménez. A review of unmanned vehicles for the detection and monitoring of marine fauna. Mar. Pollut. Bull., 140(30803631):17–29, 2019. [5] M. Bonneau, J. A. Vayssade, W. Troupe, and R. Arquet. Outdoor animal track- ing combining neural network and time-lapse cameras. Computers and Electronics in Agriculture, 168(105150), 2020. [6] R. A. Pollom and G. A. Rose. A global review of the spatial, taxonomic, and tem- poral scope of freshwater fisheries hydroacoustics research. Environmental Reviews, 24(3):333–347, 2016. [7] M. C. Lucas and E. Baras. Methods for studying spatial behaviour of freshwater fishes in the natural environment. Fish and fisheries, 1(4):283–316, 2000. [8] M. Moniruzzaman, S. M. S. Islam, M. Bennamoun, and P. Lavery. Deep learning on underwater marine object detection: a survey. In International Conference on Advanced Concepts for Intelligent Vision Systems, 2017. Springer, 2017. [9] A. R. Mahon and C. L. Jerde. Using environmental dna for invasive species surveillance and monitoring. In Marine Genomics, Methods and Protocols, pages 131–142. New York: Springer-Humana, 2016. [10] Q. Mauvisseau, M. Parrondo, M. P. Fernández, L. Garcı́a, J. L. Martı́nez, and E. Garcı́a-Vázquez. On the way for detecting and quantifying elusive species in the sea: The octopus vulgaris case study. Fisheries Research, 191:41–48, 2017. 128 [11] M. C. Yates, D. J. Fraser, , and A. M. Derry. Meta-analysis supports further refinement of edna for monitoring aquatic species-specific abundance in nature. Environmental DNA, 1:5–13, 2019. [12] B. R. Smith and J. J. Tibbles. Sea lamprey (petromyzon marinus) in lakes huron, michigan, and superior: History of invasion and control. Canadian Journal of Fisheries and Aquatic Sciences, 37:1780–1801, 1980. [13] G. L. Larson, G. C. Christie, D. A. Johnson, J. F. Koonce, K. M. Mullett, and W. P. Sullivan. The history of sea lamprey control in lake ontario and updated estimates of suppression targets. J. Great Lakes Res., 29:637–654, 2003. [14] W. P. Sullivan, G. C. Christie, F. C. Cornelius, M. F. Fodale, D. A. Johnson, and J. F. Koonce. The sea lamprey in lake erie: a case history. J. Great Lakes Res., 29:615–636, 2003. [15] T. J. Morse, M. P. Ebener, E. M. Koon, S. B. Morkert, D. A. Johnson, and D. W. Cuddy. A case history of sea lamprey control in lake huron: 1979 to 1999. J. Great Lakes Res., 29:599–614, 2003. [16] N. S. Johnson, H. T. Thompson, C. Holbrook, and J. A. Tix. Blocking and guid- ing adult sea lamprey with pulsed direct current from vertical electrodes. Fisheries Research, 150:38–48, 2014. [17] R. Hogg, S. M. Jr. Coghlan, and J. Zydlewski. Anadromous sea lampreys recolonize a maine coastal river tributary after dam removal. Transactions of the American Fisheries Society, 142:1381–1394, 2013. [18] M. L. Moser, J. B. Hume, K. K. Aronsuu, R. T. Lampman, and A. D. Jackson. Lamprey reproduction and early life history: insights from artificial propagation. In Lampreys: Biology, Conservation and Control, volume 2, pages 187–246. Springer, 2019. [19] N. Nikitina, M. Bronner-Fraser, and T. Sauka-Spengler. The sea lamprey petromy- zon marinus: a model for evolutionary and developmental biology. Cold Spring Harb Protoc., 4(1):20147008, 2009. [20] J. H. Youson and I. C. Potter. A description of the stages in the metamorphosis of the anadromous sea lamprey, petromyzon marinus l. Canadian Journal of Zoology, 57:1808–1817, 1979. [21] H. A. Dawson, B. R. Quintella, P. R. Almeida, A. J. Treble, and J. C. Jolley. The ecology of larval and metamorphosing lampreys. In Lampreys: Biology, Conservation and Control, volume 1, pages 75–125. Springer, 2015. 129 [22] T. J. Buchinger, M. J. Siefkes, B. S. Zielinski, C. O. Brant, and W. Li. Chemical cues and pheromones in the sea lamprey (petromyzon marinus). Frontiers in Zoology, 12(32):26609313, 2015. [23] S. Gambicki and G. B. Steinhart. Changes in sea lamprey size and fecundity through time in the great lakes. J. Great Lakes Res., 43:209–214, 2017. [24] V. C. Applegate. Natural history of the sea lamprey (Petromyzon marinus) in Michi- gan. PhD thesis, 1950. [25] R. D. Adams. Suction pressure measurement and behavioral observations of spawning- run sea lampreys (petromyzon marinus). Master’s thesis, Michigan, USA, 2006. [26] N. Gradwell. Hydrostatic pressure and movements of the lamprey, petromyzon, during suction olfaction and gill ventilation. Canadian Journal of Zoology, 50:1215–1223, 1972. [27] M. Liu, X. Pu, C. Jiang, T. Liu, X. Huang, L. Chen, C. Du, J. Sun, W. Hu, and Z. L. Wang. Large-area all-textile pressure sensors for monitoring human motion and physiological signals. Adv. Mater., 29(1703700), 2017. [28] L. Cai and C. Wang. Carbon nanotube flexible and stretchable electronics. Nanoscale Res. Lett., 10(320), 2015. [29] S. Gong, W. Schwalb, Y. Wang, Y. Chen, Y. Tang, J. Si, B. Shirinzadeh, and W. Cheng. A wearable and highly sensitive pressure sensor with ultrathin gold nanowires. Nat. Commun., 5(3132), 2014. [30] H. H. Chou, A. Nguyen, A. Chortos, J. W. To, C. Lu, J. Mei, T. Kurosawa, W. G. Bae, J. B. Tok, and Z. Bao. A chameleon-inspired stretchable electronic skin with interactive colour changing controlled by tactile sensing. Nat. Commun., 6(8011), 2015. [31] C. M. Boutry, M. Negre, M. Jorda, O. Vardoulis, A. Chortos, O. Khatib, and Z. Bao. A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. Sci. Robot., 3(24), 2018. [32] J. Park, J.-K. Kimb, D.-S. Kim, A. Shanmugasundaram, S. A. Park, S. Kang, S.-H. Kim, M. H. Jeong, and D.W. Lee. Wireless pressure sensor integrated with a 3d printed polymer stent for smart health monitoring. Sens. Actuators, B, 280:201–209, 2019. [33] W. M. Kier and A. M. Smith. The structure and adhesive mechanism of octopus suckers. Integr. Comp. Biol., 42(6):1146–1153, 2002. [34] A. Kremheller. The aerodynamics development of the new nissan qashqai. SAE Tech- nical Papers, (2014-01-0572), 2014. 130 [35] C. Rendl, P. Greindl, M. Haller, M. Zirkl, B. Stadlober, and P. Hartmann. Pyzoflex: printed piezoelectric pressure sensing foil. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST’12), 2012, pages 509– 518, 2012. [36] Y. Cao and N. Sepúlveda. Design of flexible piezoelectric gyroscope for structural health monitoring. Applied Physics Letters, 115:241901, 2019. [37] Y. Cao, W. Li, and N. Sepúlveda. Performance of self-powered, water-resistant bend- ing sensor using transverse piezoelectric effect of polypropylene ferroelectret polymer. IEEE Sensors Journal, 19:10327–10335, 2019. [38] Y. Cao, J. Figueroa, and W. Li. Understanding the dynamic response in ferroelectret nanogenerators to enable self-powered tactile systems and human-controlled micro- robots. Nano Energy, 63:103852, 2019. [39] Y. Cao, H. Shi, X. Tan, and N. Sepúlveda. Enabling negative pressure sensing through ferroelectret device. IEEE Sensors Letters, 6(8):1–4, 2022. [40] J. Engel, J. Chen, N. Chen, S. Pandya, and C. Liu. Multi-walled carbon nanotube filled conductive elastomers: Materials and application to micro transducers. In 19th IEEE International Conference on Micro Electro Mechanical Systems, pages 246–249. IEEE, 2006. [41] H. K. Kim, S. Lee, and K. S. Yun. Capacitive tactile sensor array for touch screen application. Sensors and Actuators A: Physical, 165(1):2–7, 2011. [42] H. Kou, L. Zhang, Q. Tan, G. Liu, W. Lv, F. Lu, H. Dong, and J. Xiong. Wireless flexible pressure sensor based on micro-patterned graphene/pdms composite. Sensors and Actuators A: Physical, 277:150–156, 2018. [43] K. Y. Lee, H. J. Yoon, T. Jiang, X. Wen, W. Seung, and S. W. Kim. Fully packaged self-powered triboelectric pressure sensor using hemispheres-array. Adv. Energy Mater., 6(1502566), 2016. [44] H. K. Lee, S. I. Chang, and E. Yoon. A flexible polymer tactile sensor: Fabrica- tion and modular expandability for large area deployment. J. Microelectromech. Sys., 15(6):1681–1686, 2006. [45] D. J. Lipomi, M. Vosgueritchian, B. C. Tee, S. L. Hellstrom, J. A. Lee, C. H. Fox, and Z. Bao. Skin-like pressure and strain sensors based on transparent elastic films of carbon nanotubes. Nat. Nanotech., 6:788–792, 2011. 131 [46] S. C. Mannsfeld, B. C. Tee, R. M. Stoltenberg, C. V. Chen, S. Barman, B. V. Muir, A. N. Sokolov, C. Reese, and Z. Bao. Highly sensitive flexible pressure sensors with microstructured rubber dielectric layers. Nat. Mater., 9:859–864, 2010. [47] B. Nie, R. Li, J. Cao, J. D. Brandt, and T. Pan. Flexible transparent iontronic film for interfacial capacitive pressure sensing. Adv. Mater., 27:6055–6062, 2015. [48] Y. Quan, X. Wei, L. Xiao, T. Wu, H. Pang, T. Liu, W. Huang, S. Wu, S. Li, and Z. Chen. Highly sensitive and stable flexible pressure sensors with micro-structured electrodes. Journal of Alloys and Compounds, 699:824–831, 2017. [49] J. I. Yoon, K. S. Choi, and S. P. Chang. A novel means of fabricating microporous structures for the dielectric layers of capacitive pressure sensor. Microelectronic Engi- neering, 179:60–66, 2017. [50] A. Bsoul, M. S. M. Ali, and K. Takahata. Piezoresistive pressure sensor using vertically aligned carbon-nanotube forests. Electron. Lett., 47(14):807–808, 2011. [51] S. A. U. Hasan, Y. Jung, S. Kim, C. L. Jung, S. Oh, J. Kim, and H. Lim. A sensitivity enhanced mwcnt/pdms tactile sensor using micropillars and low energy ar+ ion beam treatment. Sensors, 16(1), 2016. [52] X. Li, W. Huang, G. Yao, M. Gao, X. B. Wei, Z. W. Liu, H. Zhang, T. X. Gong, and B.Yu. Highly sensitive flexible tactile sensors based on microstructured multiwall carbon nanotube arrays. Scripta Materialia, 129:61–64, 2017. [53] B. Liang, W. Chen, Z. He, R. Yang, Z. Lin, H. Du, Y. Shang, A. Cao, Z. Tang, and X. Gui. Highly sensitive, flexible mems based pressure sensor with photoresist insulation layer. Small, 13(1702422), 2017. [54] L. Pan, A. Chortos, G. Yu, Y. Wang, S. Isaacson, R. Allen, Y. Shi, R. Dauskardt, and Z. Bao. An ultra-sensitive resistive pressure sensor based on hollow-sphere mi- crostructure induced elasticity in conducting polymer film. Nat. Commun., 5(3002), 2014. [55] H. Park, Y. R. Jeong, J. Yun, S. Y. Hong, S. Jin, S. J. Lee, G. Zi, and J. S. Ha. Stretch- able array of highly sensitive pressure sensors consisting of polyaniline nanofibers and au-coated polydimethylsiloxane micropillars. ACS Nano, 9(10):9974–9985, 2015. [56] T. Yang, W. Wang, H. Zhang, X. Li, J. Shi, Y. He, Q. S. Zheng, Z. Li, and H. Zhu. Tac- tile sensing system based on arrays of graphene woven microfabrics: Electromechanical behavior and electronic skin application. ACS Nano, 9(11):10867–10875, 2015. 132 [57] Y. Zhu, J. Li, H. Cai, Y. Wu, H. Ding, N. Pan, and X. Wang. Highly sensitive and skin-like pressure sensor based on asymmetric double-layered structures of reduced graphite oxide. Sensors and Actuators B: Chemical, 255(2):1262–1267, 2018. [58] G. Schwartz, B. C. Tee, J. Mei, A. L. Appleton, D. H. Kim, H. Wang, and Z. Bao. Flexible polymer transistors with high pressure sensitivity for application in electronic skin and health monitoring. Nat. Commun., 4(1859), 2013. [59] H. Wan, Y. Cao, L.-W. Lo, J. Zhao, N. Sepúlveda, and C. Wang. Flexible carbon nanotube synaptic transistor for neurological electronic skin applications. ACS Nano, 14(8):10402–101412, 2020. [60] H. Wan, J. Zhao, L.-W. Lo, Y. Cao, N. Sepúlveda, and C. Wang. Multimodal artifi- cial neurological sensory–memory system based on flexible carbon nanotube synaptic transistor. ACS Nano, 15(9):14587–14597, 2021. [61] H. B. Palmer. The capacitance of a parallel-plate capacitor by the schwartz-christoffel transformation. Electrical Eng., 56(3):363–368, 1937. [62] S.-C. Wong, P. S. Liu, J.-W. Ru, and S.-T. Lin. Interconnection capacitance models for vlsi circuits. Solid-State Electronics, 42(6):969–977, 1998. [63] Y. W. Choi, D. Kang, P. V. Pikhitsa, T. Lee, S. M. Kim, G. Lee, D. Tahk, and M. Choi. Ultra-sensitive pressure sensor based on guided straight mechanical cracks. Sci. Rep., 7(40116), 2017. [64] G. Y. Bae, S. W. Pak, D. Kim, G. Lee, D. H. Kim, Y. Chung, and K. Cho. Linearly and highly pressure-sensitive electronic skin based on a bioinspired hierarchical structural array. Adv. Mater., 28:5300–5306, 2016. [65] J. Park, Y. Lee, J. Hong, M. Ha, Y.-D. Jung, H. Lim, S. Y. Kim, and H. Ko. Giant tunneling piezoresistance of composite elastomers with interlocked microdome arrays for ultrasensitive and multimodal electronic skins. ACS Nano, 8(5):4689–4697, 2014. [66] J. Park, M. Kim, Y. Lee, H. S. Lee, and H. Ko. Fingertip skin–inspired microstructured ferroelectric skins discriminate static/dynamic pressure and temperature stimuli. Sci. Adv., 1(9):e1500661, 2015. [67] B. Y. Lee, J. Kim, H. Kim, C. Kim, and S. D. Lee. Low-cost flexible pressure sensor based on dielectric elastomer film with micro-pores. Sensors and Actuators A: Physical, 240:103–109, 2016. [68] L. Cai, S. Zhang, Y. Zhang, J. Li, J. Miao, Q. Wang, Z. Yu, and C. Wang. Direct printing for additive patterning of silver nanowires for stretchable sensor and display applications. Adv. Mater. Technol., 3(1700232), 2018. 133 [69] D. J. Lipomi, J. A. Lee, M. Vosgueritchian, B. C.-K. Tee, J. A. Bolander, and Z. Bao. Electronic properties of transparent conductive films of pedot:pss on stretchable sub- strates. Chem. Mater., 24(2):373–382, 2012. [70] Y. Wang, C. Zhu, R. Pfattner, H. Yan, L. Jin, S. Chen, F. Molina-Lopez, F. Lissel, J. Liu, N. I. Rabiah, Z. Chen, J. W. Chung, C. Linder, M. F. Toney, B. Murmann, and Z. Bao. A highly stretchable, transparent, and conductive polymer. Sci. adv., 3(3):e1602076, 2017. [71] E. Jeong, J. Lee, and D. Kim. Finger-gesture recognition glove using velostat. In the 11th International Conference on Control, Automation and Systems), 2022., pages 206–210, 2011. [72] S. Sundaram, P. Kellnhofer, and Y. Li. Learning the signatures of the human grasp using a scalable tactile glove. Nature, 569:698–702, 2019. [73] J. Tolvanen, J. Hannu, and H. Jantunen. Hybrid foam pressure sensor utilizing piezore- sistive and capacitive sensing mechanisms. IEEE Sensors Journal, 17:4735–4746, 2017. [74] B. W. Lee and H. Shin. Feasibility study of sitting posture monitoring based on piezoresistive conductive film-based flexible force sensor. IEEE Sensors J., 16(1):15– 16, 2016. [75] M. Hopkins, R. Vaidyanathan, and A. H. Mcgregor. Examination of the performance characteristics of velostat as an in-socket pressure sensor. IEEE Sensors Journal, 20:6992–7000, 2020. [76] D. Giovanelli and E. Farella. Force sensing resistor and evaluation of technology for wearable body pressure sensing. Journal of Sensors, 2016:9391850, 2016. [77] S. S. Suprapto, A. W. Setiawan, and H. Zakaria. Low-cost pressure sensor matrix using velostat. In the 5th International Conference on Instrumentation, Communi- cations, Information Technology, and Biomedical Engineering (ICICI-BME)), 2017., pages 137–140, 2017. [78] F. Y. Wu. Theory of resistor networks: the two-point resistance. J. Phys. A: Math. Gen., 37:6653–6673, 2004. [79] N. S. Izmailian, R. Kenna, and F. Y. Wu. The two-point resistance of a resistor network: a new formulation and application to the cobweb network. J. Phys. A: Math. Theory, 47:035003, 2014. [80] R. S. Saxena, N. K. Saini, and R. K. Bhan. Analysis of crosstalk in networked arrays of resistive sensors. IEEE Sensors Journal, 11:920–924, 2011. 134 [81] J. Wu, L. Wang, and J. Li. Design and crosstalk error analysis of the circuit for the 2-d networked resistive sensor array. IEEE Sensors Journal, 15:1020–1026, 2015. [82] C. Medrano-Sánchez, R. Igual-Catalán, V. H. Rodrı́guez-Ontiveros, and I. Plaza- Garcı́a. Circuit analysis of matrix-like resistor networks for eliminating crosstalk in pressure sensitive mats. IEEE Sensors Journal, 19:8027–8036, 2019. [83] J. Kaipio and E. Somersalo. Statistical and Computational Inverse Problems. Springer Science & Business Media, 2006. [84] C. Cohen-Bacrie, Y. Goussard, and R. Guardo. Regularized reconstruction in electrical impedance tomography using a variance uniformization constraint. IEEE Transactions on Medical Imaging, 16:562–571, 1997. [85] M. Gehre, T. Kluth, and A. Lipponen. Sparsity reconstruction in electrical impedance tomography: An experimental evaluation. Journal of Computational and Applied Mathematics, 236:2126–2136, 2012. [86] T. C. Pratt, L. M. O’Connor, A. G. Hallett, R. L. McLaughlin, C. Katopodis, D. B. Hayes, and R. A. Bergstedt. Balancing aquatic habitat fragmentation and control of invasive species: enhancing selective fish passage at sea lamprey control barriers. Transactions of the American Fisheries Society, 138:652–652, 2009. [87] R. L. McLaughlin, E. R. Smyth, T. Castro-Santos, M.L. Jones, M.A. Koops, T.C. Pratt, and L.-A. Vélez-Espino. Unintended consequences and trade-offs of fish passage. Fish and Fisheries, 14:580–604, 2013. [88] Hendry and R.-C. Chen. Automatic license plate recognition via sliding-window darknet-yolo deep learning. Image and Vision Computing, 87:47–56, 2019. [89] P. Chu, Z. Li, K. Lammers, R. Lu, and X. Liu. Deep learning-based apple detection using a suppression mask r-cnn. Pattern Recognition Letters, 147:206–211, 2021. [90] L. Tan, T. Huangfu, L. Wu, and W. Chen. Comparison of retinanet, ssd, and yolo v3 for real-time pill identification. BMC Medical Informatics and Decision Making, 21(324):1–11, 2021. [91] G. Guo and N. Zhang. A survey on deep learning based face recognition. Computer Vision and Image Understanding, 189(102805):1–37, 2019. [92] J. Zhong, M. Li, J. Qin, Y. Cui, K. Yang, and H. Zhang. Real-time marine animal detection using yolo-based deep learning networks in the coral reef ecosystem. In the 7th Intl. Conference on Ubiquitous Positioning, Indoor Navigation and Location-Based Services (UPINLBS), 2022., pages 301–306, 2022. 135 [93] D. Feng, C. Haase-Schütz, L. Rosenbaum, H. Hertlein, C. Gläser, F. Timm, W. Wies- beck, and K. Dietmayer. Deep multi-modal object detection and semantic segmenta- tion for autonomous driving: datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems, 22(3):1341–1360, 2021. [94] C. Negrea, D. E. Thompson, S. D. Juhnke, D. S. Fryer, and F. J. Loge. Auto- mated detection and tracking of adult pacific lampreys in underwater video collected at snake and columbia river fishways. North American Journal of Fisheries Management, 34(3):111–118, 2014. [95] H. Shi, C. M. Holbrook, Y. Cao, N. Sepúlveda, and X. Tan. Measurement of suction pressure dynamics of sea lampreys, petromyzon marinus. PLoS ONE, 16:e0247884, 2021. [96] I. González-Afanador, H. Shi, C. Holbrook, X. Tan, and N. Sepúlveda. Invasive sea lamprey detection and characterization using interdigitated electrode (ide) contact sensor. IEEE Sensors Journal, 21(24):27947–27956, 2021. [97] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014., pages 580–587. IEEE, 2014. [98] R. Girshick. Fast r-cnn. In the Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015., pages 1440–1448. IEEE, 2015. [99] S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems 28 (NIPS), pages 1–9, 2015. [100] K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask r-cnn. In the Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017., pages 2961–2969. IEEE, 2017. [101] J. Redmon, S. Divvala, R. Girshick, and Ali Farhadi. You only look once: unified, real-time object detection. In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016., pages 779–788. IEEE, 2016. [102] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg. Ssd: single shot multibox detector. In the European Conference on Computer Vision (ECCV), 2016., pages 21–37. Springer Science+Business Media, 2016. [103] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar. Focal loss for dense object detection. In the Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017., pages 2980–2988. IEEE, 2017. 136 [104] J. Redmon and A. Farhadi. Yolov3: An incremental improvement. ArXiv, abs/1804.02767, 2018. [105] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao. Yolov4: Optimal speed and accuracy of object detection. ArXiv, abs/2004.10934, 2020. [106] G. Jocher, A. Stoken, and et. al. J. Borovec. Ultralytics/yolov5: v3.1 - bug fixes and performance improvements. https://doi.org/10.5281/zenodo.4154370. [Online. v3.1, 28-October-2020]. [107] N. S. Johnson, T. J. Buchinger, and W. Li. Reproductive ecology of lampreys. In Lampreys Biology Conservation and Control, volume 1, pages 265–303. Springer, 2015. [108] 2019–2020. Database: USGS Water Data. Mean water ve- locity for discharge computation of st. mary’s river at sault ste. marie, ontario. https://waterdata.usgs.gov/monitoring- location/04127885/parameterCode=72255startDT=2019-01-01endDT=2020-01-01. [Online. Available: 2019–2020]. [109] J. H. Zar. Biostatistical analysis. 4th ed. Upper Saddle River, New Jersey: Prentice Hall, 1999. [110] H. Shi, M. Al-Rubaiai, C. M. Holbrook, J. Miao, T. Pinto, C. Wang, and Xiaobo Tan. Screen-printed soft capacitive sensors for spatial mapping of both positive and negative pressures. Adv. Funct. Mater., 29:1809116, 2019. [111] ASTM International. Astm d412-15a. https://doi.org/10.1520/D0412-15A. [accessed: December 2015]. [112] M. Mooney. A theory of large elastic deformation. Journal of Applied Physics, 11:582– 592, 1940. [113] R. S. Rivlin. Large elastic deformations of isotropic materials iv. further developments of the general theory. Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, 241:379–397, 1948. [114] O. H. Yeoh. Some forms of the strain energy function for rubber. Rubber Chemistry and technology, 66(5):754–771, 1993. [115] R. W. Ogden. Large deformation isotropic elasticity – on the correlation of theory and experiment for incompressible rubberlike solids. In Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, pages 565–584, 1972. [116] T. F. Coleman and Y. Li. An interior, trust region approach for nonlinear minimization subject to bounds. SIAM Journal on Optimization, 6:418–445, 1996. 137 [117] A. Neubeck and L. V. Gool. Efficient non-maximum suppression. In the 18th Inter- national Conference on Pattern Recognition (ICPR’06), 2006., pages 850–855. IEEE, 2006. [118] C.-Y. Wang, H.-Y. M. Liao, Y.-H. Wu, P.-Y. Chen, J.-W. Hsieh, and I-H. Yeh. Cspnet: a new backbone that can enhance learning capability of cnn. In the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020., pages 1571–1580. IEEE, 2020. [119] K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9):1904–1916, 2015. [120] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature pyramid networks for object detection. In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017., pages 2117–2125. IEEE, 2017. 138