DATA INTERPRETATION FRAMEWORKS EMPLOYING MACHINE LEARNING FOR ENERGY-LEAN DATA-DRIVEN STRUCTURAL HEALTH MONITORING WITH NOVEL SELF-POWERED SENSING TECHNOLOGY By Hadi Salehi A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Civil Engineering - Doctor of Philosophy 2019 ABSTRACT DATA INTERPRETATION FRAMEWORKS EMPLOYING MACHINE LEARNING FOR ENERGY-LEAN DATA-DRIVEN STRUCTURAL HEALTH MONITORING WITH NOVEL SELF-POWERED SENSING TECHNOLOGY By Hadi Salehi Recent advances in energy harvesting technologies have led to the evolution of self-powered structural health monitoring (SHM) techniques that are energy-lean. Concurrent to the emergence of self-powered sensing has been the development of power-efficient data communication protocols. The pulse switching architecture is among such protocols employing ultrasonic pulses for event reporting and transmitting binary data/signals through the substrate material. The uniqueness of the through-substrate self-powering is that the energy required for data computational, storage and transmission is directly harvested from the signal being sensed as well as from ambient vibrations, thus providing a promising alternative to traditional sensor systems. However, a system using such self-powered sensing technology demands dealing with power budgets for sensing and communication of binary data, resulting in missing and incomplete data received at the SHM processor due to unique time delay constraint. The nature of data thus imply the necessity for development of new data mining frameworks. This research addresses the noted issue through the development of advanced data interpretation frameworks to interpret asynchronous discrete binary and incomplete/noisy data for power-efficient SHM of plate-like structures. Finite element simulations on an aircraft stabilizer wing and structural plates were conducted to validate the proposed methodology. Further, experimental vibration tests on dynamically loaded plates were carried out to demonstrate the applicability of the approach on a realistic structure. The proposed data interpretation frameworks for data-driven SHM with discrete time-delayed binary and incomplete (noisy) data were established based on the integration of machine learning, pattern recognition, a data fusion model, probabilistic, and statistical approaches. First, it was assumed that the SHM system operates with full data availability and the constraints of the communication power budget for the sensors and the time delay were not considered. On this basis, a pattern recognition-based algorithmic framework merging an image-based pattern recognition approach using anomaly detection, statistical measures, and numerous artificial intelligence classifiers were developed for self-powered damage identification with full discrete binary data. Further, the robustness of the developed pattern recognition framework with respect to different levels of damage severity, irregular loading condition, and sensor sparsity was evaluated. An uncertainty analysis was also conducted to ascertain the effectiveness of the data analysis framework with noise contaminated data. In the next analysis phase, the effect of time delay due to the pulse switching communication article was taken into account and algorithmic frameworks detecting effect of delay were pursued. In this context, probabilistic approaches were developed to model and predict delay, whereas damage was classified through different machine learning algorithms. Further, a novel machine learning-based data interpretation frameworks that incorporate low-rank matrix completion, an image-based pattern recognition approach, a data fusion model, machine learning algorithms, and a statistical approach was developed to reconstruct/recover incomplete (sparse) data and to identify damage with reconstructed missing data. The effectiveness and robustness of the developed machine learning-based data interpretation frameworks with respect to harvested energy variations were evaluated. Numerical and experimental results demonstrate that the proposed energy-lean data-driven SHM methodology using machine learning is efficient for detecting damage from a self-powered sensing technology. Copyright by HADI SALEHI 2019 ACKNOWLEDGMENTS I would like to express my gratitude to my advisor, Dr. Rigoberto Burguño, for his support, invaluable guidance, and encouragement throughout my research and graduate studies. My success in conducting this research and becoming a hard-working researcher would not have been possible without his time, passion, and effort. I would like thank the members of my committee Dr. Subir Biswas, Dr. Arun Ross, and Dr. Mahmoodul Haq for their guidance, helpful and insightful comments, time, and support throughout my research. Specifically, I am grateful to Dr. Subir Biswas for his input on sensors and sensor networks in this collaborative research project and Dr. Arun Ross for sharing his knowledge on artificial intelligence. The friendship of Saptarshi Das has led to many interesting discussions relating to this interdisciplinary research. I am thankful for the support from the Department of Civil and Environmental Engineering. I would like to thank Laura Post and Laura Taylor for their dedication on administrative support. I would also like to acknowledge the College of Engineering for several resources during my graduate studies through fellowships, travel grants, and workshops. In addition, I acknowledge the funding from the U.S. National Science Foundation (Grant number CNS-1405273). Last, but not least, I am grateful to my wife, Roya, for her unconditional love and understanding during the past few years. I would not have been able to make it without her never- ending encouragement and support. I would like to express my deepest gratitude to my parents and my siblings for their endless love, dedication, and for the many years of support. v TABLE OF CONTENTS LIST OF TABLES ...................................................................................................................... ix LIST OF FIGURES .................................................................................................................... xi CHAPTER 1 ................................................................................................................................ 1 1 INTRODUCTION ................................................................................................................... 1 1.1 Motivation and Vision .................................................................................................... 1 1.2 Background and State of Knowledge ............................................................................. 3 1.2.1 SHM and Damage Identification ................................................................................ 3 1.2.2 Sensing Technologies for SHM .................................................................................. 5 1.2.3 Artificial Intelligence in Structural Engineering ......................................................... 7 Research Gap .................................................................................................................. 8 Research Hypothesis and Objectives ............................................................................ 10 Research Significance ................................................................................................... 11 Organization of the Dissertation ................................................................................... 14 1.3 1.4 1.5 1.6 CHAPTER 2 .............................................................................................................................. 16 2 SHM WITH ARTIFICIAL INTELLIGENCE AND SELF-POWERED SENSING ............ 16 2.1 Overview ....................................................................................................................... 16 A Through-substrate Ultrasonic Self-powered Sensor Network for SHM ................... 16 2.2 2.3 Emerging AI Techniques for SHM and Condition Assessment ................................... 20 2.3.1 Pattern Recognition ................................................................................................... 23 2.3.2 Machine Learning ..................................................................................................... 25 2.3.3 Deep Learning ........................................................................................................... 29 2.3.4 Future Directions for use of Emerging AI methods in SHM and Smart Infrastructure Monitoring ............................................................................................................................. 31 AI Algorithms and Classifiers for Energy-Lean Data-Driven SHM ............................ 35 2.4.1 Support Vector Machine ........................................................................................... 35 2.4.2 K-Nearest Neighbor .................................................................................................. 39 2.4.3 Artificial Neural Networks ....................................................................................... 41 2.4.4 Bayesian Method ...................................................................................................... 42 2.4.5 Nearest Neighbor Classifier with Two-Dimensional Principal Component Analysis .. ....................................................................................................................................43 2.4.6 Nearest Neighbor Classifier with Two-Dimensional Linear Discriminant Analysis 44 Summary ....................................................................................................................... 46 2.5 2.4 CHAPTER 3 .............................................................................................................................. 49 3 PATTERN-RECOGNITON-BASED ALGORITHMIC FRAMEWORK FOR SELF- POWERED SHM USING FULL DISCRETE BINARY SIGNALS ........................................ 49 Overview ....................................................................................................................... 49 3.1 3.2 Pattern recognition framework using anomaly detection and AI classifiers ................ 49 Numerical Study ........................................................................................................... 53 3.3 3.4 Experimental Study ....................................................................................................... 54 vi 3.5 Approach to Data/Signal Interpretation ........................................................................ 55 3.5.1 Multiple Binary Event Concept ................................................................................ 56 3.5.2 Global Response Identification ................................................................................. 57 3.5.3 Local Damage Identification ..................................................................................... 58 Damage Identification based on Finite Element Simulation Results............................ 58 3.6.1 Image-based PR Approach ....................................................................................... 58 3.6.2 Nearest Neighbor Classifier with 2DPCA ................................................................ 65 3.6.3 Nearest Neighbor Classifier with 2DLDA ................................................................ 66 Damage Identification based on Experimental Results ................................................ 66 3.7.1 Bayesian Method ...................................................................................................... 67 3.7.2 Nearest Neighbor Classifier with 2DPCA ................................................................ 69 3.7.3 Nearest Neighbor Classifier with 2DLDA ................................................................ 70 Summary ....................................................................................................................... 71 3.6 3.7 3.8 CHAPTER 4 .............................................................................................................................. 73 4 EEFECT OF IRREGULAR LOADING CONDITIONS, SENSOR SPARSITY, AND MEASUREMENT NOISE ........................................................................................................ 73 Overview ....................................................................................................................... 73 4.1 4.2 Implementation of the PR-based Algorithmic Framework with image-based PR approach and k-NN Algorithm .................................................................................................. 73 4.2.1 Image-based PR Approach Employing Anomaly Detection .................................... 75 4.2.2 Pattern Anomaly Measure Index .............................................................................. 77 4.2.3 Pattern Focal Density Measure Index ....................................................................... 77 4.2.4 K-Nearest Neighbor Method ..................................................................................... 78 Numerical Modeling and Simulations .......................................................................... 79 4.3.1 FE Simulations of a Square Plate .............................................................................. 80 4.3.2 FE Simulations of a Rectangular Plate ..................................................................... 81 4.3.3 Load Cases for FE Simulations................................................................................. 82 4.3.4 Sensor Configurations for Damage Detection Model ............................................... 83 Damage Detection Simulation Results ......................................................................... 84 4.4.1 Evaluation of the Damage Detection Strategy with Different Loading Conditions . 86 4.4.2 Evaluation of the Damage Detection Strategy with Different Sensor Configurations . ................................................................................................................................... 99 4.5 Summary ..................................................................................................................... 101 4.4 4.3 FRAMEWORKS CHAPTER 5 ............................................................................................................................ 103 5 PROBABILISTIC-BASED ALGORITHMIC FOR DAMAGE IDENTIFICATION AND CONDITION ASSESSMENT WITH DELAYED SIGNALS ..... 103 Overview ..................................................................................................................... 103 5.1 5.2 Probabilistic Framework Integrating PR and a Conditional Probability Chain .......... 103 5.2.1 Performance Verification of the Probabilistic Framework ..................................... 106 5.2.2 Theory of a Spatial-Temporal Conditional Probability Chain ................................ 109 5.2.3 Damage Indication Parameters ............................................................................... 111 5.2.4 Damage Detection based on FE Results ................................................................. 113 Probabilistic Approach for Modeling Delayed Signals .............................................. 122 5.3.1 Experimental Study ................................................................................................. 123 5.3.2 Reconstruction of Time-Delayed Signals with a Probabilistic Approach .............. 125 5.3 vii 5.3.3 Simulation Results .................................................................................................. 128 Summary ..................................................................................................................... 136 5.4 CHAPTER 6 ............................................................................................................................ 138 6 MACHINE LEARNING-BASED DATA INTERPRETATION FRAMEWORKS FOR ENERGY-LEAN SHM USING INCOMPLETE/NOISY AND SPARSE SIGNALS ........... 138 6.1 Overview ..................................................................................................................... 138 6.2 Machine Learning Framework Employing Matrix Completion, Pattern Recognition, and K-Nearest Neighbor ................................................................................................................. 138 6.2.1 Finite Element Modeling of an Aircraft Stabilizer Wing ....................................... 140 6.2.2 Performance Evaluation using Different Harvested Energy Levels ....................... 143 6.2.3 Data Fusion for Damage Identification Model ....................................................... 146 6.2.4 Learning with Low-Rank Matrix Completion ........................................................ 152 6.2.5 Classification with K-Nearest Neighbor Algorithm ............................................... 158 6.2.6 Damage Identification Results with Machine Learning Framework ...................... 159 6.3 Machine Learning Framework Employing Matrix Completion, Pattern Recognition, and Statistical Approach ................................................................................................................ 176 6.3.1 Performance Evaluation of Machine Learning Framework .................................... 176 6.3.2 Damage Classification with Statistical Approach ................................................... 177 6.3.3 Results of Self-Powered SHM with Machine Learning-based Algorithmic Framework ................................................................................................................................. 182 6.4 Summary ..................................................................................................................... 191 CHAPTER 7 ............................................................................................................................ 195 7 EFFECT OF LEARNING ALGORITHMS ON PERFORMANCE OF MACHINE LEARNING-BASED DATA MINING FRAMEWORK........................................................ 195 7.1 Overview ..................................................................................................................... 195 Data Interpretation Framework Employing Machine Learning .................................. 196 7.2 Numerical and Experimental Case Studies ................................................................. 197 7.3 7.4 Self-Powered Damage Identification Results ............................................................. 199 7.4.1 Results based on Finite Element Simulations of Aircraft Wing ............................. 199 7.4.2 Results based on Experimental Studies .................................................................. 212 Summary ..................................................................................................................... 214 7.5 CHAPTER 8 ............................................................................................................................ 216 8 CONLCUSIONS ................................................................................................................. 216 Research Contributions and Significance ................................................................... 216 8.1 8.2 Conclusions ................................................................................................................. 216 Future Research .......................................................................................................... 218 8.3 REFERENCES ........................................................................................................................ 221 viii LIST OF TABLES Table 3-1. Number of pattern occurrences based on the PR approach ......................................... 59 Table 3-2. Classification results in terms of number of pattern occurrences based on nearest neighbor classifier with feature extraction techniques using load #2 ........................................... 66 Table 3-3. Distribution of patterns from experiments on a cantilever plate using the Bayesian method........................................................................................................................................... 68 Table 3-4. Distribution of patterns from experiments on a cantilever plate using the Bayesian method........................................................................................................................................... 68 Table 3-5. Confusion matrix for experimental data based on nearest neighbor and 2DPCA ....... 69 Table 3-6. Confusion matrix for experimental data based on nearest neighbor and 2DLDA ...... 70 Table 4-1. Confusion matrix for training data .............................................................................. 94 Table 4-2. Confusion matrix for test data ..................................................................................... 94 Table 4-3. Pattern anomaly measure with load 2 .......................................................................... 95 Table 4-4. Pattern anomaly measure with load 3 .......................................................................... 96 Table 4-5. Damage detection performance based on load case 2 ................................................. 98 Table 4-6. Damage detection performance based on load case 3 ................................................. 98 Table 4-7. Damage detection performance for rectangular plate ............................................... 100 Table 4-8. Damage detection performance for square plate ....................................................... 100 Table 5-1. Damage detection accuracy for different time lags ................................................... 133 Table 5-2. Typical confusion matrix data structure used for definition of an ROC curve ......... 134 Table 6-1. Confusion matrix for validation data (matrix M1 and ECC = 0.55Θ)....................... 171 Table 6-2. Confusion matrix for test data (matrix M1 and ECC = 0.55Θ) ................................. 172 Table 6-3. Confusion matrix for validation data (matrix M2 and ECC = 0.55Θ)....................... 172 Table 6-4. Confusion matrix for test data (matrix M2 and ECC = 0.55Θ) ................................. 172 Table 6-5. Confusion matrix for validation data (matrix M1 and ECC = 0.90Θ)....................... 173 Table 6-6. Confusion matrix for test data (matrix M1 and ECC = 0.90Θ) ................................. 173 ix Table 6-7. Confusion matrix for validation data (matrix M2 and ECC = 0.90Θ)....................... 173 Table 6-8. Confusion matrix for test data (matrix M2 and ECC = 0.90Θ) ................................. 174 x LIST OF FIGURES Figure 1-1. A through-substrate self-powered sensor network ....................................................... 6 Figure 1-2. Aircraft wing with integrated self-powered sensing technology ............................... 12 Figure 1-3. A schematic of a smart city with deployed sensors ................................................... 13 Figure 1-4. Typical intelligent transportation system analytics architecture ................................ 14 Figure 2-1. Energy-lean SHM using a through-substrate ultrasonic self-powered sensor network ....................................................................................................................................................... 17 Figure 2-2. Piezoelectric harvester circuit model [6] .................................................................... 18 Figure 2-3. Factors affecting one-hop event delivery delay when using CPN in energy-harvesting- powered wireless sensor networks ................................................................................................ 19 Figure 2-4. Illustration of the interrelation of different intelligent computational techniques ..... 21 Figure 2-5. Research publications on the use of different AI branches in structural engineering 23 Figure 2-6. Schematic of a PR system .......................................................................................... 24 Figure 2-7. Tree structure of generative and discriminative PR models and algorithms ............. 25 Figure 2-8. Research publications on the use of machine learning and pattern recognition ........ 26 Figure 2-9. Machine learning categories with commonly adopted algorithms ............................ 29 Figure 2-10. Schematic of a typical CNN architecture ................................................................. 31 Figure 2-11. Schematic of a typical CNN architecture ................................................................. 32 Figure 2-12. Schematic illustration of the SVM with optimal margin and separating hyperplane ....................................................................................................................................................... 37 Figure 2-13. (a) Illustration of a typical k-NN classification, (b) A typical Voronoi diagram for two-dimensional feature space ...................................................................................................... 40 Figure 2-14. Feed-forward multilayer neural networks with tow hidden layers and back- propagation algorithm ................................................................................................................... 42 Figure 3-1. Schematic illustration of SHM strategy using image-based pattern recognition ....... 50 Figure 3-2. Illustration of loss of data resolution (a) continuous response, (b) discrete binary response......................................................................................................................................... 51 xi Figure 3-3. Geometry and sensor layout for a simply supported plate for simulation case ......... 53 Figure 3-4. Harmonic amplitudes used for dynamic loading ....................................................... 54 Figure 3-5. Details of experimental vibration test of cantilever plate: (a) Plate geometry and instrumentation layout, (b) Schematic of test setup ...................................................................... 55 Figure 3-6. Concept schematic of multiple binary event generation at sensor nodes ................... 57 Figure 3-7. Common identified patterns for simply supported plate under uniform load with harmonic amplitude #1 ................................................................................................................. 59 Figure 3-8. Classification results based on deviation of patterns method for simply supported plate due to distributed load with harmonic amplitude #2: (a) time history of new recognized patterns, (b) histogram of identified patterns............................................................................................... 60 Figure 3-9. Close-ups of deviation of patterns method time history results in Fig 7(a): (a) t = 1.5 to 2 s, (b) t = 4.5 to 5 s .................................................................................................................. 61 Figure 3-10. Spatial-temporal representation of sensor node values based on multiple thresholds according to load amplitude #2 for 4 consecutive time steps (time step = 0.01 s) ....................... 62 Figure 3-11. Classification results based on deviation of patterns method due to a localized 40% reduction in stiffness in the simply supported plate case: (a) time history of new recognized patterns, (b) Close-ups of time history results (t = 4.5 to 5 s) ...................................................... 63 Figure 3-12. Effect of a localized 40% stiffness reduction in the simply supported plate on the spatial-temporal distribution of multiple threshold values at sensor nodes for 4 consecutive time steps (time step = 0.01 s) .............................................................................................................. 64 Figure 3-13. Surface interpolation of multiple binary events according to a localized 40% stiffness reduction in the simply supported plate for 4 consecutive time steps (time step = 0.01 s) .......... 64 Figure 3-14. Frequency of normal patterns due to localized stiffness reduction in the simply supported plate using the deviation of patterns method: (a) up to 30% stiffness decrease, (b) up to 80% ............................................................................................................................................... 65 Figure 3-15. Frequency of damaged patterns identified by deviation method for different levels of localized stiffness reduction .......................................................................................................... 65 Figure 3-16. Identified patterns by Bayesian method on cantilever plate with different hole diameters: (a) 13 mm, (b) 25.5 mm .............................................................................................. 68 Figure 3-17. Identified patterns by nearest neighbor and 2DPCA method on cantilever plate for different hole diameters: (a) 13 mm, (b) 25.5 mm ........................................................................ 70 Figure 3-18. Identified patterns by nearest neighbor and 2DLDA method on cantilever plate for different hole diameters: (a) 13 mm, (b) 25.5 mm ........................................................................ 71 xii Figure 4-1. Proposed PR framework for power-efficient SHM with discrete binary data ........... 74 Figure 4-2. Illustration of image-based PR using anomaly detection ........................................... 75 Figure 4-3. Representation of sensor nodes responses as pattern ................................................. 76 Figure 4-4. Damage scenarios considered in FE model of square plate (a) single damage, (b) multiple damage ............................................................................................................................ 81 Figure 4-5. Damage scenarios considered in FE model of rectangular plate (a) single damage, (b) multiple damage ............................................................................................................................ 82 Figure 4-6. Harmonic amplitudes used for dynamic loading in FE simulations .......................... 83 Figure 4-7. Sensor layout and configurations for a clamped square plate in FE mode (dimensions in mm) ........................................................................................................................................... 84 Figure 4-8. Sensor layout and configurations for a clamped rectangular plate in FE model (dimensions in mm) ...................................................................................................................... 84 Figure 4-9. Visualization of simulation cases used for the damage detection model ................... 85 Figure 4-10. ID definition for the simulation cases ...................................................................... 86 Figure 4-11. Image-based PR methodology classification results (simulation case S-1-20-1-256) ....................................................................................................................................................... 87 Figure 4-12. Pattern anomaly measure index based on FE simulation of load case 1 and damage scenario 1 ...................................................................................................................................... 88 Figure 4-13. Damage indices obtained from pattern focal plane concept (simulation case S-1-20- 1-256) ............................................................................................................................................ 89 Figure 4-14. Image-based PR approach classification results in terms of identifying new damaged patterns:(a) simulation case S-1-40-1-256, (b) case S-1-60-1-256, (c) case S-1-80-1-256 .......... 90 Figure 4-15. Damage indices obtained from focal plane theory for new identified damaged patterns: (a) case S-1-40-1-256, (b) case S-1-80-1-256 ................................................................ 90 Figure 4-16. Image-based PR approach classification results (load case 1 and damage scenario 2) ....................................................................................................................................................... 91 Figure 4-17. Pattern anomaly measure based on FE simulation of load case 1 and damage scenario 1..................................................................................................................................................... 92 Figure 4-18. Damage indices obtained from pattern focal plane theory (load case 1 and damage scenario 2) ..................................................................................................................................... 93 xiii Figure 4-19. (a), (b), and (c): Damage indices (load case 2 and damage scenario 2), (d), (e), and (f): Damage indices (load case 3 and damage scenario 2) ............................................................ 97 Figure 4-20. Damage detection performance for damage scenario 2 and load case 3 for varying noise level: (a) Validation data, (b) Test data ............................................................................... 99 Figure 4-21. Damage detection performance for different sensor configurations with varying noise level: (a) Training data, (b) Test data .......................................................................................... 100 Figure 5-1. Flow chart of the proposed damage detection strategy using PR and conditional probability chain ......................................................................................................................... 104 Figure 5-2. Illustration of PR method based on anomaly detection ........................................... 105 Figure 5-3. Classification results of identified patterns (images) based on image-based PR approach ...................................................................................................................................... 108 Figure 5-4. Illustration of the proposed spatial-temporal conditional probability chain ............ 110 Figure 5-5. Schematic illustration of the determination of probabilities PC and PNC ............... 111 Figure 5-6. Spatial-temporal representation of sensor node values based on the probability of occurrence of binary events: (a) & (b) PC and PNC for normal pattern 1, (c) & (d) PC and PNC for damaged pattern 8 ................................................................................................................. 113 Figure 5-7. Frequency of normal patterns with time for (a) Damage class 3, (b) Damage class 5 ..................................................................................................................................................... 114 Figure 5-8. Frequency of abnormal patterns with time for (a) Damage class 3 (Patterns 7 to 11), (b) Damage class 5 (Patterns 7 to 11), (c) Damage class 3 (Patterns 12 to 16), (d) Damage class 5 (Patterns 12 to 16) ....................................................................................................................... 114 Figure 5-9. Average frequency of abnormal patterns with respect to localized stiffness reduction: (a) Patterns 4 and 6 (patterns due to delay) for damage class3, (b) Patterns 4 and 6, damage class 5, (c) Patterns 8 and 16, damage class 3, (d) Patterns 8 and 16, damage class 5 ........................ 115 Figure 5-10. Variation of mean deviation of (PC-PNC) with time for damage class 5 (a) Patterns 7, 8, and 12, (b) Patterns 13, 15, and 16 ..................................................................................... 116 Figure 5-11. Reliability factor for abnormal patterns with respect to time for patterns 7 to 11 for: (a) Damage class 3 (localized 40% stiffness decrease), and (b) Damage class 5 (localized 60% stiffness decrease) ....................................................................................................................... 118 Figure 5-12. Reliability factor for abnormal patterns with respect to time for patterns 12 to 16 for: (a) Damage class 3 (localized 40% stiffness decrease), and (b) Damage class 5 (localized 60% stiffness decrease) ....................................................................................................................... 118 xiv Figure 5-13. Surface interpolation of focal density of binary data for (a) normal pattern 1, (b) damaged pattern 8 ....................................................................................................................... 119 Figure 5-14. Surface interpolation of focal density of binary events for pattern 16 (damaged pattern) ........................................................................................................................................ 119 Figure 5-15. (a) Detection performance with respect to stiffness reduction for calibration and testing data, (b) Comparison of the model’s performance for different damage scenarios ........ 121 Figure 5-16. (a) Damage detection accuracy for damage class 3 for varying noise level, (b) Damage detection performance for the 7 damage classes with respect to noise level .............................. 122 Figure 5-17. Proposed methodology for energy-efficient SHM using a probabilistic approach 123 Figure 5-18. Strain response of the cantilever plate: (a) Node #1, and (b) Node #4 .................. 124 Figure 5-19. The PDF plot of delivery delay for sensors/strain gages 1 to 6 for experimental plate: (a) and (c) intact plate, (b) and (d) damaged plate ...................................................................... 125 Figure 5-20. Schematic of a proposed probabilistic approach for delayed signal reconstruction ..................................................................................................................................................... 127 Figure 5-21. Implementation of the proposed algorithm for signal reconstruction .................... 128 Figure 5-22. Identified patterns on experimental plate using image-based PR approach .......... 130 Figure 5-23. (a) Optimization of the kernel parameter C, (b) Optimization of the kernel parameter γ, (c) Selection of optimal kernel function, and (d) Tuning optimal kernel degree .................... 131 Figure 5-24. Confusion matrix using SVM for different size of data subsets ............................ 132 Figure 5-25. (a) ROC curves based on SVM classification algorithm for different size of data subsets, (b) a close-up view of the ROC curves ......................................................................... 134 Figure 5-26. CMC curve with SVM for different size of data subsets ....................................... 135 Figure 6-1. Proposed machine learning framework for energy-lean SHM ................................ 140 Figure 6-2. (a) Schematic of an aircraft stabilizer wing with integrated self-powered sensing system, (b) FE model of horizontal stabilizer wing, (c) Transverse acceleration distribution from FE analysis (4000th sec) .............................................................................................................. 141 Figure 6-3. (a) Total lift pressure profile across wing chord length (at aircraft speed 800 km/h), (b) Approximation of lift aerodynamic loading on aircraft horizontal stabilizer wing .................... 142 Figure 6-4. (a) Aircraft stabilizer geometry and sensor configuration, (b) Representation of sensor node values as a pattern (image matrix) ..................................................................................... 143 xv Figure 6-5. Average delivery delay response of sensor nodes for different harvested energy levels: (a) & (b) intact stabilizer, (c) & (d) damaged stabilizer ............................................................. 145 Figure 6-6. Schematic illustration of neighboring/spatial concept of the data fusion model ..... 147 Figure 6-7. Schematic illustration of spatial-temporal aspect of data fusion model .................. 147 Figure 6-8. Implementation of the proposed algorithm for constructing the mask matrix ......... 149 Figure 6-9. Schematic illustration of the proposed algorithm for constructing the mask matrix M3 (average) ..................................................................................................................................... 150 Figure 6-10. Implementation of the proposed algorithm for constructing the modified time-delayed matrix .......................................................................................................................................... 151 Figure 6-11. Schematic illustration of low-rank matrix completion technique .......................... 153 Figure 6-12. Graphical representation of SVD of a matrix X ..................................................... 155 Figure 6-13. Illustration of the structure of observed delayed matrix in terms of low-rank and sparse matrices ....................................................................................................................................... 156 Figure 6-14. Implementation of the soft-impute algorithm ........................................................ 157 Figure 6-15. Pattern classification results in term of identified patterns using image-based PR approach ...................................................................................................................................... 161 Figure 6-16. Pattern focal density of binary events for (a) pattern 1, (b) pattern 2, (c) pattern 5, (d) pattern 6, and (e) pattern 7 .......................................................................................................... 162 Figure 6-17. Number of events received at the sink for different harvested energy levels: (a), (b), (c), and (d) intact stabilizer, (e), (f), (g), and (h) damaged stabilizer ......................................... 164 Figure 6-18. Imputation accuracy for different mask matrix vs harvested energy: (a) intact stabilizer, (b) damaged stabilizer ................................................................................................ 166 Figure 6-19. Bayesian optimization algorithm results for damage detection model based on mask matrices M1 and M2: (a) & (b) ECC = 0.55Θ, (c) & (d) ECC = 1.0Θ ....................................... 168 Figure 6-20. Function evaluations vs. minimum objective values for mask matrices M1 and M2: (a) & (b) ECC = 0.55Θ, (c) & (d) ECC = 1.0Θ .......................................................................... 169 Figure 6-21. Damage detection accuracy using K-NN algorithm with different number of k: (a) & (b) ECC = 0.55Θ, (c) & (d): ECC = 1.0Θ ................................................................................... 170 Figure 6-22. Damage identification accuracy for varying mask matrices and harvested energy levels: (a) validation data, (b) test data ....................................................................................... 174 xvi Figure 6-23. Damage detection accuracy for varying noise levels based on: (a) to (c) minimum mask matrix M1, (c) to (d) average mask matrix M2 ................................................................. 175 Figure 6-24. Damage detection accuracy for varying noise levels based on: (a) to (c) minimum mask matrix M1, (c) to (d) average mask matrix M2 ................................................................. 177 Figure 6-25. Surface plot of average delivery delay for simply supported plate and damage class1 ..................................................................................................................................................... 178 Figure 6-26. The PDF plot of delivery delay for sensors 5 to 9 for numerical plate: (a) and (c) intact plate, (b) and (d) damaged plate ................................................................................................. 179 Figure 6-27. The PDF plot of delivery delay for sensors 1 to 6 for experimental plate: (a) and (c) intact plate, (b) and (d) damaged plat ......................................................................................... 180 Figure 6-28. Implementation of the proposed statistical approach ............................................. 181 Figure 6-29. Classification results of identified patterns based on image-based PR approach (numerical plate) ......................................................................................................................... 183 Figure 6-30. Imputation accuracy for different mask matrices (numerical plate) ...................... 184 Figure 6-31. Visualization of damage indices (standard deviation and kurtosis) (numerical plate): (a) and (b) intact class, (c) and (d), damage class 1, (e) and (f) damage class 4 ........................ 185 Figure 6-32. PDF plots of intact class and damage class 1 for time interval 50 -75 sec (numerical plate): (a) sensor 5, (b) sensor 6, (c) sensor 7, and (d) sensor 10 ............................................... 187 Figure 6-33. PDF plots of intact class and damage class 4 for time interval 50 -75 sec (numerical plate): (a) sensor 5, (b) sensor 6, (c) sensor 7, and (d) sensor 10 ............................................... 187 Figure 6-34. Classification results of identified patterns using image-based PR approach (experimental plate) .................................................................................................................... 189 Figure 6-35. Visualization of damage indices (standard deviation and kurtosis) (experimental plate): (a) and (b) intact plate, (c) and (d), damaged plate .......................................................... 189 Figure 6-36. PDF plots of intact and damaged cantilever plates: (a) sensing node 2, (b) sensing node 3, (c) sensing node 5, and (d) sensing node 6 .................................................................... 191 Figure 7-1. Data interpretation framework employing machine learning for self-powered SHM ..................................................................................................................................................... 197 Figure 7-2. Imputation accuracy for different mask matrix vs harvested energy: (a) intact stabilizer, (b) damaged stabilizer ................................................................................................................. 199 Figure 7-3. (a) Selection of optimal kernel function, (b) Tuning optimal kernel degree, (c) Optimization of the kernel parameter C, and (d) Optimization of the kernel parameterγ .......... 201 xvii Figure 7-4. Damage detection accuracy for varying mask matrices and harvested energy levels on validation and test data based on SVM ....................................................................................... 202 Figure 7-5. Confusion matrices for test data based on SVM: (a) Matrix M1 and ECC=0.55Θ, (b) Matrix M1 and ECC=1.0Θ, (c) Matrix M2 and ECC=0.55Θ, (d) Matrix M2 and ECC=1.0Θ .. 203 Figure 7-6. ROC curve based on SVM classification for varying mask matrices and harvested energy levels ............................................................................................................................... 204 Figure 7-7. K-NN classification accuracy for damage detection model with varying number of k: (a) & (b) ECC = 0.55Θ, (c) & (d) ECC = 1.0Θ .......................................................................... 205 Figure 7-8. Damage detection accuracy for varying mask matrices and harvested energy levels on validation and test data based on k-NN with optimal k .............................................................. 205 Figure 7-9. Confusion matrix for test data based on k-NN: (a) Matrix M1 and ECC=0.55Θ, (b) Matrix M1 and ECC=1.0Θ, (c) Matrix M2 and ECC=0.55Θ, (d) Matrix M2 and ECC=1.0Θ .. 206 Figure 7-10. ROC curve based on k-NN classification for varying mask matrices and harvested energy levels ............................................................................................................................... 207 Figure 7-11. ANN performance: (a) matrix M1 with 0.55Θ, (b) matrix M1 with 1.0Θ, (c) matrix M2 with 0.55Θ, (d) matrix M2 with 1.0Θ ................................................................................... 208 Figure 7-12. Damage detection accuracy for varying mask matrices and harvested energy levels on validation and test data based on ANN .................................................................................. 209 Figure 7-13. Confusion matrix for test data based on ANN: (a) Matrix M1 and ECC=0.55Θ, (b) Matrix M1 and ECC=1.0Θ, (c) Matrix M2 and ECC=0.55Θ, (d) Matrix M2 and ECC=1.0Θ .. 210 Figure 7-14. Damage detection accuracy for varying noise levels based on minimum mask matrix M1 and different learning algorithms: (a) to (c) Validation data, (d) to (f) Test data ................ 211 Figure 7-15. Imputation accuracy for different mask matrices ................................................... 212 Figure 7-16. ROC curve based on SVM and k-NN classification for varying mask matrices ... 213 Figure 7-17. Confusion matrices for test data based on: (a) SVM with matrix M1, (b) SVM with matrix M2, (c) k-NN with matrix M1, (d) k-NN with matrix M2 ............................................... 214 xviii CHAPTER 1 1 INTRODUCTION 1.1 Motivation and Vision Structural health monitoring (SHM) is rapidly developing as a multidisciplinary technology solution for condition assessment of civil, mechanical, and aerospace structures due to progress in sensing technologies. SHM methods that can detect and localize damage have been the subject of much study [1]. Energy availability for wireless sensor networks (WSNs) to collect and communicate data has been a major concern for SHM systems. Battery life for wireless sensor nodes is a crucial challenge that has in turn motivated the development of energy harvesting technologies that are now allowing self-powered sensor networks to become a reality [2–4]; the continuing goal being to reach pervasiveness in SHM sensing networks by minimizing power demands. Recent advances in energy harvesting technologies have led to the evolution of self- powered SHM systems that are energy-efficient. The energy-lean data-driven SHM platform presented in this research is inspired by an energy-efficient through-substrate wireless sensor network [5,6] integrating discrete floating gate self-powered sensors [7–9] and an energy-aware data communication protocol concept termed “pulse switching protocol” [10–12]. The analog piezoelectric self-powered sensor operates on a gate with a non-volatile memory that demands very low operating energy; for which it logs/records data in a discrete and asynchronous manner. The network communication part uses the noted pulse switching architecture, in which a minimal number of ultrasonic pulses/signals communicate information on the occurrence of an event and its source. Such communication is made through low-power through-substrate ultrasonic pulse networking that is powered by energy harvested from structural vibrations. However, such self- 1 powered sensing technology creates time delays on the generated data due to the power budgets demanded for sensing and communication. Further, the communicated data is in a binary format, resulting in discrete and asynchronous event-based binary information at the SHM processor. The nature of data collated from such self-powered sensing technology demands new data analysis and algorithmic frameworks for data-driven SHM. Traditional SHM approaches, e.g., vibration-based techniques [13], deal with continuous data that are physically based. A key in SHM methods with continuous data availability is the possibility of operating on the time-history data to determine response parameters. The effectiveness of such techniques thus highly depends on the availability of continuous time-history data. This is while restrictions such as measurement noise, modeling errors, environmental influences, etc., could significantly affect the data availability for the SHM system. Therefore, conventional SHM techniques cannot be effectively used for damage detection with discrete binary and noisy/limited data. In contrast, the data-driven SHM methodology presented in this dissertation is capable of overcoming the noted drawbacks. The core idea herein is to develop and evaluate advanced data interpretation frameworks to learn from noisy and missing features of time-delayed binary and missing/sparse data through artificial intelligence (AI) techniques, i.e., machine learning (ML) and pattern recognition (PR), and to detect damage based on integration of such AI methods, statistical, and probabilities approaches. It is noted that although the proposed data interpretation frameworks are developed for SHM and condition assessment with time-delayed binary and sparse signals, it is expected that the developed frameworks could be potentially used to interpret heterogeneous data collected from other disciplines and applications such as ‘smart cities’ and ‘intelligent transportation systems’. 2 1.2 Background and State of Knowledge 1.2.1 SHM and Damage Identification SHM deals with the real-time sensing and assessment of the safety and long-term sustainability and detect potential damages in civil, mechanical, and aerospace structures [1]. SHM involves monitoring of a structure through data collected from sensors, extracting damage sensitive features, and interpreting the extracted features for condition assessment of the structure/infrastructure. Numerous studies have been carried out to develop appropriate damage detection methods to extract critical structural information from measured data, as reported by Sohn et al. [1] and Doebling et al. [14]. Damage identification methods can be categorized as local-based or global-based approaches. The former refers to methods that identify damage based on monitoring structures as their component length-scales, e.g., non-destructive evaluation (NDE) techniques, while the latter implies numerical methods that consider the global vibration characteristics, e.g., vibration-based. NDE techniques [15], such as ultrasonic inspection methods and X-ray methods, are usually considered as local-based damage detection methods and conducted off-line after the damage is located. Such methods demand a priori knowledge of the damage location, while such location has to be readily accessible for testing. Clearly, such criteria cannot be guaranteed for cases in civil and aerospace engineering. On the other hand, these methods proved to be costly and time-consuming. To tackle noted challenges, vibration-based damage detection methods [15–17] have been developed as a global-based approach. For vibration-based damage detection techniques, the damage-induced variations in the physical properties, i.e., mass, damping, and stiffness, lead to detectable changes in modal properties, e.g., natural frequencies, mode shapes, modal damping, etc. Accordingly, damage can be diagnosed by interpreting the variations in vibration features of the structure. Nonetheless, environmental and 3 operating condition variability, and the physical size of the structure being monitored present notable challenges for vibration-based damage detection. A comprehensive review regarding structural damage identification and condition assessment methods can be found in studies by Farrar et al. [14,15,18,19], Sohn et al. [1], and Fan et al. [13]. SHM and damage identification methods can also be classified as ‘model-driven’ and ‘data- driven’ approaches [20–22]. A model-driven approach uses a numerical model of the structure, e.g., based on the finite element (FE) method, that correlates inconsistencies between the measured and model-generated data for damage detection. Although numerous studies have been conducted to develop model-driven approaches, these methods suffer from several shortcomings. First, the approach is computationally inefficient because it requires an iterative analysis of a computer simulation model. Second, results obtained from the simulation might not be accurate enough for precise evaluation of the structure. By contrast, in a data-driven approach the model is created through the learning gained from measured/sensed data. Damage can thus be detected by conducting a comparison among the measured data and a model. In fact, a data-driven model uses information from previously collected sensor data (i.e., training data) and employs learning algorithms for data analysis. Due to noted features, data-driven SHM approaches have been recently studied by the SHM community. As an example, Figueiredo et al. [23] investigated artificial neural networks, Mahalanobis distance, singular value decomposition techniques, and factor analysis to study environmental variability and its effect on damage detection in civil structures. Butcher et al. [24] examined the use of artificial neural networks and extreme learning machine methods for SHM in mesh-reinforced concrete structures. The study revealed that these algorithms can outperform traditional methods. Recently, a new class of data-driven SHM method was introduced by Nagarajaiah et al. [25], where the authors presented a new paradigm for damage 4 detection based on modelling and harnessing sparse and low-rank data structures. They demonstrated that the proposed method is able to effectively address structural dynamics, identification, monitoring, data sensing and management problems. The noted studies highlighted the importance of data-driven techniques for structural damage identification. Progress in self- powered sensor wireless networks for SHM have motivated the development of power-efficient SHM approaches, which can be classified as SHM data-driven methods. One such SHM system is an energy-lean data-driven SHM platform presented in this research. 1.2.2 Sensing Technologies for SHM SHM is made possible by collecting data from a sensor network installed on the monitored structure and analyzing the measurements for structural condition assessment. During the last decade, significant attention has been devoted to the deployment of new sensing technologies that can advance the state-of-art in SHM and structural monitoring. Recent development and applications of smart WSNs for SHM can be found in the published literature [26,27]. Yet, difficulties in deploying and maintaining traditional wired sensors is a fundamental drawback of using such systems. Also, managing vast amount of data collected from a dense array of wired sensors is very challenging and costly. To overcome these shortcomings, wireless sensor networks (WSNs) have been introduced and are increasingly being used as alternatives in traditional structural monitoring systems [27–32]. Clearly, non-rechargeable batteries for WSNs have limited capacity, and thus the energy required for measuring and sensing data becomes a significant concern. Harvesting power from ambient energy was thus introduced to address noted problem and it has attracted significant attention in recent years [7,33–37]. Energy harvesting technologies, which refer to convert mechanical energy into electrical energy, have been developed to extend the lifetime of WSNs by addressing the energy constraint problem [2,33,35]. Recently, self- 5 powered sensors have become a reality by overcoming the gap between the achievable scavenged energy and the energy required for sensing, computing and communication [3,36]; thus leading to the emergence of power-efficient WSNs. Recently, researchers at Michigan State University (MSU) developed an energy-aware through-substrate ultrasonic self-powered sensor network integrating piezo-floating-gate self- powered sensors and energy-efficient pulse switching protocol for SHM [6]. The proposed through-substrate ultrasonic self-powered sensor network is presented in Figure 1-1. The electrical architecture of the wireless sensors operates on gates with a non-volatile memory that demands very low energy. As a result, the sensor logs data in a discrete and asynchronous manner. The communication part, on the other hand, employs what is termed pulse switching protocol, in which a minimal number of ultrasonic pulses (i.e., compression stress waves traveling through the material substrate) are used to represent event location and forwarding information. The pulse signals received at the sink provide knowledge regarding the occurrence of the events and their location of origin, where such information will be further used for SHM, condition assessment, and decision making. Figure 1-1. A through-substrate self-powered sensor network 6 1.2.3 Artificial Intelligence in Structural Engineering Artificial intelligence (AI) is proving to be an efficient alternative approach to classical modeling techniques [38–42]. AI refers to the branch of computer science that develops algorithms and software with human-like intelligence. The term “AI” was introduced at a workshop held in Dartmouth college in 1956 [43]. AI refers to computational methods attempting to simulate human cognition capability through symbol manipulation and symbolically structured knowledge bases to solve engineering problems that defy solution using conventional methods. AI has been developed based on the interaction of various disciplines; namely, computer science, information theory, cybernetics, linguistic, and neurophysiology. Compared to traditional methods, AI offers advantages to deal with problems associated with uncertainties and is an effective aid to solve such complex problems. In addition, AI-based solutions are good alternatives to determine engineering design parameters when testing is not possible, thus resulting in significant savings in terms of human time and effort spent in experiments. AI is also able to make the process of decision making faster, decrease error rates, and increase computational efficiency. In recent years, there has been a growing interest in the use of AI in all engineering domains, and it has fueled many visions and hopes. The use of AI in civil engineering has been the topic of previous review articles. Adeli et al. [44] presented a multiparadigm learning technique, where the authors demonstrated that the performance can be notably enhanced by skillful integration of different AI branches, including neural networks, genetic algorithms, fuzzy sets, and parallel processing. An extensive study of evolutionary computation, a branch of AI, in the context of structural design was conducted by Kicinger et al. [45]. Lio et al. [46] carried out a review of studies concerning the application of metaheuristics as optimization techniques to address issues faced in the lifetime of a construction or engineering project. A survey on different AI methods 7 (e.g., fuzzy logic, evolutionary computation, neural networks, swarm intelligence, expert systems, etc.) for civil engineering was conducted by Lu et al. [47]. Shahin et al. [48] studied applications of AI in geotechnical engineering; and Saka et al. [49] conducted a survey on mathematical and metaheuristic algorithms in design optimization of steel frame structures. Adeli et al. [50] carried out a review on progress in the optimization of high-rise buildings; and a survey on the applications and methodologies of the fuzzy multiple criteria decision-making techniques was conducted by Mardani et al [51,52]. Civil and structural engineering community has witnessed an extensive growth in the use of different AI branches/methods in its diverse areas. It is acknowledged that uncertainties are an unavoidable part of structural engineering problems. For example, in seismic design earthquake demands are not known with precision. In SHM, there are uncertainties in the amplitude of the input excitation, measurement noise, and spatial density of measurements. All of the aforementioned problems can be modeled and treated as uncertainties [53]. AI is then able to deal with such uncertainty problems. The data-driven SHM methodology presented in this research is based on integration of self-powered sensing and different AI algorithms. 1.3 Research Gap A review of the literature indicates that traditional SHM techniques categorized as model-driven approaches deal with continuous data from the physical response of the structure. The hurdles for these time-based methods are dependence on the availability of the continuous time-history data and the large amount of data that needs to be analyzed. It is acknowledged that data availability for SHM can be substantially affected by limitations such as measurement noise, modeling errors, and environmental influences. Further, interpreting huge amount of data being produced by such 8 SHM approaches is costly and challenging. Therefore, even though conventional SHM techniques (e.g., vibration-based) have been widely used for damage assessment, it is not guaranteed that such methods lead to reliable and efficient damage identification models when dealing with discrete, noisy, and incomplete sensor data. On the other hand, despite several advantages offered by the through-substrate self-powered sensor network being considered in this research, there is a considerable loss of information on the collected data, i.e., loss of data resolution. Resolution can be defined, in general, by means of three components: the number of data points in the domain, the actual value at each point, and temporal availability of the data. In theory, resolution in SHM approaches based on continuous data are affected only by the number of data points (i.e., sensor nodes). Yet, the discrete binary data used in this research results in an obvious loss of resolution from the other two aspects, that is, the actual value at sensor nodes is not known and the data are missed or delayed due to lack of power at the sensor node. Time-delay issues in SHM applications can be minimized by using time synchronization protocols, and the few milliseconds of time-delay in traditional WSNs have been shown to have little impact on the accuracy of the SHM system [54–56]. Nonetheless, the expected delay in a through-substrate network running on vibration-harvested energy can be much larger, and the effect of time-delay on the SHM system cannot be disregarded since the sensor delay values are neither small nor negligible. However, no efficient approach has yet been proposed for damage detection with large values of delay provided by a smart sensing technology. The noted issues thus imply the necessity for developing novel data interpretation frameworks for SHM that accurately analyze discrete time-delayed binary and incomplete data. Further, it is essential to develop advanced data mining frameworks that embrace state-of-the-art in signal processing and data analysis techniques. 9 1.4 Research Hypothesis and Objectives The SHM platform presented in this study uses self-powered sensing units that are empowered from the energy harvested from the vibration of the monitored structure. In fact, the power required for signal sensing, storage, and communication is harvested directly from the signal being sensed. Yet, there is a considerable loss of signal/data resolution as a result of using self-powered sensing technology. Data received at the SHM processor are discrete, binary, and time-delayed, i.e., incomplete. Thus, a major obstacle in successful application and development of the self-powered SHM platform is lack of advanced algorithmic frameworks for analysis and interpretation of limited sensor data collected from noted smart self-powered technology. The main objective of the present study is to develop novel AI-based data interpretation frameworks to interpret such noisy/sparse data and to evaluate the performance of the proposed SHM system employing such data mining frameworks for damage identification in civil infrastructures and aerospace structures. The hypothesis driving the research reported herein is that advanced data interpretation frameworks based on integration of AI (i.e., machine learning and pattern recognition), and statistical/probabilistic methods can be developed and used for energy- lean data-driven SHM with discrete time-delayed binary and noisy/incomplete data (signals). To cope with the discrete nature of binary data, it is hypothesized that interpretation of discrete data over a domain resembles a pattern recognition problem due to its pixelated nature. Therefore, it is assumed that discrete binary data extracted from through-substrate sensor network using pulse- switching technology can be interpreted by pattern recognition methods for SHM purposes based on image data analysis concepts. An image-based pattern recognition framework based on anomaly detection is developed to represent self-powered sensor nodes responses as a 10 pattern/image. Thereafter, different classifiers such as Bayesian and k-nearest neighbor with different feature extraction techniques are utilized for pattern classification. To tackle the time-delayed nature of the data, it is postulated that machine learning employing matrix completion and a data fusion model can be used to learn from time-delayed and sparse data through recovery/reconstruction of the data with noise and missing/sparse features. In addition, it is assumed that a probabilistic approach can be employed to model time-delay and recover the missing/sparse signals. Finally, it is hypothesized that machine learning algorithms and statistical/probabilistic approaches can be utilized for damage detection/classification with reconstructed time-delayed binary (or recovered noisy) data. 1.5 Research Significance The concept of smart self-powered SHM has been recently gaining notable attention. Nonetheless, there are still some challenges to develop and implement a reliable power-efficient SHM system. Notably, major issues are to develop innovative solution to provide the energy required to power self-powered sensing system, and to establish advanced data interpretation frameworks for analysis of noisy and sparse data being produced by a network of self-powered sensors integrated with data transmission protocols. The present research addresses the noted issues through the development of novel data interpretation frameworks that advance the state-of-the-art in signal processing and data miming techniques for data-driven SHM integrating smart energy harvesting and energy-aware signal communication approaches. The developed energy-lean SHM platform is based on asynchronous self-powering technology, which refers to an energy-efficient scavenging technique where the energy for sensing, computation, storage, and communication is harvested from the signal being sensed. As a result, 11 the proposed SHM system effectively addresses the drawbacks associated with the deployment and maintenance of traditional wired sensors, while it possesses major advantages over conventional energy harvesting techniques and wired sensors requiring an external power source. On the other hand, the proposed through-substrate ultrasonic pulse communication architecture tackles the limitations of traditional approaches, such as packet switching, through significant energy savings in terms of transmission cost. From a data interpretation perspective, the developed data mining frameworks, which involve data preprocessing, learning, and classification phases, are established based on integration of machine learning, pattern recognition, data fusion, statistical, and probabilistic methods to accurately analyze and interpret complicated self-powered sensor data/signals. Therefore, the integrated sensing system with advanced data interpretation frameworks notably improve safety and decrease maintenance costs of civil infrastructures and aerospace structures. Figure 1-2 schematically shows the concept of an energy-lean SHM system with integrated self-powered sensing units for damage identification and condition assessment in an aircraft wing. Figure 1-2. Aircraft wing with integrated self-powered sensing technology The concept of a smart city is to use smart sensors within the city’s infrastructures to ensure sustainability, safety, and efficiency. Recently, the application of data mining frameworks and the 12 Internet of Things (IoT) paradigm to smart cities are generating much research interest. The power- efficient SHM platform presented in this research can be employed within the context of smart cities. The developed data interpretation frameworks can also be incorporated with IoT and used to interpret heterogeneous data being collected by sensors deployed in smart cities (see Figure 1-3). Furthermore, the self-powered sensing technology can be utilized to develop pioneering solutions to supply the required energy for sensors in smart cities. Figure 1-3. A schematic of a smart city with deployed sensors Finally, the data interpretation frameworks developed in this research employing machine learning can be potentially used in the areas of intelligent transportation systems and autonomous vehicles, as shown in Figure 1-4. That is, the machine learning frameworks can be used to predict drivers’ behavior, while the frameworks also have the potential to be used to advance human factors research by predicting their effect on decision making. The developed machine learning- 13 based data mining frameworks can be further modified and used as powerful decision making tools to interpret big data to understand mobility patterns in smart cities. Figure 1-4. Typical intelligent transportation system analytics architecture 1.6 Organization of the Dissertation This dissertation has been organized into eight chapters. The remainder of this dissertation is organized as follows. Chapter 2 outlines knowledge background and literature review for AI techniques, including machine learning, pattern recognition, and deep learning, for SHM and damage identification. Further, a through-substrate ultrasonic self-powered sensor network is reviewed and major advantages of using this self-powered sensing technology are discussed. Chapter 3 deals with the development of a pattern recognition (PR)-based algorithmic framework for power-efficient SHM with full discrete binary signals (i.e., signals without delay). Chapter 4 documents the evaluation on robustness and effectiveness of the SHM methodology employing the developed PR-based algorithmic framework with full discrete binary data/signals. Chapter 5 14 describes the development details of a probabilistic-based SHM strategy for interpretation of time- delayed data/signals, in which different probabilistic frameworks for predicting and modeling delay are discussed. Chapter 6 presents a novel machine learning-based data interpretation framework for energy-lean SHM using time-delayed and incomplete/sparse sensor data. Chapter 7 explores the robustness of the proposed machine learning-based data mining frameworks with respect to harvested energy availability and different learning algorithms. Finally, chapter 8 summarizes the work performed under this project, provides conclusions and discusses future research directions. 15 CHAPTER 2 2 SHM WITH ARTIFICIAL INTELLIGENCE AND SELF-POWERED SENSING 2.1 Overview This chapter presents the through-substrate self-powered wireless sensor network used for developing the proposed energy-lean SHM platform. Further, a state-of-the-art review to highlight the recent advances on use of emerging AI algorithms and classifiers for structural engineering and SHM is provided. This review identifies potential research avenues and emerging trends for employing AI methods. Theoretical concepts of several AI methods used in this research work are also described in this chapter. This chapter was partially published as a review article in the journal Engineering Structures [52]. 2.2 A Through-substrate Ultrasonic Self-powered Sensor Network for SHM The energy-lean SHM platform presented in this research is based on a through-substrate wireless sensor network [10,12,5] employing discrete floating gate self-powered sensors [7–9,57]. Self- powering is achieved by energy scavenging through piezoelectric transducers from the sensed signal, and measurements are stored as events in bins corresponding to different amplitudes. As a result, the sensor logs data in a discrete and asynchronous manner. The event statistics (i.e., binary signals) are stored directly onto a non-volatile memory formed by floating-gate elements. These memory elements can retain their contents even during brown-outs and show negligible leakage (less than -50dB over 8 year period), thus eliminating the need for a power supply when a binary signal is buffered as the sensor node. On the other hand, the communication part employs what is termed “pulse switching protocol,” in which a minimal number of ultrasonic pulses (i.e., compression stress waves traveling through the material substrate) are used to represent event 16 location and forwarding information. The primary advantage of pulse switching is the considerable energy savings in terms of transmission cost when compared to traditional approaches, such as packet switching. It represents the information to be communicated as a specific temporal pulse pattern with respect to a synchronized frame structure, thus encoding the location and forwarding information. The envisioned self-powered sensing platform (see Figure 2-1) consists of a network of low-power through-substrate ultrasonic pulse networking (TUPN) units that communicate with each other using the noted energy-lean pulse switching protocol and are powered by energy harvested from the substrate’s (i.e., structural) vibrations. Figure 2-1. Energy-lean SHM using a through-substrate ultrasonic self-powered sensor network The TUPN units are incorporated with the self-powered sensors into combined sensor nodes, or network nodes. The wireless network nodes are distributed across the structure being monitored, as illustrated in Figure 2-1(a) for a bridge girder and an aircraft wing. The event transportation network formed through the structure’s substrate itself using a cellular network abstraction is shown in Figure 2-1(b). Also, Figure 2-1(c) shows the structure of multi-hop pulse network. A TUPN detected event (based on a local measurement) prompts the generation of an ultrasonic pulse by the TUPN. The generated pulse is transported via multi-hop communication between TUPNs 17 to a sink node (or base station), where accumulative information is received from sensors across the structure to infer the structure’s condition. It is to be noted that the TUPN’s ability to forward a pulse to the network strongly depends on the locally available (harvested) power. Signals can be localized by a receiver by observing the temporal position of a received pulse with respect to a reference frame. Pulse-based communication reduces the available information to a binary format, and this turns the received information into a discrete and time-based binary set at the SHM processor. Subsequently, the pulse signals received at the sink provide knowledge regarding the occurrence of the events and their location of origin. The resulting time-delayed binary and incomplete signals are the ones to be used for structural damage detection and condition assessment. To evaluate the proposed through-substrate network, an energy harvesting simulation model was developed to translate acceleration data into energy generation in plate-like structures [6]. Figure 2-2 shows a bridge rectifier-based piezoelectric energy harvester circuit model used for the energy harvesting model. Vibration-induced accelerations are thus transformed to electric power by means of a piezoelectric module on the harvester. The electric power is rectified and stored in a super-capacitor for driving the TUPN module operation. Acceleration values adequate for the energy harvesting model were thus chosen based on data from prior studies [58,6]. Figure 2-2. Piezoelectric harvester circuit model [6] 18 The cellular pulse networking (CPN) protocol is well suited for SHM applications where the information to be transmitted can be small, i.e., essentially binary information indicating the occurrence or absence of a structural event. The CPN protocol encodes the occurrence of an event using a single pulse transmission, but communication (or data transmission) of the event through the network is not immediate. When CPN is employed in a harvested-energy-powered sensor network for SHM there can be multiple factors contributing to the latency of event information delivery at the sink; such as distance from the source to the sink, energy availability, event merging or processing, and the frame size. Figure 2-3. Factors affecting one-hop event delivery delay when using CPN in energy- harvesting-powered wireless sensor networks As shown in Figure 2-3, energy availability depends on the harvested energy as well as its consumption due to communication, idling leakage, and sensing. Pulse transmission can only happen when an event is pending and there is sufficient energy to communicate it. As a result, there can be some event buffering time between the actual event generation and its transmission, which translates to per-hop event delivery delay (see Figure 2-3). This indicates that the measured 19 signal can be communicated with an unpredictable delay. It is further noted that there is no time interval, or step, for the sensor signals since they are discrete and random based on the availability of power (i.e., signals are not continuous). 2.3 Emerging AI Techniques for SHM and Condition Assessment AI is essentially defined as the ability of a machine to mimic intelligent human behavior, thus seeking to use human-inspired algorithms for approximating conventionally defiant problems. The main goals of AI research involve knowledge representation, reasoning, automated planning, learning, natural language processing, perception, robotics, and general intelligence [38–42]. Although AI and computational intelligence/soft computing pursue a similar goal, there is a slight difference between them. According to Bezdek [38], CI is a subset of AI. It is also important to distinguish AI from data science and big data. There is indeed a substantial overlap among these methods. Data mining/science is a cross-disciplinary field used to discover valuable insights and trends in a data set. Data mining techniques focus on the discovery of unknown properties in an area where there is limited knowledge. The data set, on the other hand, is called big data if it is big in terms of volume (i.e., number of data points or features per data point), velocity (i.e., large portions of data arriving in a small amount of time for analysis and mining), or variety (i.e., different types of data such as text, speech, images, etc.). Big data thus refers to large or complex data sets that are difficult to represent using conventional data processing techniques. Machine learning, a subfield of AI, is used to design a model to learn the trends, thus focusing on prediction based on known properties learned from the training data. Deep learning, a subset of machine learning, is a tool that concentrates on learning the representations and features of the data. Figure 2-4 schematically presents the noted different intelligent techniques and their correlation. 20 Figure 2-4. Illustration of the interrelation of different intelligent computational techniques Uncertainties are an unavoidable part of structural engineering problems. For example, in seismic design earthquake demands are not known with precision. In SHM there are uncertainties in the amplitude of the input excitation, measurement noise, and spatial density of measurements. Many uncertainties also exist in the models used to predict structural response, as well as those defining constitutive behavior. Geotechnical information for foundation design purposes is determined with limited information and/or based on laboratory tests with high levels of uncertainty. All of the aforementioned problems can be modeled and treated as uncertainties [53]. AI is able to deal with such uncertainty problems. For instance, AI methods have been used to solve uncertainty problems defined within the context of damage detection and system identification using finite element model updating [59]. Model updating can be used to identify physical parameters (e.g., stiffness of a structural component) for which a reduction in value is taken to indicate damage. However, such reduction may be simply due to statistical uncertainty. Thus, it is of importance to compute the uncertainty of the estimation to distinguish whether the reduction of a parameter is due to actual damage. The use of AI methods can also result in 21 significant time and cost savings, as well as increasing computational efficiency in many structural engineering tasks. Many of the AI branches, such as machine learning (ML), pattern recognition (PR), neural networks, fuzzy logic, evolutionary computation, deep learning (DL), expert systems, probability theory, discriminant analysis, swarm optimization, metaheuristic optimization, and decision trees, have been used in structural engineering. The number of research publications showing the use of these AI methods in structural engineering over the last decade is presented in Figure 2-5. As can be seen, the use of most methods has increased during the last decade. Nevertheless, the number of studies featuring techniques such as evolutionary computation, fuzzy logic, and expert systems has not had a notable change. Even though the use of neural networks has drawn a great attention from researchers, new studies on the use of such method has also remained rather constant over the last decade. In contrast, the significant increase in studies featuring the use of ML and PR is evident. Further, deep learning architectures, e.g., convolutional neural networks (CNNs), are gaining remarkable attention among the research community over the last few years. These observations motivated this research to concentrate on ML and PR, as they are emerging as the new computational intelligence paradigms in structural engineering. 22 Figure 2-5. Research publications on the use of different AI branches in structural engineering 2.3.1 Pattern Recognition Pattern recognition (PR) is a technique in which the main goal is to classify objects into a number of classes or categories. The objects, depending on the applications, could be images, signals, hand writing, speech, or measurements to be classified [60,61]. In PR, a pattern is represented by a set of features. Concepts from statistical decision theory are used to establish decision boundaries between pattern classes. The recognition system in PR consists of two modes, namely learning (training) and classification (testing), as shown in Figure 2-6. In the learning/training mode the 23 proper features for representing the input patterns are discovered by means of the feature extraction/selection module, and the classifier is trained/calibrated to partition the feature space. In the classification mode the input patterns are assigned to one of the classes using the trained classifier; while the performance of the designed classifier, i.e., classification error rate, is evaluated by the system evaluation module. Figure 2-6. Schematic of a PR system In general, PR methods can be categorized into two main categories: supervised PR and unsupervised PR. The supervised term refers to the condition when a set of labeled training samples are available. When there is no prior information regarding the class labels and the training data are not labeled, this is known as unsupervised PR, or clustering. These terms are further discussed in the following section. Another difference in PR methods is that of generative models versus discriminative models. If the aim is to discover the distribution of patterns in the model, this denotes the generative models in PR. The task for this case is to find out how the patterns can be modeled in the class. In this regard, the density function needs to be determined based on training data. On the other hand, the goal in discriminative PR models is to determine the model that discovers the decision boundary, thus learning the function and parameters of the decision boundary. Generative and discriminative PR models along with the algorithms used are shown in Figure 2-7. 24 Figure 2-7. Tree structure of generative and discriminative PR models and algorithms During the last decade, there has been a growing interest in the application of PR to structural engineering for purposes such as SHM/damage detection, earthquake engineering and seismic design, structural reliability, structural identification, and performance evaluation. As can be seen from Figure 2-8, the most common use of PR in structural engineering has been for SHM and damage identification. In general, two main approaches are commonly considered for damage detection: the inverse approach, known as system identification, and the forward approach, which relies on extracting information from the monitored structure. The computational complexity of the inverse approach, along with the physical importance of model updating, have motivated researchers to investigate methods from the second type of approach (forward) [62]. Therefore, PR is being most frequently utilized in the context of a forward approach for damage detection and SHM. 2.3.2 Machine Learning Machine learning (ML) is a major subfield of AI (see Figure 2-4) dealing with the study, design, and development of algorithms that can learn from the data itself and make predictions using learned data [63–66]. In fact, ML refers to the capability of computers to learn without being 25 explicitly programmed. ML based models can be predictive or descriptive to achieve knowledge from the data [67,68]. The scope and potential of ML is much more general than other AI methods, although it is a subset of AI and used in various disciplines; including computer science, information theory, control computational complexity, probability and statistics, financial market, and theory and philosophy [65]. Figure 2-8. Research publications on the use of machine learning and pattern recognition It is of importance to differentiate ML from other similar AI subsets including PR and DL. In general, PR and ML are closely related areas, as they fundamentally overlap in their scope. However, PR deals with methods for classification tasks, while ML focuses on algorithms utilized for learning. In fact, the major task of PR is recognition of patterns in data and to classify them, and it does not necessarily imply learning. ML systems, on the other hand, are designed to learn by themselves. Further, DL is considered a subset of ML (see Figure 2-4), in which the system has the ability to learn features from the data. Deep learning, in fact, is a tool to learn the representation of data. Once the representation is determined, the ML problem can be solved. Indeed, deep 26 learning transforms a problem/representation with high dimensionality to a lower dimensional representation. Depending on the resources of the training dataset, ML can be categorized as supervised, unsupervised, or reinforcement learning [63,66]. Machine learning (ML) methods have been increasingly adopted over the last decade for modelling real-world problems concerning structural engineering (see Figure 2-5). This is because of their enormous capacity to capture relations among input and output data that are nonlinear or complicated to formulate mathematically. The first uses of ML techniques in structural engineering have dealt with problems such as the development of management tools for structural safety [69], and information acquisition for the design of steel members [70]. In general, ML methods have been used for SHM and damage identification, optimization, performance evaluation, structural reliability and reliability assessment, and structural parameter identification (e.g., modeling material properties of concrete). Among these, SHM is the use to attain most attention during the last decade (see Figure 2-8). 2.3.2.1 Supervised Learning The goal of supervised learning is to build a model/function to accurately predict the unknown target output of future examples. Training samples in supervised learning are labeled and the key characteristic of the learning is the existence of a teacher that provides a cost or category label for each pattern in a training dataset, thus seeking to decrease the added cost for these patterns. If the objective of the ML model is to forecast continuous target variables, the task is said to be regression. However, if the aim is to predict discrete target variables the task is known as classification. 27 2.3.2.2 Unsupervised Learning The objective of unsupervised learning is to separate the training dataset into clusters such that the data in all clusters exhibits a high level of proximity. Unlike supervised learning, the labels for data are unavailable and there is no explicit teacher. Thus, the system itself forms the clusters from the input patterns. 2.3.2.3 Reinforcement Learning In reinforcement learning, or learning with a critic, no information is given regarding the desired category signal or explicit goals. Reinforcement algorithms are forced to learn optimal goals through trial and error. In fact, in order to maximize the model’s performance, reinforcement learning allows an agent to determine the ideal behavior within a specific context. Agents receive a numerical reward as a reinforcement signal encoding the success of an action’s outcome. The goal for the agent is then to learn to select actions maximizing the accumulated reward over time. Recent research reveals the successful practical applications of ML in different fields, such as: computer vision and image processing [71–76], speech recognition [77–82], computational finance [83–85], energy production [86–88], and computational biology [89–91]. In an ML domain an algorithm has to be developed to solve problems. Different methods from various fields have been adopted for such a purpose [92,93]. Therefore, ML enables exploiting the interaction form all these fields, which in turn leads to robust solutions using various domains of knowledge. Figure 2-9 illustrates some of most prominent algorithms used in the ML domain. 28 Figure 2-9. Machine learning categories with commonly adopted algorithms 2.3.3 Deep Learning Deep learning (DL), a branch of ML, is composed of networks that can learn unsupervised from unstructured/unlabeled data. DL architecture aims to learn the feature representation of the input data. In fact, DL is based on deep neural networks, i.e., neural networks with more than one hidden layer. In such an architecture, increasing the number of layers results in a deeper network. Examples of DL architectures include convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, deep belief nets, etc. Among these, CNNs are the DL architectures that have gained the most attention among the structural engineering community during last few years. CNNs are inspired by the visual cortex of animals [94]. They have been 29 mainly used in computer science and engineering for image recognition [95–100]. Unlike standard neural networks, CNNs are capable of capturing the 2D topology of pixels, while demanding fewer computations because of a pooling process and sparsely connected neurons. Further, CNNs are able to simultaneously extract and learn optimal features from the raw data. Recent studies [101,102] have demonstrated that CNNs can outperform conventional artificial intelligence methods in both accuracy and speed. Generally, CNNs leverage the following ideas: local connectivity, parameter sharing, and pooling/subsampling of hidden units. The network consists of three layer types, namely convolution, pooling, and fully connected layers. CNNs alternate between the convolutional and pooling layers and the output is a fully-connected layer with a nonlinear classifier, e.g., softmax classifier, thus estimating the conditional probability of each class. To introduce nonlinearity in the CNNs, a rectified linear unit (ReLU) is typically used as a nonlinear activation function. In addition, among the different optimization algorithms, gradient descent algorithms are mainly used to train CNNs. The basic components of CNNs are described in the following sub-sections. A schematic of a CNN architecture for image recognition is presented in Figure 2-10, where the network consists of three convolutional layers, three pooling layers, and three fully connected layers. For all layers in the network, ReLU is used as the activation function. Further, a softmax loss layer is appended to the fully connected layers for each classification task. 30 Figure 2-10. Schematic of a typical CNN architecture 2.3.4 Future Directions for use of Emerging AI methods in SHM and Smart Infrastructure Monitoring The most common use for PR and ML has been for SHM. DL architectures have also been utilized for SHM and damage identification. It is to be expected that the use of emerging AI methods in structural engineering will increase as their potential is they are better understood and as new methods are developed. Current and emerging applications of ML, PR and DL in structural engineering are shown in Figure 2-11. DL methods emerged to interpret big data in order to identify implicit features from it, and to classify the learned features. DL-based damage detection techniques have been found to be computationally efficient. Unlike conventional ML techniques that use hand-crafted features that result in high computational complexity, DL and CNNs use optimal features learned by the network, thus increasing the classification accuracy significantly. Further, the structure of the DL architecture, specifically one-dimensional CNNs, make their mobile and low-cost hardware implementation quite feasible. Therefore, it is expected that DL will play important role in the future generation of vision-based SHM systems, i.e., those based on computer vision techniques [103–105]. 31 Figure 2-11. Schematic of a typical CNN architecture The durability of civil infrastructures has nowadays become a big issue given the number of structures that need to be repaired, and concerns on the efficiency of traditional techniques used to manage maintenance and repair actions. This situation is creating a paradigm shift toward cutting- edge technologies such as the Internet of Things (Iot) [106]. The IoT refers to a system in which WSNs mounted with intelligent software and local computing power could be effectively used for the monitoring of structures. IoT aims to increase machine-to-machine communication thru wireless integrated sensors with the goal of monitoring devices remotely and efficiently. In this new paradigm, smart devices collect data, transmit information, and process information collaboratively using cloud computing techniques. Software is also needed to extract useful information from the large amount of data that is generated. On this basis, ML could be integrated with IoT for SHM purposes [107–109]. ML can thus become an essential tool that can be applied to expand the boundaries of IoT. On the other hand, the important issue regarding the SHM of 32 structures, such as bridges, is to constantly monitor the installed sensors and to compare the new data with previous readings. It is, however, a challenging task to visit all monitored bridges given the fact that they are typically geographically distant from each other. Thus, a technology that links all sensors on the bridges to a common recording device is needed. Further, it is essential to link collected information to a centralized monitoring station that could receive all the data from the sensors through the internet. The IoT and noted artificial intelligence methods could be used to effectively address the noted difficulties. Accordingly, the IoT will enable engineers to collect data from several bridges for further analysis. ML can then be used for data analysis and interpretation. Structural health assessment employing IoT could provide a promising solution for rapid, accurate, and low-cost SHM systems. The integration of SHM, IoT, and cloud computing can lead to powerful processing of the sensed data compared to traditional SHM systems. In fact, cloud platforms can enable an SHM system to store and use data from smart monitoring devices. The structure’s health status can then be sent to an Internet server, and data stored on the server can then be monitored remotely from a mobile device and interpreted using ML. The concept of smart cities has been recently gaining attention in diverse engineering communities, and the application of the IoT paradigm to smart cities is generating research interest [110–113]. The main aim of a smart city is to make better use of public services and to reduce operational costs. In other words, the goal of a smart city is to make infrastructure smarter in order to use resources efficiently. The achievement of this goal depends on a data provided by the wireless sensor networks deployed in cities. The IoT for a smart city can provide distributed data of structural integrity measurements of monitored structures using data collected by sensors, where DL architectures, e.g., CNNs, can be used as tools to interpret data and classification [114,115]. The data collated from a city varies so much in format and quality that it is difficult for one given 33 system to effectively process all such data. The fact that every city is unique and has a different set of problems yields the need for smart data interpretation techniques. Thus, robust layers for data collection, communication protocols, data storage, etc. need to be built. DL can then be used as a viable tool for interpreting such large amounts of data. DL can be utilized to train systems to recognize patterns for large numbers of real-time networks and provide early recognition of developing network performance issues. On the other hand, the big challenge for the smart cities concept is how to deal with the large amount of time series data, a particular form of sequential data, received from connected sensors. DL architectures (e.g., CNNs) are very efficient in the analysis of sequential data. DL platforms can thus enable a system to solve optimization problems relating to smart cities and structures. The notion of a smart city is to use sensors within the city’s infrastructures to ensure sustainability, safety, and efficiency. Recent progress in nanotechnology have led to the emergence of a new class of sensors, e.g., self-sensing materials that can provide smart cities with methods to assess and monitor the condition of the infrastructure. Smart concrete, having the ability of enabling any concrete structure with self-sensing capabilities, is one of the most promising technologies [116–118]. Such functional property is achieved by correlating the variation on internal strain with the variation of appropriate material properties, e.g., electrical resistance. Sensors fabricated using a cementitious matrix with nano-inclusions of carbon nanotubes can be used for condition assessment of concrete structures and traffic monitoring in smart cities. Consequently, AI methods can be effective in the interpretation of sensor data. Other examples include new developing approaches to detect the first stages of corrosion in concrete structures. The aim is to monitor the state of concrete during the curing period, leading to concrete structures with increased lifetime and safety. To accurately monitor the strength and temperature of concrete 34 during curing, sensors are embedded in the concrete at the time of placement and measurements are communicated to smartphones through IoT. AI methods such as ML and DL can then be used to interpret the collected data for structural assessment. 2.4 AI Algorithms and Classifiers for Energy-Lean Data-Driven SHM 2.4.1 Support Vector Machine Support vector machine (SVM), proposed by Vapnik and Cortes [119], is one of the significant ML algorithms for pattern classification. SVM is able to achieve good performance as it uses the structural risk minimization principle, while it introduces the kernel trick. SVM is used in this research to evaluate the performance of the proposed damage detection methodology due to the following main reasons: • SVM has a regularization parameter to avoid over-fitting, while it generalizes the new samples very well given appropriate parameters are chosen • Unlike other AI methods (e.g., neural networks), which produce multiple solutions based on local minima, SVM is guaranteed to converge to a unique global solution • The SVM optimization technique is based on convex optimization to prevent local minima problems. • If linear decision hyperplanes are not adequate to separate the classes SVM projects the input data into a high dimension feature space, thus resulting in nonlinear classifier • SVM uses a kernel trick, thus making the user able to design different kernels for the decision function via an engineering approach • SVM is able to deal with large feature spaces (i.e., complexity does not affect the dimensionality of the feature space). 35 • The solution is sparse when dealing with large data sets (i.e., support vectors are used to specify the separating hyperplane). The problem of SVM was originated from a supervised binary classification. In binary classification, most of the solutions are evaluated by obtaining a separating hyperplane among classes. To express the SVM, it is assumed that T = yx ,{( i i :) i = ,...,1 N } denotes a training data set consisting of N number of m-dimensional extracted feature vectors 1×∈ m i Rx and the corresponding labels }1,1{−∈iy , where N is the number of training samples. The goal of SVM is to find the separating boundary between two class data through maximizing the margin between the decision/separating hyperplane and datasets as illustrated in Figure 2-12, while minimizing the misclassification. The decision hyperplane can be defined as [120]: 0=+bxwT (2-1) where w and b denote the weight vector defining the direction of the separating boundary and bias, respectively. The constraint for classification in the original feature space can be stated as: T xwy i i ( + b ≥ 1) (2-2) For the SVM, the decision function is expressed according to following equation: )( xf = Accordingly, sgn( α can be defined as: ) sgn( T + xw i ) b (2-3) sgn( α ) =  ,1  −  ,1 ≥ α 0 < α 0 (2-4) 36 Figure 2-12. Schematic illustration of the SVM with optimal margin and separating hyperplane SVM attempts to maximize the margin by minimizing w , thus resulting in the following constrained optimization problem. min ξ , w ξτ ) 1 w , ( = [min w ξ , 1 2 2 + C w N ∑ = i ξ i 1 ] (2-5) Subject to T xwy i ( i + −≥ 1) b ξξ i , i > ,0 C > ,0 = i ,...,1 N where 1τ (.), 2 . , and iξ refer to objective function, L2-norm, and slack variables, respectively. The misclassification is decreased by minimizing the non-negative slack variables in the equation above. Further, C denotes the regularization parameter that balances the significance between the maximization of margin and the minimization of the misclassification error. Introducing a Lagrange multiplier αleads to following problem: W α ) ( = min α 1 2 N ∑ ∑ = 1 i N = 1 j yy i αα j j i ( xx i , j ) − ∑ N α j = 1 j (2-6) Subject to ∑ = N i α i 1 = y i 0,0 ≤ α i ≤ C Accordingly, the corresponding decision function can be written as: 37 )( xf = ∑ = sgn( N i α i 1 xxy i ( T i + ) b ) (2-7) SVM is able to offer an alternative solution for pattern classification when the data is linearly inseparable. In this regard, SVM uses a kernel trick method that projects the data into a higher dimensional feature space, such that the data becomes separable. Let suppose Φ :(.) o R → h R represents the projection/mapping from the original into high dimensional feature space. Consequently, the inner product of the original space xT i ⋅ can be substituted by inner product of x the transformed space Φ T ( xi ) Φ⋅ )( x . Yet, it is computationally inexpensive to evaluate the inner product of the transformed space. Therefore, kernel function provides an effective path to overcome this difficulty, such that a kernel function )(⋅K satisfies the Mercer theorem. xxK ( , i ) j Φ= T ( x i ) Φ⋅ ( x ) j (2-8) Thereafter, the decision function can be defined as: )( xf = sgn( N 1∑ = i α i ), xxKy i ( i + b ) (2-9) Kernel function, in fact, defines the nonlinear mapping from the input space into a high dimensional feature space. Typical kernel functions include: • Linear, xxK ( , i =) j T x i ⋅ x j • Polynomial, xxK ( , i ) j = ( T x i ⋅ x j + d c ) • Radial basis function (RBF), xK ( , x ) j i = exp( − γ − x i x j 2 ) • Sigmoid, xxK ( , i ) j = tanh( γ ( T x i ⋅ x j + ) c ) In above-mentioned equations, c and γare the kernel parameters, and d refers to polynomial degree. 38 2.4.2 K-Nearest Neighbor Among the different AI algorithms, k-NN has been widely used in the field of damage detection and condition assessment [121–124] in structural engineering domain. The k-NN classification method is a supervised learning algorithm with an acceptable classification accuracy. In general, k-NN is a non-parametric classification approach belonging to the instance-based or lazy learning methods [125–127]. That is, it does not use the training dataset for generalization. Compared to other classification methods, such as support vector machine and conventional artificial neural networks, the training and testing steps are attained with faster speeds using a k-NN classifier. The k-NN is used for SHM and damage classification based on the following main reasons: • The k-NN makes no assumption about the data distribution, thus yielding a flexible decision boundary • In contrast with a Bayesian method, the k-NN approach does not rely on marginal distribution of features given the class, resulting in higher accuracy • The cost of the learning process with k-NN is zero • The k-NN method is robust to noisy training data • The k-NN method is effective in learning complex concepts by local approximations using simple procedures • The k-NN classifier is easy to implement and analytically tractable • The k-NN technique uses local information, yielding to highly adaptive behavior. The k-NN classification algorithm can be described through the example shown in Figure 2-13. The aim is to classify new test data (green circle) to either class 1 or class 2 for a two- dimensional PR problem. As shown in Figure 2-13(a), if k = 3 the test sample is assigned to Class 1 since there are two squares and one triangle inside the region (inner circle). If k = 7 it is classified 39 to Class 2 as the number of triangles are greater than of squares. Another important concept regarding k-NN classification can be illustrated through a Voronoi diagram (Figure 2-13 (b)). Let ܦ௡ = {ݔଵ, … , ݔ௡} denote the set of n labelled data, and ݔ́ ∈ ܦ௡ is the sample nearest to test point x. Then, the nearest neighbor rule for classifying x is defined to assign it the label associated with ݔ́ . According to this rule, Voronoi diagram partitions feature space regions (cells,) where each is labelled by the class of training point it contains. A 2D Voronoi diagram is shown in Figure 2-13(b) where the test data point has three nearest neighbors. Figure 2-13. (a) Illustration of a typical k-NN classification, (b) A typical Voronoi diagram for two-dimensional feature space The k-NN classification algorithm is formulated by assuming the pair (xi, φ(xi),) which denotes the feature vector xi and its corresponding label φ(xi); where i=1, 2,…, n and φ{1, 2, …, m}. n and m are the number of training feature vectors and the number of classes, respectively. Considering xi as an arbitrary feature vector, the distance between this feature and feature vector xj is calculated by: id ),( j = ( i xxf , j ) (2-10) where ( i xxf , j ) is a distance function that can be defined as: 40 xxf ( , i = ) j ( x i − T ) x j ∑ − x i ( x j ) (2-11) Equation (2-11) is the generalized distance, and for the case of I=∑ it denotes the Euclidean distance. The distance vector D(i) is defined by equation (2-12): ܦሺ݅ሻ = {݀ሺ݅, ݆ሻ|݅ = 1,2, … , ݊௧௘௦௧ , ݆ = 1,2, … , ݊௧௥௔௜௡} (2-12) The D(i) vector is arranged in an increasing order (Dn(i),) and the k-nearest vote vector is defined by using the first K elements as follows: ܸ = {߮ሺܦ௡ሺ݅ሻሺ1ሻ, … , ߮൫ܦ௡ሺ݅ሻሺܭሻ൯} (2-13) The classification is performed by determining the k-nearest vote vector V. In this regard, the test feature xi is classified to the class that has the most votes in V. 2.4.3 Artificial Neural Networks An artificial neural network (ANN) provides a map between the inputs and outputs through determination of the weights using input and output patterns. Detailed information regarding ANNs can be found in literature [128,129]. In this research, a supervised feed-forward neural network employing back-propagation algorithm (see Figure 2-14) was considered for pattern classification. Such network propagates inputs via each layer till an output is generated. The computed error is then transformed backwards from the output layer to input layer, while the weights are regulated/adjusted in order to minimize the error. The node weights are fixed once a predefined error is reached. 41 Figure 2-14. Feed-forward multilayer neural networks with tow hidden layers and back- propagation algorithm 2.4.4 Bayesian Method The Bayesian method is a well-known concept for establishing decision boundaries between different pattern classes. The optimal Bayes decision rule can be stated as assigning pattern x to class wi for which the conditional risk is minimum. The conditional risk is defined as [130]: xwR )| ( i = c ∑ = 1 j xwPwwL )| ). ( ( , j i j (2-14) where P(wj|x) is the posterior probability and L(wi ,wj) is the loss incurred in deciding wi when the true class is wj. For the case of a 0/1 loss function the Bayes decision rule can be simplified and stated as assigning pattern x to class wi according to the maximum posterior probability rule given by: ܲሺݓ௜|ݔሻ > ܲሺݓ௝|ݔሻ for all j ≠ (2-15) i 42 To implement the Bayesian technique, the maximum likelihood estimation method is used to determine the parameter that maximized the probability of the observed data under the resulting distribution. 2.4.5 Nearest Neighbor Classifier with Two-Dimensional Principal Component Analysis Two-dimensional principal component analysis (2DPCA) is an image feature extraction and data representation technique used in the fields of pattern recognition and computer vision [131,132]. According to image processing methods, each image can be represented as a matrix. Letting X denote an n-dimensional unitary column vector, the main idea is to project matrix (image) A onto X through a linear transformation as expressed by [131,132]: Y = AX (2-16) where Y is the projected feature vector of matrix A. The trace of the covariance matrix of the projected feature vectors can be used to characterize the total scatter of the projected samples. In this context, the following criterion was adopted: XJ ( ) = xStr ( ) (2-17) where Sx refers to the covariance matrix of the projected feature vectors of training data, and can be expressed by: = YE ( − EY )( Y − EY T ) = E [ AX − E ( AX )][ AX − E ( AX T )] = AE [( − EA ) X ][( A − EA ) X T ] (2-18) S x In equation (1-18) Y refers to an m-dimensional vector obtained by projecting image/pattern A (m×n matrix) onto n-dimensional column vector X, and E denotes the expected value. Maximizing the criterion presented in equation (2-17) is important since the aim is to maximize the total scatter of the resulting projected samples. Accordingly, tr(Sx) shall be defined by: tr ( S ) x = AEX [ T ( − EA T () A − EA )] X (2-19) 43 The 2DPCA method introduces the image covariance (scatter) matrix (St), which can be directly determined from the training datasets according to: S t N 1 = ∑ N = 1 i [ iA )( − T ] [ iAA )( ]A − (2-20) where A(i)(i=1,2,…,N) is an m×n matrix and denotes the ith training image, and A represents the average image from all training data. A = 1 N N ∑ = 1 i iA )( (2-21) Accordingly, the criterion defined in equation (2-17) can be expressed by: XJ ( T=) XSX T (2-22) The optimal projection axis Xopt is the unit vector maximizing J(X). Thus, X1 ,…, Xd (the optimal projection axes) are the orthonormal eigenvectors of the image covariance matrix (St) corresponding to the first d largest eigenvalues. Accordingly, feature extraction was done using the optimal projection vectors of 2DPCA, X1 ,…, Xd. For a given image sample A, the principal components vectors Y1 ,…, Yd are defined by equation (2-23): = Y k AX k , = k ,...,1 d (2-23) Consequently, an m×d matrix B=[Y1,…,Yd] called the feature matrix, or feature image, of the image sample A was formed using principal component vectors. A nearest neighbor classifier [133] based on an Euclidean distance was utilized for classification after the feature extraction step using feature matrices. 2.4.6 Nearest Neighbor Classifier with Two-Dimensional Linear Discriminant Analysis Two-dimensional linear discriminant analysis (2DLDA) aims to extract features that well discriminate a set of data that belongs to a number of classes [131,134]. Letting A denote an m×n 44 image, and X denote an n-dimensional column vector, the main idea is to project image A, onto X by the transformation presented in equation (2-16). As for 2DPCA, the trace of the covariance matrix of the projected feature vectors is used to characterize the total scatter of the projected samples. For this purpose, the following criterion is proposed: XJ ( =) P B P W (2-24) where PB=tr(SB) and PW=tr(SW ). It is considered that there are L known pattern classes in the training set, and M denotes the size of the training samples. The 2DLDA method introduces an image between-class scatter matrix SB and an image within-class matrix SW and that can be expressed by: S B = S W = L ∑ = 1 i YYYYN )( ( i i i − − T ) L ∑ = = 1 i XAAXAAN ][( [( ) ) i − i − i T ] (2-25) Y ( k − YY )( i T − Y i ) k = L ∑ ∑ = 1 i ∈ TA i k [( A k − XA ) i ][( A k − XA ) i T ] (2-26) L ∑ ∑ = 1 i ∈ TA i k where iA is the mean of the data matrices in class iT and A is the global mean matrix. = A i 1 N i ∑ ∈ k TA i A k (2-27) A = 1 N Also, tr(SB) and tr(SW ) can be expressed by: L ∑ ∑ = 1 i ∈ TA i k A k (2-28) Str ( ) B L T = ∑ X ( = 1 i AAN [( − T )] [( i i − i XSXXAA ])) B = T (2-29) Str ( W ) L T = ∑ ∑ X ( = 1 i ∈ TA i k [( A k − T )] A i [( A k − A i ))] X = T XSX W (2-30) 45 According to the 2DLDA method, tr(SB ) and tr(SW ) can be directly evaluated from the training samples. Thus, the criterion presented in equation (2-24) can be stated as: XJ ( =) T XSX B T XSX W (2-31) The optimal projection axis Xopt is the unitary vector maximizing J(X), and is called Fisher optimal projection axis. Thus, the Fisher optimal projection axes X1 ,…, Xd are the orthonormal eigenvectors of SW -1SB corresponding to the first d largest eigenvalues. Consequently, the Fisher projection matrix is defined as Z=[X1 ,…, Xd]. The optimal projection vectors of 2DLDA were used for feature extraction. For a given image A, the Fisher feature vectors are defined as: = Y k AX k , = k ,...,1 d (2-32) An m×d matrix C=[Y1,…,Yd] called the Fisher feature matrix of the image sample A was formed using Fisher feature vectors. After the feature extraction step, the nearest neighbor classifier was used for pattern classification. 2.5 Summary The through-substrate ultrasonic sensor network integrating piezoelectric self-powered sensors and pulse communication architecture, based on which the proposed energy-lean SHM platform is being developed, was introduced. The noted sensor network is able to harvest the essential computational, storage, and transmission power from the signal being sensed as well as ambient vibrations. The nature of data/signals provided by such a self-powered sensing technology was discussed. This chapter has also covered diverse studies on use of emerging AI methods in SHM, smart infrastructure monitoring, and condition assessment. The significance of emerging AI methods for structural engineering applications during the last decade was presented. The survey indicated that among the numerous AI methods, PR, ML, and DL have been increasingly adapted 46 and used for SHM and damage identification. It was revealed that ML, PR, and DL algorithmic techniques have the ability to learn complicated interrelations among the contributing parameters, and thus allow solving a diversity of problems that are difficult, or not possible, to solve with traditional methods. Potential research avenues for employing PR, ML, and DL were presented. Considering the emerging use of wireless sensor networks (e.g., self-powered sensor networks), ML- and PR-based models could become the next generation approaches to conduct non- destructive structural and material evaluation in SHM. ML methods are able to discover hidden information about the structure’s performance by learning the influence of various damage or degrading mechanisms and the data collected from sensors, leading to reliable and efficient SHM frameworks. It was suggested that ML and DL techniques can also be used to solve complex problems through the novel concept of the Internet of Things (IoT). On this basis, ML and DL architectures (e.g., convolutional neural networks) within the context of IoT can be used to analyze and interpret complex and big data. Besides, the integration of ML and IoT can result in the creation of novel SHM systems employing diverse and noisy sensor data. DL architectures can also be incorporated with IoT to develop unique frameworks for use in smart cities. Data interpretation systems, which are part of the noted frameworks in smart cities, can thus be optimized using such intelligent architectures. General challenges and limitations on the use of noted AI techniques were also identified. Among those limitations is the lack of rational selection of the AI method, disregarding the effect of missing/incomplete and noisy data, discarding considerations for computational efficiency, reporting classification accuracy without exploring alternative solutions to increase performance, and insufficiency presentation of the process to select optimal parameters for the AI technique. However, it was noted that by addressing the noted issues/limitations in future studies, ML, PR, 47 and DL could represent pioneering methods to increase the efficiency of many current structural engineering applications as well as for the creation of innovative uses. 48 CHAPTER 3 3 PATTERN-RECOGNITON-BASED ALGORITHMIC FRAMEWORK FOR SELF- POWERED SHM USING FULL DISCRETE BINARY SIGNALS 3.1 Overview This chapter presents the theoretical background and experimental studies, based on which the proposed pattern-recognition (PR)-based algorithmic framework is established. An image-based PR framework is developed based on integration of anomaly detection, statistical measures, and numerous AI classifiers and algorithms. The applicability and effectiveness of the self-powered damage identification strategy employing the PR-based framework was numerically and experimentally evaluated with full discrete binary signals/data, while the effect of time-delay and power budget constraint were discarded. The contents presented in this chapter were partially published in the journal Structural Control and Health Monitoring [135]. 3.2 Pattern recognition framework using anomaly detection and AI classifiers A schematic illustration of the proposed self-powered SHM strategy employing PR approach is presented in Figure 3-1. Binary events are created based on local measurements, measurement combinations, or rules. For example, the strain in one direction εxx), the combination of strains such as principal strains (ε1 and ε2), or a failure criterion. Graphical representation of the binary event data from the sensor nodes across the monitored structure describes a pixelated image, or pattern. The proposed SHM strategy begins by assuming that certain events are created (based on property defined thresholds) from the structures’ regular (in-service) response, and that the resulting patterns can be memorized and used as benchmarks for damage detection. If damage or decay occurs under consistent loading a change in structural response is anticipated, which will 49 lead to a different pattern because binary-event thresholds will be exceeded in different locations (sensor nodes). If a pattern resulting from the noted material/structural changes is recognized as new and consistent with respect to a benchmark, it is thought to be representative of damage. Figure 3-1. Schematic illustration of SHM strategy using image-based pattern recognition An important issue regarding the self-powered SHM strategy using the PR-based algorithmic framework is the loss of data resolution. Resolution can be defined, in general, by means of three components: the number of data points in the domain, the actual value at each point, and temporal availability of the data. In theory, resolution in SHM approaches based on continuous data are affected only by the number of data points, i.e., sensor nodes. However, the discrete binary data used in the proposed approach leads to an obvious loss of resolution from the other two aspects, that is, the actual value at sensor nodes is not known and the data may be missing or delayed due to lack of power at the sensor node. The effect of loss in data resolution in terms of number of data points and the actual value at each point is illustrated in Figure 3-2. 50 Figure 3-2. Illustration of loss of data resolution (a) continuous response, (b) discrete binary response According to the energy-aware pulse switching protocol, the information extracted from the sensor nodes is limited to the binary data (1 or 0) received at the sink. It was hypothesized that interpretation of such low-resolution data resembles a PR problem. Therefore, an image-based PR approach using anomaly detection is proposed to represent sensor nodes responses as a pattern and to identify abnormality and failure patterns using anomaly detection. Anomaly detection is an effort to identify unusual patterns that depart from expected behavior. Research activity aimed at the use of anomaly detection along with PR techniques for SHM applications has increased in recent years [136–139,52]. The deformed shape of the structure in normal conditions is first memorized. From this baseline knowledge, the method can classify deviation from the memorized patterns and identify new common patterns resulting from load variations or changes in material properties (e.g., damage). To implement the PR approach, the arrangement of sensor nodes (and consequently the distribution of binary values generated from structural response) was considered as a pattern/image. Consistent with image data analysis techniques, each pattern (image) was treated as a matrix and represented by specific features (binary values according to local rules) depending on the number of self-powered sensors. For instance, if the number of self-powered sensors is 51 equal to n, each pattern is represented by n features (n-dimensional PR problem). Consequently, a classifier looks for identical features at each cell. Once a new pattern becomes available, extracted features at individual cells are compared with corresponding features from memorized patterns. If all the n features are identical, the new input is classified as belonging to the same class; otherwise it is classified as a new class. By using this classification, response noise due to load variations or material degradation can be eliminated. The method also allows identifying a damage location (or location of degrading material properties), since identical features at each cell are compared and used for pattern classification. The technique can also be considered as an online supervised learning approach, since as new data (pattern) arrives the system is able to compare it with memorized patterns and also use it as a new pattern for further comparisons. Accordingly, the classifier is updated upon the interpretation of previous observations to be able to predict new patterns. It should be noted that the PR methods used in this chapter were chosen because of the pixelated nature of the binary data that resembles an image. First, an image-based PR approach based on anomaly detection is proposed as an analysis concept and used for pattern classification. Second, a Bayesian method was utilized as a classifier for pattern identification and classification. Finally, a nearest neighbor classifier [133] was used for pattern classification, for which two- dimensional principal component analysis (2DPCA) and two-dimensional linear discriminant analysis (2DLDA) techniques were used for extracting pattern/image features. The methods were chosen due to the type of data under consideration. Detailed information of noted PR methods are presented in Chapter 2. A custom program in the Matlab [140] platform was utilized for implementing the different PR algorithms. 52 3.3 Numerical Study The PR algorithms were examined through finite element (FE) simulations. FE analyses were used as a platform to evaluate the algorithms using the output at virtual sensor nodes. The FE simulations were performed using the program ABAQUS [141]. The evaluations with simulated data was made on a plate simply supported along all edges. The geometry and location of output locations (i.e., simulated sensors) are shown in Figure 3-3. The simply supported plate (Figure 3-3) was taken to be made of aluminum (E = 73100 MPa, ν = 0.33) with a square geometry of 1 m sides, and a thickness of 5 mm. Aluminum was chosen since the project’s initial scope for application to aircraft structures. However, it should be noted that the presented SHM strategy does not depend on the material properties of the monitored structure. Figure 3-3. Geometry and sensor layout for a simply supported plate for simulation case The simply supported plate in the FE simulations was subjected to a uniform pressure of 0.05 MPa with harmonic amplitude variation in time (5 s) as shown in Figure 3-4. To investigate applicability of the proposed strategy against varying loading conditions, random noise was added 53 to both harmonic amplitudes, with harmonic amplitude #2 having a higher noise level than that of harmonic amplitude #1 (see Figure 3-4). Figure 3-4. Harmonic amplitudes used for dynamic loading 3.4 Experimental Study The integrated sensor with communication technology is under fabrication and testing. However, experimental validation of the self-powered sensor and pulse communication architecture has been done separately in prior studies [5,7]. Vibration tests using conventional strain gages were thus used in this study to explore the algorithmic framework and to examine the SHM’s ability to identify damage using full binary signals in a real structure. The continuous experimental response (strains) was post-processed to extract binary events based on response thresholds. The evaluation with experimental data was made on a plate cantilevered from one side. The geometry and locations of output locations (i.e., strain gages), and a schematic of the experimental setup are shown in Figure 3-5. The cantilever plate was rectangular with length and width dimensions of 788.5 mm and 508 mm, respectively. The plate was made of aluminum 2024-T4 with 5 mm thickness, E = 73,100 MPa and ν = 0.33. Aluminum was selected since the project’s initial scope is for applications to aircraft structures. Damage on the experimental cantilever plate was introduced by creating a circular hole in the plate (see Figure 3-5). Damage severity was simulated 54 by considering holes with varying diameter, namely diameters of 13, 19 and 25.5 mm. The vibration tests were performed by subjecting the plate’s base (fixed end) with a harmonic excitation of 5 mm amplitude at a frequency of 2.3 Hz for a duration of 100 s. The noted amplitude is the relative displacement input at the clamped edge of the cantilever plate, which was mounted to the actuator of a universal testing frame. Strain data was collected during the test with a time step of 0.01 s. It is acknowledged that the modeled plate is a rather simple structural prototype. However, the proposed SHM strategy can be equally applied to real-world plate-like structures, such as a bridge girder, or an aircraft wing (see Figure 2-1). Figure 3-5. Details of experimental vibration test of cantilever plate: (a) Plate geometry and instrumentation layout, (b) Schematic of test setup 3.5 Approach to Data/Signal Interpretation According to the deviation of patterns method, the number of features on a pattern is related to the size of the sensor array. A sensitivity analysis was conducted to investigate the effect of sensor’s sparsity on the proposed approach and to obtain an adequate number of sensors for the proposed SHM system. Regular grid layouts with 9, 16, 25, 36 and 64 sensors were considered (for the square plate) and the effectiveness of the damage detection approach in terms of identifying the damage presence and location was evaluated. Results showed that the layouts shown in Figure 55 3-3and Figure 3-5 provided sufficient information for calibrating the pattern classifier. Thus, the number of virtual sensors for the simply supported plate was set at 16. In other words, each pattern (image) was represented by 16 features. The array size for the cantilever plate was 9; and thus, each pattern (image) for this case was represented with 9 features. 3.5.1 Multiple Binary Event Concept Simple pilot-type local rules for binary event generation were defined in terms of transverse displacements and maximum principal strains at the virtual sensor nodes of the simply supported plate, and longitudinal strains at the strain gage locations of the cantilever plate. According to pulse switching technology, the event information is binary. The advantage of having a one-bit system is that it is simpler in terms of communication and it requires less power, but there is less information for pattern classification. Although having more bits improves the classification accuracy, it requires more energy. On the other hand, in order to be able to determine changes in loading or structural condition, a benchmark, or normal performance level, needs to be defined. For this purpose, the concept of multiple binary events is proposed and used in this study. The concept can be implemented in the pulse switching communication technique [142,143] through variance in the signal width, frequency, or time spacing of the communication pulse. The frame size can also be extended to provide more slots for each cell such that multiple binary event types can be communicated. This allows the possibility of emitting different types of pulses that can be identified and differentiated from each other upon reception. At the sensor nodes the multiple binary event concept can be implemented by defining different event generation rules. Three event thresholds (R1, R2, and R3) were thus defined in terms of displacement or strain response at the sensor nodes to differentiate between four demand levels. It should be noted that the number of thresholds only affects the ability to differentiate between demand levels and is not related to the 56 method’s accuracy. A schematic of the multiple binary concept and the definition of local event generation rules based on the multiple binary event concept for the simply supported plate are presented in Figure 3-6. According to the proposed multiple binary event concept, the extracted features from each cell become flags that are algorithmically represented by values between 0 and 1 (0, 0.5, 0.75 and 1) as shown in Figure 3-6. Figure 3-6. Concept schematic of multiple binary event generation at sensor nodes 3.5.2 Global Response Identification Applicability of the proposed SHM strategy with PR approach was initially tested in terms of load variations and then utilized for damage detection purposes. The simulation case (square plate) was used to investigate the algorithm’s performance due to changes in loading. Accordingly, the response patterns for the plate (deformed shapes) were memorized for an initial load such that the system could recognize a change in structural response from load variations through different shapes (patterns). Three displacement values (namely 2.3, 5.5 and 10 mm for R1, R2 and R3, respectively) were considered as thresholds for the generation of binary events. The threshold 57 values were empirically chosen, based on simulations, such that sufficient binary events representing different response levels could be generated. 3.5.3 Local Damage Identification The PR methods’ ability to identify local damage was evaluated in two forms. First, localized damage on the FE model for the simply supported plate was done by gradual reduction in stiffness (elastic modulus) in a region of the plate (see Figure 3-3) over the loading time (i.e., analysis time). Localized stiffness reduction of 20 to 80% (in 10% increments) was considered. Second, damage on the experimental cantilever plate was introduced by means of a drilled hole (see Figure 3-5) and damage severity was controlled by varying the hole diameter from 13 mm to 19 mm and then 25.5 mm. The plates’ response without damage (i.e., uniform stiffness FE model and plate without a hole) was first memorized. The PR algorithms were then used to identify new patterns resulting from the introduced damage. For the simulation case (square plate), the maximum principal strain values for the generation of binary events were 100, 200 and 450 micro-strains for R1, R2 and R3, respectively. Similarly, the event thresholds for the experimental case (cantilever plate) at the strain gage locations were 80, 200 and 500 micro-strains in the longitudinal direction of the plate. The noted strain thresholds were empirically chosen to ensure that binary events representing damage could be generated. It is further noted that the values used for the square plate do not correspond to the displacement thresholds used for global response identification. 3.6 Damage Identification based on Finite Element Simulation Results 3.6.1 Image-based PR Approach Performance of the PR algorithms in terms of changes in the loading condition was evaluated in the first phase. Harmonic amplitude #1 (Figure 3-4) was first used to calibrate the classifier. The 58 plate domain was divided in regions such that a given sensor node is located at the center of the region (see Figure 3-3). The regions were assigned a value from 1 to 4 depending on the level of displacement response r as shown in Figure 3-6; where a region of value 1 implies that the sensor output r representing that region has a value less than the first threshold R1, a region with value 2 implies R1 < r < R2, etc. The anomaly detection classifier could identify 3 common patterns as shown in Figure 3-7; in which the white, green and yellow regions correspond to values 1, 2 and 3, respectively. Table 3-1 presents the number and percentage of pattern occurrences using the PR approach. It should be noted that 10% of the patterns, among the dataset of 500, were disregarded since all of the sensors were inactive. Figure 3-7. Common identified patterns for simply supported plate under uniform load with harmonic amplitude #1 Table 3-1. Number of pattern occurrences based on the PR approach Pattern 1 2 3 4 Load # 1 Constant Stiffness Occurrences Percentage (%) Occurrences Percentage (%) 283 106 23 22 65 25 5 5 133 57 278 - 28 12 60 - 59 After training the PR classifier, data generated with harmonic amplitude #2 (Figure 3-4. Harmonic amplitudes used for dynamic loading) was utilized for classification. The PR method recognized 21 patterns (18 new patterns with respect to the 3 memorized patterns from load with harmonic amplitude #1). The classification results in terms of the distribution of 10 patterns (from the set of 21 new ones) with respect to the time at which they were identified is presented in Figure 3-8(a). The frequency histogram of these patterns is also illustrated in Figure 3-8(b). Close-up views of the results in Figure 3-8(a) for portions of the time domain are shown in Figure 3-9, where it can be observed that only one pattern was recognized by the algorithm for each time step. As shown in Figure 3-9, 15 of the new patterns (from 21) had a low number of occurrences and were disregarded, leaving only 3 new patterns (4, 5, and 6) for further comparison with new inputs. Figure 3-8. Classification results based on deviation of patterns method for simply supported plate due to distributed load with harmonic amplitude #2: (a) time history of new recognized patterns, (b) histogram of identified patterns 60 Figure 3-9. Close-ups of deviation of patterns method time history results in Fig 7(a): (a) t = 1.5 to 2 s, (b) t = 4.5 to 5 s Spatial-temporal representation of the binary events based on the multiple binary event concept for the load with harmonic amplitude # 2 for four consecutive time steps of the dynamic analysis are shown in Figure 3-10. This figure presents the creation of binary events at the virtual sensor locations for different analysis time steps, which clearly shows that changes in the response of sensor nodes due to load variations can be recognized. In the second evaluation phase, performance of the PR algorithms in identifying damage was assessed. Three strain values were assumed as thresholds to recognize normal response. The classifier was calibrated (trained) for a plate with constant stiffness (healthy plate) and the uniform pressure load with amplitude #1. The number of pattern occurrences and corresponding percentages are presented in Table 3-1. The 4 common patterns identified by the PR algorithm were identical to patterns 1 to 3 shown in Figure 3-7 and pattern 4 shown in Figure 3-8(b). Thereafter, the simulated output from the damaged plate was introduced as input to the deviation of patterns algorithm. Results are provided and discussed herein for the case of 40% reduction in stiffness. The classification results in terms of the distribution of new recognized patterns with 61 respect to time is shown in Figure 3-11(a), while Figure 3-11(b) visualizes the distribution of six patterns for the time period of 4.5 to 5 seconds. These results show the occurrence of new patterns, most notably 5 and 6, as damage occurs. Accordingly, these patterns were identified as the patterns resulting from damage/material degradation. Figure 3-10. Spatial-temporal representation of sensor node values based on multiple thresholds according to load amplitude #2 for 4 consecutive time steps (time step = 0.01 s) Spatial-temporal representation of the binary events based on the multiple threshold concept was determined and shown in Figure 3-12 for 4 consecutive time steps of the dynamic analysis. The effect of material degradation on the sensor node output is clearly noticeable. In other words, stiffness reduction leads to a change in patterns, and the variation is at the same location as the region with reduced stiffness in the FE model. The sensor nodes’ multiple threshold event response was interpolated to qualitatively show physical behavior of the damaged plate using the identified 62 patterns and the results are shown in Figure 3-13, from which it can be seen that the damage location is clearly identified. Figure 3-11. Classification results based on deviation of patterns method due to a localized 40% reduction in stiffness in the simply supported plate case: (a) time history of new recognized patterns, (b) Close-ups of time history results (t = 4.5 to 5 s) An attempt was made to establish a connection between recognized patterns and structural properties of the damaged plate. In this context, the plate’s changes in flexural rigidity were assessed with the localized decrease in elastic modulus. The frequency of common patterns due to normal condition (Figure 3-7) with respect to 30% and 80% reduction in stiffness are illustrated in Figure 3-14(a) and Figure 3-14(b), respectively. It can be seen that, with the exception of pattern 2, the frequency of patterns 1 and 3 decreases with increasing stiffness reduction. However, pattern 2 is insensitive to the stiffness reduction. The frequency of new identified patterns due to localized stiffness reduction is also presented in Figure 3-15, where it can be observed that the frequency of abnormal patterns 4 and 5 (representing damage) steadily grows as flexural rigidity reduces. These classification results demonstrate that the deviation of patterns method, along with the multiple binary event concept, can be successfully used for damage detection purposes using discrete binary data. 63 Figure 3-12. Effect of a localized 40% stiffness reduction in the simply supported plate on the spatial-temporal distribution of multiple threshold values at sensor nodes for 4 consecutive time steps (time step = 0.01 s) Figure 3-13. Surface interpolation of multiple binary events according to a localized 40% stiffness reduction in the simply supported plate for 4 consecutive time steps (time step = 0.01 s) 64 Figure 3-14. Frequency of normal patterns due to localized stiffness reduction in the simply supported plate using the deviation of patterns method: (a) up to 30% stiffness decrease, (b) up to 80% Figure 3-15. Frequency of damaged patterns identified by deviation method for different levels of localized stiffness reduction 3.6.2 Nearest Neighbor Classifier with 2DPCA Data generated from the simulation case with harmonic amplitude #1 was considered as system input for the training step and three different classes/patterns were determined from the possible patterns using k-means clustering (see Table 3-1). Displacement response due to harmonic amplitude #2 was then used to extract the corresponding feature matrices using principal component vectors. The feature matrices were introduced to the PR interpretation algorithm and a nearest neighbor criterion was used for pattern classification. Accordingly, a new pattern was 65 classified by computing the distance to the nearest calibration sample in the feature space. Table 3-2 presents the results, including the number of patterns successfully classified to each of the three memorized classes. The results show that 34%, 7% and 59% of the patterns were classified to classes 1, 2 and 3, respectively, which is consistent with the results found by the deviation of patterns method. Table 3-2. Classification results in terms of number of pattern occurrences based on nearest neighbor classifier with feature extraction techniques using load #2 Pattern 1 2 3 2DPCA 2DLDA Occurrences Percentage (%) Occurrences Percentage (%) 150 32 258 34 7 59 120 37 283 27 9 64 3.6.3 Nearest Neighbor Classifier with 2DLDA Feature extraction was further performed according to the 2DLDA method, with the procedure being as previously described for 2DPCA. However, the feature matrices were extracted by means of Fisher feature vectors. The classification results are shown in Table 3-2 from which it can be seen that a nearest neighbor classifier successfully classified patterns due to loading changes, such that 27%, 9%, and 64% of the patterns were classified to each of three memorized classes. Results show the good performance of a nearest neighbor classifier along with the 2DLDA feature extraction technique to classify patterns resulting from load variations. 3.7 Damage Identification based on Experimental Results Performance of the Bayesian method and the nearest neighbor classifier (along with the 2DPCA and 2DLDA techniques) was evaluated using experimental data from a cantilevered aluminum plate (Figure 3-5). Patterns identified from strain responses of the undamaged plate were used to 66 calibrate the classifier. Consequently, strain values from tests on the damaged plate (with a hole of varying diameter) were used as input to recognize classes/patterns representing damage. 3.7.1 Bayesian Method In this study the number of partitions for k-fold cross validation was assumed as 2; thus, the Bayesian classifier was trained based on 50% of the data set and tested with the remaining 50%. To implement the Bayesian technique, k-means clustering was also performed to calibrate the classifier for common classes from the training dataset. In this context, patterns were initially classified into 5 classes. Classes 1 and 2 represent patterns in normal condition, classes 3 and 4 denote patterns due to system noise, and class 5 represents a pattern related to damage. Once the classifier was calibrated, strain data from experiments of the damaged plate was introduced to the classifier and new patterns were classified accordingly. Table 3-3 presents classification results in terms of the distribution of patterns from experiments on the damaged plates (with different diameter holes). A confusion matrix, which contains information about actual and predicted classification, was determined for each fold after performing k-fold cross validation and is shown in Table 3-4. The classification accuracies demonstrate the good performance of the Bayesian classifier in terms of detecting patterns representing noise as well as damage. The frequency of recognized patterns within each class with respect to time for plates with a 13 mm and a 25.5 mm diameter hole are shown in Figure 3-16(a) and Figure 3-16(b), respectively. The results show that the frequency of normal patterns (class 1) tends to slowly decrease with damage increase, while the frequency of normal patterns classified to class 2 do not change significantly. Conversely, the damage class pattern (5) increases with larger damage (as seen by comparing Figure 3-16(a) and Figure 3-16(b)) since new damaged patterns were identified. Variation for the class patterns related to noise (3 and 4) are observed but they do not show a 67 significant difference with damage increase and the system’s performance to detect these patterns is considered acceptable. It follows that the Bayesian classifier successfully identified new patterns resulting from damage and the frequency of patterns resulting from damage can be related to damage severity. Table 3-3. Distribution of patterns from experiments on a cantilever plate using the Bayesian method Pattern 13 mm Hole Diameter 25.5 mm Hole Diameter Occurrences Percentage (%) Occurrences Percentage (%) 1 2 3 4 5 70 154 208 46 22 14 31 42 9 4 62 114 244 50 30 12 23 49 10 6 Table 3-4. Distribution of patterns from experiments on a cantilever plate using the Bayesian True Classes 1 2 3 4 5 Sum Error (%) 1 64 2 0 0 0 66 3.0 method Predicted Classes 2 0 47 0 0 0 47 0.0 3 0 0 42 0 1 43 2.3 4 0 0 3 34 0 37 8.1 5 0 0 0 2 55 57 3.5 True Sum 64 49 45 36 56 250 3.2 Figure 3-16. Identified patterns by Bayesian method on cantilever plate with different hole diameters: (a) 13 mm, (b) 25.5 mm 68 3.7.2 Nearest Neighbor Classifier with 2DPCA Patterns identified for the experiment case were classified into the same 5 classes explained in the previous section (Bayesian method). Strains at the plate sensor nodes were used for feature extraction using the 2DPCA technique. Then, pattern classification of a data set extracted from a damaged plate was performed by means of a nearest neighbor classifier. K-fold cross validation was also implemented to evaluate model performance. The confusion matrix for the case of a damaged plate with a 25.5 mm diameter hole is shown in Table 3-5 which shows high classification accuracy, particularly for the damage pattern class (5). Figure 3-17 presents the variation of frequency of recognized patterns with time for the damaged plates. It can be seen that the frequency of normal patterns (1 and 2) reduces as the hole diameter increases and that the frequency of class 5 patterns (damaged) grows. Also, the number of occurrences of patterns from classes 3 and 4 (system noise) show less variation with respect to changes in hole diameter. These patterns were also less represented in comparison to the Bayesian method. This follows from the fact that the 2DPCA method is capable of overcoming noisy data. According to these results, a nearest neighbor classifier using extracted features based on the 2DPCA technique is capable of identifying patterns representing damage with good accuracy. Table 3-5. Confusion matrix for experimental data based on nearest neighbor and 2DPCA True Classes 1 2 3 4 5 Sum Error (%) 1 70 0 0 0 0 70 0.0 True Sum 74 46 43 27 60 250 4.0 Predicted Classes 4 0 0 0 25 3 28 10.7 5 0 1 0 0 57 58 1.7 2 0 45 0 2 0 47 4.2 3 4 0 43 0 0 47 8.5 69 Figure 3-17. Identified patterns by nearest neighbor and 2DPCA method on cantilever plate for different hole diameters: (a) 13 mm, (b) 25.5 mm 3.7.3 Nearest Neighbor Classifier with 2DLDA The confusion matrix for the PR algorithm using the 2DLDA feature extraction method is presented in Table 3-6. High classification rates were achieved and the overall average error was lower than that of the Bayesian (see Table 3-4) and 2DPCA (see Table 3-5) methods. The frequency of patterns (for each of the classes) for a damaged plate with a hole diameter 13 mm and 25.5 mm are shown in Figure 3-18(a) and Figure 3-18(b), respectively. In comparison to the results from previous methods, the 2DLDA classifier had higher variation in the patterns due to noise, and the frequency of patterns representing damage increased. This performance is attributed to the way by which the 2DLDA method discriminates between classes. Table 3-6. Confusion matrix for experimental data based on nearest neighbor and 2DLDA True Classes 1 2 3 4 5 Sum Error (%) 1 62 0 1 0 0 63 1.6 True Sum 62 42 50 31 65 250 2.4 Predicted Classes 4 0 0 0 28 0 28 0.0 5 0 0 0 2 64 66 3.0 2 0 42 0 1 0 43 2.3 3 0 0 49 0 1 50 2.0 70 Figure 3-18. Identified patterns by nearest neighbor and 2DLDA method on cantilever plate for different hole diameters: (a) 13 mm, (b) 25.5 mm 3.8 Summary This chapter presented an evaluation of the use of PR methods for the analysis of event-based binary data from novel self-powered wireless sensor networks in SHM of plate-like structures. PR methods based on image data analysis techniques were adapted and used for data interpretation and pattern identification. Simple pilot-type local rules for binary event generation were defined in terms of displacements and strains at virtual sensor nodes in simulations (finite element) and experiments; and a multi-level concept was introduced for generating event-based binary data. Performance of the proposed image-based PR methods in terms of their ability to identify effects from load variation and simulated damage was assessed. The conclusions listed below were reached: 1) The proposed PR approach employing anomaly detection can effectively recognize patterns due to variations in load amplitude and material stiffness. Since the method is based on the comparison of features at individual cells, patterns due to system noise are eliminated and potential damage can be accurately detected. 71 2) A Bayesian method was found to be a promising technique for dealing with binary data, and performance of the PR algorithm based on this classifier was acceptable in terms of identifying patterns due to local damage and data noise. 3) The 2DPCA and 2DLDA feature extraction techniques, along with a nearest neighbor classifier, can perform satisfactorily in identifying damage from binary data. 4) The classification performance of all methods was found satisfactory, but a nearest neighbor method along with the 2DLDA feature extraction technique showed better performance (smaller error). The method had a superior performance in recognizing damage because it deals directly with discrimination between classes, whereas the 2DPCA feature extraction method deals with data in its entirety for the principal components analysis. The presented chapter has demonstrated that developed PR-based algorithmic framework can be used for power-efficient SHM and damage identification for binary data sets generated from self-powered sensors employing pulse-switching networking technology. 72 CHAPTER 4 4 EEFECT OF IRREGULAR LOADING CONDITIONS, SENSOR SPARSITY, AND MEASUREMENT NOISE 4.1 Overview Chapter 3 indicated the applicability of proposed PR-based framework employing various PR methods, including image-based PR approach, Bayesian, nearest neighbor with two-dimensional principal component analysis (2DPCA), and nearest neighbor two-dimensional linear discriminant analysis (2DLDA) for damage detection in plate-like structures using full discrete binary data. Nevertheless, some of the noted PR methods were found to be computationally demanding. In addition, the effect of noise, sensor density, loading variation, damage scenarios, and the monitored structures’ aspect ratio should be investigated, as each of the aforementioned features could have a significant impact on the performance of PR framework for self-powered SHM. To address the noted important gaps, the main objective of this chapter is to present a PR framework for an energy-lean data-driven SHM system that allows damage identification with low-resolution discrete binary data, and to evaluate the robustness of the proposed framework with respect to different damage scenarios, irregular loading, sensor density, and measurement noise. The contents being presented in this chapter were partially published in the journal Intelligent Materials systems and Structures [144]. 4.2 Implementation of the PR-based Algorithmic Framework with image-based PR approach and k-NN Algorithm The interpretation of discrete and low-resolution binary data is a difficult task, and a pattern recognition (PR) framework is presented in this chapter to overcome this issue, and its 73 effectiveness and robustness was evaluated for the power-efficient SHM of plate-like structures. The proposed framework consists of the tasks listed below and illustrated in Figure 4-1, while the theoretical considerations behind the tasks are presented in following sub-sections. • • Identify normal and abnormal patterns representing different plate’s condition with an image-based PR approach employing anomaly detection Identify abnormal patterns denoting damage based on a pattern anomaly measure index • Determine damage location based on a pattern focal density measure • Compute damage classification accuracy with the k-nearest neighbor algorithm. Figure 4-1. Proposed PR framework for power-efficient SHM with discrete binary data 74 4.2.1 Image-based PR Approach Employing Anomaly Detection According to the energy-aware pulse switching protocol, the information extracted from the sensor nodes is limited to the binary data (1 or 0) received at the sink. It was hypothesized that interpretation of such low-resolution data resembles a PR problem. Therefore, an image-based PR approach using anomaly detection is proposed to represent sensor nodes responses as a pattern and to identify abnormality and failure patterns using anomaly detection. To implement the approach, the arrangement of sensor nodes (and consequently the distribution of binary values generated from structural response) was considered as a pattern/image. Consistent with image data analysis techniques, each pattern (image) was treated as a matrix and represented by specific features (binary values according to local rules). A schematic illustration of the proposed image-based PR approach employing anomaly detection for a plate structure with 16 sensor nodes is presented in Figure 4-2. Figure 4-2. Illustration of image-based PR using anomaly detection 75 The dimension of the generated matrix, which herein is the dimension of the PR problem, depends on the number of time steps and distribution of sensor nodes; while each of the matrix elements denotes a pattern’s feature (binary value) at each sensor. In other words, sensor node responses for the entire simulation time were arranged in m rows and n columns, where m denotes number of time steps and n refers to the number of sensor nodes in the network. Accordingly, if the number of sensors is n, each pattern was represented by n features at each time step. A schematic illustration for the plate structure in Figure 4-2 representing sensor responses as a pattern for each time step is presented in Figure 4-3; where the number of sensor nodes and time steps are 16 and 100, respectively. Figure 4-3. Representation of sensor nodes responses as pattern The resulting patterns from the structures’ regular (in-service) response were memorized and used as benchmarks for damage detection. A change in structural response is expected when damage or decay occurs under consistent loading, which leads to a different pattern because the binary-event thresholds are exceeded in different locations. If a pattern resulting from the noted material/structural changes is recognized as new, with respect to the benchmark, it is thought to be representative of abnormality. Further, as it was shown in previous chapter, the frequency of 76 identified patterns can be used to recognize abnormal patterns from normal ones. That is, among all the patterns identified by image-based PR approach, only the frequency of normal patterns (representing an undamaged structure) decreases with time. Therefore, normal and abnormal patterns are identified using an image-based PR approach. However, abnormal patterns could represent damage, noise, faulty sensor, etc. Therefore, damaged patterns need to be further differentiated among the abnormal ones. 4.2.2 Pattern Anomaly Measure Index A ‘pattern anomaly measure index’ is proposed to discern abnormal patterns denoting damage from those due to noise. The index is computed for a predefined time window, e.g., t=0 to t=20 s, etc., (see Figure 4-3) according to equation (4-1). If the pattern anomaly measure index increases steadily with time and damage severity it is taken to represents an actual damaged pattern; otherwise it denotes a noisy pattern. ܲܽݐݐ݁ݎ݊ ܣ݊݋݈݉ܽݕ ܯ݁ܽݏݑݎ݁ = ܰݑܾ݉݁ݎ ݋݂ ܾܽ݊݋ݎ݈݉ܽ ݌ܽݐݐ݁ݎ݊ݏ ݅݊ ݐℎ݁ ݐ݅݉݁ ݓ݅݊݀݋ݓ ܶ݋ݐ݈ܽ ݊ݑܾ݉݁ݎ ݋݂ ݌ܽݐݐ݁ݎ݊ݏ ݅݊ ݐℎ݁ ݐ݅݉݁ ݓ݅݊݀݋ݓ ሺ4 − 1ሻ 4.2.3 Pattern Focal Density Measure Index The concept of focal plane, a widely adopted image processing technique, was adapted for this study as an improved approach to determine damage location. Local functions calculate an output dataset at each sensor (cell) such that it is a function of the input value associated with that location. That is, the value of the output is directly affected by the value of a single sensor, without considering the values of neighboring sensors (cells). On the other hand, focal, or neighborhood, functions generate an output dataset where the output value at each sensor location is a function of the input value at the sensor location and the input values of the sensors in a specified neighborhood around that location [145–147]. 77 Once damaged patterns are identified, the location of damage is determined using such patterns. In this context, a potentially more effective spatial system was defined using a neighboring concept. The proposed strategy is designed to detect the location of damage using spatial measurements over the plate’s domain. A spatial fusion process integrates and extracts information (binary data) from two or more sensors. A damage index denoting the location of damage is thus defined using a focal density concept based on fused binary data. The proposed “pattern focal density measure” is determined by adding binary events (values of 1) generated at each sensor and its neighbors. Accumulated binary events were then normalized by the maximum number of neighboring sensors plus the sensor itself to determine a damage index (DI) for a sensor node. The damage index for sensor node i was determined with equation (4-2). ܦܫ ݋݂ ܵ݁݊ݏ݋ݎ ݅ = ܵݑ݉݉ܽݐ݅݋݊ ݋݂ ܾ݅݊ܽݎݕ ݁ݒ݁݊ݐݏ ݋݂ ݏ݁݊ݏ݋ݎ ݊݋݀݁ ݅ ܽ݊݀ ݅ݐݏ ݊݁݅݃ℎܾ݋ݎ ݏ݁݊ݏ݋ݎݏ ܯܽݔ݉݅ݑ݉ ݊ݑܾ݉݁ݎ ݋݂ ݊݁݅݃ℎܾ݋ݎ ݏ݁݊ݏ݋ݎݏ + ݏ݁݊ݏ݋ݎ ݊݋݀݁ ݅ ሺ4 − 2ሻ The pattern focal density measure along with the damage index was used to identify the location of damage via the concentration of event-based binary data for an individual sensor node. As proposed, if one sensor did not sense the damage, or it was far away from the damage location, the neighboring effect based on the focal plane concept helped detect the damage location. 4.2.4 K-Nearest Neighbor Method Detailed information regarding the k-NN algorithm, including introduction of the method and its implementation was presented in Section 2.4.2. Once the presence and location of damage is identified by integration of image-based PR approach, pattern anomaly measure, and pattern focal density (as highlighted in previous sub-sections,) the k-NN algorithm is used to compute the damage classification accuracy using the identified damaged patterns. The dataset for the k-NN analysis was randomly classified into three subsets: training, validation, and testing. The training 78 set was used to fit the classifier, while the validation set was used to compute the optimal k for the k-NN classifier. The best models were selected based on their performance on the validation data. Performance of the classifier with optimal k was then investigated on the test set. To implement the k-NN approach, the classifier was initially calibrated for common classes from the training dataset. The calibration dataset was thus classified into 4 classes. Class 1 represented patterns due to normal condition of the plates, Class 2 denoted noisy patterns, and Classes 3 and 4 represented patterns due to damage. A confusion matrix, which contains information about actual and predicted classification, was determined for the model to assess performance of the damage detection methodology. The performance of the damage detection model with k-NN method was thus measured using the detection performance rate defined in equation (4-3). ܦܽ݉ܽ݃݁ ܦ݁ݐ݁ܿݐ݅݋݊ ܣܿܿݑݎܽܿݕ = ܰݑܾ݉݁ݎ ݋݂ ݌ܽݐݐ݁ݎ݊ݏ ܿ݋ݎݎ݈݁ܿݐݕ ݈ܿܽݏݏ݂݅݅݁݀ ܶ݋ݐ݈ܽ ݊ݑܾ݉݁ݎ ݋݂ ݅݀݁݊ݐ݂݅݅݁݀ ݌ܽݐ݁ݐݎ݊ݏ ሺ4 − 3ሻ 4.3 Numerical Modeling and Simulations Verification of the damage detection methodology was done with data from finite element (FE) simulations on square and rectangular plates. The plates were clamped along all edges and were dynamically loaded with a uniformly distributed transverse pressure. Linear-elastic implicit dynamic analyses were performed for 100 seconds with a time step of 0.01 s. The data set size for each test case was thus 10,000. However, thirty-five percent of the patterns for which all the sensors were inactive were discarded. The dataset size for each test case thus reduced to 6,500. The reduced dataset (patterns) was then classified into 4 classes as noted in Section 4.2.4. A simple pilot-type local rule for binary event generation was defined in terms of maximum principal strains at the virtual sensor nodes (i.e., location-equivalent mesh nodes in the FE model). In this context, a threshold value R1 (250 micro-strains) was considered for the generation of 79 binary events for the square plate, while a threshold value R2 (175 micro-strains) was used for the rectangular plate. Accordingly, a binary event was generated at the sensor node if the maximum principal strain response at the virtual node exceeded the noted thresholds. Numerical simulations were performed using the FE software ABAQUS [141]. To ascertain the reliability of the proposed damage detection methodology to noisy data, different levels of noise were added to the numerical input vectors. Noise-contaminated vectors were generated with equation (4-4) and used for performance evaluation with the k-NN method. ݔ௡௢௜௦௬ = ݔ௢௥௜௚௜௡௔௟ + ߙ ∙ ݎܽ݊݀ ∙ ݔ௢௥௜௚௜௡௔௟ ሺ4 − 4ሻ In equation (4-4) α is the noise level in %, xoriginal is the input signal vector, xnoisy is the noise level in %, and rand is a MATLAB [140] function that generates random values. 4.3.1 FE Simulations of a Square Plate The considered square plate prototype has 1,000 mm sides and 5 mm thickness. The material is aluminum (E = 73 GPa, ν = 0.33, ρ = 2,708 kg/m3). The plate domain was divided into 50 × 50 elements and fully clamped boundary conditions were applied at all edges. The plate was modeled using shell elements with reduced integration (S4R). Damage was simulated by gradually decreasing the flexural stiffness (elastic modulus) of elements in a region over the loading period (i.e., analysis time) from 20% to 80% (in 20% increments). Two damage scenarios, defined in terms of different damaged regions, were considered. Figure 4-4 illustrates the geometry and different damage scenarios for the square plate. Damage scenario 1 has a single damage region (Figure 4-4(a)). The different localized stiffness reduction rates resulted in 4 simulated test cases. Damage scenario 2 has three damage regions (Figure 4-4(b)). The damage severity levels were applied equally for all three areas. That is, the localized 20% stiffness reduction was considered 80 for all the three regions for one case; while for another case, a localized 40% stiffness reduction was applied to all the regions, etc. Figure 4-4. Damage scenarios considered in FE model of square plate (a) single damage, (b) multiple damage 4.3.2 FE Simulations of a Rectangular Plate Simulations on a rectangular plate (1,000 mm × 400 mm and thickness of 5 mm) were performed to evaluate the damage detection methodology for different plate aspect ratios. This plate had the same material properties as the square one. The plate was divided into 50 × 20 elements and clamped boundary conditions were applied at all four edges. The process for simulating damage was the same as for the square plate. Two damage scenarios were considered as shown in Figure 4-5. 81 Figure 4-5. Damage scenarios considered in FE model of rectangular plate (a) single damage, (b) multiple damage 4.3.3 Load Cases for FE Simulations Performance of the damage detection methodology with respect to different loading cases (harmonic amplitudes) was evaluated. The plates were subjected to a uniform transverse pressure of 0.01 MPa with different harmonic amplitude variations in time, as shown in Figure 4-6. For the square plate, the domain was divided into 4 equal regions (Ι, II, III, and IV), as shown in Figure 4-4. Similarly, the rectangular plate’s domain was divided into 2 regions (Ι and II), as shown in Figure 4-5. Three loading cases were considered for the square plate. Case 1: uniform pressure with harmonic amplitude 1 was applied on all four regions. Case 2: pressure with harmonic amplitude 1 was applied on regions II and III, while harmonic amplitude 2 was applied to regions Ι and IV. Case 3: uniform pressure with harmonic amplitudes 1, 2, 3, and 4 were applied on regions III, IV, Ι, and II, respectively. Two load cases were used for the rectangular plate. Case 1: uniform pressure with harmonic amplitude 1 was applied to all four regions. Case 2: pressure with harmonic amplitudes 3 and 4 was applied to regions Ι and II, respectively. 82 Figure 4-6. Harmonic amplitudes used for dynamic loading in FE simulations 4.3.4 Sensor Configurations for Damage Detection Model Different sensor configurations were considered to investigate the effect of sensor density on the damage detection approach. The arrangement of sensor nodes considered are presented in Figure 4-7and Figure 4-8 for the square and rectangular plates, respectively. As shown in Figure 4-7, three sensor arrangements were considered for the square plate. According to proposed image-based PR approach (Section 4.2.1), the number of sensor nodes equals the number of pattern features (i.e., dimension of the PR problem). Thus, for the square plate, the number of sensor nodes was set as 64, 144, and 256. Accordingly, each pattern (image) was represented by 64, 144, and 256 features for each sensor configuration. Similarly, two sensor layouts were considered for the rectangular plate, where the number of sensor nodes (i.e., pattern’s features) was 48 and 72 (see Figure 4-8). 83 Figure 4-7. Sensor layout and configurations for a clamped square plate in FE mode (dimensions in mm) Figure 4-8. Sensor layout and configurations for a clamped rectangular plate in FE model (dimensions in mm) 4.4 Damage Detection Simulation Results The proposed damage detection methodology using pattern recognition (PR) was implemented in MATLAB [140]. The general workflow was as follows: 1. Finite element simulations with regions of reduced stiffness and variable loading patterns were conducted, and the maximum principal strains were extracted at the sensor nodes. 2. Binary data was generated using a threshold concept (Section 4.3) for each simulation case (e.g., square or rectangular plate, damage case, loading condition and sensor layout). 84 3. The proposed image-based PR approach employing anomaly detection was used to identify normal and abnormal patterns for each simulation scenario (Section 4.3). 4. Damage presence (i.e., damaged patterns) was detected among abnormal patterns with a pattern anomaly measure (equation (4-1)). 5. Damage location was determined using the proposed pattern focal density concept (Section 4.2.3) along with a damage index (equation (4-2)). 6. Random noise (of different levels) was assigned to the input data (equation (4-4)). 7. The k-NN classification algorithm was used to assess performance of the damage detection model with discrete binary noisy data, where accuracy was computed with equation (4-3). The objective of the study reported herein was to comprehensively evaluate the proposed PR- based damage detection methodology. Figure 4-9 shows the simulation cases considered, and Figure 4-10 presents the ID name convention used for each of them. For example, S-1-20-1-256 denotes the simulation case for the square plate having single damage (region 1) with localized 20% stiffness reduction under load case 1, and with 256 sensor nodes. Similarly, R-2&3-60-2-48 refers to the case study of a rectangular plate having multiple damage (regions 2 and 3) with localized 60% stiffness decrease under load case 2, where the number of sensing node is 48. Figure 4-9. Visualization of simulation cases used for the damage detection model 85 Figure 4-10. ID definition for the simulation cases 4.4.1 Evaluation of the Damage Detection Strategy with Different Loading Conditions The unknown nature of the input force is the major difficulty in many structural health monitoring (SHM) applications. There are many SHM algorithms that work assuming a known/regular input force but fail in real applications when faced with an irregular input force. To investigate the effectiveness and robustness of the proposed damage detection strategy in realistic conditions, irregular loading cases (see Section 4.3.3) were considered. 4.4.1.1 Square Plate with Load Case 1 (Damage Scenario 1) The plate’s response under load case 1 was first determined to identify its normal patterns. The image-based PR approach recognized 4 common patterns. Patterns 1 and 2 represent normal response conditions, while patterns 3 and 4 are due to noise. Figure 4-11(a)-(b) presents normal patterns, in which the grey regions correspond to values of 0 (inactive sensing node) and blue regions indicate values of 1 (active sensing node). Once the PR classifier was calibrated, data from the FE simulation of a damaged plate having single damage (damage scenario 1) with localized 20% stiffness reduction (see Figure 4-4(a)) was introduced to the PR method for classification. The PR approach recognized 7 new patterns with respect to the 4 previously memorized. Accordingly, patterns 5 to 11 (see Figure 4-11(c)-(i)) were identified as abnormal patterns as the 86 frequency of these patterns increased consistently with time (see Section 4.2.1). The red regions correspond to values of 1, which denote the binary events generated due to abnormality. Figure 4-11. Image-based PR methodology classification results (simulation case S-1-20-1-256) 87 The noted patterns (i.e., patterns 5 to 11) could represent damage, noise, or a faulty sensor. The pattern anomaly measure was thus computed to distinguish between patterns due to damage from those due to other irregularities. Figure 4-12 presents the pattern anomaly values (see equation (4-1)) of abnormal patterns 5 to 11 for the single damage scenario (see Figure 4-4(a)). It can be seen that pattern anomaly measure for each of the localized stiffness reduction cases increases with simulation time. This indicates that the previously identified patterns (abnormal patterns in Figure 4-11(c)-(i)) represent actual damage, as their pattern anomaly measure increases with damage severity and time. Figure 4-12. Pattern anomaly measure index based on FE simulation of load case 1 and damage scenario 1 Once the abnormal patterns representing damage were identified, the damage location was detected with the proposed pattern focal density concept (equation (4-2)). Results for the pattern focal density concept are shown in Figure 4-13, which illustrates the plate’s behavior as determined from surface interpolation of the focal density of binary events. Normal condition of the plate is evident from Figure 4-13(a) given the smooth surface of the computed density. Figure 4-13(b)-(d) show the surface interpolation of the binary data’s focal plane density for damaged patterns 7, 8, and 10, where the location of damage can be clearly discerned. In addition, the damage index (DI) of the sensing nodes corresponding to the damage location was determined for each of these 88 patterns (and shown in the figures), thus representing the severity of damage through concentrations of generated binary events. Figure 4-13. Damage indices obtained from pattern focal plane concept (simulation case S-1-20- 1-256) Damage severity was simulated by increasing the stiffness reduction’s rate. New abnormal patterns (with respect to the eight previously identified from the simulation with 20% stiffness decrease) were identified because of the increase in damage severity. It is noted that the pattern anomaly measure was also computed, confirming that the new abnormal patterns represented damage. Thus, one, five and eight new damaged patterns were recognized due to the localized stiffness reduction of 40%, 60%, and 80%, respectively. Figure 4-14 shows damaged patterns 13, 15, and 18, recognized due to stiffness decreases of 40%, 60, and 80%, respectively. The generation of binary events with increasing damage severity is obvious. The pattern focal density 89 of patterns 13 and 18, along with the corresponding damage indices, are presented in Figure 4-15. Results indicate that the damage index represents a unique characteristic for damage detection. Evidently, the damage indices of patterns 13 and 18, identified from the FE simulations with greater damage severity (localized 40% and 80% stiffness reduction), were 0.50 and 0.85; which are significantly higher than those of patterns 7 and 8 (0.12 and 0.25) recognized from simulations with lower damage severity. Figure 4-14. Image-based PR approach classification results in terms of identifying new damaged patterns:(a) simulation case S-1-40-1-256, (b) case S-1-60-1-256, (c) case S-1-80-1-256 Figure 4-15. Damage indices obtained from focal plane theory for new identified damaged patterns: (a) case S-1-40-1-256, (b) case S-1-80-1-256 90 4.4.1.2 Square Plate with Load Case 1 (Damage Scenario 2) Binary data generated from simulations of the square plate with damage scenario 2 (see Figure 4-4(b)) and different levels of stiffness reduction was used as input to the image-based PR approach. For the case of stiffness reduction of 20%, the approach identified 6 new damaged patterns (patterns 5 to 10). As expected, the number of patterns representing damage increases with increasing damage severity (stiffness reduction’s rate.) That is, patterns 11, 12 to 16, and 17 were recognized due to a localized stiffness decrease of 40%, 60%, and 80%, respectively. Thus, the presence of damage was successfully detected through the identified damaged patterns. Figure 4-16 shows some of the identified patterns for damage scenario 2. To further validate the results, the pattern anomaly measure for the damaged patterns was determined and shown in Figure 4-17. Figure 4-16. Image-based PR approach classification results (load case 1 and damage scenario 2) 91 Results of the pattern focal density concept for the case of multiple damage are shown in Figure 4-18. The plots illustrate the damage locations and indices determined from surface interpolations of the binary event’s focal plane density. The damage regions are annotated with their corresponding damage indices (DI). The interesting point is that the effect of damage progression on the focal density of binary data is evident, such that damage indices increase with an increase of damage severity. For instance, the damage index for the damaged region at the right upper part of the plate (damage region 3) is 0.25 for patterns 7, 9, and 10, and this value to 0.80 (see Figure 4-18(e)) with damage progression. Similarly, DI for damage region 4 increases from 0.14 (Figure 4-18(b)) up to 0.70 (Figure 4-18(f)). Results confirm that the damage index has a strong relation with damage progression. As seen from the results in Figure 4-18, all three damage regions (reduced stiffness regions in the FE simulations) were identified by the proposed focal density concept. Figure 4-17. Pattern anomaly measure based on FE simulation of load case 1 and damage scenario 1 To further assess the proposed damage detection strategy, a k-nearest neighbor (k-NN) classification algorithm was used for damage classification knowing the normal, damaged, and noisy patterns. As discussed before, the decreased dataset size was 6,500, and the patterns were classified into 4 classes (see Section 4.2.4), with 70% of the data was used for calibration, 15% for validation, and 15% for testing. Classification accuracy was determined (equation (4-3)) for the 92 different damage scenarios and error rates were obtained. Further, different values of k were chosen for the k-NN method and damage detection accuracy was determined for each. Yet, the optimal k value after the evaluation steps was found to be five. Figure 4-18. Damage indices obtained from pattern focal plane theory (load case 1 and damage scenario 2) To visualize the classification performance of the model on the calibration and test dataset using k-NN, the corresponding confusion matrices were determined. Table 4-1 and Table 4-2 present the confusion matrices of the square plate for damage scenario 2 with localized 80% stiffness reduction (case S-2,3&4-80-1-256). The k-NN classification accuracy for the calibration dataset was 99.95% (0.044% error,) while the accuracy for the test dataset was almost 98.2% 93 (1.74% error.) The results reveal the good performance of the proposed damage detection strategy in terms of identifying the damage. Table 4-1. Confusion matrix for training data Predicted classes True classes 1 2 3 4 1 455 2 0 0 0 0 910 0 0 3 0 0 2200 4 0 0 0 True sum 455 910 2200 2 983 985 Sum 455 910 2202 983 4550 Error (%) 0 0 0.09 0 0.044 Table 4-2. Confusion matrix for test data True classes 1 2 3 4 Predicted classes 1 97 0 0 0 2 0 193 1 5 3 0 2 471 4 0 0 6 3 197 Sum 97 199 476 203 True sum 97 195 478 205 975 Error (%) 0 3.0 1.05 2.90 1.74 4.4.1.3 Square Plate with Load Cases 2 and 3 (Damage Scenarios 1 and 2) To evaluate the robustness of the damage identification approach with respect to irregular loading, load cases 2 and 3 (defined in Section 4.3.3) were considered for the FE simulations. The process of detecting the presence of damage by identifying damaged patterns with pattern anomaly measures, and determining the location of damage via the pattern focal density concept was discussed in the prior subsections. Thus, only the results under non-harmonic loading are provided here. Six common patterns were identified, where patterns 1 to 3 denoted normal condition and patterns 4 to 6 were due to noise. After calibrating the image-based PR approach, data from simulations of the damaged plates (with single and multiple damage regions) with various damage 94 severities were fed into PR method. For damage scenario 1 with load case 2, patterns 7 to 15 were recognized as a result of simulated damage; whereas for damage scenario 2 with the same load case, 10 damaged patterns were identified (patterns 7 to 16). Similar results were obtained for load case 3, that is, the number of damaged patterns for damage scenarios 1 and 2 were 9 and 10, respectively. Thereafter, the pattern anomaly measure for each test case was computed to identify the actual damaged patterns. The pattern’s anomaly measures for the potential damaged patterns are presented in Table 4-3 and Table 4-4 for load cases 2 and 3, respectively. The increase in pattern anomaly measure with simulation time for each load case is noticeable; where for most cases these values increase with time and damage severity. Thus, the presence of damage was successfully detected using the pattern anomaly measure even under irregular loading conditions. Table 4-3. Pattern anomaly measure with load 2 Pattern anomaly measure Damage scenario Time interval (sec) Localized stiffness reduction 1 2 0 - 20 20 - 40 40 - 60 60 - 80 80 - 100 0 - 20 20 - 40 40 - 60 60 - 80 80 - 100 20% 40% 60% 80% 0.02 0.04 0.12 0.27 0.34 0.00 0.05 0.15 0.19 0.26 0.06 0.15 0.40 0.45 0.56 0.03 0.19 0.37 0.42 0.52 0.09 0.25 0.38 0.57 0.70 0.08 0.28 0.46 0.48 0.54 0.04 0.27 0.45 0.70 0.76 0.15 0.32 0.48 0.65 0.69 95 Table 4-4. Pattern anomaly measure with load 3 Damage scenario Time interval (sec) Localized stiffness reduction Pattern anomaly measure 1 2 0 - 20 20 - 40 40 - 60 60 - 80 80 - 100 0 - 20 20 - 40 40 - 60 60 - 80 80 - 100 20% 40% 60% 80% 0.00 0.00 0.07 0.05 0.14 0.05 0.06 0.13 0.14 0.15 0.01 0.06 0.11 0.17 0.20 0.10 0.13 0.16 0.18 0.19 0.01 0.14 0.27 0.40 0.45 0.14 0.24 0.30 0.36 0.41 0.02 0.14 0.33 0.42 0.50 0.17 0.27 0.32 0.43 0.46 As before, the pattern focal density concept was applied to detect damage location by plotting surface interpolations of the focal density of damaged patterns. The surface plots for damaged patterns 13, 16, and 17 (load case2) are presented in Figure 4-19(a)-(c), while the plots for patterns 7, 16, and 17 (load case 3) are shown in Figure 4-19(d)-(f), respectively. A noteworthy point is that the superposition of surface focal density plots from different patterns yields a map of all the damaged regions. As an example, damage region 1 is detected using the surface plot of pattern 7 (Figure 4-19(a)), while surface plots for patterns 16 and 17 led to detecting damage regions 2, 3, and 4. As seen from Figure 4-19(f), all three damage regions were identified by the surface plot of pattern 17 (even though the DI of region 3 is only 0.17). Therefore, the proposed pattern focal density concept allows detecting single and multiple damage locations even under irregular loading conditions. 96 Damage Location, DI=0.60 Figure 4-19. (a), (b), and (c): Damage indices (load case 2 and damage scenario 2), (d), (e), and (f): Damage indices (load case 3 and damage scenario 2) For the next analysis step, the k-NN method was used for damage classification. The optimum value of k was found to be five. Table 4-5 and Table 4-6 present the best classification results for different damage scenarios with various localized stiffness reduction rates for load cases 2 and 3, respectively. Evidently, the damage detection strategy using k-NN shows satisfactory performance on the calibration, validation, and test data, even for test cases with irregular and non-harmonic loading conditions. According to the results, damage detection performance/accuracy on the calibration data was higher compared to the classifier’s performance on the validation and testing data. The minimum performance on calibration data for load cases 2 and 3 was 97% and 94%, respectively. On the other hand, the least accuracy value on the validation data was 91% for load case 2 for a localized stiffness reduction of 80% (case S-2,3&4-80-2-256), while this value was 97 88% for load 3. Finally, results indicate good classification performance of the model on test data; that is, minimum performance was 90% for load case 2. It can be observed that the model’s precision decreases slightly with increasing stiffness reduction (damage severity) but it remains acceptable even for the test case of load case 3 with 80% localized stiffness reduction, with a damage detection performance of 89%. Table 4-5. Damage detection performance based on load case 2 Damage scenario stiffness reduction Classification accuracy (%) Calibration Validation Test 1 2 20% 40% 60% 80% 20% 40% 60% 80% 100% 99% 99% 98% 99% 98% 97% 97% 97% 95% 92% 95% 96% 94% 92% 91% 96% 94% 93% 95% 95% 94% 93% 90% Table 4-6. Damage detection performance based on load case 3 Damage scenario stiffness reduction Classification accuracy (%) Calibration Validation Test 1 2 20% 40% 60% 80% 20% 40% 60% 80% 98% 96% 94% 95% 97% 96% 96% 94% 95% 93% 93% 90% 95% 94% 90% 88% 95% 94% 92% 91% 94% 92% 91% 89% To further evaluate the proposed strategy, the calibration, validation, and test data were contaminated with random noise according to equation (4-4). Noise levels of 5% to 30% in 5% increments were considered. Figure 4-20 presents the classification results (using k-NN) for the validation and test data for damage scenario 2 and load case 3. The results demonstrate that damage detection was affected by data noise such that the model’s performance gradually decreased with 98 increasing noise level. However, the damage detection algorithm had satisfactory performance for noise levels of up to 20% (e.g., 85% accuracy on test data with a localized 80% stiffness reduction). Nonetheless, performance for higher noise levels (>25%) was not acceptable. Figure 4-20. Damage detection performance for damage scenario 2 and load case 3 for varying noise level: (a) Validation data, (b) Test data 4.4.2 Evaluation of the Damage Detection Strategy with Different Sensor Configurations The damage detection accuracy using k-NN algorithm was determined with Equation (9) for different damage scenarios and sensor densities. Figure 4-21, Table 4-7, and Table 4-8 present the classification results of the square and rectangular plates for damage scenario 2 and a localized 80% stiffness reduction under load case 1 for different number of sensors and noise levels. It can be seen that increasing the noise level did not affect the performance of the model on the calibration data. For the test data, the results trend are complicated. The classification accuracies gradually decrease with an increase in noise. The results indicate that nearly all of the models have good performance for noise levels up to 25%. This is more evident for the square and rectangular plates with 256 and 72 sensor nodes, where the model’s performance was still acceptable even for noise levels up to 30%. However, performance of the models subjected to 30% noise level was not satisfactory. The results reveal that increasing the number of sensor nodes enhances the performance of the damage detection model on calibration and test data. 99 Figure 4-21. Damage detection performance for different sensor configurations with varying noise level: (a) Training data, (b) Test data Table 4-7. Damage detection performance for rectangular plate Number of sensors Noise level Damage detection performance (%) Calibration Validation Testing 48 72 5% 10% 15% 20% 25% 30% 5% 10% 15% 20% 25% 30% 100% 99% 98% 99% 97% 95% 100% 100% 98% 96% 97% 96% 98% 94% 92% 90% 84% 81% 98% 95% 91% 89% 84% 81% 96% 94% 93% 89% 85% 82% 97% 96% 91% 92% 88% 85% Table 4-8. Damage detection performance for square plate Number of sensors Noise level Classification accuracy (%) Calibration Validation Testing 64 256 5% 10% 15% 20% 25% 30% 5% 10% 15% 20% 25% 30% 100% 100% 99% 98% 99% 98% 100% 100% 100% 100% 99% 99% 98% 96% 90% 85% 82% 79% 96% 93% 90% 87% 89% 85% 96% 93% 89% 86% 84% 82% 98% 96% 93% 92% 89% 84% 100 4.5 Summary This chapter presented a new methodology for localized damage identification using PR framework that was designed for SHM with discrete binary data. The PR-based algorithmic framework merges an image-based PR approach using anomaly detection, a pattern anomaly measure index, a pattern focal density measure index, and the k-nearest neighbor (k-NN) classification algorithm. The proposed damage detection strategy was applied to simulated scenarios, through the finite element method, of clamped aluminum plates under transverse pressure. The effect of damage severity, damage scenarios, measurement noise, loading conditions, and sensor density on the performance of the proposed damage detection method and the robustness of the approach against the noted features were evaluated. The following conclusions were reached from the study: 1) Numerical simulation results demonstrate robustness and effectiveness of the proposed damage identification methodology with non-harmonic loading conditions and relatively dense sensor configuration. 2) Validity of the method in terms of identifying the presence of damage was assessed through a proposed pattern anomaly measure. 3) Damage location was accurately identified using a proposed pattern focal density concept with corresponding damage indices. An important finding from the presented strategy is that identifying damage location through a damage index has a strong correlation with damage progression. 4) The obtained results show that the proposed damage index provides reliable information regarding the damage location in the presence of single damage, harmonic load, and dense sensor distributions. However, for non-harmonic loading and multiple damage locations, 101 the severity of damage has to be relatively high in order to successfully identify multiple damage locations. 5) The accuracy of the proposed SHM methodology employing PR was investigated using the k-NN method. Classification results revealed that damage detection performance remained satisfactory even with increased damage severity. 6) Results with noise polluted data indicated that performance of the damage detection model with harmonic loading remained acceptable up to noise levels of 25%. The results presented in this chapter demonstrate that the proposed PR-based algorithmic framework can be effectively used for interpreting discrete event-based binary data in novel wireless sensor networks for self-powered SHM. The PR framework enables a damage detection model that is effective and robust with respect to several features that can affect its performance. Results suggest that the proposed method can be applicable for real-life structures composed of metal plates. 102 CHAPTER 5 5 PROBABILISTIC-BASED ALGORITHMIC FRAMEWORKS FOR DAMAGE IDENTIFICATION AND CONDITION ASSESSMENT WITH DELAYED SIGNALS 5.1 Overview Chapters 3 and 4 presented a PR-based methodology for SHM and damage identification assuming full data availability, while the effect of power budget constraints and time data delay were disregarded. Different PR methods and global interpretation algorithms were examined only from the aspect of dealing with discreet binary data, whereas effectiveness and robustness of such methods against irregular loading, sensor sparsity, measurement noise, etc., was evaluated. The present chapter considers the effect of time delay due to the pulse switching communication protocol and algorithmic frameworks detecting the effect of delay on the data/signals is pursued. The proposed image-based PR approach and k-NN method presented in previous chapters are used and integrated with probabilistic methods. An innovative conditional probability chain was developed as part of a framework to consider the effect of data time delay on its interpretation. Further, a probabilistic approach was developed to model delay and recover delayed data. The proposed approaches were evaluated using finite element simulations and experimental studies on structural plates. The contents presented in this chapter were partially published in the journal Measurement [148]. 5.2 Probabilistic Framework Integrating PR and a Conditional Probability Chain The damage detection procedure’s development and validation is based on data obtained through the structural response simulation of a simply supported aluminum plate with the finite element method. The specification of the plate under study is presented in Figure 3-3. Cumulative 103 acceleration response at the plate sensor nodes was considered as feature vectors for training the image-based PR approach (see Chapter 3). Accordingly, each pattern was represented by 16 features. Figure 5-1. Flow chart of the proposed damage detection strategy using PR and conditional probability chain 104 Damage was simulated in the FE model by gradually decreasing the stiffness (elastic modulus) of a region in the plate (see Figure 3-3) with respect to time. Damage severity was considered by means of levels of stiffness reduction. Damage scenarios were defined as ‘classes’ and numbered from 1 to 7 for stiffness reductions of 20% to 80% (in 10% increments), respectively. As noted. A PR approach was incorporated with a conditional probability chain to predict delayed and missing signals. Finally, k-NN method was used for damage classification. Figure 5-1 shows a flow chart of the proposed damage detection strategy merging PR and a conditional probability chain. A schematic of the PR method presented in Chapters 3 and 4 is presented in Figure 5-2. Acceleration measurements at the sensor nodes are used to generate a binary pattern that consists of four features [1 0 0 1]. Thus, this example presents a 4-dimensional PR problem. Classification is then performed using the generated binary pattern. Figure 5-2. Illustration of PR method based on anomaly detection According to a through-substrate ultrasonic network, the self-powered sensor under consideration uses a piezoelectric vibration-based harvester. Therefore, the accumulated energy is proportional to the acceleration history at the harvester. Acceleration was thus chosen as the response signal in the study presented in this chapter since it could serve the dual purpose of a structural response measure as well an indicator of the available communication power budget at the sensor node. A simple pilot-type local rule for binary event generation was defined in terms of 105 acceleration thresholds R1 and R2 (namely 2.3g and 2.5g, respectively) at the sensor nodes (i.e., FE model mesh nodes). Threshold R1 was simply based on acceleration responses from the FE simulations so that binary events representing damage could be generated. Threshold R2 was chosen so that it could provide the required energy based on an energy-harvesting model to communicate the events as discussed in Section 2.1. According to the FE simulations, the maximum node transverse accelerations ranged from 20 m/sec2 to 23 m/sec2 (i.e., approximately 2 g to 2.5 g). It is clear that a higher energy harvesting profile (higher acceleration response) results in decreased delivery delays, which is essential for the pulse switching protocol. Therefore, threshold R2 was rationally set at 2.5 g. According to the threshold concept, a binary event will be generated if the accumulative acceleration response at the sensor node is higher than R1. In other words, the acceleration values at the sensor node were accumulated over time and a binary event is created when threshold R1 is exceeded. However, the event cannot be communicated until the accumulative acceleration response exceeds threshold R2. Once an event is communicated, the accumulated acceleration value of that sensor node becomes zero, as the node used its energy to communicate the event. 5.2.1 Performance Verification of the Probabilistic Framework The image-based PR approach was first used to identify patterns due to different conditions of the plate. Thereafter, the identified patterns were introduced as an input to the spatial-temporal conditional probability chain to determine damage indication parameters. Once the damage indication parameters were determined, a reliability factor for each pattern was computed using damage indicators and the presence of damage was detected by identifying the actual damaged patterns. Afterward, the location of damage was determined based on a proposed focal plane concept by computing the focal density of the binary data for each pattern. Finally, a k-NN 106 classifier was used to examine further the performance of the damage detection strategy using normal and actual damaged patterns, and to evaluate the robustness of the approach based on uncertainty analysis. Cumulative acceleration at the sensor nodes was extracted from the FE simulations to define the feature vectors for the scenarios listed below: • Case 1: An undamaged plate (constant stiffness) with full data availability • Case 2: A damaged plate (region with reduced stiffness) with full data availability • Case 3: An undamaged plate with time-delayed data • Case 4: A damaged plate with time-delayed data. A supervised PR approach employing anomaly detection was used to train the classifier using data extracted from cases 1 and 3. The normal plate’s response under a uniform 0.05 MPa pressure with harmonic amplitude was first recognized. The PR approach identified 6 common patterns (Patterns 1 to 6) as shown in Figure 5-3, in which the white and blue regions correspond to values of 0 and 1, respectively. Patterns 1 and 2 represent normal condition, while patterns 3 to 6 were due to time delay. After calibrating the classifier, data from cases 2 and 4 (damage cases) was used for classification. The PR method based on anomaly detection recognized 10 new patterns with respect to the 6 previously memorized patterns (see Figure 5-3). Accordingly, patterns 8 and 16 were identified due to damage, whereas the rest of the patterns were due to time delay and noise in the system. 107 Figure 5-3. Classification results of identified patterns (images) based on image-based PR approach After performing the classification using anomaly detection, the k-NN method was used to evaluate the damage detection strategy. For this purpose, the entire dataset was classified into two classes: a damaged class (including patterns 8 and 16) and an undamaged class (rest of the 14 patterns in Figure 5-3). Accordingly, FE simulated data of a damaged plate corresponding to one of the damaged scenarios (e.g., damage class 1) were used for calibration of the classifier. As previously noted, k-fold cross validation was used to calibrate the k-NN classifier. The number of partitions for k-fold cross validation was 2. Thus, the k-NN classifier was calibrated based on 50% of the dataset and tested with the remaining 50%. After performing k-fold cross validation and finding the best model, the classifier was calibrated using the entire dataset. Thereafter, the dataset 108 from the simulation of another damaged scenario was introduced to the PR algorithm. A PR technique using a k-NN classifier was thus used to recognize the damaged class. A detection performance (DP) parameter was used to compute the model’s performance according to equation (5-1), and the error rate (ER) for classification was determined through equation (5-2). DP = Number of Patterns Correctly Classified Total Number of Patterns (5-1) ER = Number of Patterns Misclassif ied Total Number of Patterns (5-2) 5.2.2 Theory of a Spatial-Temporal Conditional Probability Chain According to the pulse switching protocol, a sensor’s data (or value) at a given time t could be 0 (i.e., no event reported) at the data sink due to the time delay (caused by insufficient power for the multi-hop communication protocol). Values of missing data clearly need to be known to have a reliable damage detection model. To cope with this difficulty, an innovative spatial-temporal conditional probability chain was developed as part of the pattern recognition framework to take into account the effect of time delay on data interpretation algorithms. Delayed data values at the sensor nodes were predicted using previous values at the affected sensor (temporal aspect) and values of the neighbor sensors (spatial aspect) along with a reliability coefficient. A schematic of the proposed conditional probability chain is presented in Figure 5-4, which illustrates temporal integration within the sensor nodes of recognized patterns (images) and spatial integration of discrete binary data among neighbor sensors. 109 Figure 5-4. Illustration of the proposed spatial-temporal conditional probability chain The proposed chain uses the conditional probability of occurrence of binary events at each sensor (cell) with respect to the current state of its neighbors. The state of each sensor is defined as 0 or 1, such that 1 indicates that the binary event has occurred and 0 denotes an inactive sensor node. Conditional probabilities of each sensor’s state according to the state of its neighbors were determined using Bayes formula. The probability of event occurrence at each sensor depends on: (1) the state of the neighboring sensors (spatial aspect), and (2) the frequency of identical events in past time steps (temporal aspect). For instance, the conditional probability of observing 1 at a specific sensor node provided that one of the neighbors is 0 was calculated using equation (5-3). Here, P(1) was estimated based on the average frequency of observing 1 for that sensor node in previous time steps, P(0) was computed for the neighbor sensor, and P(0|1) denotes the probability that 0 and 1 occur simultaneously. Similarly, the values of P(0|1), P(1|1), and P(0|0) were calculated for each sensor node. P )0|1( = P ).1( P )1|0( P )0( (5-3) 110 5.2.3 Damage Indication Parameters Statistical parameters were defined and computed to obtain the effective indices for: (1) identifying abnormal patterns from normal ones, (2) detecting patterns representing damage among the abnormal ones, and (3) determining a reliability factor for each pattern. According to the proposed chain, a conditional probability of event occurrence was computed at each sensor node and time step. The parameters PC and PNC were thus introduced to the conditional probability chain at each sensor node using the corresponding probabilities (e.g., P(1|0), P(1|1), etc.). In general, PC represents the probability of the observed data being correct (probable) according to the probabilities calculated from previous steps, while PNC denotes the probability of the observed data not being correct (improbable) based on the prediction. For instance, assuming a conditional probability P(1|0) for sensor node i, PC for this sensor node was determined by computing the average values of P(1|0) of that sensor and its neighbors. Figure 5-5 schematically illustrates the procedure for computing PC for sensor node 6 at time t. Figure 5-5. Schematic illustration of the determination of probabilities PC and PNC 111 According to Figure 5-5, P6,7 refers to the conditional probability that the value of sensor 6 is 1 given that the value of neighbor sensor 7 is also 1, where this probability was calculated using a conditional chain. PNC was similarly computed at each sensor node. After determining PC and PNC for all the sensor nodes, each pattern was represented by spatial integration of PC and PNC of the nodes. Accordingly, the following statistical parameters were computed for each pattern for use in the damage detection model. MX = (max PC − PNC ) (5-4) MD = mean deviation ( PC − PNC ) = 16 ∑ = 1 i PC i ( PNC i − N ) (5-5) STD = Standard deviation (PC-PNC) (5-6) RMSE = 1 N 16 ∑ = 1 i ( PC − i PNC 2) i (5-7) Spatial-temporal integration of PC and PNC based on the discrete asynchronous binary data for the normal condition pattern (pattern 1) and the pattern representing damage (pattern 8) were determined for the first time step that patterns were identified, and are shown in Figure 5-6(a) to Figure 5-6(d). The interesting point in these figures is that PC is less than PNC for the sensor nodes near the reduced stiffness region for pattern 8 (see Figure 5-6(c) and Figure 5-6(d)), which indicates that the current values for these sensor nodes (which are 1) are not probable and thus their value is more likely to be 0 based on previous observations (temporal aspect). Nonetheless, the simulated damage (stiffness reduction) led to the generation of binary events. This argument shows the significance of PC and PNC for the damage detection model. However, it is important to note that as more data is accumulated the difference between PC and PNC might vary, and the predicted value at the sensor node may be consistently predicted to be 1. 112 Figure 5-6. Spatial-temporal representation of sensor node values based on the probability of occurrence of binary events: (a) & (b) PC and PNC for normal pattern 1, (c) & (d) PC and PNC for damaged pattern 8 5.2.4 Damage Detection based on FE Results 5.2.4.1 Image-based PR approach Classification Results The frequency of patterns and the mean probability deviation (equation (5-5)) were the most efficient damage indication variables. Results show that the frequency of patterns can be considered as a parameter to recognize abnormal patterns. Figure 5-7 and Figure 5-8 present the variation in frequency of normal and abnormal patterns with time for damage classes 3 and 5, respectively. It can be observed that the frequency of normal patterns noticeably decreases with time for different stiffness reduction levels. Conversely, Figure 5-8 shows how the frequency of abnormal patterns tends to increase with the simulation time; but it is noted that only the frequency 113 of patterns 8 and 16 steadily increased while that of other abnormal patterns showed varying fluctuations. Figure 5-7. Frequency of normal patterns with time for (a) Damage class 3, (b) Damage class 5 Figure 5-8. Frequency of abnormal patterns with time for (a) Damage class 3 (Patterns 7 to 11), (b) Damage class 5 (Patterns 7 to 11), (c) Damage class 3 (Patterns 12 to 16), (d) Damage class 5 (Patterns 12 to 16) To better appreciate the importance of the frequency of patterns measure, the variation of average frequencies of abnormal patterns 4, 6, 8, and 16 with respect to the increase in damage 114 severity (localized stiffness reduction) was calculated. The results are shown in Figure 5-9, which indicate that the average frequency of abnormal patterns 4 and 6 caused by time delay decreases with an increase in damage severity. Conversely, it can be determined that patterns 8 and 16 represent the actual damage since the frequency of these patterns clearly increases with increasing damage severity. Thus, the frequency of patterns is a suitable indicator to identify abnormal patterns representing actual damage from others that may be caused by system noise. Figure 5-9. Average frequency of abnormal patterns with respect to localized stiffness reduction: (a) Patterns 4 and 6 (patterns due to delay) for damage class3, (b) Patterns 4 and 6, damage class 5, (c) Patterns 8 and 16, damage class 3, (d) Patterns 8 and 16, damage class 5 The statistical parameter given by the ‘mean deviation (PC – PNC)’ was also found to be an accurate index to detect abnormal patterns that represent damage. According to Figure 5-10(a), the ‘mean deviation (PC – PNC)’ value of patterns 7 and 12 are approximately 0.35 and 0.27, respectively, nearly 70% lower than that of pattern 8 (0.59). The ‘mean deviation (PC – PNC)’ 115 value of patterns 13 and 15, shown in Figure 5-10(b) are 0.33 and 0.37, while that for pattern 16 is 0.58, or about 75% higher. It is clear that the ‘mean deviation (PC – PNC)’ value for abnormal patterns representing damage is significantly higher than for those caused by time delay and noise. This indicator is thus considered as an efficient parameter to identify abnormal patterns representing damage. It should be noted that the ‘mean deviation (PC – PNC)’ value for each pattern was almost constant during the entire analysis time. Figure 5-10. Variation of mean deviation of (PC-PNC) with time for damage class 5 (a) Patterns 7, 8, and 12, (b) Patterns 13, 15, and 16 5.2.4.2 Detection the Presence of Damage based on a Reliability Factor Concept To accurately identify damage while considering the effect of data time delay and predicting missing values, a reliability factor assigned to each pattern (or image) is proposed. Based on the presented conditional probability chain approach, previous values and neighboring values were used to predict the current pattern (image). The pattern reliability is expected to increase as more data becomes available. On the other hand, results show that the frequency of abnormal patterns increases and that of normal patterns decreases with local stiffness degradation with time (Figure 5-8 and Figure 5-9). Further, the value of ‘mean deviation (PC – PNC)’ remains constant for each pattern (Figure 5-10). Thus, an image (pattern) reliability factor was empirically defined by 116 integrating the frequency of patterns and the statistical index ‘mean deviation (PC – PNC)’ as expressed in equation (5-8). Reliability factor = (Frequency of pattern) × (Mean deviation (PC-PNC)) (5-8) Consequently, a reliability factor for each abnormal pattern was determined for the entire analysis time, and these are plotted in Figure 5-11 and Figure 5-12. Abnormal patterns with a high reliability factor are thus considered the actual damage patterns, even in the presence of time delay. According to Figure 5-11, the reliability factor of abnormal pattern 8 increases with time, while this response is not displayed by other patterns. Similarly, Figure 5-12 shows the reliability factor of pattern 16 noticeably increasing from zero to approximately 0.7, while the increase for other patterns is much smaller or negligible. The increasing image reliability factor for patterns 8 and 16 is attributed to the increasing frequency of these patterns with stiffness reduction. Thus, the results presented in Figure 5-11 and Figure 5-12 confirm that patterns 8 and 16 are patterns representing actual damage. The other patterns with a low reliability factor are classified as abnormal due to time delay and system noise and can be disregarded as damaged patterns. Based on the results shown, the reliability factor concept method is able to enhance damage detection performance by filtering out the effects of time delay and missing data. 117 Figure 5-11. Reliability factor for abnormal patterns with respect to time for patterns 7 to 11 for: (a) Damage class 3 (localized 40% stiffness decrease), and (b) Damage class 5 (localized 60% stiffness decrease) Figure 5-12. Reliability factor for abnormal patterns with respect to time for patterns 12 to 16 for: (a) Damage class 3 (localized 40% stiffness decrease), and (b) Damage class 5 (localized 60% stiffness decrease) 5.2.4.3 Detection the Location of Damage based on Pattern Focal Plane Measure To efficiently determine the location of potential damage and the state of the monitored structure, it is essential to link the binary events to physical phenomena. The focal plane concept was thus utilized to introduce a focal density measure of binary events that was determined by adding binary events (values of 1) generated at each sensor and its neighbors. Accumulated binary events were then normalized by the maximum number of neighboring cells plus the cell itself, as illustrated in Section 4.2.3. The resulting density of binary events was then used to illustrate their concentration. 118 In this study, the maximum number of neighboring cells for a given location was 8 given the sensor configuration. Figure 5-13. Surface interpolation of focal density of binary data for (a) normal pattern 1, (b) damaged pattern 8 Figure 5-14. Surface interpolation of focal density of binary events for pattern 16 (damaged pattern) 119 Results for this focal plane concept are shown in Figure 5-13 and Figure 5-14, which illustrate the plate’s behavior as determined from the surface interpolation of the focal density of binary events. The surface plot for pattern 1 (plate’s normal condition) is presented in Figure 5-13(a), while the plots for patterns 8 and 16 representing damage are shown in Figure 5-13(b) and Figure 5-14, respectively. As can be seen, the damage location (reduced stiffness region in the FE simulation) was accurately determined by the focal plane concept. 5.2.4.4 Performance Evaluation of the Probabilistic-based Damage Detection Approach using k-NN Algorithm To further assess the proposed damage detection strategy, a k-nearest neighbor (k-NN) method was used for pattern identification knowing the actual normal and damaged patterns. Identified patterns were first classified as damaged or undamaged classes. Detection performance values were then determined with equation (5-1) for different damage scenarios and error rates were obtained. Figure 5-15(a) illustrates the classification results in terms of performance of the damage detection model on the calibration and testing dataset with respect to stiffness reduction for damage class 5. Results for different damage scenarios are presented in Figure 5-15(b). As it can be observed, the damage detection strategy shows satisfactory performance on the training (calibration) and testing data. The classification accuracy for classes 3 and 5, which correspond to a 40% and 60% localized stiffness decrease, respectively, is promising. However, it can be seen that the model’s precision slightly decreases with increasing stiffness reduction, although it remains acceptable even for damage class 7 (localized 80% stiffness decrease), for which the detection performance is 94%. While the results indicate the applicability and effectiveness of the proposed PR-based SHM method, it is acknowledged that the presented results correspond to the load case considered in this study, i.e., harmonic loading. Further, even though 120 the proposed reliability index was used to take into account the effect of missing images (due to time delay or load variations), it is acknowledged that the results could be affected by irregular loading conditions. This is a subject of ongoing studies. Figure 5-15. (a) Detection performance with respect to stiffness reduction for calibration and testing data, (b) Comparison of the model’s performance for different damage scenarios 5.2.4.5 Results of Uncertainty Analysis As a final evaluation of the proposed strategy, the calibration and testing dataset were contaminated with random noise. Noise levels of 5%, to 35% in 5% increments were considered. Figure 5-16(a) presents the classification results obtained using a k-nearest neighbor classifier with various noise levels for damage class 5 (stiffness reduction up to 60%). Furthermore, the damage detection performance for all 7 damage classes was determined with respect to the noise level, and is shown in Figure 5-16(b). The results show that the model’s performance on the calibration data was not affected by the increase in noise level. Conversely, the model’s damage detection accuracy for the testing data gradually decreases with increasing noise level. The damage detection model had acceptable accuracy for noise levels up to 20%, but its performance for data with higher noise level (> 30%) was not satisfactory. 121 Figure 5-16. (a) Damage detection accuracy for damage class 3 for varying noise level, (b) Damage detection performance for the 7 damage classes with respect to noise level 5.3 Probabilistic Approach for Modeling Delayed Signals The proposed probabilistic approach presented in previous section was found to be computationally intensive. To tackle the notes issue and to take into account the effect of large values of time-delay, a new probabilistic approach is developed to reconstruct time-delayed signals and to model delay. The SHM strategy presented in Figure 5-17 is designed upon merging an image-based PR, a novel probabilistic approach, and support vector machine (SVM) algorithm. A key contribution is the development of a probabilistic approach to reconstruct/recover delayed signals. Thereafter, the SVM algorithm is used to detect the presence of damage and to determine damage detection accuracy with the reconstructed signals. 122 Figure 5-17. Proposed methodology for energy-efficient SHM using a probabilistic approach 5.3.1 Experimental Study The development and validation of the proposed SHM strategy employing a probabilistic approach is based on data obtained through experiments on a cantilever plate shown in Figure 3-5. Detailed information of experimental plate was presented in Section 3.4. The continuous experimental response (strain) from the vibration tests was post-processed to extract binary signals based on response thresholds. Consequently, strain responses were collected from the strain gages. However, the acceleration response is needed to simulate the through-substrate sensor network. To this aim, an experimentally calibrated finite element (FE) model of the cantilever test plate was used to extract the acceleration responses at the stain gage locations. The key idea is that binary signals are generated based on strain at a local level (i.e., sensor nodes or strain gages), saved in 123 the sensor cell memory, and communicated if enough power is available based on accumulated harvested energy (from acceleration at sensor nodes). Accordingly, experimental strain responses were used to generate binary signals using a threshold concept, and acceleration responses extracted from the FE simulations were used to determine harvested energy. The acceleration response and binary signals were thus used in a simulation algorithm of the through-substrate sensor network employing an energy-aware pulse switching protocol to generate the time-delayed binary signals. The generated time-delayed binary signals were finally the input to the SHM methodology. Figure 5-18. Strain response of the cantilever plate: (a) Node #1, and (b) Node #4 The strain responses from the vibration tests were used to calibrate the FE model. Simulated and experimental strain histories (for the first 4 seconds of the experiment) at nodes 1 and 4 are plotted in Figure 5-18, from which it can be observed that they are in good agreement in spite of the experimental uncertainties and noise. Similar results were observed for other strain gages (nodes), indicating that the numerical and experimental strain responses were essentially equal. With such validation of the numerical simulation, the acceleration responses at the strain gage locations were extracted from the FE model and used in the simulated pulse communication protocol for generating energy-dependent timed-delayed binary signals corresponding to the experiments. 124 5.3.2 Reconstruction of Time-Delayed Signals with a Probabilistic Approach Binary signals at the sensor nodes were created based on a threshold concept. A simple pilot-type local rule for binary signal generation was defined in terms of a strain threshold R1, namely 120 micro-strains, at the sensor nodes (i.e., strain gages locations). Consequently, a binary signal was generated if the value of the longitudinal strain at the strain gage location exceeded threshold R1. Preliminary results indicated that the layout shown in Figure 3-5(a) provided adequate information for robust data analysis. Thus, the number of sensors for the cantilever plate was set at 9, and therefore each pattern was represented with 9 features (binary values). Figure 5-19. The PDF plot of delivery delay for sensors/strain gages 1 to 6 for experimental plate: (a) and (c) intact plate, (b) and (d) damaged plate Preliminary results show that the variation of signal delay with respect to the number of event readings has a Gaussian distribution. Thus, the probability density functions (PDF) of delivery delay values at the sensor nodes were determined. The PDF plots of delivery delay for strain gages 1 to 6 obtained from experiments of the intact plate are shown in Figure 5-19(a) and (c); while 125 those for the damaged plate with a hole diameter of 19 mm are presented in Figure 5-19(b) and (d). The variations in the PDF plots of the noted sensing nodes/strain gages confirm that the distribution of event delivery delay values is Gaussian. Given the Gaussian distribution of signal delay, the PDF of each sensor node was determined. Thereafter, it was decided to develop a statistical approach using a conditional probability concept. That is, the probability that sensor node Si observes binary signal (1) at time tj should be combined with the anticipated delay function di, as shown in equation (5-9). ܲ൫ݐ௝ ௜ = 1ห݀௜൯ ሺ5 − 9ሻ In equation (5-9), ݀௜~ܰሺߤ௜, ߪ௜ ଶሻ, where ߤ௜ and ߪ௜ are average and standard deviation of signal delay for sensor node Si. The key idea behind the proposed probabilistic approach is to use the PDF values of the sensor nodes to determine the probabilities that can be further used for classification. Assuming that a binary signal is received at time tj, the aim is to determine what would be the probability that such signal comes from tj-1, tj-2, etc. For such purpose, a time lag factor parameter (l) is introduced to designate the time (i.e., number of readings) that we should go behind the current time tj to capture a delay. Figure 5-20 schematically illustrates the proposed probabilistic approach. The figure illustrates a situation in which the binary signal is observed by sensor node 4 at time t = 0.7 s. The goal is to determine the probabilities that such binary signal was actually generated at times t = 0.6, 0.5, 0.4, 0.3, 0.2, and 0.1 s, given the time lag is 0.6 s. The time lag of 0.6 s refers to the six previous readings, indicating the probabilities that the signal comes from one of the six previous readings need to be determined. The noted probabilities are written in equation (5-10). Further, Pi,tj in Figure 5-20 refers to the probabilities obtained from the PDF of sensor node i at time step tj. 126 ܲሺܸସ = 1|݆ = 0.1ሻ ≈ ܲሺ݀ସ = 0.6ሻ ܲሺܸସ = 1|݆ = 0.2ሻ ≈ ܲሺ݀ସ = 0.5ሻ ܲሺܸସ = 1|݆ = 0.3ሻ ≈ ܲሺ݀ସ = 0.4ሻ ܲሺܸସ = 1|݆ = 0.4ሻ ≈ ܲሺ݀ସ = 0.3ሻ ሺ5 − 10ሻ ܲሺܸସ = 1|݆ = 0.5ሻ ≈ ܲሺ݀ସ = 0.2ሻ ܲሺܸସ = 1|݆ = 0.6ሻ ≈ ܲሺ݀ସ = 0.1ሻ Figure 5-20. Schematic of a proposed probabilistic approach for delayed signal reconstruction 127 The implementation and different steps of the proposed algorithm are presented in Figure 5-21. Once the time delay is reconstructed using the proposed probabilistic algorithm, SVM is employed to detect damage and to classify the reconstructed signals. It is noted that the classification with SVM is based on probability values and not binary signals. Figure 5-21. Implementation of the proposed algorithm for signal reconstruction 5.3.3 Simulation Results To examine the effectiveness of the SHM method employing a probabilistic approach on a realistic structure, experimental validation was conducted using event-based time-delayed binary signals. 128 The experimental dataset for the analysis was randomly divided into three subsets: training, validation, and testing. Further, k-fold cross validation was used to prevent overfitting problems. In this research, 10-fold cross-validation (assuming k = 10) was used. Vibration tests were conducted for 10 s with a time step of 0.01; therefore, the size of dataset was 1000. To implement the SVM algorithm, the dataset (patterns) was classified into 7 classes. Classes 4 to 6 represented patterns due to normal condition of the plate, classes 1 and 3 denoted patterns due to damage, and class 2 represented noisy and time-delayed patterns. Matlab [140] was utilized for implementing the method and algorithms. The damage detection accuracy with SVM was determined according to equation (5-1). Detailed information regarding SVM was presented in Section 2.4.1. Results are presented in terms of different measures, namely: confusion matrices, the receiver operating characteristic (ROC) curve, and the cumulative match characteristic (CMC) curve. 5.3.3.1 Results of PR Approach As noted, an image-based PR was used to recognize patterns representing different conditions of the experimental plate. The identified patterns are presented in Figure 5-22, where patterns 1 and 2 denote normal conditions, pattern 3 was due to noise and time delay, and patterns 4 to 6 were recognized due to damage (i.e., damaged plate with varying hole diameters). The blue and grey regions in the figure denote active and inactive sensor nodes, respectively, whereas red regions represent the binary signals generated due to damage (hole) in a cantilever plate. 129 Figure 5-22. Identified patterns on experimental plate using image-based PR approach 5.3.3.2 Classification Results with SVM It is of importance to select the optimal hyper-parameters of the SVM algorithm. On this basis, grid-search on C and γ (i.e., kernel parameters) was conducted in this study using 10-fold cross- validation. Several pairs of (C, γ) were thus used, and the pair with best cross-validation accuracy was selected as optimal hyper-parameters. The training and validation data sets were used to optimize the regularization and kernel parameters. Different kernel functions, i.e., polynomial, radial basis function (RBF), and sigmoid, were utilized for implementing the SVM method. However, as shown in Figure 5-23(a), performance with the polynomial kernel was found to be superior compared to other kernels; where the maximum accuracy (polynomial kernel) was 94.62% and 94.56% for the validation and training data set, respectively. The optimal degree of the polynomial kernel function was determined through the optimization process, with the best degree value found being 2 (see Figure 5-23(b)). Once the polynomial kernel was selected as the optimal kernel, the grid-search with 10-fold cross validation for different pairs of (C, γ) was conducted to determine the optimum kernel parameters. Consequently, the noted hyper-parameters were tuned through the optimization 130 process. Results from the grid-search in terms of the variation of the damage detection accuracy as a function of the kernel parameters C and γ are presented in Figure 5-23(c) and (d). As can be seen, the best damage detection accuracy was achieved for a polynomial kernel with d =2, C= 10, and γ = 0.02. The noted values were thus selected and used as the optimal SVM hyper-parameters. Figure 5-23. (a) Optimization of the kernel parameter C, (b) Optimization of the kernel parameter γ, (c) Selection of optimal kernel function, and (d) Tuning optimal kernel degree As previously noted, the dataset was randomly divided into three subsets. In addition, the effectiveness of the approach with respect to size of the subsets was also explored. In this context, the dataset was divided to four different cases as follows: • Case 1: training & 10-fold cross validation (70%), test (30%) • Case 2: training & 10-fold cross validation (75%), test (25%) • Case 3: training & 10-fold cross validation (80%), test (20%) • Case 4: training & 10-fold cross validation (85%), test (15%) 131 Confusion matrices for the actual/target and predicted/output classification results using SVM were determined for different data subset sizes and shown in Figure 5-24. Results reveal that SVM is able to identify damage with good classification accuracy. Nonetheless, the best classification accuracy was achieved based for Case 2 (97.6%), where the error for damage class 1 was 8.2%. Figure 5-24. Confusion matrix using SVM for different size of data subsets The effect of time lag parameter on the performance of the SHM method was also assessed. As previously noted, the size of data set was 1000. Therefore, time lags of 3, 4, and 5 s imply that 132 300, 400, and 500 readings, respectively, were considered. Results of damage detection accuracy with different time lags based on noted cases are presented in Table 5-1, from which it can be observed that by damage detection accuracy increases by increasing the number of samples in the test set (e.g., from 92.63% to 96.24% for cases 1 to 4 and a time lag of 5 s.) The best classification result was for the time lag of 4 s, which was 97.63% for case 3.In other words, the highest classification accuracy was achieved when, after a sensor received a binary signal, the probabilities of the previous 400 readings are computed and used to reconstruct the signals. Table 5-1. Damage detection accuracy for different time lags Time lag (sec) Data pre-processing Damage detection accuracy (%) Training Ten-fold CV Test 3 4 5 Case 1 Case 2 Case 3 Case 4 Case 1 Case 2 Case 3 Case 4 Case 1 Case 2 Case 3 Case 4 71.64 73.74 75.18 75.4 94.56 94.89 95.23 95.51 92.40 92.24 92.36 92.33 69.93 72.15 73.58 74.20 94.62 94.74 94.58 95.17 92.30 91.97 91.73 91.64 78.19 79.96 83.09 85.86 95.41 96.40 97.63 97.30 92.63 93.14 94.02 96.24 To further explore the effectiveness of the SVM algorithm for the damage detection model, the receiver operating characteristic (ROC) [149,150] was determined and plotted. Confusion matrices are commonly used to describe the performance of a classifier on a set of data for which the actual values are known. Given such definition, a typical confusion matrix is shown in Table 5-2, where it defines true positives (TP), true negatives (TN), positives (P), and negatives (N). Sensitivity and specificity can then be defined according to equation (5-11): ܵ݁݊ݏ݅ݐ݅ݒ݅ݐݕ = ்௉ ௉ ܽ݊݀ ܵ݌݂݁ܿ݅݅ܿ݅ݐݕ = ்ே ே (5-11) 133 Table 5-2. Typical confusion matrix data structure used for definition of an ROC curve Actual/True Positive Negative Positive True positives (TP) False positives (FP) Prediction Negative False negatives (FN) True negatives (TN) Total Positives (P) Negatives (N) The area under the ROC curve (AUC) is a metric used to assess a classifier’s performance; such that an AUC value close to 1 indicates better classification performance. ROC curves for all cases (1 to 4) and damaged classes are shown in Figure 5-25 for a time lag of 4 s. Although good damage classification accuracy was obtained for all cases, case 3 (training & validation (80%), test (20%)) had the best performance, as its UAC value was slightly higher than the other cases. In summary, classification results, i.e., confusion matrices and ROC curves, indicate good performance of the proposed SHM methodology, even with time-delayed data. Figure 5-25. (a) ROC curves based on SVM classification algorithm for different size of data subsets, (b) a close-up view of the ROC curves 134 The cumulative match characteristic (CMC) curve [151], a rank-based metric, was computed. To determine a CMC curve, each test data is compared to each class, and the resulting scores are ranked. The rank at which a true match occurs is determined, while the probability of observing the correct classification within top ranks is computed. Plotting the noted probabilities against ranks yields a CMC curve that represents the accuracy of the SVM classifier with different ranks. To this aim, multinomial logistic regression was utilized to compute the predicted probabilities for the model. CMC curves for different size of data subsets and damaged classes are presented in Figure 5-26. Results indicate that the accuracy for the first rank for all of the cases is good, further confirming the satisfactory performance of the damage identification approach. Figure 5-26. CMC curve with SVM for different size of data subsets 135 5.4 Summary This chapter presented the results of two probabilistic-based algorithmic frameworks for energy- efficient SHM and damage detection with time-delayed and incomplete binary signals in plate-like structures. First, a framework that integrates an image-based PR approach, a spatial-temporal conditional probability chain, and k-NN algorithm is proposed for interpretation of time-delayed binary data. The presented conditional probability chain, along with a reliability factor concept, helps recognize the patterns representing actual damage even if data from one sensor, or group of sensors, is not received due to time delay. The damage detection paradigm is applied to simulated scenarios through the FE method of a simply supported aluminum plate under transverse pressure. The following conclusions were reached from the first part of this chapter: 1) Results from the numerical studies indicate that the proposed PR framework has the potential to provide satisfactory performance for damage identification in plate-like structures. 2) An important finding from the presented strategy is that the mean deviation of correct prediction probability (PC – PNC) and the frequency of patterns can be potentially used as damage indicators for the proposed damage detection model. 3) The proposed reliability factor concept represents a unique characteristic of damage detection through the model. It was shown that abnormal patterns with a high reliability factor have a strong relation with damage progression. Furthermore, damage location was identified using a proposed focal plane concept. 4) The accuracy of the proposed strategy for different damage scenarios was examined using the k-NN method, which revealed that damage detection performance remained satisfactory even with increased damage severity. In addition, based on an uncertainty 136 analysis, the performance of the damage detection strategy was shown to be good even with data noise up to 20%. Results indicated the applicability of the proposed conditional probability chain for predicting delayed signal. However, it was concluded that the proposed damage detection with a conditional probability chain is computationally demanding. To tackle the noted issue, the second part of this chapter showcases the results of a probabilistic-based algorithmic framework based on integration of an image-based PR approach, a novel probabilistic approach, and SVM algorithm. A proposed probabilistic approach allows reconstruction of the time-delayed signals through probabilistic measures, and SVM is used for damage classification with the reconstructed signals. Experimental vibration tests were conducted to explore and validate the applicability of the approach. The following conclusions were reached: 5) The distribution of the delayed signals is Gaussian. Further, results confirmed that the proposed probabilistic approach can be successfully used to reconstruct delayed signals. 6) The time lag parameter has an effect on the performance of the approach. However, such effect was found to be minimum for the set up presented in this study. 7) Results of classification, confusion matrices, ROC, and CMC curves indicated that SVM led to acceptable damage detection accuracy with noisy and delayed signals. 8) Damage classification accuracy on the test set increased with an increase in the number of data in the set. The methods and results presented in this chapter demonstrate that the self-powered SHM methodology employing developed probability approach can be effectively used for damage identification with time-delayed signals, while the approach is computationally efficient. 137 CHAPTER 6 6 MACHINE LEARNING-BASED DATA INTERPRETATION FRAMEWORKS FOR ENERGY-LEAN SHM USING INCOMPLETE/NOISY AND SPARSE SIGNALS 6.1 Overview This chapter presents machine learning (ML)-based data mining frameworks for damage identification of missing and delayed signals. On this basis, a novel ML framework integrating low-rank matrix completion, pattern recognition (PR), k-nearest neighbor, and a data fusion model is developed for data-driven SHM. Performance and accuracy of the proposed data-driven SHM strategy is investigated and tested for an aircraft horizontal stabilizer wing. The reliability of the proposed strategy with noise-polluted data is also validated. The effect of variations in harvested energy on the performance of the approach was also investigated. Further, an ML algorithmic framework incorporating low-rank matrix completion, a data fusion model, and a statistical approach is developed for damage identification. Performance and effectiveness of the proposed method for the case of dynamically loaded plates is evaluated using finite element simulations and experimental vibration tests. The material presented in this chapter was partially included in published works as an outcome of this research [152,153]. 6.2 Machine Learning Framework Employing Matrix Completion, Pattern Recognition, and K-Nearest Neighbor The numerous studies carried out in the SHM domain can be categorized as model-driven and data-driven approaches [154,155]. In a data-driven approach, the model is created through the learning gained from measured/sensed data. Damage can thus be detected by conducting a comparison among the measured data and a model. In a data-driven SHM approach the sensor- 138 measured events can be represented as a matrix. The measurements possess a low-rank structure and a sparse nature, which can be processed by mathematical tools such as sparse representation and low-rank matrix completion. Further, the inherent low-dimensional data structure of the multi- sensor data matrix can be utilized and modeled by techniques such as singular value decomposition [156]. As noted in Chapters 3 and 4 it is assumed that the analysis of discrete binary data over the two-dimensional domain of a monitored structure resembles a PR problem because of the pixelated nature of the data collected at the sink. It is further postulated that a compressed representation of the time-delayed data can be obtained by uncovering the latent structures existing in the matrix of time-delayed binary data received at the sink, thus providing a unified method for matrix completion. Therefore, the research hypothesis of the work presented in this chapter is to develop an ML framework for SHM that integrates matrix completion, PR, and a data fusion model. The proposed framework, presented in Figure 6-1, consists of three phases: data fusion, learning, and classification. The data fusion phase involves constructing a mask matrix and modified time-delayed binary matrix using a proposed data fusion model based on response correlation between sensor nodes. The modified time-delayed binary matrix constructed based on the data fusion model is introduced as input to the learning phase, in which low-rank matrix completion is used to recover/impute missing and noisy features of the modified time-delayed matrix. For the classification phase, k-NN, employing a Bayesian optimization algorithm, is used to classify the recovered time-delayed binary data and for damage detection. It is to be noted that true labels of the patterns (i.e., normal, noisy, damaged) are identified based on an image-based PR approach with full binary data, and used for validation. The proposed ML framework was implemented thru a custom program in MATLAB [157]. Detailed theory of the image-based PR 139 approach is presented in Section 3.2, while information regarding the phases of the proposed framework are provided in the following sub-sections. Figure 6-1. Proposed machine learning framework for energy-lean SHM 6.2.1 Finite Element Modeling of an Aircraft Stabilizer Wing Spatial-temporal acceleration data for the through-substrate self-powered sensor network simulation was obtained through a finite element (FE) analysis of the dynamic response of an aircraft horizontal stabilizer wing. The FE model was based on the geometry of the Boeing 737 horizontal stabilizer and the analysis was performed using the program ABAQUS [158]. The model’s geometry was simplified so that it represented the main structural components for the wing element to yield a realistic dynamic response. Shell elements with uniform thickness were used for modeling all parts. Additional stiffness from stringers was modeled by placing spars (beam elements) on the top edges of the leading and back stiffeners. Thickness for the shell elements in the box structure and stiffeners was 5 mm. This thickness value was determined to obtain realistic dynamic properties due to additional mass from non-structural components in the 140 wing. The spars were assumed to have circular cross sections of 20 mm in diameter. Material properties assigned to the model were those of aluminum 2024, with an elastic modulus of 73.1 GPa, a Poisson ratio of 0.33 and density of 2780 kg/m3. A schematic of an integrated self-powered sensing system embedded in aircraft stabilizer wing is presented in Figure 6-2(a). A view of the FE model is shown in Figure 6-2(b) and a sample distribution of transverse accelerations at the top surface nodes of the wing are shown in Figure 6-2(c). The acceleration response was used to establish the power budget for the simulation of the data communication protocol. Figure 6-2. (a) Schematic of an aircraft stabilizer wing with integrated self-powered sensing system, (b) FE model of horizontal stabilizer wing, (c) Transverse acceleration distribution from FE analysis (4000th sec) The FE dynamic simulation was performed by loading the wing under an aerodynamic loading pressure, which was estimated using FoilSim III [159] for a speed of 400 km/s and extrapolated to 800 km/s. The lift pressure distribution across the wing chord length is illustrated in Figure 6-3(a). Aerodynamic load was modeled as a harmonic perturbation of the total lift load, as schematically shown in Figure 6-3(b). The loading pressure was applied to the model as an incremental ramp with a noise perturbation of 10% over a time domain of 4000 s. The simulation neglected the rigid- body flight dynamics of the plane and thus it modeled only the relative response of the stabilizer wing. The dynamic analysis was conducted with an implicit solver. 141 Figure 6-3. (a) Total lift pressure profile across wing chord length (at aircraft speed 800 km/h), (b) Approximation of lift aerodynamic loading on aircraft horizontal stabilizer wing A simple pilot-type local rule for binary event generation was defined in terms of a strain threshold (R1) at the sensor nodes (i.e., corresponding FE model mesh nodes). In this context, a binary event was generated if the maximum principal strain at the sensor node was larger than R1. The threshold R1 (80 micro-strains) was empirically chosen, based on FE simulations, to ensure that binary events representing damage could be generated. The geometry of the aircraft stabilizer, layout, and density of the self-powered sensor nodes in the FE model is shown in Figure 6-4(a). Further, a typical representation of sensor node values as an image matrix (pattern) using the image-based PR approach is shown in Figure 6-4(b). In this study, the image-based PR approach presented in [135] was used as part of the ML framework to recognize patterns representing different conditions of the stabilizer with full data, providing valuable information for the damage detection model. A study was conducted to investigate the effect of sensor density on the proposed strategy in order to determine an adequate number of sensors for the SHM system. Layouts with 40, 80, 160, and 240 sensors were considered, and the effectiveness and accuracy of the damage detection approach, in terms of identifying damage presence and location, was evaluated. Results revealed 142 that the layout shown in Figure 6-4 provided sufficient information for calibrating the machine learning algorithm. Therefore, the number of self-powered sensors was set as 160. Consequently, each pattern (image) was represented by 160 features (binary values). To validate the damage detection strategy, damage was considered in the FE simulations by gradually reducing the elastic modulus of selected elements on the stabilizer’s skin over the analysis time, see Figure 6-4 (a). Damage scenarios were defined as ‘classes’ and numbered from 1 to 4 for stiffness reductions of 20% to 80% (in 20% increments), respectively. Figure 6-4. (a) Aircraft stabilizer geometry and sensor configuration, (b) Representation of sensor node values as a pattern (image matrix) 6.2.2 Performance Evaluation using Different Harvested Energy Levels The proposed platform for energy-lean structural health monitoring (see Figure 2-1) involves a network of low-power through-substrate ultrasonic pulse networking (TUPN) units communicating with each other through the structural substrate, and powered by energy harvested from the substrate vibrations. Harvesting energy from structural vibrations can be erratic over time 143 due to the transient nature of structural vibration intensity. In addition, the electromechanical conversion efficiency of the used harvesting modules, e.g., piezoelectric harvester, is also important in determining how much energy can be harvested from the ambient vibrations to power the sensing/communicating TUPN units. In the case of piezoelectric harvesters, the energy conversion efficiency is primarily a function of an inherent property of the modules called the electromechanical coupling constant (ECC). Both erratic vibrations and conversion efficiency can lead to variability in the energy harvesting potential. Such variability in the harvesting capacity can manifest in the monitoring network performance in terms of variable information delivery delay from the same source node to sink in different situations. In order to demonstrate the robustness of the proposed damage identification strategy against such variations in the delivery delay performance of the network, the damage detection performance in a diverse range of energy availability situations was evaluated. The inconsistency in energy availability was simulated by using different ECC values for the piezos on the associated TUPN units in separate tests*. Accordingly, four different electromechanical coupling constant values, indicated by 0.55Θ, 0.75Θ, 0.9Θ, 1.0Θ, were considered to create a variable harvesting potential. The value Θ indicates a coupling constant of 1.96x10-4 Coulomb/m, which is a reasonably high value among piezo transducers. Further, different fractions of the same (0.55, 0.75, 0.9, etc.) were used to simulate the variability in harvesting conditions from low to high. To evaluate the effect of the available harvested energy on the performance of the damage detection model, an attempt was made to determine average delivery delays for all the sensing nodes. Average delivery delay values of the sensing nodes for ECC values of 0.55Θ and 1.0Θ (i.e., lowest and highest harvested energy levels) for healthy and damaged stabilizers were computed, * The simulation was done by Dr. Biswas’s group as part of a collaborative research work. 144 and the surface plot of these values are shown in Figure 6-5. As can be seen, for the undamaged stabilizer (Figure 6-5(a) and (b)), the average delivery delay of the sensor nodes near to the damaged region is zero and only sensor nodes close to the root of the stabilizer show delivery delay variations, which is because of the binary event generation in this area. For the damaged stabilizer (Figure 6-5(c) and (d)), unlike the healthy stabilizer, sensor nodes close to the damage region also have delivery delay given the binary events are generated due to damage. The delivery delay values are not affected by different values of ECC, confirming that the variation in harvested energy does not have a notable effect on the delivery values and damage detection accuracy. An interesting point regarding these results is that the average delivery delay values of sensor nodes in the damage region is much lower than that of sensor nodes near to the wing root. This is because such sensor nodes are very close to the network sink, thus decreasing the delivery delay values. Figure 6-5. Average delivery delay response of sensor nodes for different harvested energy levels: (a) & (b) intact stabilizer, (c) & (d) damaged stabilizer 145 6.2.3 Data Fusion for Damage Identification Model According to the pulse switching architecture, the extractable information at the sink from the observed matrix, i.e., time-delayed binary data matrix X, are zeros and ones. Zero values indicate that either the real value of the sensor node (before delay) is zero, or that the value of that sensor node is one but due to time delay it is still coded as a zero. Values equal to one indicate that the real value of the sensor node before delay might be either zero or one. Interpretation of these missing and incomplete binary values without having any prior knowledge (i.e., full binary data matrix) is impossible. To tackle these challenges, one approach is to distinguish between real zero values and unknown/missed values due to delay. To this aim, a data fusion model is proposed to construct the mask matrix M and modified time-delayed binary matrix MX (modified time-delayed binary data). In a wireless sensor network, sensors are installed on a substrate and their output is affected by the host structure’s behavior. Consequently, there is a correlation between each sensor node and its neighbors. Therefore, it was intuitively inferred that spatial-temporal integration of sensor nodes can be used in the context of a data fusion model. It is noted that the spatial term refers to the neighboring effect and the temporal term represents the time aspect. A sensor fusion process integrates and extracts useful information from two or more sensors. In this context, it was decided to fuse the information given by each sensor and its neighbors to construct the matrices M and MX. To elaborate on the proposed data fusion model using the neighboring concept (i.e., spatial aspect), consider a simple square plate with 16 sensor nodes (see Figure 6-6) for which the sensor arrangement is considered as an image matrix/pattern. The sensor configuration and the neighboring nodes for sensors 1, 7, and 15 are presented in Figure 6-6 (the sensor’s neighbors are identified with red dotted lead lines, while the blue dotted lead lines denote the sensor itself). The 146 temporal aspect of the data fusion model along with the spatial term through the layout of self- powered sensors and simulation time are schematically illustrated in Figure 6-7. Specifically, this figure illustrates how the arrangement of sensors at time steps t = 5 s and t = 53 s can be represented as an image matrix using the spatial-temporal concept. Figure 6-6. Schematic illustration of neighboring/spatial concept of the data fusion model Figure 6-7. Schematic illustration of spatial-temporal aspect of data fusion model 147 Once the time-delayed matrix X is received at the sink, the first step of the data fusion phase is to construct mask matrix M with dimension m x n (m denotes a time step and n represents the number of sensors in the network) using the spatial-temporal data fusion model. The time difference between the time step (tj) at which the value of each individual sensor node Si is 1, and the latest time step of its neighbors (before tj) when their readings were 1 is determined. This rationale follows the spatial-temporal data fusion model: if the value of a sensor Si is 1, it is likely that the values of the sensor’s neighbors are 1 within a given time window. Once the noted time steps for each sensing node are identified, the, minimum, average, median, and maximum values for the time steps at which the neighbors of sensor Si reported a value of 1, given that the value of sensor Si itself is 1, are computed and used to determine the values V1, V2, V3, and V4. The values of V3 and V4 are rounded up to the nearest integer and used to compute the mask matrices M3 and M4. These values are then used to compute four mask matrices (M1, M2, M3, and M4) based on fused information. The proposed algorithm to construct the mask matrix is presented in Figure 6-8. The noted four mask matrices are further utilized to compute the accuracy of the imputation algorithm. Consequently, the optimal mask matrix is the one with highest imputation accuracy. To better visualize the noted algorithm, Figure 6-9 schematically illustrates its implementation for construction of the mask matrix based on the proposed data fusion model. The figure shows an example for which the mask matrix M3 is determined using the average values (V3) of the sensor’s neighbors for the time interval of 70 – 86 s. 148 Figure 6-8. Implementation of the proposed algorithm for constructing the mask matrix 149 Figure 6-9. Schematic illustration of the proposed algorithm for constructing the mask matrix M3 (average) 150 The modified time-delayed binary matrices (MX1, MX2, MX3, and MX4) are constructed using the corresponding mask matrices (obtained with the algorithm presented in Figure 6-8). This is done using the algorithm presented in Figure 6-10, in which a time-delayed binary matrix X observed at the sink is compared with the constructed mask matrices; thus allowing to distinguish between real zero values and unknown or missed values due to time delay. Real zero values are replaced with a small number, while missing values are represented as zero values. The resulting modified time-delayed matrix consisting of modified time-delayed binary data is the output of the data fusion phase (see Figure 6-1). The matrix is then introduced as input to the learning phase employing low-rank matrix completion to recover the missing values, reconstruct the time-delayed binary data, and to report the imputation accuracy. Among the computed matrices, the optimal mask matrix and modified time-delayed matrix are the ones with the highest imputation accuracy. Figure 6-10. Implementation of the proposed algorithm for constructing the modified time- delayed matrix 151 6.2.4 Learning with Low-Rank Matrix Completion In the present study, it is hypothesized that the values at sensor nodes that are not received at the sink due to time delay can be treated as missing values (zero values). Therefore, the problem of recovering and interpreting time-delayed binary data was expressed within the context of a matrix completion technique. The task of matrix completion is to complete the observed matrix using the values of an estimated matrix. Herein, the full-data binary matrix (i.e., binary data without delay) is referred to as the original matrix and the observed time-delayed binary matrix (i.e., time-delayed binary data) is denoted as the observed matrix. Assuming that a time-delayed binary matrix, i.e., observed matrix, has a low-rank property given a correlation between binary events, low-rank matrix completion was used to recover the matrix components. Missing values from the observed matrix X were thus recovered with the estimated time-delayed matrix Z using an imputation algorithm to determine an imputed matrix I as the output of the matrix completion process (i.e., learning phase). A conceptual illustration of the matrix completion approach for an observed time- delayed binary matrix (matrix X) is presented in Figure 6-11, which shows that the imputation accuracy for this example is 84%. Time-delayed discrete binary signals generated from a through-substrate sensor network are represented as a matrix nmX × , for which only a relatively small number of entries are observed. The aim is to fill the missing entries of the partially observed matrix through low-rank matrix completion, which can be phrased as learning an unknown parameter (a matrix nmZ × ) with very high dimensionality based on limited observations. It is thus assumed that the parameters of the matrix Z lie in a much lower dimensional manifold (matrix of low rank). On this basis, once the modified time-delayed binary matrix MX is constructed using a data fusion model, the matrix is 152 used as input to the low-rank matrix completion procedure employing an imputation algorithm to recover the time-delayed binary data matrix I. Figure 6-11. Schematic illustration of low-rank matrix completion technique For an observed (i.e., time-delayed) matrix nmX × , ⊂Ω { ,...,1 } { × ,...,1 }n m denotes the indices of observed entries. The following optimization problem is defined by finding nmZ × of (small) rank r such that the training error is small: min z ∑ ( Ω∈ ),( j i X ij − Z ij 2) ≤ δ subject to rank Z =) ( r (6-1) where 0≥δ is a regularization parameter controlling the tolerance in training error. The rank (Z) constraint makes the problem non-convex. Thus, equation 6-1 is modified to make the problem convex as: min z ∑ ( Ω∈ ),( j i X ij − Z ij 2) ≤ δ subject to ∗Z (6-2) 153 where ∗Z is the nuclear norm (trace norm), or sum of the singular values of Z. It can be concluded that ∗Z is the tightest convex relation of rank (Z). Equation (6-3) expresses the Lagrangian form of the optimization problem. min z 1 2 ∑ ( Ω∈ ),( ji X − Z ij 2) + ij λ (6-3) Z ∗ where 0≥λ is a regularization parameter that controls the nuclear norm of the minimizer λ ∧ Z . A soft-impute algorithm [156,160–162] was adapted and modified in this research as an imputation algorithm for the low-rank matrix completion task. This algorithm is computationally efficient, resulting in significant simulation time savings. The algorithm is inspired by the SVD- impute (singular value decomposition-impute) method; alternating between imputing the missing values from a current SVD and thus updating the SVD using the complete data matrix. Soft-impute for the nuclear norm regularized least squares optimization problem expressed in equation (6-3) can be scaled to large problems with m, n ≈ 105 to 106. At every iteration the value of the objective function decreases towards its minimum while it simultaneously gets closer to a set of optimal solutions for the optimization problem given in equation (6-2). The soft-impute algorithm is capable of handling matrices of large dimensions because it demands SVD computation at every iteration. It is noted that because the output of low-rank matrix completion algorithm is continuous values, thresholds were employed and tuned for the convex optimization algorithm to generate the reconstructed/recovered binary signals. Singular value decomposition (SVD) [163] is a powerful technique that has been used for many applications. Using the SVD of a matrix instead of the original matrix results in being more robust to numerical errors. The main properties of SVD include the relation of it to the rank of a 154 matrix and its optimal low-rank approximation of a matrix. In this study, the SVD method was adapted for the matrix completion task. The SVD of the data matrix ×∈ nmRX with m rows and n columns (m < n) in the defined rectangular grid containing m × n measurement positions can be represented using the following equation: X = T UDV = r ∑ = 1 i T vud kkk (6-4) where = uuU 2 [ , 1 ,..., u m ] ×∈ R mm is the left singular vector, which is an orthonormal matrix associated with the row dimension, while V = vv ,[ 1 2 v ,..., n ] ×∈ R nn denotes the right singular vector correlated with the column dimension. Further, = dD 1 [ >> ... d r > d r + 1 == ... d m ∈= ]0 × nm R contains m diagonal elements as the kth singular value. The SVD of a matrix X is the closest rank-r matrix to X. The matrix X is then assumed to be low-rank if it has only few nonzero singular values. Figure 6-12 schematically presents a representation of the SVD of a matrix, where the SVD for ai is a linear combination of the eigenvectors {vk} and the SVD for bj is determined using a linear combination of eigenvectors {uk}, while dk denotes the singular value. Figure 6-12. Graphical representation of SVD of a matrix X According to soft-impute algorithm employing SVD, at each iteration the structure of the non- sparse matrix is expressed as follows (see Figure 6-13): 155 Y = Y SP + Y LR (6-5) where SPY (sparse matrix) has the same sparsity structure as the observed matrix X , while the rank of LRY (low-rank matrix) is ~ ≤ nmr , . It is to be noted that upon convergence of the algorithm, r~is close to r (the rank of the estimated matrix Z). For the soft-impute algorithm, the matrix PΩ ( X ) with dimension m × n is a projection of the observed matrix nmX × onto the observed entries and can be written as in equation (6-6). Figure 6-13. Illustration of the structure of observed delayed matrix in terms of low-rank and sparse matrices j ) = X ij 0      if ,( i j ) Ω∈ if ,( i j ) Ω∉ (6-6) ,)( iXP ( Ω The complementary projection can also be expressed as (XP⊥ Ω ) through Therefore, the criterion can be rewritten as: ⊥ XPXP Ω + Ω ( ) ( = ) X . ∑ ( Ω∈ ,( i j ) X ij − Z ij 2 ) = XP ( Ω − ) ZP ( Ω ) 2 F (6-7) Accordingly, the optimization problem shown in equation (6-2) is rephrased as min z ∑ ,( i j ) Ω∈ XP ( Ω − ) ZP ( Ω ) 2 F ≤ δ subject to ∗Z (6-8) 156 The solution to the convex optimization problem presented in equation (6-9) is given by ˆ XSZ λ= ( ) min z 1 2 ZX − λ2 + F (6-9) Z ∗ where S λ ( X ) refers to the soft-threshold SVD and is expressed according to following equation: XS ( λ ) = Udiag λσ [( ) − 1 ++ ... λσ ( − m ]) + V T (6-10) Finally, the solution to the missing data problem in Lagrange form presented in equation (6- 11) is computed using the soft-impute algorithm. min z Zf ( λ ) = 1 2 XP ( Ω − ) ZP ( Ω ) 2 F + λ Z ∗ (6-11) The soft-impute algorithm repeatedly replaces the missing values with the current guess, and updates the guess by solving equation (6-9). The steps of the soft-impute algorithm are summarized in Figure 6-14. Figure 6-14. Implementation of the soft-impute algorithm 157 6.2.5 Classification with K-Nearest Neighbor Algorithm A supervised PR approach was used as final phase of the ML framework for classification using the imputed matrix I. In this chapter, a k-nearest neighbor (k-NN) algorithm was used to classify the recovered time-delayed data with noisy features. The benefit of this approach is that the full- data matrix is not used for calibrating/training the classifier, which is consistent with real conditions, where the full-data matrix is not available. However, the true labels of the patterns (i.e., normal, noisy, damaged) identified from the data pre-processing phase (image-based PR approach with full binary data) should be used to evaluate performance of the k-NN classifier with recovered time-delayed data for the evaluation and test steps. Detailed information regarding the k-NN method, including introduction of the method, implementation, and advantages of such method was presented in a prior study [164]. The dataset for the k-NN analysis was randomly divided into three subsets; namely, calibration (training,) validation, and testing. The calibration set was used to fit the classifier, while the validation set was used to compute the optimal k for the k-NN classifier. The best models were selected based on their performance on the validation data. Performance of the classifier with optimal k was then evaluated on the test dataset. A Bayesian optimization algorithm [165] was used to efficiently select the optimum value of k, i.e., number of neighbors, and the optimal distance for the k-NN analysis. The main idea of the Bayesian optimization algorithm is to learn Bayesian networks by means of an optimization method that uses probabilistic models, where the goal is to minimize a scalar objective function. Using the Bayesian optimization algorithm along with the k-NN method allows efficient computation of the optimal distance and a value of k that leads to obtain the best damage classification performance. 158 To implement the k-NN approach, the dataset was clustered into 7 classes. Classes 1 and 2 represented patterns due to normal condition of the stabilizer, classes 3 to 5 denoted noisy patterns, and classes 6 and 7 represented patterns due to damage simulated in the stabilizer. MATLAB [140] was utilized for implementing the k-NN analysis with Bayesian optimization. A classification performance (CP) parameter, see equation (6-12), was defined and used to evaluate the k-NN classification accuracy with noisy features. CP = Number of patterns correctly classified Total number of patterns (6-12) 6.2.6 Damage Identification Results with Machine Learning Framework The damage detection procedure’s development and validation is based on data obtained through the structural response simulation of an aircraft horizontal stabilizer with the finite element method (FEM). Acceleration and maximum principal strain responses at the stabilizer sensor nodes were extracted from the FE model. The strain response was used to generate binary events using a threshold concept, while the acceleration response was used for determining harvested energy. The key idea is that binary events at a local level (i.e., sensor node) are generated, saved in the cell memory and communicated if enough power is available based on accumulated harvested energy (acceleration at sensor nodes). Consequently, acceleration response and binary events were fed to the through-substrate sensor network employing an energy-aware pulse switching protocol to generate the delivery delayed event data. Finally, the time-delayed binary data were introduced as input to the machine learning framework for the damage detection model. Four structural condition scenarios, listed below, were considered for the data processing. • Scenario 1: Intact condition (constant stiffness) with full data availability • Scenario 2: Damaged condition (elements with reduced stiffness) with full data availability 159 • Scenario 3: Intact condition with time-delayed data • Scenario 4: Damaged condition with time-delayed data. To ascertain the reliability of the proposed damage detection methodology to measurement noise, various levels of noise were added to the numerical input vectors. To this aim, the calibrating/training, validation, and test sets from the original self-powered sensor signal were polluted with a random noise. A linear-elastic implicit dynamic analysis was performed for 4,000 seconds with a time step of 0.1; therefore, the size of the dataset was 40,000. The dataset (patterns) were then classified into 7 classes, as noted in Section 3.3. Damage classes were defined in terms of different stiffness reductions (20% to 80% in 20% increments) in a selected region (see Section 6.2.1). Results are presented here for damage class 2, i.e., localized 40% stiffness reduction. 6.2.6.1 Results of Image-based PR Approach The discrete binary data was initially used to represent sensor node responses as an image matrix/pattern, identify the true labels of patterns, and determine the presence of damage using image-based PR approach. To calibrate the image-based PR approach, the stabilizer’s normal response (e.g., due to service load) was determined to identify normal patterns. The method identified four common patterns, where patterns 1 and 2 represented normal condition, while patterns 3 and 4 were due to noise. Once the PR classifier was calibrated using data from the simulation of an intact wing, data from the FE simulation of a damaged wing was introduced to the PR approach for classification. The image-based PR approach recognized five new patterns with respect to the four previously memorized. Accordingly, patterns 5 to 7 were identified due to damage, whereas the other patterns were due to system time delay. 160 Figure 6-15. Pattern classification results in term of identified patterns using image-based PR approach Figure 6-15 illustrates the identified normal, noisy/delayed, and damaged patterns by the image-based PR approach, in which the grey and blue regions corresponds to values of 0 (inactive sensing node) and 1 (active sensing node), respectively. Further, red regions denote the binary 161 events (values of 1) generated due to simulated damage. Results reveal that the identified damaged patterns can be used to successfully detect the presence of damage. Figure 6-16. Pattern focal density of binary events for (a) pattern 1, (b) pattern 2, (c) pattern 5, (d) pattern 6, and (e) pattern 7 To detect the damage location, a potentially more effective spatial system was defined using a neighboring effect concept. A damage index denoting the location of damage was thus defined using a pattern (image) focal density concept based on fused binary data. Such concept provides a focal density measure determined by adding binary events (values of 1) generated at each sensor node and its neighbors. Accumulated binary events are then normalized by the maximum number of neighboring sensors plus the sensor itself to determine a damage index for a sensor. The surface plot for patterns 1 and 2 (stabilizer’s normal condition) are presented in Figure 6-16(a) and Figure 6-16(b), respectively; while the plots for patterns 5, 6, and 7 representing the stabilizer’s damaged 162 condition are shown in Figure 6-16(c) to Figure 6-16(e), respectively. Normal condition of the stabilizer wing is observable from the smooth surface of the computed density in the damage region, i.e., the corresponding DI is zero. By contrast, the surface interpolation of the event-based binary data’s focal density for damaged patterns 5, 6, and 7 indicate that the location of damage can be discerned by damage indices greater than zero. Therefore, the presented DI concept based on image-based PR identified the location of damage via the concentration of event-based binary data. 6.2.6.2 Damage Detection with Time-Delayed/Missing Signals and Varying Harvested Energy Levels The soft-impute algorithm was adapted and modified for matrix completion in order to recover the missing data. The algorithm was found to be computationally inexpensive as the computational time was efficient for each simulation case. Additionally, the effect of tuning parameters, i.e., regularization parameters λ and δ controlling the tolerance in training error and the nuclear norm, was investigated on the algorithm’s accuracy and runtime performance and the best parameter values were selected. Simulation results revealed that there is a unique correlation between λ and the nuclear norm such that the latter is very effective as a rank approximation. In this study soft- impute was run until it reached a tolerance of 10-6. For the simulations, the signal to noise ratio was assumed to be quite high with SNR = 1 and the corresponding results were obtained. As previously discussed, the damage detection strategy was evaluated with respect to variations in harvested energy levels using different electromechanical coupling constant (ECC) values for the piezos mounted on the aircraft horizontal stabilizer. To explore the effect of various harvested energy levels, the number of events received at the network’s sink was computed and plotted for four different ECC values (viz., 0.55Θ, 0.75Θ, 0.9Θ, 1.0Θ) and for the intact and 163 damaged stabilizers, as presented in Figure 6-17. As seen from the results, ECC values did not considerably affect the event delivery delay, as the number of events transmitted through pulse switching architecture, and received at the sink given existing time delay, does not change with an increase/decrease of the harvested energy level. Figure 6-17. Number of events received at the sink for different harvested energy levels: (a), (b), (c), and (d) intact stabilizer, (e), (f), (g), and (h) damaged stabilizer For the intact stabilizer (see Figure 6-17(a) to Figure 6-17(d)), the events received at the sink were originally generated from the area close to the wing root, where the number of events is 164 almost constant for all the ECC values. Nevertheless, for the damaged stabilizer, the events received at the network’s sink were mainly from the damaged region (see Figure 6-17(e) to Figure 6-17(h)). Thus, variations in harvested energy, i.e., different ECC values, did not significantly influence the number of events received at the sink. Therefore, the results indicate that the delivery delay values reported at the sink is robust with respect to variations in harvested energy, and therefore would not affect the damage detection performance. 6.2.6.2.1 Results of Low-Rank Matrix Completion Algorithm A soft-impute algorithm [162] was adapted and modified for the low-rank matrix completion phase. The aim was to recover the missing data due to time delay using the modified time-delayed matrices determined from the data fusion phase. The recovered/imputed matrices were obtained as output from imputation of the corrupt time-delayed data matrix X. Accuracy of the soft-impute algorithm for the intact and damaged stabilizer (decreased localized stiffness) was then computed. Results are presented here for the intact and damaged class 2 for a simulation time of 4000 s. Imputation accuracies were calculated with respect to different mask matrices (see Section 6.2.3) and various harvested energy levels, and results are presented in Figure 6-18. In the given results, M1, M2, M3, and M4, denote the mask matrices constructed based on computing the minimum, average, median, and maximum values for the time steps at which the neighbor of a typical sensor Si reported values of 1, considering that the values of sensor Si itself was 1. Results show that the highest imputation accuracies (among different combinations of mask matrices and ECC values) were obtained using the average mask matrix with ECC = 1.0Θ; that is, 92.16% and 92.34% for the intact and damaged stabilizers, respectively. Given such high imputation accuracies, the effectiveness of the imputation algorithm is considered validated. 165 Figure 6-18. Imputation accuracy for different mask matrix vs harvested energy: (a) intact stabilizer, (b) damaged stabilizer As can be observed from Figure 6-18, for the intact stabilizer the accuracy based on the minimum mask matrix (M1) was 91.36% (i.e., the imputation algorithm successfully recovered 91.36% of the missing values) for ECC = 0.75Θ; which is higher compared to the accuracy obtained based on the other mask matrices. For the damaged stabilizer this observation governed for ECC = 0.90Θ; for which the imputation accuracy for M1was 90.55%, and also higher than that of other mask matrices. Further, the imputation accuracies based on the average mask matrix (M2) for the intact stabilizer were higher in comparison with other mask matrices including the minimum mask matrix. Similarly, the average mask matrix yielded the highest imputation accuracy (see Figure 6-18(b)) for the damaged stabilizer except for the case of ECC = 0.90Θ, where highest accuracy was obtained thru the minimum mask matrix. Although the results confirm the effectiveness of the low-rank matrix completion algorithm with missing data, it is acknowledged that the reported accuracies correspond to the setup considered in this study, e.g., wing geometry, sensor grid, sink location, etc. Yet, it is expected that the imputation accuracy will not be significantly affected with variations in the noted features. 166 6.2.6.2.2 Results based on k-NN Employing Bayesian Optimization Algorithm The Bayesian optimization algorithm was used in this research to determine the optimal distance and the value of k for the k-NN classification method. As highlighted in the previous sub-section, the minimum and average mask matrices (M1 and M2) resulted in the highest imputation accuracies. Therefore, the following optimization and classification results are provided based on the noted matrices. Results from the optimization algorithm are presented in Figure 6-19 where estimated objective function value determined thru optimization algorithm for different distances is plotted with respect to the number of neighbors (i.e., NumNeighbors or k) for ECC values of 0.55*Θ and 1.0*Θ (i.e., least and maximum harvested energy levels). Results show that among the various distances, Spearman and Minkowski distances lead to optimum estimated objective function values. For instance, for the case of ECC = 0.55Θ (see Figure 6-19(a) and (b)), the best estimated objective function values were obtained using the Spearman distance. The noted values were reported as 0.037 and 0.04 for mask matrices M1 and M2, respectively; while the optimal value of k was 5 for both cases. For the case of ECC = 1.0Θ (see Figure 6-19(c) and (d)), the optimum objective functions values were estimated using the Spearman and Minkowski distances for mask matrices M1 and M2, respectively. Specifically, for mask matrix M1, the objective function was estimated as 0.032 with k = 5 using the Spearman distance (see Figure 6-19(c)). On the other hand, for mask matrix M2, the best objective function value was 0.029 with k = 7 using the Minkowski distance (see Figure 6-19(d)). For better visualization of the results of the optimization phase, minimum objective values with respect to the function evaluations based on mask matrices M1 and M2 are plotted in Figure 6-20(a) to Figure 6-20(d). The results represent the variation of the minimum observed objective and estimated minimum for various ECC values, for which the optimum objective function values noted above can be identified. 167 Figure 6-19. Bayesian optimization algorithm results for damage detection model based on mask matrices M1 and M2: (a) & (b) ECC = 0.55Θ, (c) & (d) ECC = 1.0Θ Results from the optimization process provide valuable information regarding the optimal distance and value of k for the k-NN method. However, to attain the best damage classification performance the optimum value of k was further determined by computing the damage detection accuracy with the k-NN algorithm on training, validation and test data. Accordingly, 70% of the data was used for calibration, 15% was used for validation, and 15% was used for testing. Given the total number of damage patterns (18,920) the number of damaged patterns used for testing was 2,838. Further, the damage patterns used for classification and testing were patterns 5, 6, and 7, as shown in Figure 6-15. 168 Figure 6-20. Function evaluations vs. minimum objective values for mask matrices M1 and M2: (a) & (b) ECC = 0.55Θ, (c) & (d) ECC = 1.0Θ Although the minimum and average mask matrices yielded the highest imputation accuracy, the accuracy of the damage identification approach based on the different mask matrices and harvested energy levels was determined. Nonetheless, results are presented here for ECC values of 0.55Θ and 1.0Θ. Figure 6-21 illustrates the damage detection accuracy using the k-NN algorithm for the validation and test datasets. As can be observed from Figure 6-21(a) and (b), the optimum value of k was found to be 5 for all mask matrices, which is in good agreement with results from the optimization phase. For cases with the least harvested energy (0.55Θ,) maximum accuracy was achieved using the minimum mask matrix, i.e., M1. However, for cases with the highest harvested energy level (1.0Θ,) maximum accuracy was obtained with the average mask matrix (M2). According to Figure 6-21(c), the optimum value of k for the minimum and maximum mask matrices (M1 and M4) was 5. In contrast, the optimal k value for the average mask matrix M2 was 7; while it was 10 for the median mask matrix (M3). Similar results were obtained for the test data 169 as shown in Figure 6-21(d). Results confirm the findings from the Bayesian optimization process in terms of the optimal number of k for the k-NN method. Figure 6-21. Damage detection accuracy using K-NN algorithm with different number of k: (a) & (b) ECC = 0.55Θ, (c) & (d): ECC = 1.0Θ To further evaluate the effectiveness of the damage detection model, a confusion matrix containing information about actual and predicted classification for different classes (see Section 5.3.2) was determined for the ML methodology. Table 6-1 to Table 6-4 and Table 6-5 to Table 6-8 present the k-NN classification accuracies for the ECC = 0.55Θ and ECC = 0.90Θ, respectively. Results indicate that for ECC = 0.55Θ, the classification accuracies for the validation and test data based on mask matrix M1 were 98.35% and 98.38% (see Table 6-1 and Table 6-2), respectively; which are higher compared to the accuracies obtained based on mask matrix M2, i.e., 95.39% and 95.16% for the validation and test sets (see Table 6-3 and Table 6-4). By contrast, for the case of ECC = 0.90Θ, the k-NN classification accuracies based on the average mask matrix were higher 170 than the accuracy based on the minimum mask matrix. That is, the accuracies based on the latter were 97.87% and 97.78% for the validation and test data (see Table 6-5 and Table 6-6), respectively; while accuracies achieved from the former were 98.60% and 98.76%, as presented in Table 6-7 and Table 6-8. To further evaluate the damage identification approach, results of the k-NN classification method with various mask matrices and harvested energy levels were determined and shown in Figure 6-22(a) and Figure 6-22(b) for the validation and test data, respectively. Results indicate that the average and minimum mask matrices led to the highest classification accuracy for different levels of harvested energy, e.g., for test data the maximum accuracies for ECC values of 0.55Θ, 0.75Θ, 0.90Θ, and 1.0Θ were 98.35%, 97.55%, 98.76%, and 98.07%, respectively. Evidently, the highest classification accuracy was based on the average mask matrix, i.e., 98.76% on test data and 98.60% on validation data (see Figure 6-22), for which the classification errors for class 7 (damaged scenario) were 0.21% and 0.29 for validation and test data (see Table 6-7 and Table 6-8). The pattern classification results demonstrate the good performance of the proposed damage detection strategy in terms of identifying damage with missing and noisy features. Table 6-1. Confusion matrix for validation data (matrix M1 and ECC = 0.55Θ) True classes 1 2 3 4 5 6 7 1 1979 90 0 0 9 6 9 2 20 1354 11 0 6 17 2 3 0 77 10546 0 6 41 0 Sum Error (%) 2093 5.44 1410 3.97 10670 1.16 Predicted classes 4 0 0 0 2028 0 0 0 2028 0.0 5 2 7 0 0 1214 1 5 1229 1.22 6 2 14 0 0 0 1701 1 1718 0.98 7 3 4 0 0 1 3 1572 1583 0.70 Total error 1.65% Accuracy 98.35% 171 Table 6-2. Confusion matrix for test data (matrix M1 and ECC = 0.55Θ) True classes 1 2 3 4 5 6 7 1 1984 80 0 0 3 12 11 2 10 1354 14 0 9 21 1 Sum Error (%) 2090 5.07 1409 2.91 Total error Predicted classes 3 2 63 10542 0 3 30 1 10641 0.93 1.62% 4 0 0 1 2027 0 0 0 2028 0.04 5 2 16 0 1 1212 4 3 1238 2.10 6 6 29 0 0 2 1698 0 1735 2.13 7 2 4 0 0 7 4 1571 1588 1.07 Accuracy 98.38% Table 6-3. Confusion matrix for validation data (matrix M2 and ECC = 0.55Θ) True classes 1 2 3 4 5 6 7 1 1929 8 10 0 19 55 25 Sum Error (%) 2046 5.71 Total error 2 42 2248 0 0 63 33 34 2420 7.10 Predicted classes 3 146 16 7985 0 129 11 111 8398 4.91 4 0 0 0 2375 0 0 0 2375 0.0 5 25 9 0 0 2313 99 42 2488 7.03 6 5 0 0 0 4 905 1 915 1.09 7 23 1 7 0 19 18 2021 2089 3.25 4.61% Accuracy 95.39% Table 6-4. Confusion matrix for test data (matrix M2 and ECC = 0.55Θ) True classes 1 2 3 4 5 6 7 1 1951 18 18 0 44 61 45 Sum Error (%) 2137 8.70 Total error 2 43 2222 0 0 56 38 38 2397 7.30 Predicted classes 3 94 8 7965 0 65 25 74 8231 3.23 4 0 0 1 2373 0 0 0 2374 0.04 5 40 22 5 2 2324 120 31 2554 8.61 6 15 0 1 0 19 856 11 902 5.10 7 27 12 12 0 29 21 2033 2134 4.73 4.84% Accuracy 95.16% 172 Table 6-5. Confusion matrix for validation data (matrix M1 and ECC = 0.90Θ) True classes 1 2 3 4 5 6 7 1 1902 5 9 0 0 60 1 Sum Error (%) 1977 3.79 Total error Predicted classes 2 6 1892 15 0 0 31 2 1946 2.78 3 0 6 1214 0 0 36 1 1257 3.42 4 0 0 0 2629 0 0 0 2629 0.0 5 1 36 20 0 10191 125 5 10738 1.73 6 1 5 3 0 4 965 10 988 2.32 7 10 4 8 0 0 37 1514 1573 3.75 2.13% Accuracy 97.87% Table 6-6. Confusion matrix for test data (matrix M1 and ECC = 0.90Θ) True classes 1 2 3 4 5 6 7 1 1896 11 5 0 0 64 17 Sum Error (%) 1993 4.87 Total error 2 12 1902 14 0 2 43 7 1980 3.93 Predicted classes 3 0 11 1219 0 0 44 7 1281 4.84 4 0 0 2 2627 0 0 0 2629 0.07 5 3 15 13 2 10175 109 2 10319 1.39 6 3 2 3 0 18 963 3 992 2.92 7 6 4 7 0 0 31 1487 1535 3.12 2.22% Accuracy 97.78% Table 6-7. Confusion matrix for validation data (matrix M2 and ECC = 0.90Θ) True classes 1 2 3 4 5 6 7 1 1901 10 6 0 11 18 9 Sum Error (%) 1955 2.76 Total error Predicted classes 2 4 2003 0 0 0 33 3 2043 1.96 3 4 0 2012 0 0 38 14 2068 2.70 4 0 0 0 1104 0 0 0 1104 0.0 5 82 4 8 0 10634 23 0 10751 1.09 6 3 2 13 0 0 1400 3 1421 1.47 7 3 0 0 0 0 0 1386 1389 0.21 1.14% Accuracy 98.60% 173 Table 6-8. Confusion matrix for test data (matrix M2 and ECC = 0.90Θ) True classes 1 2 3 4 5 6 7 1 1911 10 10 0 0 10 7 Sum Error (%) 1948 1.90 Total error Predicted classes 2 12 2002 0 0 0 54 5 2073 3.42 3 6 1 2007 0 0 12 10 2036 1.42 4 0 0 1 1103 0 0 0 1104 0.09 5 55 3 8 1 10644 19 0 10730 0.81 6 11 3 13 0 1 1415 2 1445 2.07 7 2 0 0 0 0 2 1389 1393 0.29 1.12% Accuracy 98.76% Figure 6-22. Damage identification accuracy for varying mask matrices and harvested energy levels: (a) validation data, (b) test data 6.2.6.2.3 Effect of Noise on Damage Identification Performance As a final evaluation of the proposed SHM strategy, the training, validation, and test data were contaminated with random noise (see Section 4). Noise levels of 5% to 20%, in 5% increments, were applied. The damage classification results obtained using a k-NN algorithm with the various noise levels for mask matrices M1 and M2 are presented in Figure 6-23. Results suggest that the damage detection model’s performance on the training data was not considerably affected by the 174 increase in noise level (see Figure 6-23(a) and (d)) such that the minimum accuracy achieved for the minimum mask matrix M1 and an ECC value of 0.75Θ was 94%. Conversely, as can be seen from Figure 6-23(b), (c), (e), and (f), the damage identification accuracy for the validation and test data gradually decreased with increasing noise level. That is, the damage detection model had almost acceptable accuracy for a noise level up to 20%, for which the minimum classification accuracy was 86% for both validation and test data based on mask matrix M2 and ECC = 0.55Θ. Results therefore show that the proposed damage detection approach is capable of identifying damage even with high noise levels. Figure 6-23. Damage detection accuracy for varying noise levels based on: (a) to (c) minimum mask matrix M1, (c) to (d) average mask matrix M2 175 6.3 Machine Learning Framework Employing Matrix Completion, Pattern Recognition, and Statistical Approach 6.3.1 Performance Evaluation of Machine Learning Framework This section presents the theory of a machine learning algorithmic framework for energy-lean SHM and damage detection with time-delayed and incomplete/sparse signals. Similar to Section 6.2, low-rank matrix completion is incorporated with an image-based PR approach as part of machine learning framework to reconstruct the delayed and incomplete signals. However, a statistical approach is developed herein for damage classification with reconstructed/recovered signals. The proposed machine learning data mining framework employing a statistical approach was evaluated through numerical and experimental studies on dynamically loaded plates, as illustrated in Sections 3.3 and 3.4. According to image-based PR approach, a simple pilot-type local rule for binary signal generation was defined in terms of the strain thresholds (R1 and R2) at the sensor nodes (i.e., corresponding FE model mesh nodes and strain gages). For the numerical study (see Figure 3-3), a binary signal was generated if the maximum principal strain at the sensor node was larger than R1. For the experimental study (see Figure 3-5), a binary signal was generated if the absolute value of the longitudinal strain at the strain gage locations exceeded threshold R2. The thresholds R1 and R2 (namely, 200 and 120 micro-strains, respectively) were empirically chosen to ensure that binary signals representing damage could be generated. Results of preliminary analyses showed that the layouts shown provided sufficient information for data collecting and calibrating/training the algorithmic framework. Thus, the number of self-powered sensors for the simply supported plate was set at 16. Consequently, each pattern (image) was represented by 16 features (binary 176 values). The array size for the cantilever plate was 9, and thus each pattern in the experimental case was represented by 9 features. The major contribution of the work presented herein is the development of a machine learning- based algorithmic framework for SHM composed of the following tasks: 1) pre-processing of time- delayed binary signals using an image-based PR approach and data fusion model, 2) reconstruction of the incomplete and time-delayed binary signals using low-rank matrix completion, and 3) identification of the sensing nodes representing damage by means of a statistical approach. A flowchart for the proposed framework is presented in Figure 6-24. Figure 6-24. Damage detection accuracy for varying noise levels based on: (a) to (c) minimum mask matrix M1, (c) to (d) average mask matrix M2 6.3.2 Damage Classification with Statistical Approach A study was conducted to investigate the characteristics of the sensor signal time delay. In this context, the delivery delay values for the 16 sensor nodes on the simply supported plate (numerical case) and 9 sensor nodes on the cantilever plate (experimental case) were determined. Surface plots of the average delivery delay based on spatial integration of the sensor nodes for the simply 177 supported plate for damage class 1 (i.e., localized stiffness reduction of 20%) is shown in Figure 6-25. As can be seen, the average delivery delay for each sensor node is affected by its distance to the sink. The average delivery value for the sensor nodes close to the sink is lower compared to those nodes that are far away from the sink. Although such value for the nodes, which are not closed to the sink, is not significantly affected by increasing the distance to the sink, this issue can be attributed to the resolution of the data (i.e., number of sensor nodes). It is expected that increasing the resolution will lead to better observe the effect of distance to the sink on the average delivery delay value of sensor nodes. Figure 6-25. Surface plot of average delivery delay for simply supported plate and damage class1 In order to characterize the signal delay and given the nature of the signals, it was initially assumed that distribution of signal delay for each sensor node could be a Gaussian-type distribution. Therefore, the probability density functions (PDF) of delivery delay values at the sensor nodes for both numerical and experimental plates were determined. The PDF plots of delivery delay for sensing nodes 5 to 9 obtained from FE simulations of the intact plate are shown 178 in Figure 6-26(a) and (c); while those for the damaged plate (damage class 1, i.e., localized stiffness reduction of 20%) are presented in Figure 6-26(b) and (d). Clearly, the PDF plots resembles a Gaussian distribution, as the plots are symmetric around the mean, and approximately 68% and 95% of values (observations) lie within 1 and 2 standard deviations of the mean, respectively. Figure 6-26. The PDF plot of delivery delay for sensors 5 to 9 for numerical plate: (a) and (c) intact plate, (b) and (d) damaged plate As can be seen, the variations in the PDF plots of the noted sensing nodes from undamaged to damaged case are noticeable, i.e., the mean values are almost identical but the standard deviation values differ. But more importantly, the results clearly indicate that the distribution of event delivery delay for both cases is Gaussian. Similar results were obtained for the test cantilever plate, where the PDF of delivery delay for sensing nodes 1 to 6 from the experiments on the intact and 179 damaged plate with a hole diameter of 19 mm are shown in Figure 6-27. Results confirm that the distribution of event delivery delay values is Gaussian. After reconstructing the time-delayed binary signals using the low-rank matrix completion algorithm (i.e., soft-impute algorithm), as illustrated in Section 6.2.4, a statistical approach is implemented to identify the damage using reconstructed signals. Statistical measures, namely, average, standard deviation, skewness, and kurtosis for each of the 16 sensor nodes (numerical case) and 9 sensor nodes (experimental case) using the reconstructed time-delayed binary signals are computed for the intact and damaged plates. Damage indices are then determined by comparing the statistical values for the intact and damaged plates. Detailed steps of the approach are presented in Figure 6-28. Figure 6-27. The PDF plot of delivery delay for sensors 1 to 6 for experimental plate: (a) and (c) intact plate, (b) and (d) damaged plat 180 Figure 6-28. Implementation of the proposed statistical approach 181 6.3.3 Results of Self-Powered SHM with Machine Learning-based Algorithmic Framework Strain and acceleration responses were extracted from the FE simulations and experiments. As noted in Section 5.3.1, strains were used to generate binary signals using a threshold concept, while the acceleration response was used for determining harvested energy. Consequently, acceleration response and binary signals were fed into a simulated through-substrate sensor network employing an energy-aware pulse switching protocol to generate the time-delayed binary signals. Finally, the time-delayed binary signals were fed as input to the machine learning algorithmic framework for damage detection. 6.3.3.1 Finite Element Simulation Results The image-based PR approach was first used to represent sensor nodes responses as pattern and identify damage based on full binary signals (signals without delay). The PR approach was calibrated such that the plate’s normal response (e.g., due to service load) was determined to identify the normal patterns. Once the PR classifier was calibrated using data from the intact plate, data from the simulation of a damaged plate was introduced to the PR approach for classification. Figure 6-29 presents classification results for the number of identified patterns for the simply supported plate, where the grey and blue regions correspond to values of 0 (inactive sensing node) and 1 (active sensing node), respectively. Red regions denote binary events/signals (values of 1) generated due to damage. Accordingly, patterns 1 and 2 represent normal conditions of the plate, while patterns 3 to 6 were due to time delay. Also, patterns 7 and 8 were identified to be from damage, as the frequency of these patterns continuously increased with simulation time (damage severity increases with simulation time). Classification results clearly indicate that sensing nodes 2, 5, and 6 imply damage for the numerical plate. It should be noted that the labels Noisy Pattern 182 in the figures refer to the patterns identified due to time delay. Further, the identified patterns occur at different times during the simulation. Figure 6-29. Classification results of identified patterns based on image-based PR approach (numerical plate) Once the preliminary results of PR approach were obtained with full signals, the reconstructed time-delayed binary matrix R containing time-delayed signals was determined as output of the low-rank matrix completion process employing imputation algorithm. Accuracy of the soft-impute algorithm for the intact and damaged plates (decreased localized stiffness) was then computed. Results are presented herein for the intact plate, and classes 1 and 4 of the damaged plates (localized 20% and 80% stiffness reduction), for the simulation time window of t = 50 s to t = 75 s. Imputation accuracies were calculated with respect to different mask matrices (see Section 3.1) and are presented in Figure 6-30. Results show that the mask matrix constructed based on computing the minimum values for the time steps at which the neighbor of a typical sensor Si reported values of 1, considering that the value of sensor Si itself was 1, resulted in the highest 183 imputation accuracy. For damage class 1 the accuracy based on the minimum mask matrix was 89.23% (i.e., the soft-impute algorithm successfully recovered and reconstructed 89.23% of the missing signals), which is higher compared to the accuracy obtained based on the other mask matrices (i.e., maximum, average, and mean mask matrices). Similar results were obtained for damage class 4 (see Figure 6-30). This observation governed for all cases (i.e., intact and damaged plates). Figure 6-30. Imputation accuracy for different mask matrices (numerical plate) A statistical approach was used to determine the presence and location of damage using the reconstructed time-delayed binary signals. Results from the statistical evaluation show that the standard deviation, kurtosis, and PDF plots using recovered time-delayed binary signals can be considered as a statistical damage criterion as presented in the algorithm in Figure 6-28. That is, for a given sensor node and damaged case, if the standard deviation value using the reconstructed signal is less than that of other sensor nodes, while the kurtosis using the reconstructed signal is higher than that of other nodes, the sensor reading could represent damage (Condition 1). However, two more conditions must be met for that sensor node to represent damage. First, the standard deviation value of the sensor for the damaged case should be lower compared to the intact case, and the kurtosis value has to be larger (Condition 2). If the noted condition is met, a third condition has to be evaluated by computing and plotting the PDF value of the sensor node using 184 reconstructed time-delayed binary signals, where the PDF plot of the sensor should significantly shift to the right once damage occurs (Condition 3). If the noted three criteria are met, the sensor node is considered to represent damage. Figure 6-31. Visualization of damage indices (standard deviation and kurtosis) (numerical plate): (a) and (b) intact class, (c) and (d), damage class 1, (e) and (f) damage class 4 185 Standard deviations and kurtosis for different sensor nodes using reconstructed time-delayed binary signals for damage classes 1 and 4 are shown in Figure 6-31. As can be observed, the standard deviation values of sensor node 6 for damage classes 1 and 4 are 0.126 and 0.123, respectively, which are lower compared to other sensor nodes. Further, the kurtosis values are 6.717 and 9.949, which are higher than that of other sensor nodes. On the other hand, the standard deviations of sensor node 6 for both classes are less than that of the undamaged case (0.135), while the kurtosis values are larger than that of the healthy case (5.287). These results indicate that sensor node 6 represents damage for classes 1 and 4 and that the damage location should be near this sensor; which is in good agreement with the results of PR approach as well as location of introduced damage (reduced stiffness) in the FE model. Nonetheless, sensor node 5 also indicates damage, especially for class 4; where, other than for node 6, its standard value (0.164) is lower in comparison to other sensor nodes and its kurtosis (8.641) is higher than that of other nodes. Thus, sensor node 5 also could represent damage when its severity is relatively high. Therefore, results confirm that among all the sensing nodes, sensor nodes 5 and 6 are the most likely to indicate the presence of damage since the results meet the first and second conditions of statistical damage criterion. To evaluate if sensing nodes 5 and 6 are true positives indicating damage, and to assess the third condition, the PDF values of these nodes and all the nodes met conditions 1 and 2 using reconstructed time-delayed binary signals were computed. The PDF plots of sensors 5, 6, 7, and 10 for the intact class and damage classes 1 and 4 are presented in Figure 6-32 and Figure 6-33, respectively. 186 Figure 6-32. PDF plots of intact class and damage class 1 for time interval 50 -75 sec (numerical plate): (a) sensor 5, (b) sensor 6, (c) sensor 7, and (d) sensor 10 Figure 6-33. PDF plots of intact class and damage class 4 for time interval 50 -75 sec (numerical plate): (a) sensor 5, (b) sensor 6, (c) sensor 7, and (d) sensor 10 187 As can be observed, among the PDF plots of different sensors, the PDF plot of sensor node 6 significantly shifts to the right once damage occurs (see Figure 6-32(b) and Figure 6-33(b)), whereas a slight shift in PDF value for sensor node 5 is noticeable (see Figure 6-32(a) and Figure 6-33(a)). It is also evident from the plots that the shift in PDF values of sensor nodes 5 and 6 becomes more significant with increasing damage severity. Such results are attributed to the fact that binary signals are created due to damage near sensor 6, which leads to an increase in the mean value of the reconstructed signals from the intact case to the damaged case for this sensor node. The slight shift of sensor node 5, a neighbor of sensor node 6, indicates that the response correlation between sensors or neighboring effect can help improve the accuracy of the damage detection approach. The obtained results show that the statistical damage criterion defined based on combination of standard deviation, kurtosis, and PDF plots using reconstructed delayed signals can be used to detect the presence and location of damage. It is also acknowledged that according to the image-based approach (see Figure 6-30) sensors 2, 5, and 6 could imply damage. However, the results presented in Figure 6-31 to Figure 6-33 demonstrate that among the noted sensors, only sensor nodes 5 and 6 suggest the presence of damage with reconstructed signals. It is important to note that similar results were obtained for different simulation time windows (i.e., different time lengths), suggesting that the approach is robust with respect to the time window size. 6.3.3.2 Experimental Validation Results To further examine the effectiveness of the damage identification method on a realistic structure, experimental validation was conducted using time-delayed binary signals. An image-based PR approach was used to identify the sensing nodes representing damage with complete signals. The classification results for the number of identified patterns for the experimental cantilever plate are presented in Figure 5-22. As noted in Section 5.3.3, patterns 1 and 2 denoted normal conditions, 188 pattern 3 was due to time delay, and patterns 4 to 6 were recognized due to damage (i.e., damaged plate with varying hole diameters). It is noted that sensing nodes 5 and 6 would imply damage. Figure 6-34. Classification results of identified patterns using image-based PR approach (experimental plate) Figure 6-35. Visualization of damage indices (standard deviation and kurtosis) (experimental plate): (a) and (b) intact plate, (c) and (d), damaged plate 189 Accuracy of the soft-impute algorithm with respect to different mask matrices for intact and damaged plates with varying hole diameters was calculated. However, results are presented here for the intact plate and a damage plate with a hole diameter of 19 mm. Results of the low-rank soft-impute are shown in Figure 6-34. Similar to the numerical plate results, the minimum mask matrix led to the highest imputation accuracy for both intact and damaged plates based on experimental test data. The maximum imputation accuracy based on the minimum mask matrix was 86.91% for the damaged plate, while it was 92.3% for the intact plate. The standard deviation and kurtosis of different sensing nodes with reconstructed time- delayed binary signals for the intact and damaged plates (19 mm diameter hole) are presented in Figure 6-35. The standard deviation of sensing node 5 for the damage plate (0.267) is lower than that of other sensing nodes. In addition, the kurtosis value (12.079) is higher compared to other sensing nodes. Evidently, the standard deviation of sensing node 5 for the damaged plate is less than that of the intact plate, i.e., 0.375, whereas the kurtosis value is larger compared to the healthy plate, i.e., 5.578. Sensing node 7 also shows a damage trend according to the first and second damage conditions; and thus, along with sensor 5, it could represent damage. It is expected for sensor node 5 to represent damage since it is close to the damage location (hole) in the experimental plate. However, it appears that node 7 shows a false positive since this node is far away from the damage location. To evaluate the third condition of the damage criterion, the PDF plots of sensing nodes 3, 5, 6, and 7 for the intact and damaged experimental plates were computed and presented in Figure 6-36, from which it can be seen that only the PDF of sensing node 5 shifts to the right (Figure 6-36 (b)). Generation of binary signals due to damage results in an increase of the mean value of the reconstructed time-delayed binary signals from the intact plate to the damaged plate for a sensing 190 node near to the damage location. Even though sensing node 7 showed a damage trend according to the first and second conditions of the damage criterion, this sensor node does not meet the third condition and thus it is not considered to represent damage. Finally, results from the experimental study validate the findings from the FE simulations, confirming that combination of standard deviation, kurtosis, and PDF of reconstructed time-delayed binary signals can be used as a statistical damage criterion to identify the presence and location of damage. Figure 6-36. PDF plots of intact and damaged cantilever plates: (a) sensing node 2, (b) sensing node 3, (c) sensing node 5, and (d) sensing node 6 6.4 Summary This chapter has presented a novel data-driven SHM strategy employing machine learning (ML) frameworks for damage identification based on the simulation of asynchronous time-delayed binary and incomplete/sparse data from a self-powered wireless sensor network integrated with an 191 energy-aware pulse switching protocol. First, a pioneering ML framework that incorporates pattern recognition (PR), low-rank matrix completion employing a soft-impute algorithm, a data fusion model, and the k-nearest neighbor (k-NN) algorithm was developed and implemented for interpretation of incomplete and time-delayed binary data. A data fusion model along with low- rank matrix completion were used to reconstruct discrete binary and time-delayed signals, whereas, classification with reconstructed signals was performed using k-NN. The damage identification strategy was applied to simulated damage scenarios on a horizontal stabilizer wing through the finite element method. Robustness of the proposed approach due to variations in harvested energy levels was also evaluated. The following conclusions were reached from the study: 1) Results confirmed that low-rank matrix completion employing a soft-impute algorithm has the ability to efficiently recover the missing and noisy features of time-delayed sensed binary data. 2) Results indicated that, among different mask matrices constructed within the ML framework, the minimum and average mask matrices led to the highest imputation and damage classification accuracy; such that the highest imputation accuracy was 92.34% and the best classification accuracy was 98.76% on the test data. 3) It is noted that the classification results were affected by considering variations in the training and testing data. That is, the highest variance of the error was almost 3.0%, indicating the maximum classification accuracy was 96% for the cases, in which various data samplings were used. 4) The accuracy of the proposed data-driven SHM method was examined using the k-NN method with missing features, and classification results indicated acceptable performance 192 of the proposed strategy. Results of a Bayesian optimization process revealed that Spearman distance was the optimal distance for k-NN analysis. 5) Results showed that the performance of the damage identification model was satisfactory even with a noise level up to 20%. 6) Results suggest that the proposed data-driven SHM strategy is robust with respect to varying harvested energy levels, as the values of time delay were not substantially affected by such variations. In the second analysis phase, an ML algorithmic framework integrating a data fusion model, low-rank matrix completion, and a statistical approach was developed and implemented for reconstruction of time-delayed binary signals and damage assessment using reconstructed signals. A proposed statistical approach was employed to detect damage with reconstructed signals. The SHM strategy was applied to simulated scenarios on a simply supported plate through the finite element method, and the applicability of the proposed method on a realistic structure was explored using vibration experimental tests. The following conclusions were reached: 7) Results based on numerical and experimental data/signals indicate satisfactory performance of the proposed algorithmic framework using limited signals for damage identification in plate-like structures. 8) The obtained results confirm that low-rank matrix completion using a soft-impute algorithm is able to efficiently reconstruct and recover the noisy features of time-delayed binary signals. 9) The proposed data fusion model represents a unique approach to enhance damage detection performance via spatial-temporal integration of sensor node values, given limited signal availability due to time delays from network communication. 193 10) The proposed statistical approach had good performance in identifying damage with reconstructed delayed signals. 11) The distribution of delivery delay resembled a Gaussian distribution. A proposed damage criterion based on the combination of the standard deviation, kurtosis, and PDF plot of the reconstructed time-delayed binary signals was shown to be appropriate for detecting the presence and location of damage. The methods and results presented in this chapter demonstrate the applicability and effectiveness of developed ML frameworks for energy-lean data-driven SHM. However, the evaluation and verification of the proposed methodology in full-scale experiments using self- powered sensors outfitted with the pulse switching communication architecture needs to be conducted. Further, the good classification results can be attributed to the fact that 194 CHAPTER 7 7 EFFECT OF LEARNING ALGORITHMS ON PERFORMANCE OF MACHINE LEARNING-BASED DATA MINING FRAMEWORK 7.1 Overview Chapter 6 presented the proposed machine learning (ML)-based data mining framework integrating matrix completion, pattern recognition (PR), and a data fusion model for energy-lean data-driven SHM methodology with noisy/sparse and incomplete signals. Yet, the effect of different ML algorithms on performance of the SHM method was not studied, even though the aforementioned feature could have a significant impact on the performance of such framework for SHM. Further, the effectiveness of the approach has not been experimentally demonstrated. To address notes issue, this chapter presents the results of robustness evaluation of the ML framework using different learning algorithms and with experimental data. On this basis, support vector machine (SVM), k-nearest neighbor (k-NN), and artificial neural networks (ANN) algorithms were used within the ML data interpretation framework to determine damage identification accuracy with sparse and delayed data. It is noted that the integrated self-powered sensor with communication technology is under design, fabrication, and testing. In addition, experimental demonstration of the self-powered sensor has been reported in recent studies [7–9,57], and the applicability of through-substrate communication protocol has been demonstrated for a sensor network on an aluminum plate [5]. Therefore, the focus of the present chapter is on evaluating the effectiveness of the proposed energy-lean SHM methodology with time-delayed binary signals generated through finite element simulations of an aircraft wing, and to validate the applicability of the SHM method on a realistic structure through vibration tests on a dynamically loaded cantilever plate. 195 7.2 Data Interpretation Framework Employing Machine Learning The proposed machine learning-based data interpretation framework presented in Figure 7-1 consists of three phases: data fusion, learning, and classification. The data fusion phase involves constructing a mask matrix and modified time-delayed binary matrix using a data fusion model based on response correlation between sensor nodes. The modified time-delayed binary matrix constructed based on the data fusion model is introduced as input to the learning phase, in which low-rank matrix completion with soft-impute algorithm is used to recover/impute missing and noisy features of the modified time-delayed matrix. For the classification phase, different ML algorithms, including SVM, ANN, and k-NN are used to classify the recovered time-delayed binary data and for damage detection. The dataset for the analysis is randomly divided into three subsets; namely, training, validation, and testing. Further, k-fold cross validation is used to prevent overfitting problems. That is, the training set is divided to k subsets of equal size, where each subset is consecutively tested using the classifier trained on the remaining k-1 subsets. Consequently, the cross-validation accuracy is determined based on the percentage of data that are correctly classified. In this research, 10-fold cross-validation (assuming k = 10) was considered. It is to be noted that sensor node responses are represented as a pattern using image-based PR approach, where true labels of the patterns (i.e., normal, noisy, damaged) identified based on full binary data are used for validation. The proposed ML-based data analysis framework was implemented thru a custom program in MATLAB. Detailed theory of the proposed machine learning data mining framework is presented in Section 6.2. 196 Figure 7-1. Data interpretation framework employing machine learning for self-powered SHM 7.3 Numerical and Experimental Case Studies Finite element simulations are performed on an aircraft wing stabilizer, as illustrated in Section 6.2.1 (see Figure 6-2). Experimental vibration tests are conducted on a cantilever plate, as presented in 3.4 (see Figure 3-5). For the numerical case, a simple pilot-type local rule for binary event generation was defined in terms of a strain threshold (R1) at the sensor nodes (i.e., corresponding FE model mesh nodes). Thus, a binary event (value of 1) was generated if the maximum principal strain at the sensor node was larger than R1. The threshold R1 (80 micro- strains) was empirically chosen, based on the FE simulations, to ensure that binary events 197 representing damage could be generated. Damage scenarios were defined as ‘classes’ and numbered from 1 to 4 for stiffness reductions of 20% to 80% (in 20% increments), respectively. The geometry of the aircraft stabilizer wing, the layout of the self-powered sensor nodes, the region of simulated damage in the FE model, and a typical representation of sensor node values as a pattern are shown in Figure 6-4. For the experimental case, the local rule for binary event generation at the sensor nodes (i.e., strain gages locations) was defined in terms of a strain threshold R2 set at 120 micro-strains. Acceleration and maximum principal strain responses at the stabilizer wing sensor nodes were extracted from the FE analysis. The strain response was used to generate binary events using a threshold concept, while the acceleration response was used for determining harvested energy. The acceleration response and binary events were then fed to a simulated through-substrate self- powered sensor network employing an energy-aware pulse switching protocol to generate the time-delayed binary data. Finally, the time-delayed binary data/signals was introduced as input to the SHM methodology employing ML-based data interpretation framework for damage detection. For the vibration tests, as it was illustrated in Section 5.3.1, strain response was collected from the strain gages, while the acceleration response at the strain gages locations were taken from the numerical simulation and used in the pulse communication protocol for generating timed-delayed binary data for the experimental study. To implement the machine learning algorithms, the dataset for the numerical case study (i.e., aircraft stabilizer wing) was classified into 7 classes. Classes 1 and 2 represented patterns due to normal condition of the wing, classes 3 to 5 denoted noisy and time-delayed patterns, and classes 6 and 7 represented patterns due to damage. For the experimental case study (i.e., cantilever plate), the data was classified to 4 classes. Classes 1 and 2 presented normal condition of the plate, class 198 3 denoted the noisy patterns, and class 4 represented patterns identified due to damage. The damage detection accuracy with different ML algorithms (namely, SVM, k-NN, and ANN) with noisy features was determined according to equation (7-1). ܦܽ݉ܽ݃݁ ܦ݁ݐ݁ܿݐ݅݋݊ ܣܿܿݑݎܽܿݕ = ܰݑܾ݉݁ݎ ݋݂ ݌ܽݐݐ݁ݎ݊ݏ ܿ݋ݎݎ݁ܿݐ݈ݕ ݈ܿܽݏݏ݂݅݅݁݀ ܶ݋ݐ݈ܽ ݊ݑܾ݉݁ݎ ݋݂ ݌ܽݐݐ݁ݎ݊ݏ ሺ7 − 1ሻ 7.4 Self-Powered Damage Identification Results 7.4.1 Results based on Finite Element Simulations of Aircraft Wing 7.4.1.1 Results of Low-Rank Matrix Completion Algorithm Accuracy of the low-rank matrix completion algorithm employing soft-impute algorithm for the intact and damaged stabilizer (decreased localized stiffness) was computed. Results are presented here for the intact and damaged class 2 for a simulation time t = 4000 s. Imputation accuracies were calculated with respect to different mask matrices and various harvested energy levels, and results are presented in Figure 7-2. Figure 7-2. Imputation accuracy for different mask matrix vs harvested energy: (a) intact stabilizer, (b) damaged stabilizer 199 In the given results, M1, M2, M3, and M4, denote the mask matrices constructed based on computing the minimum, average, median, and maximum values for the time steps at which the neighbor of a typical sensor Si reported values of 1, considering that the values of sensor Si itself was 1. Results show that the highest imputation accuracies (among different combinations of mask matrices and ECC values) were obtained using the minimum mask matrix with ECC = 0.75Θ and ECC = 1.0Θ. Consequently, the highest imputation accuracy for the intact and damaged stabilizers are 96.61% and 96.52%, respectively .This indicates that the soft-impute algorithm successfully recovered 96.61% and 96.52% of the missing values. Given such high imputation accuracies, the effectiveness of the low-rank matrix completion technique employing imputation algorithm is considered validated. 7.4.1.2 Results of SVM Algorithm Different kernel functions, i.e., polynomial, radial basis function (RBF), and sigmoid, were used for implementing the SVM algorithm. However, performance of the SVM with a polynomial kernel was found to be superior compared to other kernels as shown in Figure 7-3(a). As shown, the accuracy based on polynomial kernel was 98.21% and 98.14%, respectively, on training and validation data set. The optimal degree of the polynomial kernel function was determined through an optimization process, determining that the best degree was 2 (see Figure 7-3(b)). Once the polynomial kernel was selected as the optimal kernel, a gird-search with 10-fold cross validation was conducted for different pairs of (C, γ) to determine the optimum kernel hyper- parameters with best cross-validation accuracy. The training and validation data set were used to optimize the regularization and kernel parameters. Consequently, the kernel hyper-parameters were tuned through such optimization process. Results of the optimization process in terms of the damage detection accuracy as a function of the kernel parameters C and γ are presented in Figure 200 7-3(c) and (d). As it can be observed, the best damage detection accuracy was achieved for a polynomial kernel, with d =2, C= 20, and γ= 1. Therefore, the noted values were used as optimal hyper-parameters in the SVM algorithm. Figure 7-3. (a) Selection of optimal kernel function, (b) Tuning optimal kernel degree, (c) Optimization of the kernel parameter C, and (d) Optimization of the kernel parameterγ Results of the damage detection accuracy employing the SVM algorithm with various mask matrices and harvested energy levels were determined and shown in Figure 7-4 for the validation and test data. Results indicate that the minimum (M1) and average (M2) mask matrices led to highest classification accuracy for different levels of harvested energy. Clearly, the highest classification accuracy on the validation and test data was 99.70%, which was based on the minimum mask matrix (see Figure 7-4). 201 Figure 7-4. Damage detection accuracy for varying mask matrices and harvested energy levels on validation and test data based on SVM Confusion matrices for the actual/target and predicted/output classification results using SVM were determined for different mask matrices and harvested energy levels. However, classification results presented in Figure 7-5 are only for the minimum and average mask matrices with ECC values of 0.55Θ and 1.0Θ (i.e., least and maximum energy levels), since these matrices led to highest imputation and classification accuracy. Results reveal that SVM is able to identify damage with satisfactory classification accuracy. As an example, the classification errors based on the minimum mask matrix for class 7 (damaged case) and for least and maximum energy levels were 1.2% and 1.7%, respectively (see Figure 7-5(a) and (b)). To further explore the effectiveness of the SVM algorithm for the damage detection model, the receiver operating characteristic (ROC) was determined and plotted. An ROC curve plots sensitivity as a function of specificity for all test combinations, illustrating a classifier’s trade-off between true positives and false negatives. A higher value of sensitivity for a given specificity denotes better performance. The area under the ROC curve (AUC) is a metric used for assessing a 202 classifier’s performance; such that an AUC value close to 1 indicates better classification performance (i.e., a test set with an AUC value of 1 is perfectly accurate). Figure 7-5. Confusion matrices for test data based on SVM: (a) Matrix M1 and ECC=0.55Θ, (b) Matrix M1 and ECC=1.0Θ, (c) Matrix M2 and ECC=0.55Θ, (d) Matrix M2 and ECC=1.0Θ ROC curves for class 7 (damaged scenario) based on minimum, average, and median mask matrices, and ECC values of = 0.55Θ and 1.0Θ are presented in Figure 7-6. It can be observed that the minimum mask matrix (M1) led to best classification performance, as the AUC values based 203 on this matrix are higher compared to other mask matrices (i.e., for ECC = 0.55Θ the AUC values are 98.89%, 98.85%, and 94.92% based on mask matrices M1, M2, and M3, respectively). For ECC = 1.0Θ, mask matrix M1 also led to the highest AUC value (i.e., 99.61%) compared to other matrices. Classification results, i.e., confusion matrices and ROC curves, indicate the good performance of the proposed ML methodology employing SVM in terms of identifying damage, even with noisy and time-delayed data. Figure 7-6. ROC curve based on SVM classification for varying mask matrices and harvested energy levels 7.4.1.3 Results of k-NN Algorithm For the k-NN analysis, the calibration/training data set was used to fit the classifier, whereas the validation set was used to compute the optimal k for the k-NN classifier. The best models were selected based on their performance on the validation data. Performance of the classifier with optimal k was thus investigated on the test data. In this study, to select the optimal distance and number of neighbors (k), the accuracy of the k-NN classifier on training, validation, and test set was computed. Although the minimum and average mask matrices led to the highest imputation accuracy, the accuracy of the classifier based on different mask matrices and harvested energy levels was determined. However, results are presented here for ECC values of 0.55Θ and 1.0Θ. Further, the optimal distance for the k-NN classifier, through the optimization process, was found 204 to be Spearman. The classification accuracy for the validation set based on optimal distance is shown in Figure 7-7, from which the optimum value of k for different mask matrices can be observed. For the least and maximum harvested energy (i.e., ECC = 0.55Θ and 1.0Θ), maximum accuracy was achieved using the minimum mask matrix (M1). According to Figure 7-7(a), for all mask matrices, the optimum value of k was 5. Yet, for ECC = 1.0Θ (see Figure 7-7(b)), and the average mask matrix (M2,) the optimal k value was 7; while it was 5 for the other mask matrices. Figure 7-7. K-NN classification accuracy for damage detection model with varying number of k: (a) & (b) ECC = 0.55Θ, (c) & (d) ECC = 1.0Θ Figure 7-8. Damage detection accuracy for varying mask matrices and harvested energy levels on validation and test data based on k-NN with optimal k 205 Figure 7-9. Confusion matrix for test data based on k-NN: (a) Matrix M1 and ECC=0.55Θ, (b) Matrix M1 and ECC=1.0Θ, (c) Matrix M2 and ECC=0.55Θ, (d) Matrix M2 and ECC=1.0Θ Figure 7-8 illustrates the damage detection accuracy employing k-NN method, from which it can be observed that minimum mask matrix M1 led to highest classification accuracy for both validation and test data, i.e., 98.90% and 99.10% based on ECC values of 0.75Θ and 1.0Θ, respectively. The confusion matrices shown in Figure 7-9 present the detailed classification accuracy on test data. As can be seen, the classification errors for class 7 (damage class) are 1.0% 206 and 0.8% based on minimum mask matrix, and least and highest harvested energy level (see Figure 7-9(a) and (b)). The classification errors for class 7 based on average mask matrix, however, are higher, i.e., 3.6% and 2.9% for ECC values of 0.55Θ and 1.0Θ. Similar trend was obtained for class 6 representing damage phase, such that the maximum accuracies were achieved based on matrix M1 for different energy availability. Therefore, results confirm that the minimum mask matrix M1 lead to highest classification accuracy for damaged classes. Once the optimal k value and distance were determined, the accuracy of the damage detection model on test data using optimal parameters was computed. Further, ROC curves for the k-NN classifier for ECC values of 0.55Θ and 1.0Θ are presented in Figure 7-10, from which it can be observed that the best classification accuracy for ECC = 0.55Θ was achieved using mask matrix M1; while for the ECC = 1.0Θ, the best accuracy was obtained using M2. Results confirm that M1 and M2 led to the highest classification accuracy for validation and test data. With such high classification accuracies, good performance of the damage detection approach using k-NN was demonstrated. Figure 7-10. ROC curve based on k-NN classification for varying mask matrices and harvested energy levels 207 7.4.1.4 Results of ANN Algorithm For the ANN analysis, a three-layer (i.e., two-hidden-layers, each with 10 neurons) feed-forward neural network using back-propagation algorithm was used. This network architecture was chosen based on an evaluation of different ANN configurations, whose performance was based in terms of the cross entropy error against the number of epochs. Such performance decreased as the network was calibrated. In other words, the error reduced after more epochs of training. A lower cross entropy implied the better performance of an ANN. Classification results with ANN indicating the network’s performance in terms of cross entropy loss with respect to number of epochs are presented in Figure 7-11. Results herein are presented for minimum and average mask matrices based on the least and maximum energy level (i.e., ECC values of 0.55Θ and 1.0Θ). Figure 7-11. ANN performance: (a) matrix M1 with 0.55Θ, (b) matrix M1 with 1.0Θ, (c) matrix M2 with 0.55Θ, (d) matrix M2 with 1.0Θ 208 As can be observed, the best performance is based on the highest energy level (ECC = 1.0Θ) using mask matrix M1, where the cross entropy error, i.e., 0.0017592, was achieved at epoch 162, which is smaller compared to other reported epochs (see Figure 7-11). Nonetheless, results indicate the good cross entropy errors; thus confirming the acceptable performance of the ANN. Figure 7-12. Damage detection accuracy for varying mask matrices and harvested energy levels on validation and test data based on ANN Damage detection performance in terms of ANN accuracy with respect to different mask matrices and energy levels are shown in Figure 7-12, where the best classification results were obtained based on minimum mask matrix M1. To better visualize the results, the confusion matrices were determined and presented in Figure 7-13. Evidently, the highest classification accuracy was based on the minimum mas matrix and ECC = 1.0Θ, i.e., 99.60% on validation and test data (see Figure 7-13), for which the classification error on test data for classes 6 and 7 representing damage were 2.1% and 0.80% (see Figure 7-13(b)). Such classification results based on ECC = 1.0Θ were achieved for the average mask matrix M2, for which the classification accuracy was 98.70% on test data, while the classification error for class 6 presenting damage scenario was 0.90%, which is higher compared to classification error for class 7 (3.4% based on 209 highest energy level). The pattern classification results thus demonstrate the good performance of the proposed approach employing ANN with missing data. Figure 7-13. Confusion matrix for test data based on ANN: (a) Matrix M1 and ECC=0.55Θ, (b) Matrix M1 and ECC=1.0Θ, (c) Matrix M2 and ECC=0.55Θ, (d) Matrix M2 and ECC=1.0Θ 7.4.1.5 Results of Uncertainty Analysis To evaluate the reliability of the machine learning-based data interpretation framework with different learning algorithms, the validation and test data were contaminated with random noise. 210 Noise levels of 5% to 20%, in 5% increments, were considered. The classification results using ANN, k-NN, and SVM methods with various noise levels for the optimal mask matrix M1 are presented in Figure 7-14. Figure 7-14. Damage detection accuracy for varying noise levels based on minimum mask matrix M1 and different learning algorithms: (a) to (c) Validation data, (d) to (f) Test data Results suggest that the damage detection model’s performance on the validation and test data was affected by the increase in noise level, such that for the ANN method the minimum accuracies (90%) were achieved for all the considered ECC values when the noise level was 20%, see Figure 7-14(d). As can be seen from Figure 7-14(e), the minimum accuracy based on the k-NN algorithm and the same noise level was 87% for the ECC value of 0.90Θ. Similar results were obtained for the SVM analysis, in which the minimum classification accuracy on test data was 91% (see Figure 7-14(f)). Results indicate that although the damage detection accuracy gradually decreased with 211 increasing noise level, the proposed damage detection approach employing different machine learning algorithms is still capable of identifying the damage phase even with high noise levels. 7.4.2 Results based on Experimental Studies Accuracy of the imputation algorithm with respect to different mask matrices for intact and damaged plates with varying hole diameters was calculated, and results are presented here for the intact plate and the damage plate with a 19 mm diameter hole. The imputation accuracies from the low-rank matrix completion process employing are shown in Figure 7-15. Similar to the numerical study (i.e., aircraft stabilizer wing) results, the minimum mask matrix led to the highest imputation accuracy for both intact and damaged plates. The maximum imputation accuracy based on minimum mask matrix was 95.84% for the damaged plate. These results indicate acceptable performance of the low-rank matrix completion algorithm in terms of recovering the missing/incomplete data. Figure 7-15. Imputation accuracy for different mask matrices Once the time-delayed matrix was recovered, different ML algorithms, including SVM, k- NN, and ANN, were used to determine the damage detection accuracy with experimental time- delayed binary data. Results are presented here based on SVM and k-NN. The hyper-parameters of the SVM algorithm (i.e., kernel function, polynomial degree, C and γ) and the optimal distance 212 and number of k for the k-NN algorithm were identical to the FE simulations. ROC curves for SVM and k-NN algorithms were determined and are plotted in Figure 7-16, from which it can be seen that both ML algorithms resulted in good damage classification accuracy with high AUC values. Figure 7-16. ROC curve based on SVM and k-NN classification for varying mask matrices Confusion matrices for the test data set based on the SVM and k-NN algorithms were also determined using M1 and M2 and are presented in Figure 7-17. Clearly, the accuracy of the damage detection approach using both algorithms is acceptable, having low classification errors. Therefore, results from the experimental study of the cantilever plate validate the findings from the FE simulations of the aircraft stabilizer wing, confirming that the proposed data interpretation framework employing SVM, k-NN, and ANN can be effectively used for SHM and damage identification with missing and incomplete data. In addition, results confirm that the minimum and average mask matrices constructed within the ML methodology lead to highest imputation and classification accuracies. It is noted that the noted classification results can be affected by using different data sampling such that the accuracy decreases by almost 3%, still indicating the acceptable performance of the machine learning framework. 213 Figure 7-17. Confusion matrices for test data based on: (a) SVM with matrix M1, (b) SVM with matrix M2, (c) k-NN with matrix M1, (d) k-NN with matrix M2 7.5 Summary This chapter presented results of machine learning-based data mining framework with different learning algorithms, including SVM, k-NN, and ANN. The framework’s viability for varying machine learning algorithms was evaluated. The damage detection strategy was applied to simulated scenarios on an aircraft horizontal stabilizer wing through the finite element method and 214 on dynamic experiments on a cantilever plate. The following conclusions were reached from this chapter: 1) Numerical and experimental results indicate that, among different mask matrices constructed within the ML methodology, the minimum and average mask matrices led to the highest imputation and classification accuracies. The highest imputation accuracies were 96.61% and 95.84%, and the best classification accuracies were 99.70% and 99.80 % based on an SVM algorithm for the numerical and experimental data, respectively. 2) It is noted that highest variance of the error was 3.0% by considering different data samplings. That is, the highest classification accuracies with SVM were 96.50% and 96.20% for the numerical and experimental data, respectively. 3) Results of pattern classification, confusion matrices, and ROC curves confirmed that all the evaluated ML algorithms led to acceptable damage detection accuracy with noisy and missing signals. The presented chapter demonstrated that the proposed data-driven SHM methodology employing machine learning framework can be effectively used for data interpretation of asynchronous time-delayed binary and incomplete/noisy data sets from novel though-substrate self-powered wireless sensor networks. 215 CHAPTER 8 8 CONLCUSIONS 8.1 Research Contributions and Significance The research presented in this dissertation has contributed to the development of a new class of SHM technique called energy-lean data-driven SHM. Novel data interpretation frameworks employing machine learning were developed for self-powered SHM with discrete asynchronous delayed binary and incomplete/sparse signals. The proposed data-driven SHM system effectively addresses the drawbacks associated with the deployment and maintenance of traditional wired sensors, thus significantly improving safety and decreasing maintenance costs of civil and aerospace structures. Development of the noted data-driven SHM platform could promote the use of self-powered sensing for SHM and condition assessment in civil infrastructure. The research findings are expected to pave a way for use of machine learning and pattern recognition for various emerging applications in structural engineering domain. Namely, the proposed data-driven SHM methodology employing machine learning can be effectively used for damage diagnosis with noisy and sparse/missing data for smart infrastructure monitoring. Further, the developed machine learning-based data mining frameworks can be employed beyond the SHM domain; including the analysis and interpretation of incomplete and missing experimental test data due to faulty sensors, noise, and uncertainties. 8.2 Conclusions The following conclusions were reached based on the material, findings, and discussions provided throughout this dissertation: 216 1) The pattern recognition (PR)-based algorithmic framework based on the integration of an image-based PR approach using anomaly detection and various artificial classifiers can be used to represent sensor node response as pattern/image, and to detect damage from the recognized patterns in plate-like structures. The PR framework has the ability to successfully identify abnormality among patterns using discrete delayed binary and noisy signals/data provided by a self-powered sensing technology. The PR framework is robust with respect to non-harmonic/irregular loading conditions, sensor configurations, measurement noise, and various damage severities. 2) The probabilistic-based approach employing the spatial-temporal conditional probability chain can be used for predicting delayed signals. Further, a data mining framework incorporating image-based PR approach, a novel probabilistic approach, and SVM algorithm can be successfully used to model delay and reconstruct missing signals/data collected from smart self-powered sensors. 3) The developed machine learning-based data interpretation framework based on the integration of low-rank matrix completion, PR approach, data fusion model, and k-NN algorithm can be effectively used for energy-lean SHM with incomplete, sparse, and missing signals/data from a through-substrate self-powered sensor network. The machine learning framework is robust with respect to variations in harvested energy and measurement noise. Further, the proposed data analysis framework has the ability to employ different learning algorithms such as SVM, k-NN, and ANN, while performing satisfactorily in terms of reconstruction of missing signal and classification of damage in civil and aerospace structures. 217 4) The proposed machine learning-based algorithmic framework (merging low-rank matrix completion, PR approach, data fusion model, and the statistical approach) can be employed to recover and reconstruct missing/delayed and sparse data for self-powered damage identification in structural plates. 5) The developed energy-lean data-driven SHM platform employing machine learning-based data interpretation frameworks and probabilistic-based algorithmic frameworks can be efficiently used for damage detection and condition assessment in civil and aerospace structures when dealing with asynchronous discrete binary, noisy, and sparse data. 8.3 Future Research The current study and investigation was motivated by the increased interest in using the artificial intelligence for civil engineering applications. In this context, this dissertation focused on exploring the use of machine learning and pattern recognition for data-driven SHM. Applicability and effectiveness of the proposed machine learning-based data interpretation frameworks for energy-lean long-term SHM with novel self-powered sensing technology was evaluated and demonstrated. Nevertheless, there are still some challenges that are the focus of future research. Namely, the applicability of the proposed SHM strategy in full-scale experiments using self- powered sensors outfitted with the energy-aware pulse switching architecture needs to be evaluated. Besides, the ability of the approach for various types of structures and types of damage could be investigated. In this study, supervised machine learning and pattern recognition algorithms were used for damage identification and classification with labeled signals/data. Future research could focus on the use of unsupervised machine learning algorithms for damage prediction for the cases where class targets are unknown. Reinforcement learning algorithms may also be used for condition assessment and decision making for the cases where an agent is needed 218 to determine the ideal behavior within a specific context. More, future research topic may focus on the use of recurrent neural networks for time series prediction take into account the effect of time delay. Studies on optimal sensor placement would also be an interesting future research topic. In addition, it is noted that for the SHM application in the wing, true positive rate is more important compared to false positive rate, since the former refers to an outcome where the machine learning classifier correctly predicts the positive class. Yet, future studies could be conducted to determine the acceptable threshold for true positive rate. Finally, it is noted that for evaluation of the developed data mining frameworks, different data samplings were considered, for which the corresponding variance of errors were determined. Future study would explore the effect of different data sampling methods on the performance of the frameworks. In addition to the problem addressed in this dissertation, potential research avenues for employing pattern recognition, machine learning, and deep learning in emerging structural engineering applications were also identified. Considering the emerging use of wireless sensor networks (e.g., self-powered sensor networks), machine learning-based models could become the next generation approaches to conduct non-destructive structural and material evaluation in SHM. Machine learning methods are able to discover hidden information about the structure’s performance by learning the influence of various damage or degrading mechanisms and the data collected from sensors, leading to reliable and efficient SHM frameworks. Future research topics may focus on the use of machine learning to solve complex problems through the novel concept of the Internet of Things (IoT). On this basis, use of machine learning and deep learning architectures (e.g., convolutional neural networks) within the context of IoT to analyze and interpret complex and big data could be a potential future research topic. Further, exploring the ability of proposed machine learning frameworks integrated with IoT for SHM systems with 219 diverse and noisy sensor data could be an interesting future topic. Finally, future research topics may focus on the use of the developed machine learning frameworks as data-driven techniques for intelligent infrastructure monitoring and vision-based SHM in in smart cities. 220 REFERENCES 221 REFERENCES [1] Sohn H, Farrar CR, Hemez FM, Shunk DD, Stinemates DW, Nadler BR, et al. A review of structural health monitoring literature: 1996–2001. Los Alamos Natl Lab USA 2003. [2] Roundy S, Wright PK, Rabaey J. A study of low level vibrations as a power source for wireless sensor nodes. Comput Commun 2003;26:1131–44. [3] Elvin NG, Lajnef N, Elvin AA. Feasibility of structural monitoring with vibration powered sensors. Smart Mater Struct 2006;15:977. [4] Park G, Rosing T, Todd MD, Farrar CR, Hodgkiss W. Energy harvesting for structural health monitoring sensor networks. J Infrastruct Syst 2008;14:64–79. [5] Das S, Lorenz S, Dong B, Huo Q, Biswas S. Through-substrate event reporting using harvested energy in ultrasound sensor networks, IEEE; 2015, p. 1–7. [6] Das S, Salehi H, Shi Y, Chakrabartty S, Burgueno R, Biswas S. Towards packet-less for energy-harvesting structures. Comput Commun ultrasonic sensor networks 2017;101:94–105. doi:10.1016/j.comcom.2016.11.001. [7] Chenling Huang, Lajnef N, Chakrabartty S. Calibration and Characterization of Self- Powered Floating-Gate Usage Monitor With Single Electron per Second Operational Limit. IEEE Trans Circuits Syst Regul Pap 2010;57:556–67. doi:10.1109/TCSI.2009.2024976. [8] Zhou L, Chakrabartty S. Linearization of CMOS Hot-Electron Injectors for Self-Powered Monitoring of Biomechanical Strain Variations. IEEE Trans Biomed Circuits Syst 2017;11:446–54. [9] Zhou L, Abraham AC, Tang SY, Chakrabartty S. A 5 nw quasi-linear CMOS hot-electron injector for self-powered monitoring of biomechanical strain variations. IEEE Trans Biomed Circuits Syst 2016;10:1143–51. [10] Huo Q, Biswas S, Plummer Jr A. Pulse Switching for Static Event Sensing in Sensor Networks, IEEE; 2011, p. 1–6. [11] Huo Q, Rao J, Biswas S. Pulse switching: Toward a packet-less protocol paradigm for event sensing. IEEE Trans Mob Comput 2013;12:35–50. [12] Huo Q, Dong B, Biswas S. A pulse switching paradigm for ultra low power cellular sensor networks. Pervasive Mob Comput 2014;13:221–45. [13] Fan W, Qiao P. Vibration-based damage identification methods: a review and comparative study. Struct Health Monit 2011;10:83–111. [14] Doebling SW, Farrar CR, Prime MB. A summary review of vibration-based damage identification methods. Shock Vib Dig 1998;30:91–105. [15] Farrar CR, Worden K. An introduction to structural health monitoring. Philos Trans R Soc Lond Math Phys Eng Sci 2007;365:303–15. 222 [16] Doebling SW, Farrar CR, Prime MB, Shevitz DW. Damage identification and health monitoring of structural and mechanical systems from changes in their vibration characteristics: a literature review. Los Alamos National Lab., NM (United States); 1996. [17] Carden EP, Fanning P. Vibration based condition monitoring: a review. Struct Health Monit 2004;3:355–77. [18] Farrar C. Historical overview of structural health monitoring. Lect Notes Struct Health Monit Using Stat Pattern Recognit 2001. [19] Farrar CR, Park G, Allen DW, Todd MD. Sensor network paradigms for structural health monitoring. Struct Control Health Monit 2006;13:210–25. [20] Khoa NL, Zhang B, Wang Y, Chen F, Mustapha S. Robust dimensionality reduction and damage detection approaches in structural health monitoring. Struct Health Monit 2014;13:406–17. [21] Smarsly K, Dragos K, Wiggenbrock J. Machine learning techniques for structural health monitoring, 2016, p. 5–8. [22] Taffese WZ, Sistonen E. Machine learning for durability and service-life assessment of reinforced concrete structures: Recent advances and future directions. Autom Constr 2017;77:1–14. [23] Figueiredo E, Park G, Farrar CR, Worden K, Figueiras J. Machine learning algorithms for damage detection under operational and environmental variability. Struct Health Monit 2011;10:559–72. [24] Butcher JB, Day CR, Austin JC, Haycock PW, Verstraeten D, Schrauwen B. Defect Detection in Reinforced Concrete Using Random Neural Architectures: Defect detection in reinforced concrete using random neural architectures. Comput-Aided Civ Infrastruct Eng 2014;29:191–207. doi:10.1111/mice.12039. [25] Nagarajaiah S, Yang Y. Modeling and harnessing sparse and low‐rank data structure: a new paradigm for structural dynamics, identification, damage detection, and health monitoring. Struct Control Health Monit 2017;24. [26] Yun C-B, Min J. Smart sensing, monitoring, and damage detection for civil infrastructures. KSCE J Civ Eng 2011;15:1–14. [27] Sundaram BA, Ravisankar K, Senthil R, Parivallal S. Wireless sensors for structural health monitoring and damage detection techniques. Curr Sci 2013:1496–505. [28] Lynch JP, Sundararajan A, Law KH, Kiremidjian AS, Kenny T, Carryer E. Embedment of structural monitoring algorithms in a wireless sensing unit. Struct Eng Mech 2003;15:285– 97. [29] Lynch J. Overview of wireless sensors for real-time health monitoring of civil structures, 2004, p. 10–1. [30] Spencer BF, Ruiz‐Sandoval ME, Kurata N. Smart sensing technology: opportunities and challenges. Struct Control Health Monit 2004;11:349–68. [31] Lynch JP. Design of a wireless active sensing unit for localized structural health monitoring. Struct Control Health Monit 2005;12:405–23. 223 [32] Lynch JP, Loh KJ. A summary review of wireless sensors and sensor networks for structural health monitoring. Shock Vib Dig 2006;38:91–130. [33] Elvin NG, Elvin AA, Spector M. A self-powered mechanical strain energy sensor. Smart Mater Struct 2001;10:293. [34] Sirohi J, Chopra I. Fundamental understanding of piezoelectric strain sensors. J Intell Mater Syst Struct 2000;11:246–57. [35] Rahimi M, Shah H, Sukhatme GS, Heideman J, Estrin D. Studying the feasibility of energy harvesting in a mobile sensor network. vol. 1, IEEE; 2003, p. 19–24. [36] Elvin N, Elvin A, Choi D. A self-powered damage detection sensor. J Strain Anal Eng Des 2003;38:115–24. [37] Lajnef N, Rhimi M, Chatti K, Mhamdi L, Faridazar F. Toward an integrated smart sensing system and data interpretation techniques for pavement fatigue monitoring. Comput Civ Infrastruct Eng 2011;26:513–23. [38] Bezdek J. What is Computational Intelligence? Comput Intell Imitating Life JM Zurada RJ Marks II CJ Robinson Eds IEEE Press 1994:1–12. [39] Artificial Intelligence: A New Synthesis. Elsevier; 1998. doi:10.1016/C2009-0-27773-7. [40] Computational intelligence: a logical approach. Choice Rev Online 1998;35:35-5701-35– 5701. doi:10.5860/CHOICE.35-5701. [41] Kurzweil R. The Age of Spiritual Machines: When Computers Exceed Human Intelligence. New York, NY: Penguin Books; 2000. [42] Luger GF. Artificial Intelligence: Structures and Strategies for Complex Problem Solving. 6 edition. Boston: Pearson; 2008. [43] Russell SJ, Norvig P, Canny JF, Malik JM, Edwards DD. Artificial intelligence: a modern approach. vol. 2. Prentice hall Upper Saddle River; 2003. [44] Adeli H, Hung S-L. Machine learning: neural networks, genetic algorithms, and fuzzy systems. John Wiley & Sons, Inc.; 1994. [45] Kicinger R, Arciszewski T, Jong KD. Evolutionary computation and structural design: A 2005;83:1943–78. state-of-the-art. Comput Struct survey doi:10.1016/j.compstruc.2005.03.002. the of [46] Liao TW, Egbelu P, Sarker B, Leu S. Metaheuristics for project and construction management–A state-of-the-art review. Autom Constr 2011;20:491–505. [47] Lu P, Chen S, Zheng Y. Artificial Intelligence in Civil Engineering. Math Probl Eng 2012;2012:1–22. doi:10.1155/2012/145974. [48] Shahin MA. Artificial intelligence in geotechnical engineering: applications, modeling aspects, and future directions. Metaheuristics Water Geotech. Transp. Eng., Elsevier; 2013, p. 169–204. [49] Saka MP, Geem ZW. Mathematical and metaheuristic applications in design optimization of steel frame structures: an extensive review. Math Probl Eng 2013;2013. 224 [50] Aldwaik M, Adeli H. Advances in optimization of highrise building structures. Struct Multidiscip Optim 2014;50:899–919. [51] Mardani A, Jusoh A, Zavadskas EK. Fuzzy multiple criteria decision-making techniques and applications–Two decades review from 1994 to 2014. Expert Syst Appl 2015;42:4126– 48. [52] Salehi H, Burgueño R. Emerging artificial intelligence methods in structural engineering. Eng Struct 2018;171:170–89. [53] Yuen K-V. Bayesian methods for structural dynamics and civil engineering. John Wiley & Sons; 2010. [54] Xu N, Rangwala S, Chintalapudi KK, Ganesan D, Broad A, Govindan R, et al. A Wireless Sensor Network For Structural Monitoring. Proc. 2Nd Int. Conf. Embed. Networked Sens. Syst., New York, NY, USA: ACM; 2004, p. 13–24. [55] Kim S, Pakzad S, Culler D, Demmel J, Fenves G, Glaser S, et al. Health Monitoring of Civil Infrastructures Using Wireless Sensor Networks. Proc. 6th Int. Conf. Inf. Process. Sens. Netw., New York, NY, USA: ACM; 2007, p. 254–263. doi:10.1145/1236360.1236395. [56] Wang Y, Law KH. Structural Control with Multi-Subnet Wireless Sensing Feedback: Experimental Validation of Time-Delayed Decentralized H* Control Design. Adv Struct Eng 2011;14:25–39. doi:10.1260/1369-4332.14.1.25. [57] Huang C, Chakrabartty S. An asynchronous analog self-powered CMOS sensor-data-logger with a 13.56 MHz RF programming interface. IEEE J Solid-State Circuits 2012;47:476–89. [58] Anton SR, Inman DJ. Vibration energy harvesting for unmanned aerial vehicles. vol. 6928, International Society for Optics and Photonics; 2008, p. 692824. [59] Lombaert G, Moaveni B, He X, Conte JP. Damage identification of a seven-story reinforced concrete shear wall building using Bayesian model updating. Proc IMAC-XXVII 2009:9– 12. [60] Duda RO, Hart PE, Stork DG. Pattern Classification. 2 edition. New York: Wiley- Interscience; 2000. [61] theodoridis S, Koutroumbas K. Pattern Recognition. 4th Edition. Elsevier; 2008. [62] Santos JP, Cremona C, Orcesi AD, Silveira P. Early damage detection based on pattern recognition and data fusion. J Struct Eng 2016;143:04016162. [63] Bishop C. Pattern Recognition and Machine Learning. Springer N Y 2007. [64] Cherkassky V, Mulier FM. Learning from data: concepts, theory, and methods. John Wiley & Sons; 2007. [65] Michalski RS, Carbonell JG, Mitchell TM. Machine learning: An artificial intelligence approach. Springer Science & Business Media; 2013. [66] Alpaydin E. Introduction to machine learning. MIT press; 2014. [67] Robert C. Machine learning, a probabilistic perspective 2014. [68] Marsland S. Machine learning: an algorithmic perspective. CRC press; 2015. 225 [69] Stone J, Blockley D, Pilsworth B. Towards machine learning from case histories. Civ Eng Syst 1989;6:129–35. [70] Arciszewski T, Mustafa M, Ziarko W. A methodology of design knowledge acquisition for use in learning expert systems. Int J Man-Mach Stud 1987;27:23–32. [71] Tong S, Chang E. Support vector machine active learning for image retrieval, ACM Press; 2001, p. 107. doi:10.1145/500141.500159. [72] Sebe N. Machine learning in computer vision. vol. 29. Springer Science & Business Media; 2005. [73] Rosten E, Drummond T. Machine Learning for High-Speed Corner Detection. In: Leonardis A, Bischof H, Pinz A, editors. Comput. Vis. – ECCV 2006, vol. 3951, Berlin, Heidelberg: Springer Berlin Heidelberg; 2006, p. 430–43. doi:10.1007/11744023_34. [74] Lézoray O, Charrier C, Cardot H, Lefèvre S. Machine learning in image processing 2008. [75] Bradski G, Kaehler A. Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, Inc.; 2008. [76] Di K, Li W, Yue Z, Sun Y, Liu Y. A machine learning approach to crater detection from topographic data. Adv Space Res 2014;54:2419–29. doi:10.1016/j.asr.2014.08.018. [77] Duygulu P, Barnard K, de Freitas JF, Forsyth DA. Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary, Springer; 2002, p. 97–112. [78] Dede G, Sazlı MH. Speech recognition with artificial neural networks. Digit Signal Process 2010;20:763–8. [79] Hinton G, Deng L, Yu D, Dahl G, Mohamed A, Jaitly N, et al. Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups. IEEE Signal Process Mag 2012;29:82–97. doi:10.1109/MSP.2012.2205597. [80] Deng L, Hinton G, Kingsbury B. New types of deep neural network learning for speech recognition and related applications: an overview, IEEE; 2013, p. 8599–603. doi:10.1109/ICASSP.2013.6639344. [81] Graves A, Mohamed A, Hinton G. Speech recognition with deep recurrent neural networks, IEEE; 2013, p. 6645–9. doi:10.1109/ICASSP.2013.6638947. [82] Deng L, Li X. Machine learning paradigms for speech recognition: An overview. IEEE Trans Audio Speech Lang Process 2013;21:1060–89. [83] LeBaron B. Agent-based computational finance. Handb Comput Econ 2006;2:1187–233. [84] Brabazon A, O’Neill M. Natural computing in computational finance. vol. 100. Springer Science & Business Media; 2008. [85] Harris T. Credit scoring using the clustered support vector machine. Expert Syst Appl 2015;42:741–50. [86] Sharma N, Sharma P, Irwin D, Shenoy P. Predicting solar generation from weather forecasts using machine learning, IEEE; 2011, p. 528–33. [87] Marvuglia A, Messineo A. Monitoring of wind farms’ power curves using machine learning techniques. Appl Energy 2012;98:574–83. 226 [88] Wan C, Xu Z, Pinson P, Dong ZY, Wong KP. Probabilistic forecasting of wind power generation using extreme learning machine. IEEE Trans Power Syst 2014;29:1033–44. [89] Needham CJ, Bradford JR, Bulpitt AJ, Westhead DR. A primer on learning in Bayesian networks for computational biology. PLoS Comput Biol 2007;3:e129. [90] Ben-Hur A, Ong CS, Sonnenburg S, Schölkopf B, Rätsch G. Support vector machines and kernels for computational biology. PLoS Comput Biol 2008;4:e1000173. [91] Che D, Liu Q, Rasheed K, Tao X. Decision tree and ensemble learning algorithms with their applications in bioinformatics. Softw. Tools Algorithms Biol. Syst., Springer; 2011, p. 191– 9. [92] Reich Y. Machine learning techniques for civil engineering problems. Comput Civ Infrastruct Eng 1997;12:295–310. [93] Kanevski M, Pozdnoukhov A, Timonin V. Machine learning for spatial environmental data: theory, applications, and software. EPFL press; 2009. [94] Ciresan DC, Meier U, Masci J, Maria Gambardella L, Schmidhuber J. Flexible, high performance convolutional neural networks for image classification. vol. 22, Barcelona, Spain; 2011, p. 1237. [95] Zeiler MD, Fergus R. Visualizing and understanding convolutional networks, Springer; 2014, p. 818–33. [96] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. ArXiv Prepr ArXiv14091556 2014. [97] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis 2015;115:211–52. [98] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions, Cvpr; 2015. [99] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition, 2016, p. 770– 8. [100] Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, et al. Recent advances in convolutional neural networks. Pattern Recognit 2017. [101] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks, 2012, p. 1097–105. [102] Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L. Large-scale video classification with convolutional neural networks, 2014, p. 1725–32. [103] Cha Y, Choi W, Büyüköztürk O. Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks. Comput Civ Infrastruct Eng 2017;32:361–78. [104] Cha Y-J, Choi W. Vision-Based Concrete Crack Detection Using a Convolutional Neural Network. Dyn. Civ. Struct. Vol. 2, Springer; 2017, p. 71–3. [105] Kong X, Li J. Vision‐Based Fatigue Crack Detection of Steel Structures Using Video Feature Tracking. Comput Civ Infrastruct Eng 2018. 227 [106] Atzori L, Iera A, Morabito G. The internet of things: A survey. Comput Netw 2010;54:2787–805. [107] Abdelgawad A, Yelamarthi K. Structural health monitoring: Internet of things application, IEEE; 2016, p. 1–4. [108] Abdelgawad A, Yelamarthi K. Internet of Things (IoT) Platform for Structure Health Monitoring. Wirel Commun Mob Comput 2017;2017. [109] Tokognon CA, Gao B, Tian GY, Yan Y. Structural health monitoring framework based on Internet of Things: A survey. IEEE Internet Things J 2017;4:619–35. [110] Schaffers H, Komninos N, Pallot M, Trousse B, Nilsson M, Oliveira A. Smart cities and the future internet: Towards cooperation frameworks for open innovation, Springer; 2011, p. 431–46. [111] Zanella A, Bui N, Castellani A, Vangelista L, Zorzi M. Internet of things for smart cities. IEEE Internet Things J 2014;1:22–32. [112] Perera C, Zaslavsky A, Christen P, Georgakopoulos D. Sensing as a service model for smart cities supported by internet of things. Trans Emerg Telecommun Technol 2014;25:81–93. [113] Kim T, Ramos C, Mohammed S. Smart city and IoT 2017. [114] Wang L, Sng D. Deep learning algorithms with applications to video analytics for a smart city: A survey. ArXiv Prepr ArXiv151203131 2015. [115] Chin J, Callaghan V, Lam I. Understanding and personalising smart city services using machine learning, The Internet-of-Things and Big Data, IEEE; 2017, p. 2050–5. [116] Han B, Yu X, Ou J. Self-sensing concrete in smart structures. Butterworth-Heinemann; 2014. [117] Konsta-Gdoutos MS, Aza CA. Self sensing carbon nanotube (CNT) and nanofiber (CNF) cementitious composites for real time damage assessment in smart structures. Cem Concr Compos 2014;53:162–9. [118] Gupta S, Gonzalez JG, Loh KJ. Self-sensing concrete enabled by nano-engineered cement- aggregate interfaces. Struct Health Monit 2017;16:309–23. [119] Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995;20:273–97. [120] Vapnik V. Statistical learning theory. 1998. Wiley, New York; 1998. [121] Lee Y. Handwritten digit recognition using k nearest-neighbor, radial-basis function, and backpropagation neural networks. Neural Comput 1991;3:440–9. [122] Denoeux T. A k-nearest neighbor classification rule based on Dempster-Shafer theory. IEEE Trans Syst Man Cybern 1995;25:804–13. [123] Gong L, Wang C, Wu F, Zhang J, Zhang H, Li Q. Earthquake-induced building damage detection with post-event sub-meter vhr terrasar-X staring spotlight imagery. Remote Sens 2016;8:887. [124] Vitola J, Pozo F, Tibaduiza DA, Anaya M. A sensor data fusion system based on k-nearest neighbor pattern classification for structural health monitoring applications. Sensors 2017;17:417. 228 [125] Lee Y. Handwritten Digit Recognition Using K Nearest-neighbor, Radial-basis Function, and Backpropagation Neural Networks. Neural Comput 1991;3:440–449. [126] Denoeux T. A k-nearest neighbor classification rule based on Dempster-Shafer theory. IEEE Trans Syst Man Cybern 1995;25:804–13. [127] Jianping Gou TX. A Novel Weighted Voting for K-Nearest Neighbor Rule. J Comput 2011;6:833–40. [128] Lopes Jr V, Park G, Cudney HH, Inman DJ. Impedance-based structural health monitoring with artificial neural networks. J Intell Mater Syst Struct 2000;11:206–14. [129] Lee JJ, Lee JW, Yi JH, Yun CB, Jung HY. Neural networks-based damage detection for bridges considering errors in baseline finite element models. J Sound Vib 2005;280:555– 78. [130] Jain AK, Duin RPW, Mao J. Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 2000;22:4–37. [131] Wang L, Wang X, Feng J. On image matrix based feature extraction algorithms. IEEE Trans Syst Man Cybern Part B Cybern 2006;36:194–7. [132] Yang J, Zhang D, Frangi AF, Yang J-Y. Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 2004;26:131–7. [133] Kuncheva LI, Jain LC. Nearest neighbor classifier: Simultaneous editing and feature selection. Pattern Recognit Lett 1999;20:1149–56. [134] Li M, Yuan B. 2D-LDA: A statistical linear discriminant analysis for image matrix. Pattern Recognit Lett 2005;26:527–32. [135] Salehi H, Das S, Chakrabartty S, Biswas S, Burgueño R. Structural damage identification using image-based pattern recognition on event-based binary data generated from self- powered sensor networks. Struct Control Health Monit 2018:e2135. doi:10.1002/stc.2135. [136] Basharat A, Gritai A, Shah M. Learning object motion patterns for anomaly detection and improved object detection, IEEE; 2008, p. 1–8. [137] Lankewicz L, Benard M. Real-time anomaly detection using a nonparametric pattern recognition approach, IEEE; 1991, p. 80–9. [138] Sagha H, Bayati H, Millán J del R, Chavarriaga R. On-line anomaly detection and resilience in classifier ensembles. Pattern Recognit Lett 2013;34:1916–27. [139] Moshtaghi M, Leckie C, Karunasekera S, Rajasegarar S. An adaptive elliptical anomaly detection model for wireless sensor networks. Comput Netw 2014;64:195–207. [140] MATLAB,The mathworks, Inc, Natick, MA 2014. [141] Dassault Systemes Simulia Corp. Abaqus analysis users manual, Abaqus documentation, version 6.11, Dassault Systemes, Providence, RI, 2013. n.d. [142] Huo Q, Rao J, Biswas S. Pulse switching: Toward a packet-less protocol paradigm for event sensing. IEEE Trans Mob Comput 2013;12:35–50. 229 [143] Huo Q, Dong B, Biswas S. A cellular pulse switching architecture for binary event sensing. 2012 IEEE Glob. Commun. Conf. GLOBECOM 379-384, 2012, p. 379–84. [144] Salehi H, Burgueño R. Pattern recognition framework using asynchronous discrete binary data for condition and damage assessment in plate-like structures. J Intell Mater Syst Struct 2019:1045389X19835948. [145] Njuguna R, Gruev V. Linear current mode image sensor with focal plane spatial image processing. Proc. 2010 IEEE Int. Symp. Circuits Syst. ISCAS, 2010, p. 4265–8. [146] Olyaei A, Genov R. Focal-Plane Spatially Oversampling CMOS Image Compression Sensor. IEEE Trans Circuits Syst Regul Pap 2007;54:26–34. [147] Luo Z, Xiong J, Yang J. A Model-based Analyzing and Calculating of the Focal Plane Array for Passive Millimeter Wave Imaging System. Proc. Int. Conf. Commun. Circuits Syst. 629- 632 2006, n.d. [148] Salehi H, Burgueño R, Das S, Biswas S, Chakrabartty S. Localized Damage Identification of Plate-Like Structures With Time-Delayed Binary Data From a Self-Powered Sensor Network, American Society of Mechanical Engineers; 2017, p. V002T05A008- V002T05A008. [149] Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett 2006;27:861–74. [150] Bradley AP. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit 1997;30:1145–59. [151] Bowyer KW, Chang K, Flynn P. A survey of approaches and challenges in 3D and multi- modal 3D+ 2D face recognition. Comput Vis Image Underst 2006;101:1–15. [152] Salehi H, Das S, Chakrabartty S, Biswas S, Burgueño R. Damage identification in aircraft structures with self‐powered sensing technology: A machine learning approach. Struct Control Health Monit 2018;25:e2262. [153] Salehi H, Das S, Chakrabartty S, Biswas S, Burgueño R. An algorithmic framework for reconstruction of time-delayed and incomplete binary signals from an energy-lean structural health monitoring system. Eng Struct 2019;180:603–20. [154] Gui G, Pan H, Lin Z, Li Y, Yuan Z. Data-driven support vector machine with optimization techniques for structural health monitoring and damage detection. KSCE J Civ Eng 2017;21:523–34. [155] Ying Y, Garrett Jr JH, Oppenheim IJ, Soibelman L, Harley JB, Shi J, et al. Toward data- driven structural health monitoring: application of machine learning and signal processing to damage detection. J Comput Civ Eng 2012;27:667–80. [156] Candès EJ, Recht B. Exact matrix completion via convex optimization. Found Comput Math 2009;9:717. [157] Matlab. Mathworks Inc Natick MA 2014. [158] Dassault Systems Simulia Corp. Abaqus analysis user manual 2013. [159] Benson T. FoilSim III. Natl Aeronaut Space Adm NASA Glenn Res CenterCleveland OH 2014. 230 [160] Candès EJ, Tao T. The power of convex relaxation: Near-optimal matrix completion. IEEE Trans Inf Theory 2010;56:2053–80. [161] Cai J-F, Candès EJ, Shen Z. A singular value thresholding algorithm for matrix completion. SIAM J Optim 2010;20:1956–82. [162] Mazumder R, Hastie T, Tibshirani R. Spectral regularization algorithms for learning large incomplete matrices. J Mach Learn Res 2010;11:2287–322. [163] Hastie T, Mazumder R, Lee J, Zadeh R. Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares. ArXiv14102596 Stat 2014. [164] Salehi H, Das S, Chakrabartty S, Biswas S, Burgueño R. A machine-learning approach for damage detection in aircraft structures using self-powered sensor data. vol. 10168, International Society for Optics and Photonics; 2017, p. 101680X. [165] Pelikan M, Goldberg DE, Cantú-Paz E. BOA: The Bayesian optimization algorithm, Morgan Kaufmann Publishers Inc.; 1999, p. 525–32. 231