i    DEVELOPMENT OF A STRUCTURED ILLUMINATION REFLECTANCE IMAGING SYSTEM FOR ENHANCED DETECTION OF SUBSURFACE AND SURFACE DEFECTS IN APPLE FRUIT   By Richard Li              A THESIS Submitted to  Michigan State University  in partial fulfillment of the requirements  for the degree of Biosystems Engineering  Master of Science 2016 ii  ABSTRACT DEVELOPMENT OF A STRUCTURED ILLUMINATION REFLECTANCE IMAGING SYSTEM FOR ENHANCED DETECTION OF SUBSURFACE AND SURFACE DEFECTS IN APPLE FRUIT By Richard Li A novel method of defect detection in apple fruit was developed using structured illumination reflectance imaging (SIRI). SIRI provided the ability to recover depth specific information within a fruit, which could potentially enhance defect detection. Preliminary tests were performed on a Nylon test material using a newly constructed SIRI system and results revealed that the SIRI system was capable of detecting subsurface inhomogeneities and also enhancing sample surface features. The SIRI system was applied for detection of bruises with various severities consistently outperformed the uniform illumination images, with overall detection rates ranging between 70-100% even in fresh bruises which had not yet visibly developed on the fruit surface. The SIRI system was also able to detect bruises that naturally occurred during infield sorting operations. The detection rates ranged from 81.1%-85.1% for , and were lower at 62.7%-69.5% for . SIRI was then applied to detect various surface defects on It was found that defects such as insect damage, black scab, sunburn and rot could be detected but with low detection rates. Linear discriminant analysis of the various defect types revealed that distinct features present in the SIRI images made classification of defective apples possible. The SIRI method has demonstrated its potential in successfully detecting fresh bruises and certain surface defects in apple fruit and should warrant further research into quality assessment of fresh fruit. iii  This thesis is dedicated to my mother and father. Thank you for your constant support and words of encouragement. Without your love and support, I would not have grown into the person I am today nor would I have found the strength to continue with my graduate studiesiv  ACKNOWLEDGEMENTS  I would like to express my sincere gratitude to my graduate advisor, Dr. Renfu Lu, for his continuous guidance throughout my two years as a graduate student. Without which, I would not have been able to complete this work. My sincere thanks also to committee members Dr. Kirk Dolan and Dr. Daniel Guyer for their support both during and outside of graduate coursework which have aided in strengthening my knowledge and this work. I am also grateful toward all of the instructors with whom I have had the pleasure of receiving lectures under during my studies at the university.  An enormous thank you to the Department of Biosystems Engineering at Michigan State University for supporting me financially during the past two years of my studies. Without this aid, I would not have been able to seek higher education at this institution. I would also like to thank my friends and colleagues with whom I have shared my findings with for their feedback and advice. Your support and friendship has made my time spent at Michigan State University all the more enjoyable   v  TABLE OF CONTENTS  LIST OF TABLES ............................................................................................................................................ vii LIST OF FIGURES ........................................................................................................................................... ix KEY TO ABBREVIATIONS ..............................................................................................................................xiii INTRODUCTION ............................................................................................................................................. 2 Objectives ................................................................................................................................................. 5 CHAPTER 1: LITERATURE REVIEW ................................................................................................................. 7 1.1 Defect Types ........................................................................................................................................ 7 1.2 Insect Infestation ................................................................................................................................ 7 1.3 Disease ................................................................................................................................................ 8 1.4 Physiological Disorders ....................................................................................................................... 8 1.5 Bruising ............................................................................................................................................... 8 1.6 Methods of Quality Assessment ......................................................................................................... 9 1.6.1 Impact Analysis .............................................................................................................................. 10 1.6.2 Acoustic Analysis ............................................................................................................................ 10 1.6.3 X-ray Imaging and Computed Tomography ................................................................................... 11 1.6.4 Magnetic Resonance Imaging ........................................................................................................ 13 1.6.5 Machine Vision and Spectroscopy ................................................................................................. 13 1.6.6 Hyperspectral Imaging ................................................................................................................... 15 1.7 Spatial Frequency Domain Imaging .................................................................................................. 17 CHAPTER 2: CONSTRUCTION AND PRELIMINARY EXPERIMENTS OF A STRUCTURED ILLUMINATION REFLECTANCE IMAGING (SIRI) SYSTEM....................................................................................................... 22 2.1 Introduction and Theory ................................................................................................................... 22 2.2 Major Components for Generating Structured Illumination ............................................................ 28 2.3 Preliminary Study on a Synthetic Test Sample ................................................................................. 35 2.3.1 A Synthetic Test Sample ................................................................................................................. 35 2.3.2 System Setup.................................................................................................................................. 37 2.3.3 Creation of Structured Illumination Patterns ................................................................................ 38 2.3.4 Image Acquisition ........................................................................................................................... 38 2.3.5 Image Demodulation and Processing ............................................................................................ 41 2.4 Preliminary Study on Apple Samples ................................................................................................ 43 CHAPTER 3: BRUISE DETECTION EXPERIMENT O ES ......... 46 3.1 SIRI System Adjustments .................................................................................................................. 47 3.2 Apple Sample Preparation ................................................................................................................ 47 3.3 Image Acquisition .............................................................................................................................. 48 3.4 Image Organization ........................................................................................................................... 50 3.5 Image Preprocessing and Demodulation .......................................................................................... 51 3.6 Bruise Detection Algorithm ............................................................................................................... 51 vi  3.7 Results ............................................................................................................................................... 55 3.8 Evaluation of SIRI System Performance ............................................................................................ 56 3.9 Accuracy of Detected Bruise Areas ................................................................................................... 62 CHAPTER 4: EVALUATION OF APPLE BRUISES FROM THE INFIELD SORTING MACHINE ............................. 68 4.1 Introduction ...................................................................................................................................... 68 4.2 Infield Sorting Machine ..................................................................................................................... 68 4.3 Apple Samples ................................................................................................................................... 70 4.4 Running the Infield Sorting Machine ................................................................................................ 72 4.5 Image Acquisition .............................................................................................................................. 73 4.6 Post-System Hand Evaluation ........................................................................................................... 74 4.7 SIRI Image Processing........................................................................................................................ 75 4.8 Bruise Detection Results ................................................................................................................... 76 CHAPTER 5: DETECTION OF SURFACE DEFECTS OAPPLES .......................................... 82 5.1 Introduction ...................................................................................................................................... 82 5.2 Defective Apple Samples .................................................................................................................. 83 5.3 SIRI System Setup .............................................................................................................................. 91 5.4 Image Acquisition .............................................................................................................................. 93 5.5 Image Demodulation ........................................................................................................................ 94 5.6 Image Processing for Defect Enhancement .................................................................................... 106 5.7 Defect Segmentation ...................................................................................................................... 109 5.8 Discriminant Model Analysis ........................................................................................................... 116 5.9 Results ............................................................................................................................................. 117 CONCLUSION AND FUTURE WORK ........................................................................................................... 128 APPENDIX .................................................................................................................................................. 132 BIBLIOGRAPHY .......................................................................................................................................... 137    vii  LIST OF TABLES  Table 4.1. Detection rates (%) obtained under SIRI by either spatial frequency 0.10 mm-1 or 0.15 mm-1 for  ........................... 78  Table 4.2. Detection rates (%) obtained under SIRI by either spatial frequency 0.10 mm-1 or 0.15 mm-1 for  ........................... 78  Table 4.3. Detection rates (%) obtained under SIRI by either spatial frequency 0.10 mm-1 or 0.15 mm-1 for  ........................... 78  Table 4.4. Detection rates (%) obtained under SIRI by either spatial frequency 0.10 mm-1 or 0.15 mm-1 for apple bruises identified prior to sorting through the infield machine. ........................... 78  Table 4.5. Overall detection rates (%) based on the total number of visually identified bruises (both old and new) by either spatial frequency 0.10 mm-1 or 0.15 mm-1 and total number of SIRI detected bruises before and after samples were run through the sorting system................................................................ 79  Table 4.6. The number of additional bruises detected by SIRI which were not identified by hand during peeled fruit bruise evaluation. .................................................................................................................... 80  Table 5.1. Defect detection rates under SIRI. ........................................................................................... 119  Table 5.2 Defect detection rates under uniform illumination. ................................................................. 120  Table 5.3. Defect detection rates (%) achie .. 121  Table 5.4. Average percent differences in the detected and true d ............... 122   ............. 122  -one out cross-validation linear discriminant model. ....................................................................................................... 125  -one out cross-validation linear discriminant model. ....................................................................................................... 125  mination using a leave-one out cross-validation linear discriminant model. ...................................................................... 126  Table 5.9. Confusion matrix of defect leave-one out cross-validation linear discriminant model. ...................................................................... 126  Table 5.10. Confusion matrix of SIRI cross validated linear discriminant model using selected feature set  ......................................................................................................................................... 126 viii   Table 5.11. Confusion matrix of SIRI cross validated linear discriminant model using selected feature set  ........................................................................................................................................ 126  Table 5.12. Confusion matrix of cross validated linear discriminant model for uniform illumination using  ........................................................................................................ 127  Table 5.13. Confusion matrix of cross validated linear discriminant model for uniform illumination using  ....................................................................................................... 127  Table A1. Average bruise detection rates (in percent) of low impact apples at 0-1, 4-6 and 24 hours after initial impact. ............................................................................................................................................ 133  Table A2. Average bruise detection rates (in percent) of medium impact apples 0-1, 4-6 and 24 hours after initial impact. .................................................................................................................................... 133  Table A3. Average bruise detection rates (in percent) of high impact apples 0-1, 4-6 and 24 hours after initial impact. ............................................................................................................................................ 133  Table A4. Standard deviation of bruise detection rates (in percent) of two replicates of low impact apples at 0-1, 4-6 and 24 hours after initial impact. ................................................................................. 134  Table A5. Standard deviation of bruise detection rates (in percent) of two replicates of medium impact apples at 0-1, 4-6 and 24 hours after initial impact. ................................................................................. 134  Table A6. Standard deviation of bruise detection rates (in percent) of two replicates of high impact apples at 0-1, 4-6 and 24 hours after initial impact. ................................................................................. 134  Table A7. Average differences (in percent) between detected and measured low impact bruise area. . 135  Table A8. Average differences (in percent) between detected and measured medium impact bruise area. .................................................................................................................................................................. 135  Table A9. Average differences (in percent) between detected and measured high impact bruise area. 135  Table A10. Standard deviations of the average differences (in percent) between two replicates of detected and measured low impact bruise areas. ................................................................................... 136  Table A11. Standard deviations of the average differences (in percent) between two replicates of detected and measured medium impact bruise areas. ............................................................................ 136  Table A12. Standard deviations of the average differences (in percent) between two replicates of detected and measured medium impact bruise areas. ............................................................................ 136   ix  LIST OF FIGURES  Figure 2.1. Comparison of components and illumination outputs between a machine vision and structured illumination imaging system. .................................................................................................... 22  Figure 2.2. Comparison of light attenuation and penetration depth of (a) low and (b) high spatial frequency sinusoidal illumination patterns in a turbid tissue medium. ..................................................... 25  Figure 2.3. Three-phase image demodulation process used to retrieve alternating component (AC) and direct component (DC) images from three raw SIRI images. ..................................................................... 27  Figure 2.4. Schematic of the structured illumination reflectance imaging (SIRI) system. .......................... 29  Figure 2.5. Oriel Instruments QTH lamp housing (left) and radiometric power supply (bottom right) used for the structured illumination reflectance imaging system. ..................................................................... 30  Figure 2.6. CEL5500 digital light projector with an optical fiber input (Digital Light Innovations, Austin, TX, USA) for generating structured illumination patterns. ......................................................................... 31  Figure 2.7. CCD camera with zoom lens used in the structured illumination reflectance imaging system. .................................................................................................................................................................... 32  Figure 2.8. Complete structured illumination reflectance imaging (SIRI) system (front panels of the imaging chamber removed). ....................................................................................................................... 34  Figure 2.9. Positions and diameters of holes drilled into the nylon test material...................................... 36  Figure 2.10. Real image of the Nylon test block used in preliminary tests. ............................................... 37  Figure 2.11. Raw phase-shifted structured illumination reflectance images (at 0o, 120o, and 240o pattern phase shifts) of the Nylon test sample acquired at spatial frequency 0.05 mm-1. ..................................... 41  Figure 2.12. Demodulated SIRI images of the Nylon test sample at spatial frequencies of 0.0 (i.e., direct component or DC), 0.02, 0.04, 0.05, 0.06, 0.08, 0.10 and 0.12 mm-1. ...................................................... 42  Figure 2.13. A pendulum impact device for creating bruises on apples. .................................................... 44  ial frequencies 0, 0.03, 0.05, 0.08, 0.10, 0.15, 0.18, 0.20, 0.25 mm-1. ........................................................... 45  Figure 3.1. Sample height adjustment procedure for acquiring reflectance images from apples under structured illumination. .............................................................................................................................. 50  Figure 3.2. Apple image mask used for the removal of the image background. ........................................ 52 x  Figure 3.3. Flowchart of image processing procedures used to detect bruised regions on demodulated apple images. .............................................................................................................................................. 54  (b) apples at spatial frequencies of 0.10 mm-1, 0.15 mm-1, and 0.25 mm-1. ...................... 55  Figure 3.5. Bruise detection rates (in percent) and 2 standard deviations of the detection rates (whiskers) of 2 replicates of 10 low impact apples at 0-1, 4-6, and 24 hours after initial impact under uniform (0 cycle/mm) and sinusoidal pattern illumination at three spatial frequencies. .......................... 59  Figure 3.6. Bruise detection rates (in percent) ) and 2 standard deviations of the detection rates (whiskers) of 2 replicates of 10 medium impact apples at 0-1, 4-6, and 24 hours after initial impact under uniform (0 cycle/mm) and sinusoidal pattern illumination at three spatial frequencies. .......................... 59  Figure 3.7. Average bruise detection rates (in percent) ) and 2 standard deviations of the detection rates (whiskers) of 2 replicates of 10 high impact apples at 0-1, 4-6, and 24 hours after initial impact under uniform (0 cycle/mm) and sinusoidal pattern illumination at three spatial frequencies. .......................... 60  Figure 3.8. Overall bruise detection rates (in percent) and 2 standard deviations of the detection rates (whiskers) of low impact apples at 0-1, 4-6, and 24 hours after initial impact, where overall detection represents detection rates where at least one spatial frequency of SIRI could successfully detect a bruise. ......................................................................................................................................................... 60  Figure 3.9. Overall bruise detection rates (in percent) and 2 standard deviations of the detection rates (whiskers) of medium impact apples at 0-1, 4-6, and 24 hours after initial impact, where overall detection represents detection rates where at least one spatial frequency of SIRI could successfully detect a bruise. ........................................................................................................................................... 61  Figure 3.10. Overall bruise detection rates (in percent) and 2 standard deviations of the detection rates (whiskers) of high impact apples at 0-1, 4-6, and 24 hours after initial impact, where overall detection represents detection rates where at least one spatial frequency of SIRI could successfully detect a bruise. ......................................................................................................................................................... 61  Figure 3.11. Average differences (in percent) and 2 standard deviations (whiskers) between detected and measured low impact bruise area of two replicates of 10 apple samples. ......................................... 64  Figure 3.12. Average differences (in percent) and 2 standard deviations (whiskers) between detected and measured medium impact bruise area of two replicates of 10 apple samples. .................................. 65  Figure 3.13. Average differences (in percent) and 2 standard deviations (whiskers) between detected and measured high impact bruise area of two replicates of 10 apple samples. ........................................ 65  Figure 4.1. Schematic of the infield sorting machine system (courtesy of Dr. Renfu Lu). .......................... 69  Figure 4.2. Partial view of the infield sorting machine used to transport apples harvested by hand to the onboard sorting system (courtesy of Dr. Renfu Lu). ................................................................................... 70  sorting. ..................... 71 xi  Figure 4.4. Empty bin in which the apple samples were deposited into after running through the infield sorting system.  .................................................................................................................... 73  Figure  .......................................................... 84   .......................................................................................... 85  .......................................................................................... 86  ........................................................................ 87   ...................................................................................... 88  ple samples. ............................................................................. 89   ........................................................................ 90   ..................................... 90   ............................................................................................... 91  Figure 5.10. Demodulated (b) low spatial frequencies. ......................................................................................................................... 97  Figure 5.11. (b) low spatial frequencies. ......................................................................................................................... 98  Figure spatial frequencies. ..................................................................................................................................... 99  Figure low spatial frequencies. ............................................................................................................................ 100  Ffrequencies and (b) low spatial frequencies. ............................................................................................ 101  high spatial frequencies and (b) low spatial frequencies. ........................................................................ 102  (b) low spatial frequencies. ....................................................................................................................... 103  spatial frequencies. ................................................................................................................................... 104  (b) low spatial frequencies. ....................................................................................................................... 105 xii   ...................................................................... 107  Figure 5.20. Comparison of defect contrast in raw (a), division (b), ratio (c) and PC ratio (d) black scab  ................................................................................................................................ 108  Figure 5.21. Comparison of (a) high frequency rot principal component ratio images (spatial frequency 0.40 mm-1) and (b) low frequency images (spatial frequency 0.15 mm-1). ............................................... 109  Figure 5.22. Flow chart of the image processing operations used for surface defect segmentation. ..... 110  Figure 5.23. Resultant images from the image processing procedures (a) and final segmentation results  ................................................................................................................. 112  Figure 5.24. Resultant images from the image processing procedures (a) and final segmentation results  ........................................................................................................ 113  Figure 5.25. Resultant images from the image processing procedures (a) and final segmentation results . ..................................................................................................................... 114  Figure 5.26. Resultant images from the image processing procedures (a) and final segmentation results (b) of rotting region ...................................................................................................... 114  Figure 5.27. Resultant images from the image processing procedures (a) and final segmentation results (b)  ............................................................................................. 115   xiii  KEY TO ABBREVIATIONS  HSI  Hyperspectral Imaging SFDI  Spatial Frequency Domain Imaging NMR  Nuclear Magnetic Resonance CT  Computed Tomography MRI  Magnetic Resonance Imaging VIS-NIR  Visible and Near Infrared ROI  Region of Interest CTIS  Computed Tomography Imaging Spectrometer DC  Direct Component AC  Alternating Component DLP  Digital Light Projector QTH  Quartz Tungsten Halogen CCD  Charged Coupled Device SIRI  Structured Illumination Reflectance Imaging DMD  Digital Mirror Device PCA  Principal Component Analysis PC  Principal Component 2  INTRODUCTION  Food loss and/or waste due to inferior quality or improper postharvest handling causes huge economic loss in the fruit industry annually. The United States Department of Agriculture Economic Research Service reported that 31% of the available retail food supply went uneaten by consumers in 2010. An estimated $30 billion was lost in uneaten produce alone (Buzby et al., 2014). Defects present on produce at the time of sale typically discourage consumers from making a purchase. Visually inferior fruit may still be used in alternative food processes if sorted properly and prevented from reaching the fresh market. Therefore, successful defect detection may play a critical role in reducing food loss. Consumers have expectations of products meeting certain standards of quality when they purchase produce. It is essential that producers meet these expectations in order to promote customer loyalty and improved profitability. Traditionally, fruit quality inspection is done manually by trained workers. While this method can achieve high performance in identifying defective products, it is inconsistent and labor intensive. Therefore, there is a demand for automatic, rapid and nondestructive fruit quality inspection technology in the industry. Current automated methods of quality inspection employed in the industry include machine vision and spectroscopy. Machine vision systems are rapid, easy to implement and can acquire images from the entire surface of a sample, however these systems still struggle with detecting subtle defects which may not be visible on the surface of a sample. Spectroscopy, on the other hand, has demonstrated its capability to determine internal quality attributes of fruit and can also be performed rapidly. However, spectroscopy usually makes point measurements, which may not be suitable for detecting defects that are localized or spatially 3  distributed. Hyperspectral imaging (HSI) is an emerging technology that combines the capabilities of machine vision and spectroscopy. In research, HSI has demonstrated the ability to acquire information from both the surface and the internal tissue of a fruit sample (Ariana & Lu, 2008a; ElMasry et al., 2009; Lu, 2007). HSI has been used in research for quality attribute evaluation of a myriad of fruit, however there are still obstacles preventing the technology from commercial implementation at the present. Primarily, acquiring and processing large volumes of HSI data is too time consuming to meet industrial demands. Additionally, the spectral information acquired may not be easily related to properties buried within the fruit tissue due to unknown depths of light penetration. Research conducted on specific fruit attributes has addressed image processing speed issues by identifying key wavelengths for use in multispectral imaging systems, however the issue of controlling light penetration still remains. A common defect that remains challenging to detect is tissue bruising. Subsurface defects such as bruising may not be detectable on the surface of fruit, especially while the damage is fresh. Shahin et al. (2002) examined bruises aged 1 to 30 days using X-ray imaging and artificial neural network classification and found that new bruises could not be reliably separated using this methodology. Bruise detection may be performed accurately through careful selection of wavelengths. ElMasry et al. (2008) utilized a hyperspectral imaging system to determine key wavelengths at which bruises of all ages may be ss was slow and not suited for online grading. Huang et al. (2015) went beyond HSI wavelength selection and constructed an online le of accurately classifying freshly bruised apples less than 1 hour old at rates of 90.4%, while the samples were stationary, however, performance dropped to 74.6% during online grading. While 4  hyper- and/or multi-spectral imaging techniques hold promise for bruise detection, there remain challenges in implementing the techniques. Wavelength selection alone may not be entirely sufficient for detecting certain defects. Information pertaining to specific defect types, such as bruising, may be further enhanced through the ability to confine light to a specific depth where the defect occurs.  In this research, spatial frequency domain imaging (SFDI) was proposed for enhancing bruise detection due to its ability to control light penetration in biological tissue (Anderson et al., 2007; Bassi et al., 2008; Cuccia et al., 2005; Weber et al., 2009; Weber et al., 2011). SFDI utilizes structured illumination patterns as a light source rather than the uniformly distributed light sources currently implemented in conventional imaging systems. Biomedical research has demonstrated that altering the spatial frequency of the projected illumination patterns allows for control of the depth, to which the incident light penetrates biological tissue (Cuccia et al., 2005). The technique could yield similar results for fruit and other food products as those reported in biomedical research. Much of the research utilizing SFDI has focused on determining the optical properties of imaged samples, and it needs to acquire phase-shifted images for multiple spatial frequencies and use inverse algorithms, which would take long imaging and processing time. Rather than interrogating fruit tissue for determining optical properties, reflectance images of structured illumination patterns may offer a direct, faster and simpler method for detection of depth-specific defects. For our application of SFDI to fruit defect detection, determination of the optical properties of a sample is not the goal. Instead, changes in the reflectance generated under structured illumination due to the turbidity of the tissue and the presence of internal defects are of greater interest. As such, identification of defective tissue 5  would be attempted directly on reflectance images acquired under structured illumination. This technique would be named and referred to as structured illumination reflectance imaging (SIRI) rather than SFDI throughout the experiments performed.  Through selection of appropriate spatial frequencies of illumination, SIRI should provide a method of confining light penetration to specific depths of the fruit. Due to its depth-specific nature, bruising was selected as the main focus of our research as this defect would best demonstrate the advantages that SIRI may possess when compared to conventional, uniform illumination. Obtaining depth-specific information should improve the detection of subsurface bruising. The ability to confine light to the top layers of a sample may also improve upon surface defect detection. Therefore, the detection of surface defects in apple fruit was also explored. Objectives  The goal of utilizing SFDI in defect detection is to take advantage of the ability to control light penetration depth in fruit tissue and to better resolve defect features acquired in subsurface and surface level images. We hypothesize that by selecting specific spatial frequencies of illumination, modulated light can be used to detect subsurface fruit defects, such as tissue bruising, with higher accuracy than uniform illumination. The potential to confine light to the sample surface may also improve detection of surface defects. Implementation of a SIRI system may result in improvements in detecting both internal and surface-level defects present in fruit. The specific objectives of the research were to:  Design and construct an imaging system capable of generating structured illumination patterns and acquiring reflectance images; 6   Observe and evaluate the effects of various lighting patterns and spatial frequencies on light penetration depth in a test material and determine key spatial frequencies that are most suitable for subsurface defect detection;  Apply the imaging system and develop a defect detection algorithm to demonstrate its ability to detect bruised tissue in apple fruit compared to uniform illumination; and  Explore the potential of structured illumination for surface defect detection in apples.   7  CHAPTER 1: LITERATURE REVIEW  1.1 Defect Types  There are many forms of defects, both internal and external, that may take place at the surface of, and within, fruit. These defects arise due to a myriad of factors such as growing environment, insect infestation, post-harvest storage and handling conditions, or form over time. While external defects may be visible and easy to detect under hand inspection, subtle external defects and internal defects are difficult or impossible to see on the surface of a damaged fruit and require special inspection techniques that can interrogate the internal structure of the sample in order to detect them. It is thus crucial to understand how each type of internal defect affects the internal structure of a sample to better identify these imperfections using various quality detection methods. 1.2 Insect Infestation  A common form of surface and internal defects found in fruit is damage caused by insects. Caterpillars, codling moths, aphids and other insects feed upon growing fruit and may leave visible damage on the surface. However, some insects such as codling moth larvae burrow deep into the fruit tissue leaving behind only a small hole. These internal defects may be invisible to an inspection worker, and may fail to be detected by a machine vision system that analyzes only the surface of a sample. The extent of the internal damage caused by burrowing pests can be large and will negatively affect consumer satisfaction to a large extent. Therefore, enhanced external and internal inspection techniques are needed to ensure high fruit quality. 8  1.3 Disease  Fruit are also susceptible to diseases that cause tissue damage. Mold, mildew and rot alter the firmness and texture of infected fruit leaving them soft and watery. If consumed by humans, diseased fruit may cause illness. The symptoms of disease are typically visible on the surface of a sample; large brown spots, soft and wrinkled tissue, and speckles of discoloration are all surface indications of disease. These defects are more easily detectable by the naked eye and surface analyzing techniques such as machine vision; however, if a disease affects the interior of a sample, a more effective inspection technique is required. 1.4 Physiological Disorders  Physiological disorders also negatively impact fruit quality. Poor nutrient intake and growing conditions can lead to various physiological conditions that impact fruit texture and taste. Examples of these disorders include bitter pit, internal breakdown, Braeburn browning, and core flush. These defects originate within the fruit and cannot be seen externally. These defects are commonly overlooked during sorting and packaging because the infected fruit have not yet begun to display external symptoms. Internal quality detection methods are required in order to catch these defects before damaged fruit are shipped to market.  1.5 Bruising  During harvest and postharvest handling and storage, fruit are subject to mechanical stress. Bruising of fruit is a very common defect, and it can occur if harvesting methods are too rough with the collected products or when excessive vibration or impacting force is exerted on the fruit 9  during transportation and postharvest handling. Bruises are typically considered a cosmetic downgrade, but can leave fruit susceptible to rot. Bruises can often be detected on the surface of a fruit and may extend into the tissue. External inspection by hand may be sufficient to detect bruising, however this defect is challenging to detect through automated methods. Apples may form new bruises as they are handled during the sorting process. The discoloration associated with bruising does not form immediately and fresh bruises may go undetected due to the lack of visible, external symptoms. The damage resides beneath the surface of the fruit, therefore methods capable of examining subsurface tissue are most appropriate for detecting bruises. 1.6 Methods of Quality Assessment    Traditionally, food quality inspection is done manually by trained workers. However, this process is time-consuming, laborious, costly, and subject to inconsistencies. Yet many operations presently running still utilize this method (Elmasry et al., 2012). The development and implementation of an automated quality assessment system would greatly reduce the labor, cost, and error associated with human inspection and may increase the efficiency of the entire production line. Nondestructive methods for quality and/or defect detection include impact analysis, sonic vibration, nuclear magnetic resonance (NMR), X-ray, computed tomography and optical techniques (Abbott, 1999). While all methods have demonstrated the ability to determine internal fruit quality, optical techniques currently provide the most rapid and cost effective approach for implementation in commercial online grading systems.   10  1.6.1 Impact Analysis  Impact analysis of fruit involves striking the surface of a sample by an external load and measuring the resulting force-time response from the fruit. If a load is below a certain threshold, the surface will eventually return to its initial state, thus this method may be applied nondestructively. Acco, 1882). Impact analysis is conducted using a probe with a spherical tip made from a material with known physical properties. A sensor measures the maximal force during contact with the sample. Firmness of the sample is then calculated from these measured values. Alternatively, impact analysis can be done by dropping fruit onto a hard surface or by using compressed air. Impact analysis is not sensitive to fruit shape and has been applied to a broad range of fruits such as peaches (Delwiche et al., 1987), tomatoes (De Ketelaere & De Baerdemaeker, 2001), apples (Shmulevich et al., 2003), and kiwifruit (Ragni et al., 2010).  However, this method may not work well in products with high firmness as the deformation after impact will be very small and the contact time is very short, resulting in a low signal to noise ratio. 1.6.2 Acoustic Analysis  It is possible to determine fruit quality by using sonic or acoustic vibrations. Acoustic waves can be transmitted, reflected and refracted as they travel through a material. By analyzing the properties of waves as they travel inside a sample, fruit quality properties can be related to wave properties such as propagation velocity, attenuation and reflection. At a particular frequency a material will vibrate more rigorously, and this frequency is known as the resonance 11  method of interrogating fruit mechanical properties is fast and can also be accurate, however this technique is not as useful for fruit with irregular shapes (non-spherical) (Nicolai et al., 2014). Applications of vibration analysis on fruit have mostly been evaluated for apples (Abbott, 2010), melons (Sun, 2010) and tomatoes (De Ketelaere & De Baerdemaeker, 2001) due to their spherical shapes. Additionally, soft fruit tissue dampens vibrations and lessens the effectiveness of this technique for determining fruit quality. Thus, while sonic vibration is a promising quality assessment method for certain fruit, it is not widely applied mainly because the firmness measured by the technique is not consistently correlated with the standard destructive method and with the sensory evaluation of firmness by humans. 1.6.3 X-ray Imaging and Computed Tomography  X-ray imaging has been applied to determine internal fruit quality. An image is acquired from a sample by transmitting X-ray radiation through it. Different tissues or structures within the sample with a sufficiently different absorption coefficient than its surroundings can be distinguished in the resulting image. This technique is useful for identifying internal defects within fruit samples. X-ray radiography has been used to investigate internal defects such as watercore in apples (Kim & Schatzki, 2010) and other internal disorders in fruit and vegetables (Haff & Toyofuku, 2008). The method is fast and can be implemented on sorting lines.  However, the cost of implementing an X-ray system may be quite high when compared to other optical imaging techniques and additional steps must be taken in order to ensure shielding from X-ray radiation (Nicolai et al., 2014). X-ray inspection also has a social stigma attached to it. Consumers 12  have a negative perception about irradiated foods and their safety for consumption. Although products that have been inspected using X-ray imaging are exposed to a much smaller dose of radiation compared to irradiated foods, consumers are still hesitant to purchase these products. There is also concern with equipment operators being exposed to radiation. Again, the dosage of radiation a worker is exposed to through using this system is very low, however the negative reputation associated with radiation still raises concerns.   Computed tomography (CT) can be used in conjunction with X-ray imaging. CT uses a mathematical algorithm to recreate a 3D image from multiple images of an object taken from different angles. CT systems are used in medicine for nondestructive visualization of tissue. While medical CT scanners are capable of acquiring images in seconds, they are comprised of sophisticated moving components with high equipment costs. Simplified CT systems can be made at reduced costs, however they are still relatively expensive for food and agricultural product inspection. Lammertyn et al. (2003) applied X-ray CT to the detection of internal defects in pears and were able to visualize tissue browning and cavity formation. X-ray CT was also able to detect codling moth feeding tunnels in apples and cherries (Hansen et al., 2005).  Donis-González et al. (2013) used a medical CT scanner for classifying internal chestnut quality and found that the CT imaging system was capable of producing high-resolution and high-contrast images of the internal structure of chestnuts. While image acquisition and reconstruction times for the chestnuts were quite fast (1.8 seconds), they are not rapid enough for implementation in high volume sorting lines. Overall, X-ray and CT imaging are promising methods for internal defect detection, but the social stigma surrounding X-ray radiation, slow image acquisition times and the high cost of implementing these systems prevent them from mass implementation in the 13  food industry. Therefore, a simplified and radiation-free method of visualizing internal produce tissue is needed or preferred to reduce imaging times and ensure consumer satisfaction. 1.6.4 Magnetic Resonance Imaging  Internal defect detection can also be done using magnetic resonance imaging (MRI). MRI is a nondestructive imaging technique that uses wavelengths in the radio-frequency range to interact with fruit tissue. Protons present in the water of a fruit are spun by applying a strong magnetic field to the sample. This results in the ability to monitor the spatial distribution of proton density within a sample, and information about the internal structure of a sample can be visualized (Nicolai et al., 2014). MRI is well suited for biological materials which are rich in proton sources such as fat, oil, salt and water. Therefore the MRI technique can detect defects that are strongly associated with the water content of a sample, such as watercore in apples (Cho et al., 2008), internal browning in apples (Gonzalez et al., 2001), fruit mealiness (Marigheto et al., 2008), and chilling injury (Hernández-Sánchez et al., 2004). Physical properties such as size, shape and volume can also be determined using MRI (Létal et al., 2003). MRI has potential for online grading of fruit, however image acquisition speeds are slow and the cost of implementing a system is high, even higher than that of a CT system (Nicolai et al., 2014). 1.6.5 Machine Vision and Spectroscopy  While all of the previous methods have shown promise for the detection of internal fruit defects, they have issues regarding their speed and cost of implementation, which prevent them from practical implementation in fruit sorting lines in the industry. Methods of fruit quality assessment that are both rapid and cost effective are optical techniques that utilize light in the 14  visible to near-infrared (VIS-NIR) region of the electromagnetic spectrum. These techniques include machine vision and spectroscopy, both of which are currently used in commercial grading lines. Another optical technique that shows great promise for rapid nondestructive quality assessment of fruit is hyperspectral imaging (HSI), which is essentially a combination of machine vision and spectroscopy.  A machine (or computer) vision system consists of a digital camera connected to a computer with software for image acquisition and processing. Images acquired by a machine vision system display the intensity of colors (red, green, and blue), in the case of color imaging, or greyscale intensity in the configuration of broadband or monochromatic imaging, across certain regions of the imaged sample. Image processing algorithms are then applied to the images in order to classify each region of the fruit based on the corresponding color or greyscale intensities. This is typically accomplished by setting a threshold on the intensity values of one of the wavebands. Imaging systems are capable of examining the entire surface of a sample and external properties such as size, shape, color, surface texture, and external defects can be easily determined. As such, machine vision systems capable of sorting produce based on these external features are readily available from major sorting line manufacturers (Nicolai et al., 2014). However, chemical properties and internal defects of a product are generally undetectable by machine vision methods, because they are usually wavelength dependent and their detection would require spectral information or information at specific wavelengths.   Spectroscopy offers the ability to nondestructively investigate the internal and chemical properties of a product by acquiring spectral information about a region of interest (ROI) on a sample. Fruits and vegetables are opaque objects in the VIS-NIR spectrum. When light interacts 15  with these objects, photons are absorbed and scattered by the sample. The physical structures and compositions of the sample tissue are responsible for the scattering of light, while absorption is mainly dependent on the chemical components present in a sample. Generally, there are fewer features present in the scattering spectra, while the acquired absorption spectra may have different peaks that are related to the amount of a specific chemical constituent present in a sample. NIR spectroscopy has been successfully applied for the nondestructive measurement of soluble solids content of apples (Lammertyn et al., 1998), cherries (Lu, 2001), sugar beets (Pan et al., 2015), kiwifruit (McGlone & Kawano, 1998), and various other fruits. Spectroscopy focuses on a relatively small portion of the product such that the region inspected may not be representative of the whole sample. Because spectroscopy does not directly measure specific chemical components or structural features, it has to rely on the establishment of a calibration model that relates to the reference measurements by other standard methods, for predicting a specific chemical composition or quality attribute. Furthermore, spectroscopy does not provide spatial information about a sample and the spectral information acquired may not be pinpointed to a specific region of the sample.  1.6.6 Hyperspectral Imaging   Hyperspectral imaging provides spatial information as regular imaging systems do, as well as spectral information for each pixel in the image acquired. Through the use of the principles behind both computer imaging and spectroscopy, a hyperspectral imaging (HSI) system could simultaneously (Kim et al., 2011). The potential that hyperspectral imaging has for 16  nondestructively determining food quality has led to significant interest in developing the technology for implementation in production lines over the past decade. Hyperspectral imaging acquires an enormous amount of information from a sample in comparison to machine vision, however not all of the information produced is useful. In addition, the image acquisition times needed for a hyperspectral imaging system are currently too time consuming for rapid online assessment. Hence, much research has focused on reducing the image dimensionality and identifying important features and optimal wavebands so that the technique can be implemented using less elaborate systems (e.g., multispectral imaging systems) (Lu et al., 2011; Pan et al., 2015; Xing et al., 2008). Developing techniques to quickly analyze and extract information from hyperspectral imaging is currently the greatest hurdle keeping the technology from commercial implementation in the food industry; however, if efficient image processing techniques and key wavelengths of interest can be identified for the imaging systems, the technology would be unrivaled in on-line food quality detection (Feng & Sun, 2012). In addition to the high dimensionality of the data acquired by HSI, the inability of controlling light penetration depths in samples also exist with HSI systems as that with machine vision or spectroscopy systems. The popularly used reflectance imaging mode restricts the data acquired by the HSI system to the surface of a sample. Although a new sensing mode of combining reflectance and transmittance has been proposed for detecting both external and internal quality attributes of pickling cucumbers (Ariana & Lu, 2008a, 2008b), it is challenging to implement this sensing mode for online quality inspection of other fruits and vegetables. In order to ensure that fruit quality is acceptable throughout the entirety of a sample, a method for controlled light penetration must be considered.  17  1.7 Spatial Frequency Domain Imaging   Light penetration depth in fruit varies depending on the wavelength of light used for illumination. Research on determining fruit firmness and soluble solids content which are related to internal structure and composition has showed that certain wavelengths of light are more effective at penetrating samples than others (Osborne et al., 1993; Park et al., 2003). However, these wavelengths may not penetrate deeply enough into a sample in order to gather information from the core. While certain wavelengths of light are able to pass through a sample completely, the signal captured by the imaging system of such wavelengths are typically weak. Increasing the intensity of the illumination source may be a solution to the light penetration problem, but it could cause damage to some samples. Furthermore, many internal defects are spatially distributed or localized in samples and, therefore, are difficult to detect using conventional machine vision, spectroscopy or HSI techniques, because biological materials like fruit are highly scattering media over the visible or near-infrared region. A method of imaging or mapping the subsurface or internal structural or chemical properties of biological tissue using the visible or near-infrared light has been studied in the field of biomedical imaging, which can control light penetration depth in the tissue by utilizing structured illumination. Spatial frequency domain imaging (SFDI) utilizes projections of structured illumination as opposed to uniform light currently used in machine vision or HSI systems.  Biological tissues such as skin, brain and breast tissue have been imaged using SFDI for quantitatively mapping absorption and reduced scattering coefficients in order to detect abnormalities or inhomogeneities present in tissue. Bassi et al. (2009) utilized spatially 18  modulated light for the detection of inhomogeneities in diffusive media. The study revealed that inhomogeneities present in phantom tissue samples distorted the amplitude and phase of spatially modulated light. Detecting the change in phase allowed for accurate localization of the inhomogeneities. Weber et al. (2011) estimated tissue absorption and scattering coefficients using SFDI and a computed-tomography imaging spectrometer (CTIS). Hyperspectral images of tissue phantoms were captured using the CTIS. Two spatial frequencies 0 mm-1 (uniform illumination) and 0.29 mm-1 (or 3.45 mm/period) were used in imaging. Two spatial frequency projections were needed in order to separate absorption from scattering, typically one spatial frequency is zero and the other is nonzero. Weber et al. (2011) reported that low spatial frequencies are most sensitive to changes in absorption while higher frequencies have sensitivity to scattering changes, but greatly reduced sensitivity to absorption. Cuccia et al. (2005) explored the effects of spatial frequency on light penetration depth on Siloxane with heterogeneities embedded at varying depths.  Modulation images at 42 spatial frequencies ranging from 0 to 0.63 mm-1 revealed that as the spatial frequency of illumination increased, the deeply embedded object became decreasingly apparent, until only the superficial object was visible. The depth at which light penetrates the tissue was controlled by varying the spatial frequency of illumination. However, depth-specific information may not be extracted directly from the patterned reflectance images acquired using the technique. The information encoded in the raw structured illumination images is interwoven into two components, an alternating component (AC) and direct component (DC); and the depth-specific information is contained within the AC, while the average intensity response of the sample, similar to that of a uniform illumination image, is contained in the DC. In order to separate these two components an image demodulation step 19  needs to be performed, which requires three phase-shifted images to be taken (Bassi et al., 2008).  Sinusoidal illumination patterns are the most commonly used form of structured lighting due to simplified extraction of optical properties using a three-phase demodulation equation.  Similar results may be expected, if SFDI were applied to fruit tissue. Based on previous results, defects that lie beneath the skin of a fruit such as bruised tissue may become more apparent under structured illumination than in conventional imaging methods. Conclusions drawn from the previous studies also seem to suggest that surface features may also be enhanced through the use of SFDI at high spatial frequencies.  To our knowledge, Anderson et al. (2007) were the first to apply SFDI to the detection of defects in agricultural products. In their stored in a refrigerator for 25 hours for the bruises to develop. Reflectance measurements were made with an SFDI system at 32 wavelengths between 650 and 980 nm at nine equally-spaced spatial frequencies from 0.0149 to 0.1344 mm-1. Absorption and scattering coefficients of the bruised regions were recovered through demodulating the structured reflectance images and using a light propagation model. This methodology yielded scattering coefficients that were clearly distinguishable between bruised and non-bruised regions on a sample. Distinctions could also be made between varying levels of bruise severity. A disadvantage to the technique used in the study was that images took a long time (6 minutes per imaged area) to acquire. This is far too slow for industrial applications, however image acquisition times can be shortened by reducing the number of wavelengths and spatial frequencies used in imaging. In the previous study, SFDI was used to determine the modulation transfer function of the apples and the effects that the spatial frequency of illumination had on light penetration in apple tissue was not discussed. This 20  experiment is one of the only studies that has applied SFDI to quality assessment of agricultural goods.  Ariana and Lu (2010) have demonstrated the ability to identify defective cucumber tissue through the use of laser scattering images without the need of recovering the optical properties of each fruit. That study has demonstrated that internal defect detection can be done directly on scattering images of samples rather than through determining their optical properties. A similar approach may be used in determining the presence of internal defects using SFDI reflectance images, where defects may be identified without the need of using an inverse algorithm for a light transfer model, thus simplifying and potentially speeding up the defect identification process. The theoretical promise that SFDI possesses for detection of internal tissue seemed to be ideal for the detection of depth-specific defects in fresh fruits and vegetables. One specific defect that remains a challenge for conventional, uniform illumination imaging techniques is tissue bruising in apple fruit. Point measurement techniques such as spectroscopy may not necessarily take measurements from the bruised area. Fresh bruises in particular are difficult to detect as the bruised regions have not yet developed an obvious discoloration at the surface. Thus, machine vision and other surface inspection methods struggle with identifying fresh bruises due to the lack of contrast between healthy and damaged tissue. HSI has demonstrated potential in detecting bruises. However, image acquisition speed and processing are slow, and the age of the bruise has been shown to affect detection performance of HSI (Lu, 2003). The aforementioned quality assessment methods all utilize uniformly distributed illumination sources to acquire images. The resulting data may contain information attributed to tissue regions not necessarily associated with the bruise. The ability of structured illumination imaging to confine light 21  penetration to specific depths may provide a unique advantage in isolating the acquired information to subsurface bruised tissue. Obtaining more pertinent information related to the bruises may improve detection. The specific defect of tissue bruising may best highlight the advantages that structured illumination techniques provide when compared to uniform illumination methods. Therefore, the detection of subsurface bruising in apple fruit was selected as the main focus of this study due to its depth-specific nature and the challenges it presents to conventional imaging techniques currently used in the industry. Confining illumination to the surface of a sample may also improve upon surface defect detection as SFDI has demonstrated the ability to enhance image feature resolution at higher spatial frequencies (Neil et al., 1997). Image acquisition under structured illumination may hold potential for detection a variety of defects present in fresh produce, however this technique has rarely been applied in the field of agricultural quality assessment. The application of a newly constructed structured illumination reflectance imaging (SIRI) system to the detection of bruises and surface defects in apple fruit was investigated and the results of those exploratory studies are presented in this work.    22  CHAPTER 2: CONSTRUCTION AND PRELIMINARY EXPERIMENTS OF A STRUCTURED ILLUMINATION REFLECTANCE IMAGING (SIRI) SYSTEM  2.1 Introduction and Theory  The SFDI technique applied in biomedical research offers the ability to control the depth at which light penetrates biological tissue through utilizing structured illumination patterns. Structured illumination differs from many conventional imaging techniques which utilize uniformly distributed light sources. Figure 2.1 provides a comparison between the conventional imaging systems such as machine vision and hyperspectral imaging and a structured illumination imaging system.     As mentioned in chapter 1, the spatial frequency of illumination determines the depth the incident light reaches. The spatial frequency of a pattern refers to the number of projected pattern periods per millimeter in the direction of light modulation of that illumination pattern. Anderson et al. (2007) 23  However their method of identifying bruised regions required recovery of the optical properties of the fruit tissue. Rather than applying time consuming computations using inverse algorithms, we propose a more direct method of bruise detection through image processing on reflectance images acquired under the same structured illumination. Thus, rather than referring to our method as SFDI, we will utilize the term structured illumination reflectance imaging (SIRI) instead. Before moving on to describing the SIRI system constructed, the theory behind structured illumination and its ability to control light penetration needs to be discussed.  Weber et al. (2009) provided a derivation of the relationship between the spatial frequency of illumination and its depth of penetration in a two-layer, turbid material. The model describing a spatially modulated planar wave and depth of light penetration is based on the time independent, steady-state, diffusion equation (1)  (1) where  ,  is the reduced scattering coefficient,  is the optical absorption coefficient,  is the fluence rate and q is the source term.  To adapt this model for a sinusoidal light source, a sinusoidal source function, S, as defined in Eq. (2), is substituted for q in Eq. (1)   (2) 24  where is the spatial frequency and is the spatial phase of the source. When observed by a camera, the fluence rate varies only with depth (z), therefore the diffusion equation reduces to a one dimensional expression with substitution of S for q,   (3) Here we define where  is the light attenuation coefficient which is the inverse of the average depth of light penetration and is dependent on both wavelength and spatial frequency. Solving for  we arrive at   (4) It is shown in Eq. (4) that the light penetration depth is inversely related to the spatial frequency of the source; lower spatial frequencies penetrate deeper into turbid media than higher frequencies. The main mechanism by which spatial frequency affects light penetration is that the intensity of a structured light pattern attenuates at different rates within a turbid medium attenuates and can no longer penetrate into the sample is where the light is confined to. Low spatial frequencies attenuate at a lower rate than high spatial frequencies. Therefore, low spatial frequencies are able to penetrate more deeply into a turbid material. Figure 2.2 provides a visual comparison of the amplitude attenuation and light penetration depths of sinusoidal patterns with different spatial frequencies in tissue. 25      Neil et al. (1997) theorized that a microscope can only efficiently image portions of an object within focus. By projecting structured illumination upon a turbid sample, the resulting image captured only portions of the sample which were in focus. The rate of attenuation with defocus of the projected pattern on the sample surface depended upon the spatial frequency of illumination. Therefore, the spatial frequency of illumination may also enhance the ability to spatially resolve resulting reflectance images resulting in cleaner, in-focus images. This enhanced ability to resolve object features will be referred to as a simulated increase in image resolution throughout this work. Bassi et al. (2008) demonstrated this effect for turbid phantom samples on the macroscopic scale using a CCD camera and sinusoidal illumination patterns. It was demonstrated that higher spatial frequencies were able to significantly increase the spatial resolution and contrast of inhomogeneities within the phantom than lower frequencies and uniform illumination. 26   The depth-specific information contained within raw reflectance images acquired under structured illumination cannot be directly retrieved from a single SIRI image. The raw SIRI images contain information encoded in two components, an alternating component (AC) and direct component (DC). The resulting intensities in the raw SIRI image is a sum of the DC and AC responses. The DC is reflectance image representing the average intensity of the sinusoidal illumination pattern used in image acquisition and is analogous to a conventional, uniform illumination image. The AC image contains the measurement of the amplitude of reflected intensity of the sinusoidal modulated pattern, which attenuates at varying rates depending on the spatial frequency of the pattern. Thus, the AC image contains the depth specific information related to spatial frequency of illumination used for image acquisition. The separation of the AC and DC is aSIRI image processing. In order to retrieve the AC and DC images from a SIRI image, a three-phase demodulation process is required where three sinusoidal fringe pattern projections are recorded per spatial frequency. For each of the three projections, the sinusoidal pattern is phase-shifted 120o from the last. The AC and DC images are recovered using the following equations  (5)  (6) where ,  and  each represent one of the three phase-shifted reflectance images acquired at each spatial frequency (Bassi et al., 2008). Figure 2.3 provides a visual aid of retrieving the AC and DC images using the three-phase demodulation process.  27      In order to verify the relationship between fringe pattern spatial frequency and light penetration depth as well as to assess the potential of this technique for defect detection in agricultural products, preliminary steps were needed. Firstly, a SIRI system capable of acquiring structured reflectance images would be required. Following that, the SIRI system needed to be tested on a controlled test material to observe and verify the relationship between light penetration depth, spatial resolution and spatial frequency. 28  2.2 Major Components for Generating Structured Illumination   The SIRI system we wished to implement would function similarly to a conventional machine vision system. The sample would be illuminated by a light source and a camera would then acquire images of the illuminated scene followed by image processing on a computer. The key difference between the SIRI and a conventional machine vision system is the need for structured light in SIRI. Therefore, a special component is needed in order to physically generate light patterns. In our constructed system, a digital light projector (DLP) was used to create and project structured illumination patterns onto the samples. The other key component used would be: a quartz tungsten halogen (QTH) light source, a charge-coupled device (CCD) camera and a computer. This imaging system would be setup to acquire reflectance images similar to how conventional machine vision systems operate therefore the technique employed by this imaging system will be referred to as structured illumination reflectance imaging (SIRI). The schematic of the SIRI system is displayed in figure 2.4.  29     A 250-W quartz-tungsten halogen (QTH) lamp (Ushio Inc., Japan) with an output in the VIS-NIR wave range was used as the illumination source in the system. The structure (Oriel Instruments, Stratford, CT,  USA) that the QTH lamp was housed as shown in figure 2.5 allowed for the attachment of light filters for wavelength selection if needed. The light generated from the QTH would need to be guided via optical fiber from the housing to its destination. The QTH was operated through a radiometric power supply console (Oriel Instruments, Stratford, CT, USA) which controlled the intensity output of the light source. The power supply was able to maintain a set level of power to generate a stable light output. The indicator on the power supply displayed the power output of the equipment in real time. Steady state output was achieved when the 30  power supply displayed the desired power output the equipment was set to and no longer fluctuated.     Sinusoidal illumination patterns were generated using a DLP. The key component present in DLPs that redirect light into the desired configuration is the digital mirror device (DMD). Any DLP equipped with a DMD would be sufficient for generating structured illumination patterns, however most commercial models utilize a built-in LED as the device light source. For future studies the ability to implement any custom light source was desired, therefore a DLP capable of light fiber input was needed. The DLi CEL5500-Fiber (Digital Light Innovations, Austin, TX, USA) DLP shown in figure 2.6 was selected as the specific DLP to be implemented in the SIRI system because it was designed with an optical fiber input component. With the CEL5500, a fiber optic 31  cable could be used to channel light from the QTH into the DLP. Software bundled with the DLP was used to load and control illumination patterns into the DLP memory for projection. The software was capable of creating timelines for displaying a sequence of pre-loaded patterns at variable time intervals as well as individually in a slide show fashion. The latter method was used to display illumination patterns while collecting images to ensure that each pattern projected had an appropriate exposure time under the camera.     A 12-bit CCD camera (Sensicam QE, Cooke Corporation, Auburn Hills, MI, USA) was used to acquire grayscale reflectance images from the illuminated samples. Compatible software was installed on the computer to control the CCD camera. The exposure time, image size, resolution and image acquisition were all controlled via the software. A zoom lens (Navitar TV Zoom 7000, Navitar Inc., Rochester, NY, USA) was attached to the CCD to acquire images in focus and to reduce distortion caused by the native camera lens. Crossed linear polarizers were attached to 32  both the lens of the DLP and the zoom lens (Fig. 2.3) to prevent CCD saturation caused by the specular reflection of light on the surface of the samples. Figure 2.7 presents the camera as well as the zoom lens used in the SIRI system. The CCD was equipped with a cooler to mitigate thermal effects on image collection. A warm-up period was needed prior to image collection in order to allow the CCD cooling system to reach a desired temperature. Indicator lights on the CCD notified the user when the equipment was ready for image collection.    The camera and DLP were both controlled via the bundled software using a desktop computer (Dell Precision T7610, Dell Inc., Round Rock, TX, USA). Sinusoidal bitmaps of structured illumination patterns to be projected through the DLP were created in MATLAB (The Mathworks Inc., Natick, MA, USA).  The system was constructed inside of a light proof enclosure of plastic panels lined with black felt to prevent stray light from entering the scene. In order to acquire images in focus, both the DLP and camera were attached to adjustable fixtures so that the positions of each component could be adjusted separately to capture high quality images. The 33  camera was attached to an easel capable of making horizontal adjustments in position and the easel was fixed to a motorized stand capable of vertical movement. The DLP was attached to a also constructed using a similar vertically adjustable component. Figure 2.8 shows the entire SIRI system constructed from the components previously listed. The front panels of the enclosure have been removed to showcase the system, otherwise they are normally in place to isolate the system from the outside environment. 34    35  2.3 Preliminary Study on a Synthetic Test Sample   Due to the lack of results reported on the use of structured illumination techniques in agricultural products, there is a knowledge gap in applying this technology for rapid defect detection in fruits and vegetables. The constructed system would need to be tested to observe any potential it may have in identifying embedded defects in turbid media such as fruit tissue. Therefore, a preliminary study was conducted to confirm the general effect of spatial frequency on light penetration depth and resolution of internal defects. The preliminary study sought to replicate the results obtained by Cuccia and Bevilacqua (2008), where lower spatial frequencies provided better light penetration in a turbid medium. The tasks to be completed in this early work were: 1) create a synthetic sample that simulates defects present in an otherwise homogeneous material; and 2) observe the effects of spatial frequency on resolving the simulated defects beneath the surface of the test material using the SIRI system. The results obtained in these initial images would later be applied to attempt to identify real defective tissue by applying the SIRI system using promising spatial frequencies on bruised apples. The outcomes of this work would reveal the defect types that the SIRI system would best identify as well as reveal the key spatial frequency range to do so. 2.3.1 A Synthetic Test Sample  Prior to acquiring images in real fruit samples, the system was tested on a synthetic control material in order to observe the general effect that structured illumination had on light penetration. The material used to create the synthetic sample was cut from a slab of highly scattering nylon. The dimensions of the cut sample were 100 mm x 100 mm x 336   cylindrical holes with varying diameters were bored into the material normal to one of the 100 mm x 30 mm faces of the sample. In order to attempt to resolve defects at various depths within a diffuse material, the centers of the holes were placed at various depths below one of the adjacent 100 mm x 100 mm faces. Both defect size and depth were considered as factors which may have influenced defect detection. Therefore, holes drilled into the nylon block were made in various sizes.  Figure 2.9 provides the relative positions and diameters of each hole drilled into the test material. The holes were filled with a 1,000 times diluted mixture of water and light absorbing India ink (Higgins) and sealed to simulate defective tissue. The completed sample (minus the ink) is displayed in figure 2.10.   37     2.3.2 System Setup  Prior to taking images with the SIRI system, adjustments to the instrument setup needed to be made in order to ensure high quality images could be collected. The DLP was set to a height of 250 mm from the sample imaging stage in order to ensure that the entire surface of the sample would be illuminated. The camera was angled 15o from vertical to capture the full sampling area. The distance from the stage to camera zoom lens was set to 400 mm where images acquired by the camera were in focus.    38  2.3.3 Creation of Structured Illumination Patterns  The sinusoidal illumination patterns to be utilized in the SIRI system were generated in MATLAB (The Mathworks, Inc., Natick, MA, USA). Prior to collecting images, bitmap images of the illumination patterns were imported into the DLP memory. In order to create sinusoidal intensity patterns, the bitmaps created in MATLAB needed to be in 8-bit resolution as this was the only input accepted by the DLP. In order to observe the general effects that spatial frequency of structured illumination had on light penetration, a wide range of spatial frequency patterns were created. The spatial frequencies ranged from 0.01 mm-1 to 0.60 mm-1. While it was possible to create higher frequencies in MATLAB, the resolution of the projected patterns deteriorated in acquired images at spatial frequencies around 0.40 mm-1. The lens of the DLP was not able to project very high frequencies clearly which introduced noise and negatively impacted image quality. In addition to the projector lens, the camera lens also caused slight distortion in capturing the real world scene. In order to ensure that the acquired images had high resolutions, the spatial frequencies used in this preliminary study were limited to a maximum of 0.30 mm-1. 2.3.4 Image Acquisition  The nylon test sample was placed on the imaging stage of the constructed SIRI system directly beneath the DLP. The height of the sample stage was adjusted such that the projected patterns occupied an area of 10 cm x 7 cm at the surface of the sample where the intensity of the patterns oscillated along the 10 cm side. This was done to ensure that the intended spatial frequencies of the generated sinusoidal patterns were maintained and undistorted by the projection optics. Prior to collecting images, the camera was turned on and allowed to warm up 39  to a steady state temperature. The warmup time was crucial for keeping dark current noise, caused by thermal effects on the sensing element of the CCD, to a minimum in acquired images. Indication of steady-state was provided by the CCD when the sensors within the instrument determine that a steady-state temperature has been achieved by the cooling system. The QTH light source output was controlled by a 300 W radiometric power supply (Oriel Instruments, Stratford, CT, USA). The intensity of the light source was set to a steady output of 150 W in this experiment. The light emanating from the QTH was unfiltered and travelled from the light source allowed us to load the bitmap patterns created in MATLAB into the DLP memory and setup a specific order to display the patterns. Resolving the depth specific information contained in the structured reflectance images required three phase-shifted images in order to perform image demodulation. Therefore, for each spatial frequency used in the study, three separate 120o phase-shifted images were created. Once the camera had reached a steady state response, the PCOCam imaging software (PCO, Kelheim, Germany) was initiated. The program was able to display a live signal streaming from the camera and adjustments to exposure time and viewing area could be made. The exposure time used in this preliminary study was determined using an auto-exposure setting available in the software. The auto-exposure setting would adjust the exposure time for each image acquired such that the resulting signal was not saturated. The full viewing area capability of the camera was used to capture images. In addition to exposure time, the intensity of the response signal captured by the camera could also be controlled through adjusting the aperture of its zoom lens. A Spectralon panel (Labsphere, Inc., North Sutton, NH, USA) with a 98% reflectance rate was used to determine the aperture setting which would 40  acquire a high signal to noise ratio without saturating the CCD. The final step prior to collecting reflectance images of the sample were to gather reference images of the Spectralon panel and the dark signal response. While the intensity of the bitmaps created in MATLAB follows a consistent pattern, in the real scene created by the DLP the actual intensity output of the DLP may not be completely consistent with the input pattern, therefore causing the projected patterns to be something other than sinusoidal. The white and dark calibration images were to be used for correcting any non-uniformities that may originate from the projection optics such that the resulting SIRI images maintained a sinusoidal intensity distribution. Twenty white reference images were acquired using a uniformly distributed projection pattern on the Spectralon panel. And 20 dark reference images were acquired after cutting off the light source from the DLP. Following reference image acquisitions, reflectance images of the nylon test sample were collected. The patterns displayed by the DLP were controlled manually through the software. One reflectance image was captured by the camera per display pattern. Collected images were stored on the computer connected to the SIRI system. The raw, phase-shifted reflectance images appeared as they do in figure 2.11. Once images of every spatial frequency and their respective phases were acquired, the depth-specific information contained in the raw images were extracted through an image processing procedure.  41     2.3.5 Image Demodulation and Processing   Raw reflectance images were imported into MATLAB for preprocessing and demodulation. Preprocessing was performed prior to demodulation on the raw images. A region of interest (ROI) was selected to remove most of the background present in the raw images. A Gaussian filter was applied to the ROIs to remove hot pixels and high frequency noise. Once each raw image had been cropped and filtered, the same demodulation process used by Bassi et al. (2008) was followed using equations (5) and (6) to recover the AC and DC images from each spatial frequency tested. Figure 2.12 displays the resulting DC and AC images recovered from each spatial frequency of structured illumination used in the experiment.  42     The DC image represents the average intensity response of the sample under structured illumination and thus are very similar to uniform illumination images. Since the amplitude of each frequency pattern projected onto the surface of the sample was equivalent, the DC recovered from each spatial frequency are nearly identical. The ink-filled holes absorbed the incident light and contrasted strongly against the nylon material. All three defects are visible in the DC image, however the two objects that were drilled in close proximity to one another on the left side of the sample seem to blend into a single wide object. The light was able to penetrate deeply enough into the test material to reach these defects, however the DC signal was not able to resolve each object individually. As the spatial frequency increases, the center object, which was drilled the deepest beneath the sample surface, starts to fade as the light penetration decreased with increasing spatial frequency. However, objects closer to the surface of the sample are highly resolved. At the spatial frequency of 0.06 mm-1, the two holes drilled on the left side of the 43  sample were clearly distinguishable, while the deep center hole faded considerably. When the spatial frequency further increased to 0.12 mm-1, the light was confined mostly to the surface of the sample and the defects were no longer clearly visible. Instead, the surface texture of the nylon block was resolved. The general effect of spatial frequency on light penetration was clearly demonstrated in these test images. The ink in the nylon samples possessed dramatically different optical properties than the surrounding nylon, which caused sharp contrast between the two materials in the resulting SIRI images. Defects such as tissue bruising should also create regions of contrasting optical properties in real fruit. It was hypothesized that subsurface tissue bruising in apple fruit would be suitable for enhanced detection under SIRI.  The next step in this preliminary study was to apply SIRI to real apple samples in order to observe the potential of structured illumination for detecting bruises in apple fruit. 2.4 Preliminary Study on Apple Samples   imaging. The two different colored apple varieties were selected in order to test the assumption that the color of the fruit skin ty to tissue optical properties. Bruises were inflicted upon the apples using the impact pendulum shown in figure 2.13. The swinging arm of the pendulum consisted of a 483 mm long steel rod, where one end was fixed to a pivot and the other had a wood sphere 74.20 mm in diameter and weighing 161.65 g. The same pendulum was used in a previous bruising study with modifications made to the impacting end (Lu, 2003). The sphere was raised and dropped from a height such that an impact energy of 1.0 J was applied to the surface of each apple sample.   44     SIRI images were acquired immediately following impact under the same spatial frequencies used on the nylon test material. Once all raw images were acquired, ROI selection and image filtering were performed before image demodulation. Figure 2.14 displays the resulting DC and AC images acquired across a wide range of spatial frequencies for both apples.  Once again, the DC images represent what would be seen under uniform illumination where bruises were invisible. Bruised regions became more apparent as the spatial frequency increased. The lower frequencies of 0.03 and 0.05 mm-1 were not able to resolve the bruised regions with high contrast, suggesting that these spatial frequencies are too low to acquire depth-specific information pertaining to bruises in apples. The bruises display the highest level of visible contrast at the spatial frequency range of 0.10 to 0.25 mm-1. At frequencies greater than 0.15 45  mm-1 the surface features of the fruit begin to appear in the AC images due to the light penetration being confined closer to the surface of each sample. While the higher spatial frequency images provide a greater level of contrast between bruised and intact tissue, the appearance of surface features such as skin texture and fruit lenticels introduce noise to the image. Therefore, these features may complicate bruise detection attempts. Based on these preliminary results, three key spatial frequencies, i.e., 0.10, 0.15 and 0.25 mm-1, were identified,     The results of this preliminary study, along with the one for the synthetic sample, demonstrated that SIRI has potential in detecting subsurface tissue bruising in apple fruit. In order to more extensively study the ability of the SIRI system in detecting bruises of various severities and ages, a full experiment exploring bruise detection was designed, which is described in Chapters 3 and 4. 46  CHAPTER 3: BRUISE DETECTION EXPERIMENT ON  AND  APPLES    The preliminary study conducted on the nylon test material and two apple samples fruit. The spatial frequencies between 0.10 and 0.25 mm-1 were able to detect bruises immediately following impact, whereas in uniform illumination images, the bruised regions were invisible. Fresh bruises are one specific type of defect that conventional machine vision systems demonstrating its performance in fresh bruise detection over conventional uniform illumination systems. An extensive study on bruise detection using the SIRI sfactors such as apple variety, bruise severity and bruise age were studied to determine their effects on SIRI performance. Uniform illumination images were also acquired for comparison of bruise detection performance. A bruise detection algorithm was developed for the automatic detection of bruised regions in both SIRI and uniform illumination images, where the algorithm would attempt to highlight the bruised regions in each image. Based on the resulting algorithm regions returned, a successful detection rate was determined under each spatial frequency used in the experiment. These detection rates under structured illumination and their comparison to uniform illumination rates would demonstrate the enhanced ability to detect fresh bruises under the SIRI system.    47  3.1 SIRI System Adjustments  The constructed SIRI system was left unmodified from the previous preliminary study for this bruise study. The same 10 cm x 7 cm projection area was maintained throughout the entire experiment in order to preserve the spatial frequencies being projected onto the samples used previously. Based on the preliminary results, three spatial frequencies were selected to collect SIRI images under: 0.10, 0.15 and 0.25 mm-1. These three spatial frequencies effectively covered the range from where fresh bruises became just visible under SIRI to where surface features of the fruit begin to appear. Three phase-shifted sinusoidal patterns were loaded into the DLP memory per spatial frequency with an additional uniform pattern. Therefore, a total of 10 projection patterns were used per sample in the experiment.  The power supply settings for the QTH light source were set to a constant 150 W output, identical to the settings used in the previous study. 3.2 Apple Sample Preparation   The two varieties were chosen based on their fruit skin had a lighter skin tone which contrasted more visibly with developed bruised tissue, while skin colors were selected to better assess the capability of the SIRI system for detecting subsurface damage regardless of fruit skin color. The apples used in the experiment were hand harvested at the horticultural research farm south of the Michigan State University campus in Holt, Michigan during the normal harvest season of 2014. Sixty apples of each variety were selected for a total 48  of 120 apples. Harvested apples were carefully inspected by hand prior to imaging to ensure that no visible pre-existing bruises or surface defects were present on the fruit. The 60 apples of each variety were separated into three groups of 20, where each individual group would be subjected to one of three different levels of impact energy, i.e., low (0.16 J), medium (0.33 J) and high (1.11 J), to generate three different severities of bruising in each group of apples. The same swinging arm pendulum used in the preliminary study was used to apply impact forces on the apples. The amount of impact energy applied to an apple was to be controlled by varying the height at which the ball end was lifted prior to dropping. Apples were placed into cold storage for 48 hours at 40C (400F) before starting the experiment.  The fruit were removed from storage and left out to equilibrate to room temperature (~24 0C) before applying impacts to generate bruising.  The samples were kept organized in cardboard cup trays which were marked with sample numbers as well as the impact type to be applied to that group.  3.3 Image Acquisition   White and dark reference images were collected following the exact same procedure presented in the previous study in order to correct for non-uniformities in pattern projection. was adjusted so that the intensity response from the Spectralon panel was just below saturation to ensure that a high signal to noise ratio would be acquired from the apple samples. Following the acquisition of reference images, the first apple sample to be imaged was impacted and then immediately placed onto the imaging stage for image collection. The height of the sample stage was adjusted such that the projection area of the illumination patterns would cover an area of 10 x 7 cm. A simple, fixed height marker constructed out of metal rods was used to adjust the 49  stage to the appropriate height between apple samples as shown in figure 3.1. Nine SIRI images and one uniform illumination image were acquired for a total of 10 images collected per sample. The images were saved with a sample number and phase number to keep track of the specific details of each image acquired. Following image collection, the sample would be returned to the cup tray and the next sample would be impacted for imaging. Bruise development is not an instantaneous process as the discoloration of the damaged tissue forms over time. While the SIRI technique in the preliminary study had demonstrated the capability of detecting bruises ability to detect them. Therefore, each group of 20 apples were imaged multiple times over three time intervals after bruising: 0-1 hours, 4-6 hours and 24 hours. Following the collection of images 4-6 hours after impact, the apples were covered with a plastic bag to prevent moisture loss and left at room temperature overnight. On the following day after 24 hours had passed since initial impact, the final set of images were collected. The experiment spanned 4 days in total. On the first day, the 0-1 hour and 4-6 hour images sets were acquired for medium and high impact groups of both varieties and the remaining 24 hour image set was acquired on the following day. Image collection for the light impact group started following the collection of the 24 hour image set of the previous two groups. Immediately following the acquisition of images 24 hours after impact, the diameters of the bruises that had developed on each set of apples were physically measured by hand using a digital caliper. These values were recorded for future use in comparing the area of bruised region detected by an automatic detection algorithm to the true areas recorded by hand. 50     3.4 Image Organization   The images from the experiment were saved into folders separating each group by impact force and variety. To simplify the demodulation process, the images were reorganized by phase for easier access. The images in each time set were placed into folders according to the phase they were acquired at. An additional folder for the uniform illumination images was created for each image set. The MATLAB code to be used for image processing could easily access each folder to pull out the appropriate images needed though this reorganization. Once the images were properly sorted into their appropriate folders, the demodulation process was the next step to be performed.  51  3.5 Image Preprocessing and Demodulation   Prior to performing image demodulation, some preprocessing operations were applied to each image in order to remove noise and background. A specific region of interest (ROI) of each image was cropped in MATLAB to remove most of the background present in each apple image. The resulting ROI was then subjected to further image processing steps. A low pass, Gaussian frequency domain filter with standard deviation of 0.5 and cut-off frequency set to 45 Hz was applied to each ROI to remove high frequency noise present in the images. This step resulted in ROIs with smoother sinusoidal responses than in the original raw image, which would allow for a higher quality demodulation. Uniformity corrections to each ROI were performed by using the white and dark reference images acquired prior to sample imaging, which are given by the following equation:  (7) where R is the corrected reflectance image, I is the original SIRI image, D is the dark reference image and W is the reflectance image of the Spectralon panel. The demodulation equations presented in the preliminary study (Chapter 2) were used to demodulate the preprocessed images into their respective AC and DC parts. Both components of each sample were used for automatic bruise detection. The demodulated images were saved into cell arrays in MATLAB for easy access in further processing steps. 3.6 Bruise Detection Algorithm  Following image demodulation, the apple images were further preprocessed to improve contrast and remove noise prior to bruise detection. First, the remaining background left over by 52  the ROI selection process was removed through creating image masks of the apple fruit. Masks were created through segmentation of the apple fruit areas by setting a global threshold. Figure 3.2 displays the resulting mask created from the segmentation process. Masks were created for each sample and multiplied with the demodulated images to remove all pixels other than those of the apple ROI.     The raw AC images obtained from demodulation did not provide a high level of contrast. Therefore, it was difficult to detect bruised regions using the raw AC alone. An image processing step was performed to increase the contrast between bruised and intact tissue utilizing both AC and DC images. The following equation was used to generate ratio images for each spatial frequency acquired:  (8) 53  where is the ratio image, ACi is the AC of spatial frequency i, and DCi is the DC of spatial frequency i. This operation yielded images with enhanced contrast between healthy and damaged apple regions for improved performance in detection.   Canny edge detection was performed on smoothed ratio images to locate the edges of the bruised regions. Morphological operations of dilation followed by opening were performed to connect nearby edge pixels and remove smaller objects in order to further reduce noise. The remaining edge pixels were inspected by a circular Hough transform algorithm to find evidence of bruises. As seen in figure 3.3, the perimeters of the apple bruise form a circular region. The algorithm treated each pixel location of the edge detected image as the center point of a circle with radius r, where r was a range of radii passed into the algorithm by the user. Based on physical measurements of the resulting bruise diameters, the r used during detection ranged from 25 to 55 pixels in order to encompass all possible bruise areas created during the experiment. The circular Hough transform drew all possible circles within the range of radii at each pixel location and then counted the number of foreground pixels in the edge detected image that fell on the drawn circumferences and stored the count in an accumulator matrix. Once the algorithm passed through every pixel in the image, the center location and radius that accumulated the highest number of counts provided the greatest evidence for a circle. These values were then used to draw a boundary around the detected bruised region. 54    55  3.7 Results  Figure 3.4 displays resulting DC and AC images acquired from the samples of each apple variety. Similar to the results obtained in the preliminary study, the bruises are invisible in the DC images which represent the average intensity response of the apple to each illumination pattern. The AC images contain the depth-specific information captured by the SIRI system and the bruises are visible in figure 3.4 at all of the spatial frequencies used in imaging. The effects of spatial frequency on light penetration are also observable as surface features of the samples begin to appear at higher spatial frequencies where light penetration was confined closer to the surface of the fruit.      56  3.8 Evaluation of SIRI System Performance  The apple data previously obtained was separated into two replicate sets of 10 apples per impact energy. Figures 3.5-3.7 present the average bruise detection rates for low, medium and high impact apples under structured and uniform illumination of two replicates of apple sample data. The circular regions returned by the detection algorithm were visually judged and compared to ground truth bruise images in order to determine whether the bruised region on each apple was successfully detected. The detection rates were calculated based on how many bruised regions out of the 10 apple samples were successfully detected in each replicate per impact group of apples. Success rates under each individual spatial frequency and uniform illumination as well as an overall detection rate were obtained. The overall detection rates are presented in figures 3.8-3.10. A sample contributed to the overall detection rate if the bruised region was successfully detected by at least one spatial frequency, excluding uniform illumination. The numerical values of the individual spatial frequency and overall detection rates as well as the standard deviations between the two replicates are reported in appendix tables A1-A6.  Detection rates under uniform illumination were quite low across all levels of impact, reaching a maximum detection rate of 50%. Successful bruise detection was consistently higher under structured illumination when compared to uniform illumination in all sets of images. Images acquired under spatial frequencies of 0.10 mm-1 and 0.15 mm-1 had similar detection rates, both much higher than those observed under uniform illumination. Slightly higher detection rates were obtained under 0.10 mm-1 than 0.15 mm-1 patterns, while performance under 0.25 mm-1 illumination was consistently lower than 0.10 mm-1 and 0.15 mm-1 at all levels of bruising severity and development. This suggests that the depth to which light penetrated the 57  fruit tissue under 0.10 mm-1 illumination was the closest to the true depth at which bruised tissue -1, light penetration was shallower than at the lower frequencies and the decrease in detection may be explained by the appearance of surface features in the images such as fruit lenticels seen in figure 3.4. Due to the higher rate of light intensity attenuation at high spatial frequencies, the light penetration was confined closer to the surface layers of fruit tissue. The resulting image may still contain bruise features, but the detected surface features add noise that may not be entirely removed. While the rates fell under 0.25 mm-1 illumination, this spatial frequency still consistently outperformed uniform illumination.  Bruise detection rates increased with bruise severity. Detection rates of the individual spatial frequencies all increased in medium impact apples compared to low impact apples and the highest individual performances were observed in the high impact group. The increase in performance from medium to high impact was less significant, however. Overall detection rates were similar across all levels of bruise severity. Based on the results, the SIRI system should be able to detect bruises of all sizes and severities. However, if the SIRI system were to be limited to projecting only one spatial frequency, the severity of tissue damage may factor into the mance as light bruises possibly pose a challenge for the system. Therefore there may be a tradeoff between system accuracy and speed.  The greatest amount of variation between detection rates arose when comparing rates obtained at different times after initial impact. As bruises developed, the optical properties of the damaged tissue began to change and the appearance of bruises under the SIRI system transformed over time.  Changes in detection performance over bruise development time have 58  also been reported in previous bruise studies utilizing other imaging methods (Huang et al., 2015; Upchurch et al., 1994; Xing et al., 2006). SIRI was able to detect bruised regions in AC images immediately after impact with overall detection rates ranging from 85-100%.  Bruises were not visible under uniform illumination immediately following impact, but after 4-6 hours, slight discoloration began to visibly form on the skins of the fruit as the bruises developed. The contrast between healthy and damaged tissue appeared to increase in the AC images as well, resulting in darkened bruised regions. As a result, overall detection rates as well as individual spatial frequency rates typically increased after bruised regions had developed for 4-6 hours after initial impact. An increase in successful detection was also observed in uniform illumination images, but the detection algorithm was still unable to achieve high rates. After the bruises had developed for 24 hours, a drop in performance in all sets of apples was observed. The bruised regions exhibited reduced contrast with surrounding tissue and the algorithm began to struggle with detection. Results obtained from images acquired 0-1 hours and 4-6 hours after initial impact remained similar in both varieties. However, larger drops in the detection rates were observephenomenon may be due to variations in the bruise development process between different apple varietie   59       60        61        62  3.9 Accuracy of Detected Bruise Areas  Percent differences of detected bruise areas are presented in Figures 3.11-3.13. After bruise detection was performed on the sample images, images with a successful detection were used to examine the accuracy of the SIRI system in isolating bruised regions. The circular regions returned by the Hough Transform were segmented and an area was calculated and stored for comparison against a ground truth. A true area was obtained through physical measurements of the major and minor axes of the bruised regions on the apple fruit 24 hours after impact. A percent difference was calculated between the segmented Hough Transform areas and the true areas. In the low impact sample set (figure 3.11), the average accuracies reported ranged from 14.7% to 61.8%. The magnitude of the differences decreased in all images acquired 4-6 hours after impact when compared to accuracies obtained in images acquired 0-1 hours after impact. The accuracy of the algorithm in detecting bruised areas appears to follow a similar trend as its ability to successfully detect bruised regions, where peak performance was observed in images obtained 4-6 hours after impact. Percent differences obtained after 24 hours appear to be much greater than in previous images for  apples. Percent differences also increased  It was previously variety, while thSince the detection algorithm struggled with detecting the 24 hour old bruises, poorer accuracy may have arisen as a result. For both the low impact and medium impact groups, the average differences were all positive, indicating that the Hough Transform returned areas which tended to be larger than the true areas of the defects. For the medium impact group, the magnitudes of 63  the area differences appear to be smaller than those observed in the low impact group, providing further evidence that bruise severity affects the performance of the SIRI system. The percent differences reported in the high impact group were drastically different from the previous two with nearly all of the average differences being negative. Here, the Hough Transform tended to underestimate the sizes of the bruises. This may have been a result of the range of radii input as parameters into the detection algorithm where the upper limit of the radii range was typically estimate was consistently smaller than the true area of the bruise, resulting in negative percent differences in measured bruise area. The magnitudes of the high impact percent differences, -6 hours after impact. While the algorithm seems to have switched from overestimating areas to underestimating them, the accuracy of area measurement continued the trend of improving with bruise severity. The standard deviations obtained for the percent area differences were typically less than 20%. The algorithm seems to perform with reasonable consistency, although it was not highly accurate. Some deviations between replicates were quite high reaching as high as 30.9% in the low impact energy apples. The large fluctuations observed in the area differences could be reflective of how the circular Hough transform algorithm performed. The circular Hough transform only returns a single perfectly circular region which it judged to be the best fit in each image. The true bruise areas may not be perfectly circular and therefore large regions of the true bruise regions may not have been included in the area retuned by the Hough transform. While this is one of the weaknesses of the bruise detection algorithm, the issue is related to image post-64  processing. Improvements can be made to the bruise detection algorithm to acquire more accurate measurements of bruise area, but this study has clearly demonstrated the enhanced detection performance of structured illumination compared to uniform illumination, although the technique is not ready for commercial real-time applications. The numerical values of the average percent differences and their standard deviations are presented in the appendix tables A7-A12.    65       66  Based on the overall detection rates reported, the SIRI system was able to achieve high fresh bruise detection rates in the two apple varieties tested. Xing et al. (2006) reported bruise 0.16 J impact using VIS-NIR spectroscopy. After selection of key wavelengths based on hyperspectral data, the multispectral imaging system described by Huang et al. (2015) reported detection rates of 90.4% and 92.3% in The SIRI system was able to achieve higher overall and individual spatial frequency detection rates than both of these imaging techniques for fresh bruises. However, the results reported in this work are exploratory and extensive further testing of the SIRI system is needed before conclusive evidence of improved detection performance can be provided. Additionally, the speed at which the Hough transform was able to perform detection was at an average rate of 1.52 seconds per image. The process also required about 1 second to acquire all of the spatial frequency images needed for demodulation. The current process used in detecting bruise regions is still too slow to meet commercial demands. The imaging techniques used by Xing et al. (2006) and Huang et al. (2015) were able to collect images at a much faster rate than the current SIRI system.  Nevertheless, the on of fresh bruises in apple fruit. As an exploratory study on the potential of SIRI for the detection of subsurface defects, the system demonstrated its ability to outperform conventional machine vision. Optimization of the bruise detection algorithm could also improve the speed of the system, however in the grand scheme of developing this imaging technique, a focus should be placed first created in this experiment were under controlled conditions and may not represent bruises that 67  may occur in real world situations.  An opportunity to test the SIRI system on realistic bruises arose when an infield sorting machine being developed inhouse was to be evaluated for bruising potential, which is presented in Chapter 4.   68  CHAPTER 4: EVALUATION OF APPLE BRUISES FROM THE INFIELD SORTING MACHINE  4.1 Introduction   The SIRI system demonstrated improved bruise detection when compared to uniform illumination in the previous experiment. However, the bruises identified in the study were all created under controlled conditions. Therefore, the bruises detected may not be representative of naturally occurring bruises that may be encountered in the real situation. The opportunity to obtain supplementary data on the performance of SIRI on realistic bruises was presented during an evaluation of an infield sorting system which was being developed by our group independent of the SIRI system. The infield sorting system was a mobile machine capable of transporting and sorting apples based on fruit appearance.  The goal of the system evaluation was to determine whether the sorting machine created bruises while samples passed through the system and were deposited into bins. SIRI attempted to identify bruised regions which may have formed as a result of impacts experienced during sorting and deposition. This, along with multiple hand inspections of the apple samples for comparison, was used to determine the bruise detection potential of SIRI. The results reported in this study will focus on the performance of SIRI for bruise detection rather than the evaluation of the infield sorting machine. 4.2 Infield Sorting Machine  An infield apple sorting machine was being developed as a separate project during the time that experiments using the SIRI system were being carried out. The machine was designed to sort apples during harvest using a built-in machine vision system. Apples placed onto the 69  system travel via conveyor belts into three separate lanes of cup holders for imaging. Each apple was graded by the imaging system and then dropped into an appropriate bin. Figure 4.1 shows the infield sorting machine system with its various components labelled. Figure 4.2 displays a real image of the conveyors used by the sorting system.  Throughout the entire sorting process the apples experience drops while being transferred between different system locations. Bruising that may occur after the samples pass through the machine vision system will go undetected by the machine resulting in potentially misclassified fruit. Therefore, an experiment to evaluate the ic situations encountered during harvest and sorting. Rather than attempting to discover new potential applications of SIRI, this study served more as a supplement to the results obtained from the previous bruise study.     70      4.3 Apple Samples   This experiment was performed during the month of March in 2016, during which apples were no longer available for hand harvest. Instead, the apple samples were acquired from a commercial packinghouse in Michigan. Two bushels each of Fuji and Gala apples were supplied by the packinghouse. The apples were collected prior to sorting and therefore potentially had preexisting bruises. The apples were packed into cardboard boxes and placed into cold storage at 30C (36oF) for 6-7 days before the start of the experiment. On the morning of the first experiment day, one bushel of apples of each variety was removed from cold storage and examined by hand to detect any preexisting bruises. Detected preexisting bruises were marked 71  with a permanent marker and the bruise diameters were measured using a digital caliper and recorded on a spreadsheet. Each apple was marked with a sample number and placed into a cup tray for organization as shown in figure 4.3.  After the visual inspection, the apples were left to warm at room temperature for 2 hours before running them through the infield sorting machine. The experiment using the infield sorting machine was carried out over two days and the total number of samples used in the experiment were 90 Fuji and 115 Gala apples on day 1 followed by 92 Fuji and 118 Galaapples on day 2.       72  4.4 Running the Infield Sorting Machine   Operating the infield sorting machine was simply turning the machine on and having the conveyor system active. Two different bin conditions were used in the experiment: 1) an empty bin shown in figure 4.4 and 2) a prefilled bin with a layer of apples already present. The different bin situations would allow for observation of bruising potential between apples and the wood bin when the sorting machine is filling an empty bin and the bruising potential between apples and apples when the bin is partially filled. The drop height between the bin filling mechanism and the surface of the bin, or the apple layer, if present, was adjusted automatically through height sensors and remained constant throughout the experiment. The experiment spanned two days where one of the two bin situations was tested per day. The first day of the experiment used the partially filled bin scenario. Extra  apples were picked while harvesting fruit for the SIRI bruise experiment and had been kept in cold storage for 6 months. The Golden Delicious apples previously harvested were visually distinct from the Fuji and Gala apples used for bruise evaluation in this experiment and therefore ensured that each apple sample could be identified and recovered after running through the machine. A two-apple thick layer of Golden Delicious fruit, approximately 4 bushels, was filled into the bin prior to running the test samples through the system. The apple samples were run through the sorting machine one variety at a time to prevent confusion during recovery.  After all of the apples had fallen into the bin, they were recovered carefully by hand and placed back onto their respective cup trays and transported to the SIRI system for image acquisition. The following day, the experiment was run using an empty bin for sample deposition where the samples made contact with the wood floor of the bins rather than with Golden Delicious apple fruit.    73     4.5 Image Acquisition   The SIRI system was left in the same configuration for infield apple image collection as in the previous bruise study. The Fuji and Gala apples were on average larger than the Golden Delicious and Delicious apples previously used. Therefore, some adjustments to the sample stage height were made to ensure that the entire apple surfaces would be covered by the DLP projection and camera. Based on previously obtained results, only two spatial frequencies, i.e., 0.10 and 0.15 mm-1, were selected to acquire images. These two spatial frequencies previously yielded high detection rates for bruises and were deemed adequate for this experiment. Unlike the previous bruise experiment where the bruises were created using a controlled method for consistent results, the bruises resulting from the infield sorting machine could occur in various sizes as well as in various locations. In order to detect all possible bruise areas, multiple images 74  needed to be acquired from various views of each sample. Images were acquired over a span of 8 hours after the samples were run through the system. SIRI images were collected from three different views of each fruit each rotated 120o from the last. Bruises that existed prior to running the apples through the infield sorting system were noted on a spreadsheet as to which image view they were contained in for future reference. A total of 18 images, 6 per sample view (2 frequencies, 3 phases), were acquired under the SIRI system per apple. On each day of the experiment one bushel of each variety was imaged using the SIRI system using identical settings. to a shared data storage device. After image acquisition was completed, the apples were returned to their respective trays and awaited post-imaging, manual evaluation. 4.6 Post-System Hand Evaluation   After SIRI image collection, the apple samples were visually inspected by hand again to detect bruises that did not exist prior to running the fruit through the infield sorting machine. Similar to the pre-examination of apples, each new bruise had its diameter measured by a digital caliper and was recorded on a spreadsheet. After this inspection, each fruit was peeled and evaluated once more to check if any bruises that were invisible underneath the skin of the apples could be identified. These bruises were also recorded albeit separately from the new bruises detected while the skin was still attached to the fruit. The locations of the recorded bruises were not tracked on the spreadsheets, therefore the bruises detected by the SIRI system could be verified through the number of bruises detected by hand and not the specific locations on each 75  fruit; however, the recorded diameters of the bruises helped with determining whether bruises detected by the SIRI system could feasibly match with those recorded. 4.7 SIRI Image Processing   The SIRI images were processed in a similar manner as the images in the previous bruise study were. Each collected image was sorted into an appropriate folder by its specific view and pattern phase for easy access. A label of A, B or C was applied to each view acquired during image collection and respective folders were created for storage. Each image was cropped to include an ROI and filtered to remove high frequency noise. Three-phase demodulation was then applied to each sample view and spatial frequency to obtain AC and DC images. Two processing operations were applied to the AC and DC images in attempts to create the highest level of contrast possible. First, simple division images were created by taking the quotient of the AC and DC images. This operation created contrasts which were easier to visually identify, therefore these images were useful in visual judgement of bruise detection by the SIRI system. The second operation applied to the demodulated images was the same ratio operation used in the bruise study. The ratio images performed most favorably in the previous study when employing the circular Hough transform to find circular edge pixels. However, the bruises created by the infield sorting machine were far more varied than the controlled bruises detected in the last experiment. There was no guarantee that the resulting bruises would be circular in shape and therefore, the circular Hough transform would not be the most appropriate approach to bruise detection.  While the division and ratio operations improved the contrast between bruised and non-bruised regions, an automatic bruise detection algorithm was still difficult to implement. The contrast 76  was not strong enough to obtain consistent segmentation results between bruised and healthy tissue and samples with many bruises in close proximity tended to causes large chunks of segmented regions which could not be distinguished into individual bruises. Therefore visual judgement was necessary to determine if each recorded bruised region was present in SIRI images. 4.8 Bruise Detection Results   The number of pre-existing bruises manually machine sorting as well as their detection rates under SIRI are presented in tables 4.1 and 4.2. The results obtained on bruises which formed after the apples were run through the system are in tables 4.3 and 4.4. The spreadsheet which recorded the marked bruises which had occurred prior to running the machine had to be referred to in order to distinguish pre-existing bruises and bruises which resulted from the sorting machine. Overall rates were calculated based on the total number of hand identified bruises compared to the total number of bruises identified under SIRI and are presented in table 4.5. bruises, both pre-existing and newly formed, were far more likely to bruise.  High detection rates  under the SIRI system with overall rates greater than 80%. The Fuji apples performed less favorably as the detection rates fell below 70%.  The performance of the SIRI system differed between the two varieties used in this experiment. Gala apples yielded higher rates in all categories over the Fuji variety. The previous bruising study using Golden Delicious and Delicious apples did not exhibit a significant difference in detection rates between the two 77  varieties. The previous bruising experiment sought to test if skin color impacted the performance of SIRI and was deemed not to noticeably affect bruise detection. However, based on these results it seems that apple variety impacts SIRI in some manner. The different physiological conditions for each variety may have existed at the time of testing, which would have resulted in different responds to tissue damage, and thus may have affected the optical properties of bruised regions differently between the two varieties. Therefore, due to the limited range of spatial frequencies used in this experiment, the optimum settings may not have been used for identifying bruised regions in each variety used, especially for the Fuji apples which were lower in detection rates. In the previous bruise study, the age of a bruise factored into its detectability where bruises older than 24 hours saw decreased detection rates. Therefore, it was expected that older bruises identified before sorting would be more difficult to detect than newly formed bruises. However, the detection rates did not differ significantly between pre-existing and newly formed bruises, suggesting that the age of the bruises did not influence the detection of naturally formed bruises as greatly as it did in the previous bruise study. One possibility may be due to the larger number of samples used in this study. Image acquisition of the entire sample set took a much longer time (> 8 hours) when compared to the relatively quick 30 mins it took to collect a set of images in the previous work. The newly formed bruises may have developed to a similar level as the pre-existing ones during the long imaging process, thus resulting in similarly appearing bruised regions. Another issue that may have affected detection rates in both varieties is that it was difficult to distinguish between bruises that had been identified before running the samples through the machine and those found later. The bruises identified earlier were marked and made note of on a spreadsheet; however, some markings did not appear clearly in the 78  demodulated images and bruises spread over several different surface views could not be tracked with certainty in some cases. Therefore, there may have been some redundancy in counting identified bruises. Also the regions that some bruises occurred in happened to fall near the edges of fruit in some images, which were not completely visible due to the curvature of the apple, and some regions of the apple surface may not have been captured in any of the three views due to positioning errors. These errors may have led to missed bruises that were not captured in any images or perhaps confusion in matching detected bruises to pre-existing ones on the spreadsheet.  Bin Status # Apples # Hand Detected Bruises # Detected Bruises Detection Rate Empty 92 85 57 67.1 Partially Filled 90 58 40 69.0    Bin Status # Apples # Hand Detected Bruises # Detected Bruises Detection Rate Empty 118 199 171 85.9 Partially Filled 115 163 149 91.4   Bin Status # Apples # Hand Detected Bruises # Detected Bruises Detection Rate Empty 92 184 130 70.7 Partially Filled 90 95 56 58.9    Bin Status # Apples # Hand Detected Bruises # Detected Bruises Detection Rate Empty 118 245 207 84.5 Partially Filled 115 260 194 74.6  79    Fuji Gala  Empty Partially Filled Empty Partially Filled Overall Bruise Detection Rate 69.5 62.7 85.1 81.1   There were instances where over-classification occurred. In some SIRI images the number of bruises detected exceeded the number of bruises marked by hand. Some of the defects marked as bruises on the spreadsheet were in reality other types of mechanical damage such as lines of indentation in the fruit skin due to compression or puncture wounds that broke the skin. However, these defects were easily distinguishable in the ratio and division images used to judge defect detection. Therefore it is possible that the SIRI system was able to detect bruises that were overlooked during post-imaging evaluation. Bruises that were undetected during inspection of the apple skins after imaging may have been cut off the fruit during peeling. If the bruise did not extend deeply into the fruit tissue the instrument used to skin the skin of the samples may have cut too deeply into the fruit and removed the entire bruise, thus leaving it unmarked for validation. Table 4.6 contains the number of unmarked bruises detected by the SIRI system in each bin scenario.  There were far fewer extra bruises detected in the Fuji variety than in the Gala apples. This may provide further evidence that the two varieties have distinct differences in bruise development. If the previous assumption that the peeling instrument cut too deep and removed the entire bruises in some apple samples is true, then the increased rate at which extraneous bruises were detected Gala bruises occur more shallowly than in Fuji thicker than the skins of . A peeling cut that 80  skin may also explain why the skin may provide more protection against bruising. This is not the only possibility as to why differences between apple varieties occurred. Information on the maturity, growing location, harvest dates, etc. were unknown when the samples were obtained. All of these factors may have impacted the bruising susceptibility and bruise detection of each fruit. More undocumented bruises were detected in the apple groups that were dropped into empty bins. This would have been expected as the empty bin scenario resulted in higher counts of bruise occurrence than the layered case. Therefore, the overall creation of the previously mentioned shallow bruises may have also been higher.  Fuji Gala Empty Bin Partially Filled Bin Empty Bin Partially Filled Bin 17 12 56 28   ability to detect multiple naturally occurring bruise types in apple fruit. Challenges encountered throughout this process included unimaged portions of the sample surface area, difficulty in segmentation of bruised areas, and the distinguishing of individual bruises from aggregated areas of damaged tissue. Additionally, the need to physically rotate each sample multiple times by hand in order to acquire images of the entire fruit surface areas greatly increased the imaging time requirements. However, rather high detection rates were achieved in the Gala variety despite the challenges SIRI faced in this experiment. The lower performance observed in the Fuji apples suggested that 81  fruit variety impacts the performance of the SIRI technique. The performance of bruise detection in Fuji apples may have been improved if other spatial frequencies were used for imaging. In the future, for optimized implementation of this method, key spatial frequencies need to be determined for specific products. The results of this experiment and the previous one have displayed the potential that SIRI possesses in detecting subsurface bruising in apples of various varieties. There are still many challenges in regards to image acquisition speed, selection of key spatial frequencies and improving image processing algorithms for improved defect segmentation and detection as highlighted by this infield sorting machine study. However, the promise this method has shown warrants further efforts into developing and optimizing the technique. To further investigate the potential that SIRI offers in defect detection, different types of fruit defects needed to be introduced to the system. Therefore, following the infield sorting machine study, a new experiment focusing on surface defect detection was designed, which is described in the next chapter.    82  CHAPTER 5: DETECTION OF SURFACE APPLES  5.1 Introduction   The research presented in Chapters 3 and 4 has been mainly focused on the detection of bruises that occur to the tissue under the fruit skin, which are not readily visible. However, the preliminary study on a test material, as presented in Chapter 2, also demonstrated that the SIRI system could have great potential for detecting surface defects in turbid media using high spatial frequencies. Both theoretical analysis (Chapter 2) and previous studies have shown that high spatial frequency illumination patterns are capable of revealing features confined close to, or at, the surface of a sample. Moreover, SIRI is also known to enhance the ability to resolve surface features of a sample (Neil et al., 1997). These properties of SIRI suggest that it could be useful in enhancing surface features, which otherwise do not show up clearly in images acquired using conventional machine vision technique. Therefore, a further experiment was designed, in which the SIRI system was applied to detect surface defects of apple fruit at higher spatial frequencies. The performance of SIRI was evaluated on the detection of various types of surface defects that commonly occur on apple fruit. Spatial frequencies higher than those used in the bruise studies were investigated as previous observations revealed that high frequencies confine light penetration to the surface of the samples. This chapter presents the experiment and image processing algorithms for detection of the various surface defects in apples.  83  5.2 Defective Apple Samples   The apples used in this experiment were collected from the same packinghouse as the surface defects were removed from the sorting line by the human inspectors. The defects on the fruit were identified through manual inspection, not machine vision. Approximately one bushel (roughly 100 fruit) of each variety were acquired. The apples were removed from the sorting line after a cleaning and waxing operation was performed. Therefore, the samples used in this experiment exhibited a greater level of specular reflection due to the presence of the wax coatings. Apple selection was not made on specific types of surface defects and were collected as they came off the sorting line until one bushel was acquired. This resulted in a random assortment of various types of surface defects which would need to be identified and sorted by hand prior to imaging.  The apples were stored in a cold room at 00C (32oF) for approximately three weeks before the start of the experiment. Hand inspection was performed one day before imaging to identify the specific defects each apple possessed. The apples in each variety were organized into separate groups according to defect type. The samples between varieties were kept separate even if common defect types occurred in both varieties. This was done to determine if the apple variety affected SIRI performance. Some apples did not contain any noticeable defects and were removed from the experiment. A total of nine defect types were  Between the two varieties of fruit, two common defect types were observed. 84  The defect present on each fruit was determined by visual identification of symptoms apples were grouped into the following four groups: insect damage, sun burn or scab, streak and mildew.   surface mildew. The symptoms of this defect is the presence of web-like scarring on the fruit surface. This defect may have been caused by exposure to low temperatures while the apples mildew disorder. The scabbing can be seen as a network of yellow fruit tissue running across the surfaces of the fruit.     85   Another commonly occurring surface defect present in damage. A multitude of insects may feed on the fruit tissue while they remain in the orchard, therefore the damage caused by the pests may result in a variety of defects with differing appearances.  A majority of the insect defects were identified by the presence of an entrance wound on the fruit surface, where the pest may have burrowed into the fruit. Individual holes were quite small and difficult to spot by eye, however these burrows tended to occur in clusters which made the defects more visible. Some insects feed on the surface of apples and do not dig into the tissue. The damage left by such insects appear as scars of indented and discolored tissue.  Samples afflicted by insect damage are shown in figure 5.2. There were a bearing insect damage.     86   A defect that was similar in appearance to insect damage occurred in some fruit. This defect had the appearance of blackened, scabbed tissue on the surface of the apples. There were no signs of entry wounds into the fruit tissue, therefore they were classified as a separate defect from the previous insect category. Figure 5.3 shows the samples possessing these defects. Blackened skin tissue is indicative of damage caused by over exposure to sunlight. The defects present on these apples would be referred to as sun burns. Seven cases of sun burn       The final surface des were linear streaks of scabbed tissue. The scabs had distinct textures and coloration from the surrounding apple skin. The firmness and discoloration of the streaks were similar to that of the mildew defect type, only 87  the defect seemed to occur as a line rather than in a web pattern. Figure 5.4 provides images of the 18 streak defect  described.      Five variety. Both fruit varieties exhibited the streak defect and damage caused by insects or other natural causes. Three types of surface variety were black scab, rot and lenticel damage.  Mechanical  . Mechanical damage may have occurred during rough handling of the fruit. These defects occurred at the same depth as many insect defects should be detected similarly. Mechanical 88  damage was identified in a total of 27 samples. Figure 5.5 provides some examples of the types       The streak de symptoms were nearly identical in both apple varieties, where a streak of scabbed, discolored in figure 5.6.  89      specks on the apple skins or as abnormally large lenticels compared to the rest of the fruit. There were not clear symptoms that could be used to identify a specific defect in these samples. The defects were subtle and difficult to identify even through close visual inspection.  Despite being unable to identify a specific disorder within these samples, they were considered in this experiment because they possessed some distinct lenticel appearances from the other samples. There were only five   90      A frequently occurring defect identified in the areas of cracked black tissue on the fruit surface. These defects would be referred to as black scab. These defects were quite obvious when observed on the fruit surface. Some cases occurred as one large scab while other samples had multiple smaller sized black scabs clustered near each other. This disorder is a disease caused by a fungal infection of the fruit tree. The black scab ples with 18 samples. The appearance of black scab is presented in figure 5.8.    91   uth, this defect occurred within the fruit, however the symptoms were visible on the surface as soft, darkened tissue that often occurred as sunken and/or shriveled areas on the surface. Therefore, this defect was still of merit as confining light penetration to the fruit surface may still enhance detectability. A total of five y and are displayed in figure 5.9.      Once the defects had been identified and sorted in each apple variety, the samples were replaced into cold storage for three days before the start of image collection. 5.3 SIRI System Setup   The configuration used for surface defect detection was mostly the same as that used in the bruise experiment. The relative positions of the DLP, camera and sample stage were kept the same as in previous experiments. As previously mentioned, the resolution of the DLP projector deteriorated at frequencies exceeding 30-40 mm-1 while using a 10 x 7 cm projection area. The frequencies to be investigated in this experiment would reach up to 60 mm-1, and it was therefore necessary to adjust the projection area to allow for sharp illumination patterns at higher frequencies. This was achieved by bringing the DLP closer to the sample stage so as to shrink the 92  projection area to roughly 7 x 4 cm, where projected patterns even at 60 mm-1 appeared clearly on a flat surface. The closer proximity of the DLP to the stage interfered with the camera view causing parts of the DLP hardware to appear in the collected images. To avoid the appearance of any parts of the DLP in the field of view, the camera angle was adjusted from 15o to 20o and the horizontal position was shifted closer toward the DLP in order to acquire an image of the full sample. The camera height was also increased from 64 inches from the ground up to 70 inches to keep the stage within an appropriate focal distance of the zoom lens. The same rod instrument used in previous experiments to maintain a constant stage height was adjusted and used to ensure that the surface of the samples on the stage were set to a consistent height. A new element was introduced to the system in the form of a spectral filter. Based on previous studies in defect detection, light in the near-infrared (NIR) wavelength range may be better suited for detecting defects in apple fruit reflectance images (Aneshansley et al., 1997). By filtering out the visible portion of the QTH light source, the SIRI system could acquire images in the NIR region only. The CCD camera used in the SIRI system was sensitive to the NIR wavelength region and therefore, could acquire images in that spectral region. The light filter used was a 50 x 50 mm square long-pass filter with a cut-off wavelength of 700 nm (Edmund Optics, Barrington, NJ, USA). The filter was placed in an adapter attached to the light source housing and filtered the illumination source prior to entering the DLP. Since the filter greatly reduced the intensity output of the DLP, the exposure setting of the CCD camera was increased to 0.5 s for acquiring strong image signals.  The power output of the light source was also adjusted from 150 to 200 W to further improve image signal.  93  The increased specular reflection of the apple fruit surfaces due to the presence of a wax coating led to some challenges in image acquisition. While the two crossed linear polarizers attached to the DLP and CCD zoom lens were able to greatly reduce the specular reflection, a slight amount still remained in each image. This issue would need to be resolved through image processing following image demodulation. Since the goal of this study was to identify surface defects, high spatial frequencies were selected in order to confine the incident light to the surface of the fruit. Four spatial frequencies of 0.30, 0.40, 0.50 and 0.60 mm-1 were chosen for image the peak frequency of 0.25 mm-1 started to reveal apple lenticel features in the demodulated images. Therefore 0.30 mm-1 was chosen as a spatial frequency where reflected light begins to contain a significant amount of information pertaining to the surface of the sample. 5.4 Image Acquisition   The sorted apple samples were removed from cold storage two hours before image acquisition to allow the samples to equilibrate to room temperature. The QTH light source was switched on and allowed to warm up to a constant output of 200 W. The CCD was also turned on and allowed time to reach a steady-state temperature prior to image collection. Once the camera and light source reached steady output, the DLP was turned on and loaded with the appropriate spatial frequency patterns. A uniform illumination pattern was projected first to acquire reference images. Twenty white reference images of the Spectralon panel and 20 dark reference images were acquired as they were done in previous studies. Once the reference images were collected, the system was ready for acquisition of apple sample images.   94   The procedure used in collecting sample images was identical to that used in the bruise experiment. The samples were placed onto the stage cup and the stage height was adjusted using the rod mechanism. Images were collected at all phase shifts at 0, 0.30, 0.40, 0.50 and 0.60 mm-1 and saved to appropriate folders labelled with the variety and defect type. All sample images were collected on the same day. The images were acquired one variety at a time where a second set of reference images were collected after imaging for one variety was completed. Once imaging was completed, the apple samples were returned to cold storage, as they would be needed for additional imaging at a later date. It was later decided that images collected at lower frequencies, like those used in the bruise experiment, may aid in enhancing some defect types. The same procedure was therefore followed to collect additional SIRI images at spatial frequencies 0.10, 0.15 and 0.20 mm-1.  5.5 Image Demodulation   The SIRI images were demodulated following the same procedure used in all previous experiments and inspected to gain a preliminary understanding of which defects may have been enhanced through the imaging technique. Figures 5.10-5.18 display the raw DC and AC images of some samples acquired from each defect type at both high and low spatial frequencies. The resulting raw images reveal a varying degree of information pertaining to each defect type. Some defects were enhanced, while others did not seem to gain much benefit from the SIRI technique.  samples, the insect and streak defects appear more apparent in the high frequency AC images than in the DC images (Figures 5.10 and 5.13), suggesting that SIRI was capable of enhancing the surface features of the fruit. The insect damage was not enhanced compared to the DC image under low frequency illumination. The mildew and sun scab AC images do not seem 95  to exhibit any increase in contrast as a result of SIRI using either range of spatial frequencies (Figures 5.11 and 5.12)through SIRI (Figures 5.14-5.18). The exception was in the apples with lenticel defects (Figure 5.16), where the defective lenticels cannot be distinguished from the normal ones. In all cases, the lenticels of the apples appear in the AC images, while they are invisible in the DCs. As the spatial frequency of the ACs increase, the texture of the images appears to become more complex as if the physical texture of the fruit skin also becomes visible in the images. These results, at a glance, suggest that the high spatial frequencies used to collect images were able to confine the incident light to the surface of the samples and the lower range of frequencies used did not introduce any significant surface features when compared to the DC images. Therefore, information pertaining to the defects which occur at the surface should be contained within the high frequency AC images. The exception to this trend is the rot defect where the tissue damage extended within the fruit and thus, the low spatial frequency images reveal more information pertaining to rot. The general trend observed was that the reflectance intensity of the overall fruit area decreased with increasing spatial frequency. This may possibly be explained by the increased rate of light intensity attenuation within the fruit tissue due to higher spatial frequency as discussed in chapter 2. In images with more obvious defects, the intensity of the defect itself did not seem to be affected by the increase in frequency. Other than revealing surface features which were invisible in the DC images, the effects of spatial frequency did not seem to influence defect visibility as significantly as it did with bruises. The lack of clarity of certain defects in the high frequency raw images may be due to the abundant number of fruit lenticels appearing in the images which act as noise interfering with the defects and in some cases mask the defects 96  from view. Furthermore, the specular reflectance caused by the wax coating on the fruit is quite evident in some images. The initial appearance of the raw AC images suggested that segmentation of the surface defects would be more difficult than bruise segmentation due to the presence of bright lenticels and a region of specular reflection.  97      98     99      100    101     102    103    104     105       106  5.6 Image Processing for Defect Enhancement   The raw AC images resulting from demodulation provided insight toward which defects may be enhanced through high spatial frequency imaging. The next step was to apply morphological operations to the AC images in order to improve the contrast between the defect and surrounding tissue, much like what was done during the bruise study. Division and ratio images were created using the AC and DC images, where division images were simply the quotient of the AC and DC images and ratios images were created following equation (9) in chapter 3. These operations were able to increase the defect contrast in some cases, while in others they did not seem to affect contrast significantly. The defects that division and ratio e two The lenticel and mildew defects were not visible in the transformed images. In addition to contrast enhancement, the lenticels of the fruit appear less prominently in the transformed images, however they are still present and may interfere with segmentation. The specular reflection is also greatly reduced, although small areas of high intensity can still be observed in some images. The contrast of the defective tissue was enhanced in the division and ratio images and the lenticels were less pronounced when compared to the raw AC images.     An alternative approach to enhancing defect contrast was through the use of principal component analysis (PCA) on the five spatial frequencies used in imaging. Principal components were obtained which represented the information contained within all of the SIRI images 107  acquired. The second principal component (PC) image seemed to contain the most information pertaining to the defects present in the images, while PC 3, 4, and 5 seem to embody the noise and some subtle texture features contained in the SIRI images. The first PC appeared to represent the DC signal present in each SIRI image. Figure 5.19 displays the PCs obtained from some of the apples afflicted with black scab. The contrast between the scabs and the surrounding tissue is greatest in PC2, while the PC1 images appear to be identical to the DC images previously shown.      A ratio of the first two PCs was created through dividing PC 2 by PC 1 in an attempt to further improve contrast. Compared to PC 2 alone, the ratio images using both PC 1 and 2 demonstrate greatly improved contrast. The level of improvement varied between defect types, but the PC ratio images typically resulted in the highest contrast. Figure 5.20 displays the resulting images for a black scab afflicted apple obtained through each image processing operation. 108      The AC images resulting from the lower spatial frequencies (0.10, 0.15, and 0.20 mm-1) were able to reveal some features of the surface defects. The general region where the surface defects occurred were enhanced through SIRI at the lower frequencies, however, the overall resolution of the exact defect shapes were not as high as in the higher spatial frequency images. A possible advantage of these lower frequency images is that the apple lenticels are not as prominent, thus the images are much less noisy. Defect segmentation would be attempted on high spatial frequency images due to the higher defect resolution which may improve accuracy. However, there was one case where the lower frequencies outperformed the higher ones. The rot defect was enhanced to a greater degree in the lower frequency images. Figure 5.21 shows the PC ratio rot images of both low and high spatial frequency SIRI.   109      5.7 Defect Segmentation  Segmentation of the defects present in the PC ratio images was attempted through thresholding operations. First, an image mask was applied to each PC ratio image to segment the apple area from the background. Once the apple areas had been isolated, an intensity threshold e threshold returned by this method did not perform well and manual adjustments were made until the resulting binary image appeared to isolate the defective regions well. Following thresholding of the image, an opening operation was performed to remove foreground objects which were either too large or too small to be considered defects. Some remnants of the perimeter of the apple remained in certain images, and the opening operation successfully removed these areas. A hole filling operation was then applied to fill in any remaining blobs of the binary image. The remaining pixels after all image processing operations were applied were considered as detected defective area. Figure 5.22 provides a flow chart of the image processing steps used in defect segmentation.  110      The results of segmentation varied with defect type, and in some cases the defective tissue did not contrast strongly enough for isolation. The defects that segmentation operations failed in were mildew, lenticel damage and the streaks present in both apple varieties. In the cases of mildew and lenticel damage, the defect is inherently tied to the surface texture of the fruit. The PC ratio operation alleviated some of the noise created by the presence of the apple lenticels; however, since the mildew and lenticel defects rely on the appearance of such features, this operation effectively masked these defect types. Therefore, it was determined that lenticel defects could not be noticeably enhanced and segmented through SIRI. The contrast of the rot defects oAC images. The SIRI images acquired under the lower spatial frequencies of 0.10, 0.15, and 0.20 mm-1 resulted in higher levels of contrast when the PC ratio operation was applied. This result 111  was expected because similar to bruising, the most severely afflicted area of the fruit is within the apple, while visible symptoms of rot may appear on the surface. Therefore, while higher spatial frequencies are capable of detecting lower spatial frequencies are ultimately better suited for their detection. An interesting observation of the lower frequency rot images is the presence of both low and high intensity contrast in the defective tissue. The brighter pixels in the images correspond to regions where the defective tissue was closer to the surface of the sample and the darker pixels correspond to damage within the fruit. Due to the multi-depth nature of the rot defect, this phenomena was not observed in any other defect type. Furthermore, the rotten apples were not internally inspected to verify the true extent of the rot defect. There may have been mold tissue present, as rotten fruit are susceptible to mold infection, which may have responded differently to the structured illumination than softened, apple tissue. The streak defect in both varieties had the most inconsistent results out of all the defect types observed. The streak defects in the raw AC images did not appear with consistent contrast with the surrounding tissue. In the other detectable defect types, the defective tissue appeared consistently with either lower or higher pixel intensity than the healthy tissue. In the case of the streak defect, some AC images contained streaks with a higher intensity than surrounding tissue, while other images had streaks of lower intensity. Segmentation of the streak defects could not be achieved using a single universal algorithm.  The PC ratio operation was able to successfully enhance defect contrast in the black scab, insect, sun scab, mechanical damage and rot defect images such that segmentation could be performed to isolate the defective tissue regions in each sample. Figures 5.23-5.27 displays 112  the results of the attempted defect segmentations. The segmented region areas would be compared to ground truth areas traced by hand in order to gauge detection accuracy.   113       114       115      The detection rates achieved using the PC ratio images did not perform as well as expected and saw lower detection rates than the uniform illumination image set. Therefore, an alternative image processing procedure was performed where ratio images were created using the AC and DC images just as they were in the two bruising experiments performed. The same thresholding and size filtering 116  operations were applied to the AC/DC ratio images as the PC ratio images with slight adjustments to threshold levels and the detection rates were determined. 5.8 Discriminant Model Analysis  SIRI was capable of achieving relatively high detection rates in certain defect types, while in others, the method struggled. Defect identification through segmentation may not be the ideal method of defect identification. Instead, samples with different defect types may be differentiated through discriminant analysis.  SIRI image features were extracted and used to create discriminant models for classifying each sample into its appropriate defect type. Based on the success rate of classification, the SIRI method can be judged on its ability to enhance certain features of each defect type. Various properties contained within each raw SIRI image were calculated and extracted as features for use in defect classification. The features extracted consisted of: mean intensity, standard deviation, entropy, contrast, energy, correlation, homogeneity, mean gradient magnitude, and direction. These features were calculated five times per sample, once for each spatial frequency and DC image available, for a total of 45 features per sample. The features were stored in an n x 45 element matrix, where n represents the number of samples per apple variety. A class label representing the sample defect type was assigned to iscriminant models were created for each apple variety. First, a linear discriminant model was trained using all of the available feature data. Following initial observations on the performance of these models on the same data used in training, a separate model was trained and evaluated using leave-one-out cross validation. There was a significant drop in performance once the models were cross-validated. To improve 117  classification rates, sequential backwards and sequential forward feature selection were performed to determine which features were most useful for distinguishing defect types. In order to determine if a certain feature would remain in the final feature set, a criterion was set. The criterion used during the feature selection process was the misclassification rate obtained when a certain feature was introduced (in forward selection) or removed (in backward selection) from the set. The feature selection algorithm would train and cross-validate a new discriminant model numerous times using a various set of features and calculate the resulting misclassification rate. A feature would be included in the final set only if the misclassification rate decreased as a result of adding or subtracting that feature from the raw set. Feature selection was completed once the misclassification criterion was minimized, the features used to obtain that minimized criterion would become the final selected feature set. Each method of sequential feature selection yielded a final, minimized criterion value. The method which resulted in the smallest misclassification rate would be the final set of features for use in model training. The success of classifying different defect types would be based on the results of a final discriminant model trained and cross-validated using the selected features.  5.9 Results   True defect areas were traced by hand using the DC images of the apples through Photoshop. The traced regions were filled in with white pixels and segmented in Matlab to acquire the area, in pixels, of each defective region. The number of foreground pixels was then counted in each of the segmentation results previously obtained. The resulting segmentation and ground truth areas were compared through a percent difference calculation to determine how accurately the sample 118  successfully detected using SIRI was determined by how much of the true defective area could be segmented. Due to the appearance of lenticels features in SIRI images, the segmented images may contain extraneous blobs representing noise rather than the actual defective tissue. The defective tissue may still have been successfully segmented in addition with the noise. Therefore, in order to determine whether at least the true defective regions were extracted, the segmented region blobs were subtracted from the true area images. The extra noise contained in the segmentation images would not appear in these difference images and only pixels which were undetected by the segmentation attempt would remain. The percentage of leftover, undetected pixels compared to the true area was calculated and a threshold value was set such that if the majority of pixels was detected in the SIRI images, the segmentation could be counted as a success.  Table 5.1 lists the detection rates achieved in each defect group under SIRI. The rot defect is distinct in nature due to the presence of subsurface tissue affecting the appearance of the defect in SIRI images. The ground truth for the rot defect was determined based solely on what here may not contain the internal aspects of rot. This may explain the relatively low detection results achieved for the rot defect. In order to compare the performance of SIRI to uniform illumination, defect detection was also attempted on the uniform illumination images acquired during the experiment. It would not have been feasible to compare the performance of the two methods by applying the same set of thresholding levels and filter parameters used in defect segmentation for SIRI to the uniform illumination images. Therefore new thresholding and image opening operation values were used to achieve segmentation results specifically for uniform 119  illumination images. The uniform illumination defect detection algorithm following defect segmentation was identical to the one used by the SIRI images where the segmented defect areas were compared to the traced ground truth regions. The detection rates achieved using uniform illumination images are presented in table 5.2. The detection rates were all higher under uniform illumination with the exception of rot detection which tied with SIRI. These results suggest that SIRI offers no advantage to surface defect detection and may actually decrease performance. One possible cause of the lower performance is likely due to the appearance of extraneous surface features at high spatial frequencies. The presence of the fruit lenticels made isolation of the defective regions extremely difficult. Some defect types such as insect damage occurred in similar sizes as the lenticels. Median filtering of the images prior to defect segmentation may aid in removing the noise, but similarly sized defect areas would also be removed as a result. The uniform illumination images did not exhibit any lenticel features, therefore, the segmentation process was much simpler to perform and the defective tissue was cleanly separable from the surrounding areas. Nonetheless, SIRI was able to demonstrate that it could enhance the ability to spatially resolve surface features through the use of high spatial frequencies. In this case, SIRI enhanced too many features and the information of interest was lost among extraneous features revealed by the technique.   Fuji  Gala  Defect Type Insect Damage Sun Scab Mechanical Damage Black Scab Rot Detection Rate (%) 80.0 85.7 55.6 77.8 20.0   120    Fuji Gala Defect Type Insect Damage Sun Scab Mechanical Damage Black Scab Rot Detection Rate (%) 86.7 100.0 77.8 83.3 20.0   The detection rates obtained using the AC/DC ratio images are presented in table 5.3. The detection performance of this image set more closely matched that of the uniform illumination set where the detection rates for insect damage and sun scab were identical between the two image sets. SIRI was able to outperform uniform illumination in the black scab and rot defect types. However, the mechanical defect type was still unable to match uniform illumination. One possibility of decreased performance may be that the defects in the mechanical group were more subtle and prone to being blocked out by the enhancement of non-defective features. The results of the AC/DC images also lent some insight in which range of spatial frequencies may be most appropriate for specific defect types as the detection performance of each individual spatial frequency tested could be obtained unlike in the PC ratio image set. It was observed that the detection rates for sun scab, and black scab saw an increase in detection as the spatial frequency of illumination increased toward the upper range used in the experiment. While the insect damage defect saw greater performance at the lower end of the spatial frequency range used. Mechanical damage detection seems to peak at 0.40 mm-1 and falls off on either side of that spatial frequency. The SIRI detection rates of the rot defect type are based on images acquired at the low spatial frequencies (0.10, 0.15, 0.20 mm-1) used in the experiment, which revealed more information related to the defect as the rot damage extended within the fruit and was invisible under high spatial frequency patterns. Here SIRI has reinforced its enhanced performance for subsurface defect detection as uniform illumination was not able to detect rot 121  damage beneath the fruit surface. The AC/DC images provide more favorable evidence than the PC ratio images for the use of SIRI in surface defect detection. SIRI was still unable to distinctively outperform uniform illumination; however, there were cases where detection rates were enhanced by the technique, especially in the rot defect.   Fuji Gala Frequency Insect Damage Sun Scab Mechanical Black Scab Rot Planar 86.7 100.0 77.8 83.3 20.0 0.30 86.7 85.7 55.6 83.3 *60.0 0.40 86.7 85.7 70.4 88.9 *60.0 0.50 73.3 100.0 59.3 88.9 *60.0 0.60 73.3 100.0 59.3 88.9  *The SIRI detection rates reported for the rot defect were obtained from images acquired at low spatial frequencies of 0.10, 0.15 and 0.25 mm-1 in ascending order from top to bottom of table 5.5.    Tables 5.4 and 5.5 present the average percent differences between the measured defect and true defect areas. The differences were typically very high, with some exceptions where the segmentation was quite accurate. In many cases the attempted defect segmentation contained blobs of noise, such as the apple lenticels, in addition to the true defective tissue. The data suggests that neither SIRI nor uniform illumination could accurately pinpoint where the boundaries of each defect were, but was able to instead identify the general regions where defective tissue may be. Based on these observations it may not be feasible to attempt to accurately measure surface defects with SIRI. The presence of the fruit lenticels and other skin textures which were enhanced through the SIRI technique had detrimental effects on detection in this study. However, SIRI did seem to outperform uniform illumination in accurately measuring the area of detected defects. While the effects of high spatial frequencies created noise and made defect detection more difficult, it may still enhance the clarity of the boundaries of the 122  surface defects making successful detections more accurate than their uniform illumination counterparts. Therefore, the effects of high spatial frequency on enhancing the resolution of the surface layer images was demonstrated and consistent with results obtained in the preliminary study. If direct defect segmentation is to be the goal of applying SIRI to surface defects, a more advanced algorithm capable of removing unwanted surface features needs to be developed. However, on average, the SIRI images typically had more accurate estimates of defect area than the uniform illumination images.  Percent Difference in Area (%) Illumination Type Insect Sun Scab Uniform 88.6 25.7 Sinusoidal 65.7 -3.6   Percent Difference in Area (%) Illumination Type Mechanical Black Scab Rot Uniform 121.6 89.4 -36.6 Sinusoidal 27.6 63.8 28.1   The results of each step of the discriminant analysis process are presented in tables 5.6-5.13. Tables 5.6 and 5.7 provide the true positive SIRI classification rates of each defect type in bold when all 45 features were considered in training. To obtain reliable measures of the distinctions between defect types, a cross-validated model was created. The model did not achieve high classification rates as observed in tables 5.6 and 5.7. However when observing individual class performances, some classes performed relatively well. The mildew defect, which could not be detected previously through segmentation, had the highest classification rate among the Fuji apples. Therefore, while the exact areas of the defect could not be extracted, 123  features which describe the defect may still be extracted and identified in the SIRI images. The same features were extracted from the uniform illumination images of the defective samples and a separate discriminant model was cross-validated. The results of uniform-illumination discriminant classification are presented in tables 5.8 and 5.9. The overall detection rate of the ly higher than the 56.9% achieved by the SIRI albeit less significant, at 63.8% compared to the 58.6% achieved using uniform illumination images. The individual clasunder uniform illumination with the exception of the streak defect where uniform illumination images could not successfully identify any streak defects, while the SIRI images were able to detect them to some degree.  Feature selection was performed to attempt to improve model performance using fewer features. The results of the newly trained and cross validated models saw improvement, as shown in tables 5.10-5.13under SIRI SIRI images reached a relatively high rate of 75.9%. A similar trend was observed when comparing the performance under SIRI with uniform illumination uniform illumination between uniform and structured illumination in individual clasvariety after feature selection as the largest difference in classification performance between the two methods was in observed in the mildew defect type. The other defect classification rates were not significantly different. This suggests that SIRI does not offer a distinct advantage over 124  uniform difference in classification performance after feature selection was performed. The SIRI classification rates saw greater individual defect classification rates in all classes other than the mechanical defect type, which was not significantly higher under uniform illumination. Once again the streak defect achieved a 0% classification rate under uniform illumination whereas SIRI achieved 50%. SIRI seems to demonstrate some potential of chapter 4 that biological differences between apple varieties may impact the performance of the SIRI system. The differences in defect classification performance between the two varieties when compared to uniform illumination may further support this previous claim. While the classification was not extremely high, it seems that the feature extracted from each defect type under SIRI do possess enough distinctions for feasible classification. Blasco et al. (2007) were able to achieve surface defect classification rates of up to 87% in citrus fruit using color images alone. When their color machine vision system was used in conjunction with NIR, ultra-violet and fluorescent imaging, the system could achieve successful detection rates of up to 95%. It seems that surface defect detection through structured illumination alone is not sufficient and that the rates achieved under SIRI could potentially be improved through obtaining additional feature information by combining the method with other technologies. Color is a feature that is often used in identifying defective samples (Leemans & Destain, 2004; Pydipati et al., 2006) and is often included in multispectral applications. The images acquired in this experiment were all done so under a broad wave range in the NIR region while using a grayscale CCD. Therefore, color features were not an option in this study. The SIRI method has the potential to combine with multispectral 125  systems offering the ability to acquire images at various spectral frequencies as well as spatial frequencies. Images obtained under such a system would allow for more features to be extracted, and could also potentially reveal more visible distinctions than just SIRI under a broad wave range. Therefore, improvements could still be made to increase the accuracy of both defect segmentation and classification. Until these adjustments are made and tested, it is unknown how the SIRI system will perform compared to uniform illumination under more optimal conditions. However, as an exploratory study into the potential of SIRI for the detection of surface defects in apple fruit, this experiment has demonstrated that the technique may possess some potential depending on the variety of fruit used.    Classification Rates (%)  Insect Mildew Sun Scab Streak Insect 53.3 26.7 20.0 0.0 Mildew 3.1 71.9 0.0 25.0 Sun Scab 14.3 28.6 28.6 28.6 Streak 5.6 50.0 0.0 44.4 Overall 56.9   Classification Rates (%)  Black Scab Mechanical Rot Streak Black Scab 66.7 16.7 11.1 5.6 Mechanical 7.4 70.4 3.7 18.5 Rot 0.0 40.0 40.0 20.0 Streak 25.0 25.0 0.0 50.0 Overall 63.8      126   Classification Rates (%)  Insect Mildew Sun Scab Streak Insect 60.0 20.0 20.0 0.0 Mildew 12.5 84.4 3.1 0.0 Sun Scab 28.6 42.9 28.6 0.0 Streak 0.0 33.3 0.0   66.7 Overall 69.4   Classification Rates (%)  Black Scab Mechanical Rot Streak Black Scab   61.1 38.9 0.0 0.0 Mechanic 7.4   74.1 3.7 14.8 Rot 0.0 40.0   60 0.0 Streak 12.5 87.5 0.0   0.0 Overall   58.6   Classification Rates (%)  Insect Mildew Sun Scab Streak Insect 60.0 20.0 20.0 0.0 Mildew 6.3 78.1 0.0 15.6 Sun Scab 42.9 14.3 28.6 14.2 Streak 0.0 44.4 0.0 55.6 Overall 63.9   Classification Rates (%)  Black Scab Mechanical Rot Streak Black Scab 77.8 16.7 5.6 0.0 Mechanical 7.4 81.5 0.0 11.1 Rot 0.0 20.0 80.0 0.0 Streak 37.5 12.5 0.0 50.0 Overall 75.9    127   Classification Rates (%)  Insect Mildew Sun Scab Streak Insect   60.0 33.3 6.7 0.0 Mildew 3.1   96.9 0.0 0.0 Sun Scab 42.9 28.6   28.6 0.0 Streak 0.0 33.3 0.0   66.7 Overall 75.0   Classification Rates (%)  Black Scab Mechanical Rot Streak Black Scab   55.6 44.4 0.0 0.0 Mechanic 7.4   85.2 3.7 3.7 Rot 0.0 40.0 60.0 0.0 Streak 0.0 100.0 0.0   0.0 Overall 62.1    128  CONCLUSION AND FUTURE WORK   This study explored the potential of SIRI as a new means for detecting subsurface and surface defects of apples. The basic underlying principle of controlling light penetration through the spatial frequency of projection allowed for the enhancement of subsurface and surface defects on apples. Preliminary tests on a highly diffusing synthetic material demonstrated that spatial frequency directly affected the light penetration and spatial resolution of foreign objects embedded in the material at different depths; low spatial frequencies penetrated deeper into the material and better revealed internal features, while higher spatial frequencies confined light and enhanced spatial resolution of defects closer to the surface of the sample. These important observations prompted further experiments on applying SIRI for enhancing fresh bruise detecting bruises in images acquired immediately after bruising, something which conventional uniform illumination was unable to do. SIRI outperformed conventional illumination even in done on bruises naturally incurred to apples run through an infield sorting machine. The various shapes, sizes and number of bruises that occurred in the test apples made detection much more difficult. However, the SIRI system was still able to achieve relative success with detection rates ranging from 81.1% - . A noticeable decrease in bruise detection rate was - 69.5%. These results suggested that apple variety may impact the performance of SIRI, which was not observed in the controlled bruising study. Differences in fruit physiology may impact which spatial frequencies are best suited for defect detection. Therefore, key spatial frequencies may need to be 129  discovered for optimizing defect detection in specific products. Because SIRI offered the ability to control light penetration not only within fruit tissue, but also potentially at the surface, a separate experiment was performed on apples with a variety of surface defects. The SIRI images acquired under high spatial frequency patterns did not enhance the defects as much as that was observed in the bruising studies. However, the features of fruit lenticels were enhanced by image demodulation, which had hindered the identification of defects during image processing. SIRI was able to detect the general defect regions with relative success ranging from 20% - 85.7%, however small defects and those which appeared on the surface as abnormal textures could not be identified based solely on the intensity distribution of the reflectance images. Furthermore, the performance under uniform illumination exceeded the results obtained under SIRI. While the technique demonstrated its ability to enhance surface features, the enhancement of undesired fruit lenticels served as a disadvantage instead of aiding the detection of surface defects. Identifying defects through a discriminant analysis model made detecting each defect type possible. Although SIRI did not achieve high rates of detection, the results suggest that the defect information was still present in the images. Classification rates achieved using uniform illumination  when compared to SIRI results. This provides further evidence that apple variety impacts SIRI performance and that for some varieties of fruit, SIRI may enhance the detection of surface defects. Thus, there is still potential in using this technique for surface defect detection.   The SIRI system used in all experiments throughout the work has shown promise in detecting defects in fruit tissue. However, the current methodology used to acquire and demodulate the reflectance images is too slow to satisfy commercial needs. The main weakness 130  of the current SIRI system is the method of image demodulation used, where three phase-shifted images are required to separate the DC and AC of each spatial frequency used in image collection. The demodulation step itself is fast, however the time needed to acquire each individual image significantly slows the system down. Additionally, to acquire the phase-shifted images, the current system requires the sample to remain stationary during the image acquisition in order to preserve the location of the sample in the scene. These requirements prevent the current SIRI system from achieving the high speed performance needed in the industry. These issues may all be mitigated through reducing the number of images required for the demodulation process. One solution would be developing a new demodulation technique that requires fewer, if any, phase-shifts so as to reduce the total number of images needed. Alternatively, it may be possible to combine multiple projection patterns into one composite pattern, allowing for simultaneous acquisition of various spatial frequency information in one image. The combination of a new demodulation process requiring fewer phase shifts and the ability to combine multiple spatial frequency patterns into one would significantly increase the image acquisition speed.  While improving the speed of the SIRI system should be a priority in order to make this technique practical for food quality inspection, attention should also be paid to improving the algorithm for defect detection. All experiments using the SIRI system acquired images using a broad wavelength range. This was done in order to explore the potential of the effects of spatial frequency on light penetration alone. It has been demonstrated in previous studies that specific spectral frequencies of illumination improve the detection of certain defects. The current SIRI system is capable of utilizing different light sources or a filter to select specific wavelengths of light for imaging. It is possible that the combination of both spatial and spectral frequency 131  information may improve the detection of fruit defects to a level above what neither method can achieve alone.  The results of the bruise studies have demonstrated that SIRI is indeed capable of consistently outperforming uniform illumination in detecting fresh bruises. The surface defect experiment presented less clear-cut results, as SIRI did not exhibit exceptionally high levels of performance in identifying defective tissue in apple fruit. Nevertheless, the technique was still able to demonstrate its ability to confine light penetration to specific tissue layers, and at the same time it enhanced the resolution of surface features of the samples, even though this also resulted in a noisier image. The SIRI images for surface defects contained specific defect features and, hence, better defect classification is still possible through further interrogation of the data, when compared to uniform illumination.  In conclusion, the SIRI technique is promising in detecting surface defects as well as subsurface defects. SIRI has shown its potential in measuring properties of fruit tissue at various depths. The system is relatively simple when compared to the instrumentation required for other optical measuring techniques. By combining with multispectral imaging technique, SIRI has potential for increased performance. Although at the present, SIRI is still too slow for commercial implementation for fruit sorting and grading, with the improvements proposed above, it may eventually achieve the speeds that are needed for commercial applications. The technique is relatively new, and it has had limited exploration in the field of agriculture. We hope the exploratory results obtained in these studies will bring new attention to the technique for further exploration of its full potential for a wider range of agricultural and food products.  132             APPENDIX   133   Variety Time after Impact Detection Rate for Each Spatial Frequency  Uniform 0.1 mm-1 0.15mm-1 0.25mm-1 Overall* Golden Delicious 0-1 Hours 0 75 70 45 90 Golden Delicious 4-6 Hours 30 75 80 55 85 Golden Delicious 24 Hours 0 40 35 20 50 Delicious 0-1 Hours 5 75 70 25 95 Delicious 4-6 Hours 10 70 80 45 90 Delicious 24 Hours 10 60 55 20 75    Variety Time after Impact Detection Rate for Each Spatial Frequency  Uniform 0.1 mm-1 0.15mm-1 0.25mm-1 Overall* Golden Delicious 0-1 Hours 5 95 100 65 100 Golden Delicious 4-6 Hours 50 95 90 65 100 Golden Delicious 24 Hours 20 50 50 50 70 Delicious 0-1 Hours 0 75 65 30 85 Delicious 4-6 Hours 10 95 90 55 95 Delicious 24 Hours 25 85 75 55 95    Variety Time after Impact Detection Rate for Each Spatial Frequency  Uniform 0.1 mm-1 0.15mm-1 0.25mm-1 Overall* Golden Delicious 0-1 Hours 5 95 90 60 95 Golden Delicious 4-6 Hours 50 100 100 80 100 Golden Delicious 24 Hours 25 85 65 40 100 Delicious 0-1 Hours 0 85 70 40 90 Delicious 4-6 Hours 25 100 100 65 100 Delicious 24 Hours 25 90 80 40 95       134   Variety Time after Impact Standard Deviation of Each Spatial Frequency  Uniform 0.1 mm-1 0.15mm-1 0.25mm-1 Overall* Golden Delicious 0-1 Hours 0 5 10 5 0 Golden Delicious 4-6 Hours 10 15 10 5 15 Golden Delicious 24 Hours 0 0 5 10 0 Delicious 0-1 Hours 5 5 10 15 5 Delicious 4-6 Hours 10 0 10 5 5 Delicious 24 Hours 10 10 5 0 15    Variety Time after Impact Standard Deviation of Each Spatial Frequency  Uniform 0.1 mm-1 0.15mm-1 0.25mm-1 Overall* Golden Delicious 0-1 Hours 5 5 0 5 0 Golden Delicious 4-6 Hours 10 5 10 5 0 Golden Delicious 24 Hours 10 10 10 0 10 Delicious 0-1 Hours 0 5 25 20 5 Delicious 4-6 Hours 10 5 10 5 5 Delicious 24 Hours 5 5 5 5 10    Variety Time after Impact Standard Deviation of Each Spatial Frequency  Uniform 0.1 mm-1 0.15mm-1 0.25mm-1 Overall* Golden Delicious 0-1 Hours 5 5 0 10 5 Golden Delicious 4-6 Hours 20 0 0 0 0 Golden Delicious 24 Hours 15 5 15 0 5 Delicious 0-1 Hours 0 5 0 10 0 Delicious 4-6 Hours 15 0 0 15 0 Delicious 24 Hours 15 0 10 0 5   135   Variety Time after Impact Spatial Frequency 0.1 mm-1 0.15mm-1 0.25mm-1 Golden Delicious 0-1 Hours 18.5 29.1 21.5 Golden Delicious 4-6 Hours 14.7 18.3 14.1 Golden Delicious 24 Hours 48.8 61.9 23.1 Delicious 0-1 Hours 26.2 23.9 71.8 Delicious 4-6 Hours 25.3 26.0 16.1 Delicious 24 Hours 54.4 59.7 49.7   Variety Time after Impact Spatial Frequency 0.1 mm-1 0.15mm-1 0.25mm-1 Golden Delicious 0-1 Hours 11.7 11.8 9.5 Golden Delicious 4-6 Hours 7.4 10.3 4.0 Golden Delicious 24 Hours 61.0 53.9 51.9 Delicious 0-1 Hours 27.1 50.2 28.2 Delicious 4-6 Hours 7.2 8.5 5.9 Delicious 24 Hours 20.5 14.3 19.9   Variety Time after Impact Spatial Frequency 0.1 mm-1 0.15mm-1 0.25mm-1 Golden Delicious 0-1 Hours -9.8 -17.1 -8.2 Golden Delicious 4-6 Hours -21.7 -20.2 -19.2 Golden Delicious 24 Hours -7.8 -8.0 2.9 Delicious 0-1 Hours -2.2 -5.5 -12.4 Delicious 4-6 Hours 1.0 0.6 -10.5 Delicious 24 Hours -3.3 -9.2 -12.4      136   Variety Time after Impact Standard Deviation of Each Spatial Frequency 0.1 mm-1 0.15mm-1 0.25mm-1 Golden Delicious 0-1 Hours 0.7 7.3 2.5 Golden Delicious 4-6 Hours 6.8 4.9 10.8 Golden Delicious 24 Hours 11.5 30.9 19.9 Delicious 0-1 Hours 12.8 11.0 23.7 Delicious 4-6 Hours 14.9 14.6 4.3 Delicious 24 Hours 6.8 15.0 18.3   Variety Time after Impact Standard Deviation of Each Spatial Frequency 0.1 mm-1 0.15mm-1 0.25mm-1 Golden Delicious 0-1 Hours 12.6 14.3 7.9 Golden Delicious 4-6 Hours 6.3 9.4 3.3 Golden Delicious 24 Hours 10.7 20.8 4.1 Delicious 0-1 Hours 12.7 1.4 7.9 Delicious 4-6 Hours 7.5 5.7 4.5 Delicious 24 Hours 7.0 2.3 14.6   Variety Time after Impact Standard Deviation of Each Spatial Frequency 0.1 mm-1 0.15mm-1 0.25mm-1 Golden Delicious 0-1 Hours 12.2 8.4 1.5 Golden Delicious 4-6 Hours 4.6 7.0 2.1 Golden Delicious 24 Hours 4.7 1.6 3.2 Delicious 0-1 Hours 4.2 2.6 2.6 Delicious 4-6 Hours 8.1 9.0 2.4 Delicious 24 Hours 5.6 6.6 3.2   137  BIBLIOGRAPHY138  BIBLIOGRAPHY Abbott. (2010). Relationship of Sonic Resonant Frequency to Compression Tests and Magness-Taylor Firmness of Apples During Refrigerated Storage. Transactions of the ASABE.   Abbott, J. A. (1999). Quality measurement of fruits and vegetables. Postharvest Biology and Technology, 15(3), 207-225. doi:http://dx.doi.org/10.1016/S0925-5214(98)00086-6  Anderson, E. R., Cuccia, D. J., & Durkin, A. J. (2007). Detection of bruises on golden delicious apples using spatial- frequency-domain imaging.  Aneshansley, D., Throop, J., & Upchurch, B. (1997). Reflectance spectra of surface defects on apples. Paper presented at the Proceedings of the Sensors for Nondestructive Testing International Conference, Northeast Regional Agricultural Engineering Service.  Ariana, D. P., & Lu, R. (2008a). Quality evaluation of pickling cucumbers using hyperspectral reflectance and transmittance imagingPart II. Performance of a prototype. Sensing and Instrumentation for Food Quality and Safety, 2(3), 152-160. doi:10.1007/s11694-008-9058-9  Ariana, D. P., & Lu, R. (2008b). Quality evaluation of pickling cucumbers using hyperspectral reflectance and transmittance imaging: Part I. Development of a prototype. Sensing and Instrumentation for Food Quality and Safety, 2(3), 144-151. doi:10.1007/s11694-008-9057-x  Ariana, D. P., & Lu, R. (2010). DETECTION OF INTERNAL DEFECT IN PICKLING CUCUMBERS USING LASER SCATTERING IMAGE ANALYSIS. XVIIth World Congress of the International Commission of Agricultural and Biosystems Engineering (CIGR).   Bassi, Cuccia, D. J., Durkin, A. J., & Tromberg, B. J. (2008). Spatial shift of spatially modulated light projected on turbid media. Journal of the Optical Society of America A, 25(11), 2833-2839. doi:10.1364/JOSAA.25.002833  Bassi, A., D'Andrea, C., Valentini, G., Cubeddu, R., & Arridge, S. (2009). Detection of inhomogeneities in diffusive media using spatially modulated light. Opt Lett, 34(14), 2156-2158. doi:10.1364/OL.34.002156  Blasco, J., Aleixos, N., Gómez, J., & Moltó, E. (2007). Citrus sorting by identification of the most common defects using multispectral computer vision. Journal of Food Engineering, 83(3), 384-393. doi:http://dx.doi.org/10.1016/j.jfoodeng.2007.03.027  Buzby, J. C., Farah-Wells, H., & Hyman, J. (2014). The estimated amount, value, and calories of postharvest food losses at the retail and consumer levels in the United States. USDA-ERS Economic Information Bulletin(121).     139  Cho, B.-K., Chayaprasert, W., & Stroshine, R. L. (2008). Effects of internal browning and watercore on low field (5.4 MHz) proton magnetic resonance measurements of T2 values of whole apples. Postharvest Biology and Technology, 47(1), 81-89. doi:http://dx.doi.org/10.1016/j.postharvbio.2007.05.018  Cuccia, Bevilacqua, F., Durkin, A. J., & Tromberg, B. J. (2005). Modulated imaging: quantitative analysis and tomography of turbid media in the spatial-frequency domain. Opt Lett, 30(11), 1354-1356. doi:10.1364/OL.30.001354  De Ketelaere, B., & De Baerdemaeker, J. (2001). Tomato firmness estimation using vibration measurements. Mathematics and Computers in Simulation, 56(45), 385-394. doi:http://dx.doi.org/10.1016/S0378-4754(01)00309-3  Delwiche, M. J., McDonald, T., & Bowers, S. V. (1987). Determination of peach firmness by analysis of impact forces. Transactions of the ASAE, 30(1), 249-0254.   Donis-González, I. R., Guyer, D. E., Leiva-Valenzuela, G. A., & Burns, J. (2013). Assessment of chestnut (Castanea spp.) slice quality using color images. Journal of Food Engineering, 115(3), 407-414. doi:http://dx.doi.org/10.1016/j.jfoodeng.2012.09.017  Elmasry, G., Kamruzzaman, M., Sun, D.-W., & Allen, P. (2012). Principles and applications of hyperspectral imaging in quality evaluation of agro-food products: a review. Critical reviews in food science and nutrition, 52(11), 999.   ElMasry, G., Wang, N., & Vigneault, C. (2009). Detecting chilling injury in Red Delicious apple using hyperspectral imaging and neural networks. Postharvest Biology and Technology, 52(1), 1-8. doi:10.1016/j.postharvbio.2008.11.008  ElMasry, G., Wang, N., Vigneault, C., Qiao, J., & ElSayed, A. (2008). Early detection of apple bruises on different background colors using hyperspectral imaging. LWT - Food Science and Technology, 41(2), 337-345. doi:10.1016/j.lwt.2007.02.022  Feng, Y.-Z., & Sun, D.-W. (2012). Application of hyperspectral imaging in food safety inspection and control: a review. Critical reviews in food science and nutrition, 52(11), 1039.   Gonzalez, J. J., Valle, R. C., Bobroff, S., Biasi, W. V., Mitcham, E. J., & McCarthy, M. J. (2001). Detection Postharvest Biology and Technology, 22(2), 179-188. doi:http://dx.doi.org/10.1016/S0925-5214(00)00183-6  Haff, R. P., & Toyofuku, N. (2008). X-ray detection of defects and contaminants in the food industry. Sensing and Instrumentation for Food Quality and Safety, 2(4), 262-273. doi:10.1007/s11694-008-9059-8  Hansen, J. D., Schlaman, D. W., Haff, R. P., & Yee, W. L. (2005). Potential postharvest use of radiography to detect internal pests in deciduous tree fruits. Journal of Entomological Science, 40(3), 255-262.   140  Hernández-Sánchez, N., Barreiro, P., Ruiz-Altisent, M., Ruiz-Cabello, J., & Fernández-Valle, M. E. (2004). Detection of freeze injury in oranges by magnetic resonance imaging of moving samples. Applied Magnetic Resonance, 26(3), 431-445. doi:10.1007/BF03166814  Huang, W., Li, J., Wang, Q., & Chen, L. (2015). Development of a multispectral imaging system for online detection of bruises on apples. Journal of Food Engineering, 146, 62-71. doi:http://dx.doi.org/10.1016/j.jfoodeng.2014.09.002  Kim, & Schatzki. (2010). APPLE WATERCORE SORTING SYSTEM USING X-RAY IMAGERY: I. ALGORITHM DEVELOPMENT. Transactions of the ASABE.   Kim, M., Chao, K., Chan, D., Jun, W., Lefcourt, A., Delwiche, S., . . . Lee, K. (2011). Line-scan hyperspectral imaging platform for agro-food safety and quality evaluation: System enhancement and characterization. Transactions of the ASABE, 54(2), 703-711.   Lammertyn, J., Dresselaers, T., Van Hecke, P., Jancsók, P., Wevers, M., & Nicolaï, B. M. (2003). MRI and x-Magnetic Resonance Imaging, 21(7), 805-815. doi:10.1016/s0730-725x(03)00105-x  Lammertyn, J., Nicolaï, B., Ooms, K., Smedt, V. D., & Baerdemaeker, J. D. (1998). NON-DESTRUCTIVE MEASUREMENT OF ACIDITY, SOLUBLE SOLIDS, AND FIRMNESS OF JONAGOLD APPLES USING NIR-SPECTROSCOPY. 41(4).   Leemans, V., & Destain, M. F. (2004). A real-time grading method of apples based on features extracted from defects. Journal of Food Engineering, 61(1), 83-89. doi:http://dx.doi.org/10.1016/S0260-8774(03)00189-4  ripening and storage. LWT - Food Science and Technology, 36(7), 719-727. doi:http://dx.doi.org/10.1016/S0023-6438(03)00099-9  Lu, R. (2001). Predicting firmness and sugar content of sweet cherries using near-infrared diffuse reflectance spectroscopy. Transactions of the ASAE, 44(5), 1265-1271.   Lu, R. (2003). DETECTION OF BRUISES ON APPLES USING NEARINFRARED HYPERSPECTRAL IMAGING. Transactions of the ASAE, 46(2), 523. doi:10.13031/2013.12941  Lu, R. (2007). Nondestructive measurement of firmness and soluble solids content for apple fruit using hyperspectral scattering images. Sensing and Instrumentation for Food Quality and Safety, 1(1), 19-27. doi:10.1007/s11694-006-9002-9  Lu, R., Ariana, D. P., & Cen, H. (2011). Optical absorption and scattering properties of normal and defective pickling cucumbers for 7001000 nm. Sensing and Instrumentation for Food Quality and Safety, 5(2), 51-56. doi:10.1007/s11694-011-9108-6  Marigheto, N., Venturi, L., & Hills, B. (2008). Two-dimensional NMR relaxation studies of apple quality. Postharvest Biology and Technology, 48(3), 331-340. doi:http://dx.doi.org/10.1016/j.postharvbio.2007.11.002 141  McGlone, V. A., & Kawano, S. (1998). Firmness, dry-matter and soluble-solids assessment of postharvest kiwifruit by NIR spectroscopy. Postharvest Biology and Technology, 13(2), 131-141. doi:http://dx.doi.org/10.1016/S0925-5214(98)00007-6  ng optical sectioning by using structured light in a conventional microscope. Opt Lett, 22(24), 1905-1907. doi:10.1364/OL.22.001905  Nicolai, B. M., Defraeye, T., De Ketelaere, B., Herremans, E., Hertog, M. L., Saeys, W., . . . Verboven, P. (2014). Nondestructive measurement of fruit and vegetable quality. Annu Rev Food Sci Technol, 5, 285-312. doi:10.1146/annurev-food-030713-092410  Osborne, B. G., Fearn, T., & Hindle, P. H. (1993). Practical NIR spectroscopy with applications in food and beverage analysis: Longman scientific and technical.  Pan, Zhu, Q., Lu, R., & McGrath, J. M. (2015). Determination of sucrose content in sugar beet by portable visible and near-infrared spectroscopy. Food Chem, 167, 264-271. doi:10.1016/j.foodchem.2014.06.117  Park, B., Abbott, J. A., Lee, K., Choi, C., & Choi, K. (2003). Near-infrared diffuse reflectance for quantitative and qualitative measurement of soluble solids and firmness of Delicious and Gala apples. Transactions of the ASAE, 46(6), 1721.   Pydipati, R., Burks, T. F., & Lee, W. S. (2006). Identification of citrus disease using color texture features and discriminant analysis. Computers and Electronics in Agriculture, 52(12), 49-59. doi:http://dx.doi.org/10.1016/j.compag.2006.01.004  Ragni, L., Berardinelli, A., & Guarnieri, A. (2010). Impact device for measuring the flesh firmness of kiwifruits. Journal of Food Engineering, 96(4), 591-597. doi:http://dx.doi.org/10.1016/j.jfoodeng.2009.09.006  Shahin, M., Tollner, E., McClendon, R., & Arabnia, H. (2002). Apple classification based on surface bruises using image processing and neural networks. Transactions of the ASAE, 45(5), 1619.   Shmulevich, I., Galili, N., & Howarth, M. S. (2003). Nondestructive dynamic testing of apples for firmness evaluation. Postharvest Biology and Technology, 29(3), 287-299. doi:http://dx.doi.org/10.1016/S0925-5214(03)00039-5  Sun, D.-W. (2010). Hyperspectral Imaging for Food Quality Analysis and Control. San Diego: Academic Press [Imprint].  Upchurch, B. L., Throop, J. A., & Aneshansley, D. J. (1994). Influence of time, bruise-type, and severity on near-infrared reflectance from apple surfaces for automatic bruise detection. Transactions of the American Society of Agricultural Engineers, 37(5), 1571-1575.   Weber, J. R., Cuccia, D. J., Durkin, A. J., & Tromberg, B. J. (2009). Noncontact imaging of absorption and  Scattering in layered tissue using spatially modulated structured light. Journal of Applied Physics, 105(10), 102028. doi:10.1063/1.3116135 142  Weber, J. R., Cuccia, D. J., Johnson, W. R., Bearman, G. H., Durkin, A. J., Hsu, M., . . . Tromberg, B. J. (2011). Multispectral imaging of tissue absorption and scattering using spatial frequency domain imaging and a computed-tomography imaging spectrometer. J Biomed Opt, 16(1), 011015. doi:10.1117/1.3528628  Computers and Electronics in Agriculture, 52(12), 11-20. doi:http://dx.doi.org/10.1016/j.compag.2006.01.006  Xing, J., Guyer, D., Ariana, D., & Lu, R. (2008). Determining optimal wavebands using genetic algorithm for detection of internal insect infestation in tart cherry. Sensing and Instrumentation for Food Quality and Safety, 2(3), 161-167. doi:10.1007/s11694-008-9047-z