Computer vision technology coupled with uniform illumination is still unsatisfactory in detecting some defects of apples, e.g., subsurface tissue bruising. Structured illumination (SI) provides an alternative illumination scheme, which is advantageous in obtaining high-contrast/resolution images and controlling light penetration for depth-resolved imaging. The objective of this research was therefore to develop a new imaging modality based on SI, which is hereinafter referred to as structured-illumination reflectance imaging (SIRI), to enhance defect detection of apples. A SIRI platform, capable of acquiring broadband and multispectral images, was constructed by using a digital micro-mirror device based digital light projector. Using sinusoidally-modulated illumination patterns, SIRI acquires phase-shifted pattern images from samples, which are then demodulated into three sets of images, direct component (DC) and amplitude component (AC), and phase, each of which provides different image features. An image formation model and a general demodulation methodology for SIRI were proposed for assessing the features of the technique regarding image contrast and resolution and light penetration. Experiments for standard targets and apple samples showed that AC resulted in higher image resolution and contrast but lower light penetration than DC, and that its effectiveness in defect detection depended on defect type, fruit variety or surface morphology, and spatial frequency of illumination patterns. Demodulation is a critical step in obtaining DC and AC images, and conventional approaches require a minimum of three phase-shifted pattern images. New demodulation approaches based on spiral phase transform (SPT) and Gram-Schmidt orthonormalization, which required only two phase-shifted pattern images, were proposed to improve the SIRI image acquisition speed and lay a foundation for future real-time implementation of the technique. In addition, a new approach, based on solving a linear system composed of phase-shifted pattern images, was also proposed for demodulating composite sinusoidal patterns that contain more than one frequency components. Furthermore, a phase analysis methodology, based upon phase demodulation, phase unwrapping, phase-to-height conversion and in-plane calibrations, was proposed for reconstructing the three-dimensional (3-D) geometry of apples from acquired SIRI pattern images to aid in detection of surface concavities (i.e., stem/calyx regions) and enhance defect detection.Experiments were conducted on detection of subsurface bruising in apples of four varieties by using a multispectral SIRI system, in conjunction with the SPT demodulation approach. The six wavebands between 710 and 810 nm yielded the overall detection errors of 11.7-14.2%. Further experiments were conducted on detection of general surface and subsurface defects for apples of two varieties by using SIRI images at 730 nm, aided with machine learning algorithms. Compared to DC and AC images used individually, their combinations and or plus RT (ratio, the division of AC by DC) images achieved significantly better classification results, and convolutional neural network (CNN) resulted in the highest overall accuracies of 98% for both varieties of apples. Finally, an exploratory study was conducted on using SIRI, based on the two-phase SPT demodulation, to acquire pattern images from moving samples to evaluate the potential of the technique for future real-time applications, which accentuated the need for a high-speed camera for SIRI to operate at a practically acceptable sample moving speed. This research has demonstrated the feasibility and potential of SIRI as a new versatile and effective tool for enhancing defect detection of apples. More research, however, is needed to improve the instrumentation and software of SIRI for real-time, practical applications.
Read
