Advanced Driver-Assistance Systems (ADAS) and autonomous systems, in general, such as emerging autonomous vehicles rely heavily on visual data and state-of-the-art deep learning approaches to classify and localize objects such as pedestrians, traffic signs and lights, and other nearby cars, to assist the corresponding vehicles maneuver safely in their environments. However, due to the well-known domain shift problem, the performance of object detection methods could degrade rather significantly under challenging scenarios such as low light and adverse weather conditions. The domain shift problem arises due to the difference between the distributions of source data used for training in comparison with target data used during realistic testing scenarios. The area of domain adaptation has been instrumental in addressing the domain shift problem encountered by many applications. In fact, domain adaptation frameworks for object detection methods have been providing powerful tools for handling a variety of underlying changes in probability distribution between training and testing data. In this dissertation, we first propose a novel integrated Generative-model based unsupervised training and Domain Adaptation (GDA) framework that improves the performance of a region-proposal based object detector under challenging scenarios. In particular, we exploit unsupervised image-to-image translation to generate annotated visuals that are representatives of a target challenging domain. Then, we use these generated annotated visuals in addition to unlabeled target domain data to train a domain adaptive region-proposal based object detector. We show that using this integrated approach outperforms both methods, unsupervised image translation, and domain adaptation, when they are used separately.℗ Despite the popularity of region-proposal based object detectors, such as Faster R-CNN and many of its variants, these detectors suffer from a long inference time. Therefore, such approaches are not the optimal choice for time-critical, real-time applications such as autonomous driving. As a result, in the second part of this dissertation, we propose a novel MultiScale Domain Adaptive YOLO (MS-DAYOLO) framework for the popular state-of-the-art real time object detector YOLO. MS-DAYOLO employs multiple domain adaptation paths and corresponding domain classifiers at different scales of the recently introduced YOLOv4 object detector. Building on our baseline MS-DAYOLO architecture, we introduce three novel deep learning architectures for a Domain Adaptation Network (DAN) that generates domain-invariant features. In particular, we propose a Progressive Feature Reduction, a Unified Domain Classifier, and an Integrated architecture.While RGB cameras represent the most popular imaging sensors used by ADAS systems and autonomous vehicles due to cost and related practical reasons, employing other modalities such as thermal and gated imaging sensors can significantly improve the detection performance under challenging conditions. However, these other types of sensors are expensive, and incorporating them into ADAS and autonomous vehicle platforms may cause design and manufacturing challenges. As a result, in the third part of this dissertation, we propose a new framework that utilizes Cross Modality Knowledge Distillation (CMKD) to improve the performance of RGB-only pedestrian detection in low light and adverse weather conditions without increasing computational complexity during inference. Specifically, we develop two CMKD methods that rely on feature-based knowledge distillation and adversarial training to transfer knowledge from a detector (teacher) that is trained using multiple modalities to a single modality detector (student) that is trained using RGB images only.℗ To validate the proposed approaches, we train and test them using popular datasets captured by vehicles driving under different conditions including challenging scenarios. Our experiments with the proposed approaches show significant improvements in object detection performance in comparison with state-of-the-art methods.
Read
