Search results

Title: OPTIMIZATION OF LARGE SCALE ITERATIVE EIGENSOLVERS
Creator: Afibuzzaman, Md
Date: 2021
Collection: Electronic Theses & Dissertations
Description: Sparse matrix computations, in the form of solvers for systems of linear equations, eigenvalue problem or matrix factorizations constitute the main kernel in problems from fields as diverse as computational fluid dynamics, quantum many body problems, machine learning and graph analytics. Iterative eigensolvers have been preferred over the regular method because the regular method not being feasible with industrial sized matrices. Although dense linear algebra libraries like BLAS, LAPACK,...
Show moreSparse matrix computations, in the form of solvers for systems of linear equations, eigenvalue problem or matrix factorizations constitute the main kernel in problems from fields as diverse as computational fluid dynamics, quantum many body problems, machine learning and graph analytics. Iterative eigensolvers have been preferred over the regular method because the regular method not being feasible with industrial sized matrices. Although dense linear algebra libraries like BLAS, LAPACK, SCALAPACK are well established and some vendor optimized implementation like mkl from Intel or Cray Libsci exist, it is not the same case for sparse linear algebra which is lagging far behind. The main reason behind slow progress in the standardization of sparse linear algebra or library development is the different forms and properties depending on the application area. It is worsened for deep memory hierarchies of modern architectures due to low arithmetic intensities and memory bound computations. Minimization of data movement and fast access to the matrix are critical in this case. Since the current technology is driven by deep memory architectures where we get the increased capacity at the expense of increased latency and decreased bandwidth when we go further from the processors. The key to achieve high performance in sparse matrix computations in deep memory hierarchy is to minimize data movement across layers of the memory and overlap data movement with computations. My thesis work contributes towards addressing the algorithmic challenges and developing a computational infrastructure to achieve high performance in scientific applications for both shared memory and distributed memory architectures. For this purpose, I started working on optimizing a blocked eigensolver and optimized specific computational kernels which uses a new storage format. Using this optimization as a building block, we introduce a shared memory task parallel framework focusing on optimizing the entire solvers rather than a specific kernel. Before extending this shared memory implementation to a distributed memory architecture, I simulated the communication pattern and overheads of a large scale distributed memory application and then I introduce the communication tasks in the framework to overlap communication and computation. Additionally, I also tried to find a custom scheduler for the tasks using a graph partitioner. To get acquainted with high performance computing and parallel libraries, I started my PhD journey with optimizing a DFT code named Sky3D where I used dense matrix libraries. Despite there might not be any single solution for this problem, I tried to find an optimized solution. Though the large distributed memory application MFDn is kind of the driver project of the thesis, but the framework we developed is not confined to MFDn only, rather it can be used for other scientific applications too. The output of this thesis is the task parallel HPC infrastructure that we envisioned for both shared and distributed memory architectures.
Show less

Title: Deep Convolutional Networks for Modeling Geo-Spatio-Temporal Relationships and Extremes
Creator: Wilson, Tyler
Date: 2021
Collection: Electronic Theses & Dissertations
Description: Geo-spatio-temporal data are valuable for a broad range of applications including traffic forecasting, weather prediction, detection of epidemic outbreaks, and crime monitoring. Data driven approaches to these problems must address several fundamental challenges such as handling the %The two we focus on are the importance ofgeo-spatio-temporal relationships and extreme events. Another recent technological shift has been the success of deep learning especially in applications such as computer...
Show moreGeo-spatio-temporal data are valuable for a broad range of applications including traffic forecasting, weather prediction, detection of epidemic outbreaks, and crime monitoring. Data driven approaches to these problems must address several fundamental challenges such as handling the %The two we focus on are the importance ofgeo-spatio-temporal relationships and extreme events. Another recent technological shift has been the success of deep learning especially in applications such as computer vision, speech recognition, and natural language processing. In this work, we argue that deep learning is a promising approach for many geo-spatio-temporal problems and highlight how it can be used to address the challenges of modeling geo-spatio-temporal relationships and extremes. Though previous research has established techniques for modeling spatio-temporal relationships, these approaches are often limited to gridded spatial data with fixed-length feature vectors and considered only spatial relationships among the features, while ignoring the relationships among model parameters.We begin by describing how the spatial and temporal relationships for non-gridded spatial data can be modeled simultaneously by coupling the graph convolutional network with a long short-term memory (LSTM) network. Unlike previous research, our framework treats the adjacency matrix associated with the spatial data as a model parameter that can be learned from data, with constraints on its sparsity and rank to reduce the number of estimated parameters.Further, we show that the learned adjacency matrix may reveal useful information about the dominant spatial relationships that exist within the data. Second, we explore the varieties of spatial relationships that may exist in a geo-spatial prediction task. Specifically, we distinguish between spatial relationships among predictors and the spatial relationships among model parameters at different locations. We demonstrate an approach for modeling spatial dependencies among model parameters using graph convolution and provide guidance on when convolution of each type can be effectively applied. We evaluate our proposed approach on a climate downscaling and weather prediction tasks. Next, we introduce DeepGPD, a novel deep learning framework for predicting the distribution of geo-spatio-temporal extreme events. We draw on research in extreme value theory and use the generalized Pareto distribution (GPD) to model the distribution of excesses over a threshold. The GPD is integrated into our deep learning framework to learn the distribution of future excess values while incorporating the geo-spatio-temporal relationships present in the data. This requires a novel reparameterization of the GPD to ensure that its constraints are satisfied by the outputs of the neural network. We demonstrate the effectiveness of our proposed approach on a real-world precipitation data set. DeepGPD also employs a deep set architecture to handle the variable-sized feature sets corresponding to excess values from previous time steps as its predictors. Finally, we extend the DeepGPD formulation to simultaneously predict the distribution of extreme events and accurately infer their point estimates. Doing so requires modeling the full distribution of the data not just its extreme values. We propose DEMM, a deep mixture model for modeling the distribution of both excess and non-excess values. To ensure the point estimation of DEMM is a feasible value, new constraints on the output of the neural network are introduced, which requires a new reparameterization of the model parameters of the GPD. We conclude by discussing possibilities for further research at the intersection of deep learning and geo-spatio-temporal data.
Show less

Title: Dissertation : novel parallel algorithms and performance optimization techniques for the multi-level fast multipole algorithm
Creator: Lingg, Michael
Date: 2020
Collection: Electronic Theses & Dissertations
Description: Since Sir Issac Newton determined that characterizing orbits of celestial objects required considering the gravitational interactions among all bodies in the system, the N-Body problem has been a very important tool in physics simulations. Expanding on the early use of the classical N-Body problem for gravitational simulations, the method has proven invaluable in fluid dynamics, molecular simulations and data analytics. The extension of the classical N-Body problem to solve the Helmholtz...
Show moreSince Sir Issac Newton determined that characterizing orbits of celestial objects required considering the gravitational interactions among all bodies in the system, the N-Body problem has been a very important tool in physics simulations. Expanding on the early use of the classical N-Body problem for gravitational simulations, the method has proven invaluable in fluid dynamics, molecular simulations and data analytics. The extension of the classical N-Body problem to solve the Helmholtz equation for groups of particles with oscillatory interactions has allowed for simulations that assist in antenna design, radar cross section prediction, reduction of engine noise, and medical devices that utilize sound waves, to name a sample of possible applications. While N-Body simulations are extremely valuable, the computational cost of directly evaluating interactions among all pairs grows quadratically with the number of particles, rendering large scale simulations infeasible even on the most powerful supercomputers. The Fast Multipole Method (FMM) and the broader class of tree algorithms that it belongs to have significantly reduced the computational complexity of N-body simulations, while providing controllable accuracy guarantees. While FMM provided a significant boost, N-body problems tackled by scientists and engineers continue to grow larger in size, necessitating the development of efficient parallel algorithms and implementations to run on supercomputers. The Laplace variant of FMM, which is used to treat the classical N-body problem, has been extensively researched and optimized to the extent that Laplace FMM codes can scale to tens of thousands of processors for simulations involving over trillion particles. In contrast, the Multi-Level Fast Multipole Algorithm (MLFMA), which is aimed for the Helmholtz kernel variant of FMM, lags significantly behind in efficiency and scaling. The added complexity of an oscillatory potential results in much more intricate data dependency patterns and load balancing requirements among parallel processes, making algorithms and optimizations developed for Laplace FMM mostly ineffective for MLFMA. In this thesis, we propose novel parallel algorithms and performance optimization techniques to improve the performance of MLFMA on modern computer architectures. Proposed algorithms and performance optimizations range from efficient leveraging of the memory hierarchy on multi-core processors to an investigation of the benefits of the emerging concept of task parallelism for MLFMA, and to significant reductions of communication overheads and load imbalances in large scale computations. Parallel algorithms for distributed memory parallel MLFMA are also accompanied by detailed complexity analyses and performance models. We describe efficient implementations of all proposed algorithms and optimization techniques, and analyze their impact in detail. In particular, we show that our work yields significant speedups and much improved scalability compared to existing methods for MLFMA in large geometries designed to test the range of the problem space, as well as in real world problems.
Show less

Title: The Evolution of Fundamental Neural Circuits for Cognition in Silico
Creator: Tehrani-Saleh, Ali
Date: 2021
Collection: Electronic Theses & Dissertations
Description: Despite decades of research on intelligence and fundamental components of cognition, we still know very little about the structure and functionality of nervous systems. Questions in cognition and intelligent behavior are addressed by scientists in the fields of behavioral biology, neuroscience, psychology, and computer science. Yet it is difficult to reverse engineer observed sophisticated intelligent behaviors in animals and even more difficult to understand their underlying mechanisms.In...
Show moreDespite decades of research on intelligence and fundamental components of cognition, we still know very little about the structure and functionality of nervous systems. Questions in cognition and intelligent behavior are addressed by scientists in the fields of behavioral biology, neuroscience, psychology, and computer science. Yet it is difficult to reverse engineer observed sophisticated intelligent behaviors in animals and even more difficult to understand their underlying mechanisms.In this dissertation, I use a recently-developed neuroevolution platform -called Markov brain networks- in which Darwinian selection is used to evolve both structure and functionality of digital brains. I use this platform to study some of the most fundamental cognitive neural circuits: 1) visual motion detection, 2) collision-avoidance based on visual motion cues, 3) sound localization, and 4) time perception. In particular, I investigate both the selective pressures and environmental conditions in the evolution of these cognitive components, as well as the circuitry and computations behind them. This dissertation lays the groundwork for an evolutionary agent-based method to study the neural circuits for cognition in silico.
Show less

Title: Optimal Learning of Deployment and Search Strategies for Robotic Teams
Creator: Wei, Lai
Date: 2021
Collection: Electronic Theses & Dissertations
Description: In the problem of optimal learning, the dilemma of exploration and exploitation stems from the fact that gathering information and exploiting it are, in many cases, two mutually exclusive activities. The key to optimal learning is to strike a balance between exploration and exploitation. The Multi-Armed Bandit (MAB) problem is a prototypical example of such an explore-exploit tradeoff, in which a decision-maker sequentially allocates a single resource by repeatedly choosing one among a set of...
Show moreIn the problem of optimal learning, the dilemma of exploration and exploitation stems from the fact that gathering information and exploiting it are, in many cases, two mutually exclusive activities. The key to optimal learning is to strike a balance between exploration and exploitation. The Multi-Armed Bandit (MAB) problem is a prototypical example of such an explore-exploit tradeoff, in which a decision-maker sequentially allocates a single resource by repeatedly choosing one among a set of options that provide stochastic rewards. The MAB setup has been applied in many robotics problems such as foraging, surveillance, and target search, wherein the task of robots can be modeled as collecting stochastic rewards. The theoretical work of this dissertation is based on the MAB setup and three problem variations, namely heavy-tailed bandits, nonstationary bandits, and multi-player bandits, are studied. The first two variations capture two key features of stochastic feedback in complex and uncertain environments: heavy-tailed distributions and nonstationarity; while the last one addresses the problem of achieving coordination in uncertain environments. We design several algorithms that are robust to heavy-tailed distributions and nonstationary environments. Besides, two distributed policies that require no communication among agents are designed for the multi-player stochastic bandits in a piece-wise stationary environment.The MAB problems provide a natural framework to study robotic search problems. The above variations of the MAB problems directly map to robotic search tasks in which a robot team searches for a target from a fixed set of view-points (arms). We further focus on the class of search problems involving the search of an unknown number of targets in a large or continuous space. We view the multi-target search problem as a hot-spots identification problem in which, instead of the global maximum of the field, all locations with a value greater than a threshold need to be identified. We consider a robot moving in 3D space with a downward-facing camera sensor. We model the robot's sensing output using a multi-fidelity Gaussian Process (GP) that systematically describes the sensing information available at different altitudes from the floor. Based on the sensing model, we design a novel algorithm that (i) addresses the coverage-accuracy tradeoff: sampling at a location farther from the floor provides a wider field of view but less accurate measurements, (ii) computes an occupancy map of the floor within a prescribed accuracy and quickly eliminates unoccupied regions from the search space, and (iii) travels efficiently to collect the required samples for target detection. We rigorously analyze the algorithm and establish formal guarantees on the target detection accuracy and the detection time.An approach to extend the single robot search policy to multiple robots is to partition the environment into multiple regions such that workload is equitably distributed among all regions and then assign a robot to each region. The coverage control focuses on such equitable partitioning and the workload is equivalent to the so-called service demands in the coverage control literature. In particular, we study the adaptive coverage control problem, in which the demands of robotic service within the environment are modeled as a GP. To optimize the coverage of service demands in the environment, the team of robots aims to partition the environment and achieve a configuration that minimizes the coverage cost, which is a measure of the average distance of a service demand from the nearest robot. The robots need to address the explore-exploit tradeoff: to minimize coverage cost, they need to gather information about demands within the environment, whereas information gathering deviates them from maintaining a good coverage configuration. We propose an algorithm that schedules learning and coverage epochs such that its emphasis gradually shifts from exploration to exploitation while never fully ceasing to learn. Using a novel definition of coverage regret, we analyze the algorithm and characterizes its coverage performance over a finite time horizon.
Show less

Title: Evolving Phenotypically Plastic Digital Organisms
Creator: Lalejini, Alexander
Date: 2021
Collection: Electronic Theses & Dissertations
Description: The ability to dynamically respond to cues from the environment is a fundamental feature of most adaptive systems. In biological systems, changes to an organism based on environmental cues is called phenotypic plasticity. Indeed, phenotypic plasticity underlies many of the adaptive traits and developmental patterns found in nature and serves as a key mechanism for responding to spatially or temporally variable environments. Most computer programs require phenotypic plasticity, as they must...
Show moreThe ability to dynamically respond to cues from the environment is a fundamental feature of most adaptive systems. In biological systems, changes to an organism based on environmental cues is called phenotypic plasticity. Indeed, phenotypic plasticity underlies many of the adaptive traits and developmental patterns found in nature and serves as a key mechanism for responding to spatially or temporally variable environments. Most computer programs require phenotypic plasticity, as they must respond dynamically to stimuli such as user input, sensor data, et cetera. As such, phenotypic plasticity also has practical applications in genetic programming, wherein we apply the natural principles of evolution to automatically synthesize computer programs rather than writing them by hand. In this dissertation, I achieve two synergistic aims: (1) I use populations of self-replicating computer programs (digital organisms) to empirically study the conditions under which adaptive phenotypic plasticity evolves and how its evolution shapes subsequent evolutionary outcomes; and (2) I transfer insights from biology to develop novel genetic programming techniques in order to evolve more responsive (i.e., phenotypically plastic) computer programs. First, I illustrate the importance of mutation rate, environmental change, and partially-plastic building blocks for the evolution of adaptive plasticity. Next, I show that adaptive phenotypic plasticity stabilizes populations against environmental change, allowing them to more easily retain novel adaptive traits. Finally, I improve our ability to evolve phenotypically plastic computer programs with three novel genetic programming techniques: (1) SignalGP, which provides mechanisms to control code expression based on environmental cues, (2) tag-based genetic regulation to adjust code expression based on current context, and (3) tag-accessed memory to provide more dynamic mechanisms for storing data.
Show less

Title: Face Recognition : Representation, Intrinsic Dimensionality, Capacity, and Demographic Bias
Creator: Gong, Sixue
Date: 2021
Collection: Electronic Theses & Dissertations
Description: Face recognition is a widely adopted technology with numerous applications, such as mobile phone unlock, mobile payment, surveillance, social media and law enforcement. There has been tremendous progress in enhancing the accuracy of face recognition systems over the past few decades, much of which can be attributed to deep learning. Despite this progress, several fundamental problems in face recognition still remain unsolved. These problems include finding a salient representation, estimating...
Show moreFace recognition is a widely adopted technology with numerous applications, such as mobile phone unlock, mobile payment, surveillance, social media and law enforcement. There has been tremendous progress in enhancing the accuracy of face recognition systems over the past few decades, much of which can be attributed to deep learning. Despite this progress, several fundamental problems in face recognition still remain unsolved. These problems include finding a salient representation, estimating intrinsic dimensionality, representation capacity, and demographic bias. With growing applications of face recognition, the need for an accurate, robust, compact and fair representation is evident.In this thesis, we first develop algorithms to obtain practical estimates of intrinsic dimensionality of face representations, and propose a new dimensionality reduction method to project feature vectors from ambient space to intrinsic space. Based on the study in intrinsic dimensionality, we then estimate capacity of face representation, casting the face capacity estimation problem under the information theoretic framework of capacity of a Gaussian noise channel. Numerical experiments on unconstrained faces (IJB-C) provide a capacity upper bound of 27,000 for FaceNet and 84,000 for SphereFace representation at 1% FAR. In the second part of the thesis, we address the demographic bias problem in face recognition systems where errors are lower on certain cohorts belonging to specific demographic groups. We propose two de-biasing frameworks that extract feature representations to improve fairness in face recognition. Experiments on benchmark face datasets (RFW, LFW, IJB-A, and IJB-C) show that our approaches are able to mitigate face recognition bias on various demographic groups (biasness drops from 6.83 to 5.07) as well as maintain the competitive performance (i.e., 99.75% on LFW, and 93.70% TAR @ 0.1% FAR on IJB-C). Lastly, we explore the global distribution of deep face representations derived from correlations between image samples of within-class and cross-class to enhance the discriminativeness of face representation of each identity in the embedding space. Our new approach to face representation achieves state-of-the-art performance for both verification and identification tasks on benchmark datasets (99.78% on LFW, 93.40% on CPLFW, 98.41% on CFP-FP, 96.2% TAR @ 0.01% FAR and 95.3% Rank-1 accuracy on IJB-C). Since, the primary techniques we employ in this dissertation are not specific to faces only, we believe our research can be extended to other problems in computer vision, for example, general image classification and representation learning.
Show less

Title: CS1 AND GENDER : UNDERSTANDING EFFECTS OF BACKGROUND AND SELF- EFFICACY ON ACHIEVEMENT AND INTEREST
Creator: Sands, Philip
Date: 2021
Collection: Electronic Theses & Dissertations
Description: Over the past 20 years, the field of computer science has experienced a growth in studentinterest. Despite this increase in participation rates, longstanding gender gaps persist in computer science. Recent research has examined a wide variety of individual factors (e.g., self-efficacy, sense of belonging, etc.) that impact student interest and achievement in computer science; however, these factors are rarely considered in the context of existing learning theories. In this correlational study...
Show moreOver the past 20 years, the field of computer science has experienced a growth in studentinterest. Despite this increase in participation rates, longstanding gender gaps persist in computer science. Recent research has examined a wide variety of individual factors (e.g., self-efficacy, sense of belonging, etc.) that impact student interest and achievement in computer science; however, these factors are rarely considered in the context of existing learning theories. In this correlational study, I explored the relationship between prior knowledge of computer programming, self-efficacy, and the sources of self-efficacy as they differed by gender in a theoretical model of achievement and interest for students in first-year computer science (CS1) courses. This model was based on prior work from Bandura (1997) and others exploring self- efficacy and social cognitive theory in the context of mathematics and science fields. Using cross-sectional data from N=182 CS1 students at two universities, structural regressions were conducted between factors impacting CS1 students across the entire population and for men (N=108) and women (N=70) individually. This data was then used to address the following research questions. (1A) How do prior knowledge of computer programming, the sources of self- efficacy, and self-efficacy for computing predict CS1 achievement and student intentions to continue study in CS? (1B) How does self-efficacy mediate the relationship between student prior knowledge of computer programming and achievement in CS1? (1C) How are thoserelationships moderated by gender? (2) How does feedback in the form of student grades impact intention to continue in CS when considering gender as a moderating factor? For all students, student self-efficacy for CS positively impacted CS1 achievement and post-CS1 interest. Aligning with past research, self-efficacy was derived largely from mastery experiences, with vicarious experiences and social persuasions also contributing to a moderate degree. Social persuasions had a negative effect on self-efficacy, which diverged from research in other fields. The relationship between prior knowledge of computer programming and CS1 achievement was not mediated by self-efficacy and had a small positive effect. For women, vicarious experiences played a stronger role in defining student self-efficacy in CS. Additionally, while the importance of self-efficacy on achievement was similar to that for men, self-efficacy and achievement both played a much stronger role in determining student interest in CS for women. All these findings are in need of further exploration as the analysis was underpowered due to a small, COVID-19 impacted sample size. Future work should focus on the role of feedback on student self-efficacy, the potential misalignment of CS1 feedback and social network feedback, and interventions that address student beliefs about CS abilities to increase opportunities for authentic mastery and vicarious experiences.
Show less

Title: Contributions to Fingerprint Recognition
Creator: Engelsma, Joshua James
Date: 2021
Collection: Electronic Theses & Dissertations
Description: From the early days of the mid to late nineteenth century when scientific research first began to focus on fingerprints, to the present day fingerprint recognition systems we find deployed on our day to day devices, the science of fingerprint recognition has come a long way. In spite of this progress, there remains challenging problems to be solved. This thesis highlights a few of these problems, and proposes solutions to address them. One area of further research that must be conducted on...
Show moreFrom the early days of the mid to late nineteenth century when scientific research first began to focus on fingerprints, to the present day fingerprint recognition systems we find deployed on our day to day devices, the science of fingerprint recognition has come a long way. In spite of this progress, there remains challenging problems to be solved. This thesis highlights a few of these problems, and proposes solutions to address them. One area of further research that must be conducted on fingerprint recognition systems is that of robust, operational evaluations. In chapter two of this thesis, we show how the current practices of using calibration patterns to evaluate fingerprint readers are limited. We then propose a realistic fake finger called the Universal Target. The Universal Target is a realistic, 3D, fake finger (or phantom) which can be imaged by all major types of fingerprint sensing technologies. We show the entire manufacturing (molding and casting) process for fabricating the Universal Targets. Then, we show a series of evaluations which demonstrate how the Universal Targets can be used to operationally evaluate current commercial fingerprint readers. Our Universal Target is a significant step forward in enabling more realistic, standardized evaluations of fingerprint readers. In our third chapter, we shift gears from improving the evaluation standards of fingerprint readers to instead focus on the security of fingerprint readers. In particular, we turn our attention towards detecting fake fingerprint (spoof) attacks. To do so, we open source a fingerprint reader (built from low-cost ubiquitous components), called RaspiReader. RaspiReader is a high-resolution fingerprint reader customized with both direct-view imaging and FTIR imaging in order to better detect fingerprint spoofs. We show through a number of experiments that RaspiReader enables state-of-the-art fingerprint spoof detection accuracy. We also demonstrate that RaspiReader enables better generalization to what are known as "unseen attacks" (those attacks which were not seen during training of the spoof detector). Finally, we show that fingerprints captured by RaspiReader are completely compatible with images captured by legacy fingerprint readers for matching.In chapter four, we move on to propose a major improvement to the fingerprint feature extraction and matching sub-modules of fingerprint recognition systems. In particular, we propose a deep network, called DeepPrint, to extract a 200 byte fixed-length fingerprint representation. While prevailing fingerprint matchers primarily utilize minutiae points and expensive graph matching algorithms for comparison, two DeepPrint representations can be compared with only 192 multiplications and 191 additions. This is extremely useful for large scale search where potentially billions of pairwise fingerprint comparisons must be made. The DeepPrint representation also enables practical encrypted matching using a fully homomorphic encryption scheme. This enables better protection of the fingerprint templates which are stored in the database. While discriminative fixed-length representations are available for both face and iris recognition, such a representation has eluded fingerprint recognition. This chapter aims to fill that void.Finally, we conclude our thesis by working to extend fingerprint recognition to all ages. While current fingerprint recognition systems are being used by billions of teenagers and adults around the world, the youngest people among us remain disenfranchised. In particular, modern day fingerprint recognition systems do not work well on infants and young children. In this penultimate chapter, we aim to rectify this major shortcoming. To that end, we prototype a high-resolution (1900 ppi) infant fingerprint reader. Then, we track and fingerprint 315 infants (under the age of 3 months at enrollment) at the Dayalbagh Children's Hospital in Agra India over the course of 1 year (4 different sessions). To match the infant fingerprints, we develop our own high-resolution infant fingerprint matcher. Our experimental results demonstrate significant promise for the extension of fingerprint recognition to all ages. This work has the potential for major global good as all young infants and children could be given a verifiable digital identity for better vaccination tracking as a child and for government benefits and assistance as an adult. In summary, this thesis makes major contributions to the entire end-to-end fingerprint recognition system and extends its use case to all ages.
Show less

Title: Towards Robust and Secure Face Recognition : Defense Against Physical and Digital Attacks
Creator: Deb, Debayan
Date: 2021
Collection: Electronic Theses & Dissertations
Description: The accuracy, usability, and touchless acquisition of state-of-the-art automated face recognition systems (AFR) have led to their ubiquitous adoption in a plethora of domains, including mobile phone unlock, access control systems, and payment services. Despite impressive recognition performance, prevailing AFR systems remain vulnerable to the growing threat of face attacks which can be launched in both physical and digital domains. Face attacks can be broadly classified into three attack...
Show moreThe accuracy, usability, and touchless acquisition of state-of-the-art automated face recognition systems (AFR) have led to their ubiquitous adoption in a plethora of domains, including mobile phone unlock, access control systems, and payment services. Despite impressive recognition performance, prevailing AFR systems remain vulnerable to the growing threat of face attacks which can be launched in both physical and digital domains. Face attacks can be broadly classified into three attack categories: (i) Spoof attacks: artifacts in the physical domain (e.g., 3D masks, eye glasses, replaying videos), (ii) Adversarial attacks: imperceptible noises added to probes for evading AFR systems, and (iii) Digital manipulation attacks: entirely or partially modified photo-realistic faces using generative models. Each of these categories is composed of different attack types. For example, each spoof medium, e.g., 3D mask and makeup, constitutes one attack type. Likewise, in adversarial and digital manipulation attacks, each attack model, designed by unique objectives and losses, may be considered as one attack type. Thus, the attack categories and types form a 2-layer tree structure encompassing the diverse attacks. Such a tree will inevitably grow in the future. Given the growing dissemination of ``fake news” and "deepfakes", the research community and social media platforms alike are pushing towards generalizable defense against continuously evolving and sophisticated face attacks. In this dissertation, we first propose a set of defense methods that achieve state-of-the-art performance in detecting attack types within individual attack categories, both physical (e.g., face spoofs) and digital (e.g., adversarial faces and digital manipulation), then introduce a method for simultaneously safeguarding against each attack.First, in an effort to impart generalizability and interpretability to face spoof detection systems, we propose a new face anti-spoofing framework specifically designed to detect unknown spoof types, namely, Self-Supervised Regional Fully Convolutional Network (SSR-FCN), that is trained to learn local discriminative cues from a face image in a self-supervised manner. The proposed framework improves generalizability while maintaining the computational efficiency of holistic face anti-spoofing approaches (< 4 ms on a Nvidia GTX 1080Ti GPU). The proposed method is also interpretable since it localizes which parts of the face are labeled as spoofs. Experimental results show that SSR-FCN can achieve True Detection Rate (TDR) = 65% @ 2.0% False Detection Rate (FDR) when evaluated on a dataset comprising of 13 different spoof types under unknown attacks while achieving competitive performances under standard benchmark face anti-spoofing datasets (Oulu-NPU, CASIA-MFSD, and Replay-Attack).Next, we address the problem of defending against adversarial attacks. We first propose, AdvFaces, an automated adversarial face synthesis method that learns to generate minimal perturbations in the salient facial regions. Once AdvFaces is trained, it can automatically evade state-of-the-art face matchers with attack success rates as high as 97.22% and 24.30% at 0.1% FAR for obfuscation and impersonation attacks, respectively. We then propose a new self-supervised adversarial defense framework, namely FaceGuard, that can automatically detect, localize, and purify a wide variety of adversarial faces without utilizing pre-computed adversarial training samples. FaceGuard automatically synthesizes diverse adversarial faces, enabling a classifier to learn to distinguish them from bona fide faces. Concurrently, a purifier attempts to remove the adversarial perturbations in the image space. FaceGuard can achieve 99.81%, 98.73%, and 99.35% detection accuracies on LFW, CelebA, and FFHQ, respectively, on six unseen adversarial attack types.Finally, we take the first steps towards safeguarding AFR systems against face attacks in both physical and digital domains. We propose a new unified face attack detection framework, namely UniFAD, which automatically clusters similar attacks and employs a multi-task learning framework to learn salient features to distinguish between bona fides and coherent attack types. The proposed UniFAD can detect face attacks from 25 attack types across all 3 attack categories with TDR = 94.73% @ 0.2% FDR on a large fake face dataset, namely GrandFake. Further, UniFAD can identify whether attacks are adversarial, digitally manipulated, or contain spoof artifacts, with 97.37% classification accuracy.
Show less

Title: Online Learning Algorithms for Mining Trajectory data and their Applications
Creator: Wang, Ding
Date: 2021
Collection: Electronic Theses & Dissertations
Description: Trajectories are spatio-temporal data that represent traces of moving objects, such as humans, migrating animals, vehicles, and tropical cyclones. In addition to the geo-location information, a trajectory data often contain other (non-spatial) features describing the states of the moving objects. The time-varying geo-location and state information would collectively characterize a trajectory dataset, which can be harnessed to understand the dynamics of the moving objects. This thesis focuses...
Show moreTrajectories are spatio-temporal data that represent traces of moving objects, such as humans, migrating animals, vehicles, and tropical cyclones. In addition to the geo-location information, a trajectory data often contain other (non-spatial) features describing the states of the moving objects. The time-varying geo-location and state information would collectively characterize a trajectory dataset, which can be harnessed to understand the dynamics of the moving objects. This thesis focuses on the development of efficient and accurate machine learning algorithms for forecasting the future trajectory path and state of a moving object. Although many methods have been developed in recent years, there are still numerous challenges that have not been sufficiently addressed by existing methods, which hamper their effectiveness when applied to critical applications such as hurricane prediction. These challenges include their difficulties in terms of handling concept drifts, error propagation in long-term forecasts, missing values, and nonlinearities in the data. In this thesis, I present a family of online learning algorithms to address these challenges. Online learning is an effective approach as it can efficiently fit new observations while adapting to concept drifts present in the data. First, I proposed an online learning framework called OMuLeT for long-term forecasting of the trajectory paths of moving objects. OMuLeT employs an online learning with restart strategy to incrementally update the weights of its predictive model as new observation data become available. It can also handle missing values in the data using a novel weight renormalization strategy.Second, I introduced the OOR framework to predict the future state of the moving object. Since the state can be represented by ordinal values, OOR employs a novel ordinal loss function to train its model. In addition, the framework was extended to OOQR to accommodate a quantile loss function to improve its prediction accuracy for larger values on the ordinal scale. Furthermore, I also developed the OOR-ε and OOQR-ε frameworks to generate real-valued state predictions using the ε insensitivity loss function.Third, I developed an online learning framework called JOHAN, that simultaneously predicts the location and state of the moving object. JOHAN generates its predictions by leveraging the relationship between the state and location information. JOHAN utilizes a quantile loss function to bias the algorithm towards predicting more accurately large categorical values in terms of the state of the moving object, say, for a high intensity hurricane.Finally, I present a deep learning framework to capture non-linear relationships in trajectory data. The proposed DTP framework employs a TDM approach for imputing missing values, coupled with an LSTM architecture for dynamic path prediction. In addition, the framework was extended to ODTP, which applied an online learning setting to address concept drifts present in the trajectory data.As proof of concept, the proposed algorithms were applied to the hurricane prediction task. Both OMuLeT and ODTP were used to predict the future trajectory path of a hurricane up to 48 hours lead time. Experimental results showed that OMuLeT and ODTP outperformed various baseline methods, including the official forecasts produced by the U.S. National Hurricane Center. OOR was applied to predict the intensity of a hurricane up to 48 hours in advance. Experimental results showed that OOR outperformed various state-of-the-art online learning methods and can generate predictions close to the NHC official forecasts. Since hurricane intensity prediction is a notoriously hard problem, JOHAN was applied to improve its prediction accuracy by leveraging the trajectory information, particularly for high intensity hurricanes that are near landfall.
Show less

Title: Replaying Life's Virtual Tape : Examining the Role of History in Experiments with Digital Organisms
Creator: Bundy, Jason Nyerere
Date: 2021
Collection: Electronic Theses & Dissertations
Description: Evolution is a complex process with a simple recipe. Evolutionary change involves three essential “ingredients” interacting over many generations: adaptation (selection), chance (random variation), and history (inheritance). In 1989’s Wonderful Life, the late paleontologist Stephen Jay Gould advocated for the importance of historical contingency—the way unique events throughout history influence future possibilities—using a clever thought experiment of “replaying life’s tape”. But not...
Show moreEvolution is a complex process with a simple recipe. Evolutionary change involves three essential “ingredients” interacting over many generations: adaptation (selection), chance (random variation), and history (inheritance). In 1989’s Wonderful Life, the late paleontologist Stephen Jay Gould advocated for the importance of historical contingency—the way unique events throughout history influence future possibilities—using a clever thought experiment of “replaying life’s tape”. But not everyone was convinced. Some believed that chance was the primary driver of evolutionary change, while others insisted that natural selection was the most powerful influence. Since then, “replaying life’s tape” has become a core method in experimental evolution for measuring the relative contributions of adaptation, chance, and history. In this dissertation, I focus on the effects associated with history in evolving populations of digital organisms—computer programs that self-replicate, mutate, compete, and evolve in virtual environments. In Chapter 1, I discuss the philosophical significance of Gould’s thought experiment and its influence on experimental methods. I argue that his thought experiment was a challenge to anthropocentric reasoning about natural history that is still popular, particularly outside of the scientific community. In this regard, it was his way of advocating for a “radical” view of evolution. In Chapter 2—Richard Lenski, Charles Ofria, and I describe a two-phase, virtual, “long-term” evolution experiment with digital organisms using the Avida software. In Phase I, we evolved 10 replicate populations, in parallel, from a single genotype for around 65,000 generations. This part of the experiment is similar to the design of Lenski’s E. coli Long-term Evolution Experiment (LTEE). We isolated the dominant genotype from each population around 3,000 generations (shallow history) into Phase I and then again at the end of Phase I (deep history). In Phase II, we evolved 10 populations from each of the genotypes we isolated from Phase I in two new environments, one similar and one dissimilar to the old environment used for Phase I. Following Phase II, we estimated the contributions of adaptation, chance, and history to the evolution of fitness and genome length in each new environment. This unique experimental design allowed us to see how the contributions of adaptation, chance, and history changed as we extended the depth of history from Phase I. We were also able to determine whether the results depended on the extent of environmental change (similar or dissimilar new environment). In Chapter 3, we report an extended analysis of the experiment from the previous chapter to further examine how extensive adaptation to the Phase I environment shaped the evolution of replicates during Phase II. We show how the form of pleiotropy (antagonistic or synergistic) between the old (Phase I) and new (Phase II) habitats was influenced by the depth of history from Phase I (shallow or deep) and the extent of environmental change (similar or dissimilar new environment). In the final chapter Zachary Blount, Richard Lenski, and I describe an exercise we developed using the educational version of Avida (Avida-ED). The exercise features a two-phase, “replaying life’s tape” activity. Students are able to explore how the unique history of founders that we pre-evolved during Phase I influences the acquisition of new functions by descendent populations during Phase II, which the students perform during the activity.
Show less

Title: Solving Computationally Expensive Problems Using Surrogate-Assisted Optimization : Methods and Applications
Creator: Blank, Julian
Date: 2022
Collection: Electronic Theses & Dissertations
Description: Optimization is omnipresent in many research areas and has become a critical component across industries. However, while researchers often focus on a theoretical analysis or convergence proof of an optimization algorithm, practitioners face various other challenges in real-world applications. This thesis focuses on one of the biggest challenges when applying optimization in practice: computational expense, often caused by the necessity of calling a third-party software package. To address the...
Show moreOptimization is omnipresent in many research areas and has become a critical component across industries. However, while researchers often focus on a theoretical analysis or convergence proof of an optimization algorithm, practitioners face various other challenges in real-world applications. This thesis focuses on one of the biggest challenges when applying optimization in practice: computational expense, often caused by the necessity of calling a third-party software package. To address the time-consuming evaluation, we propose a generalizable probabilistic surrogate-assisted framework that dynamically incorporates predictions of approximation models. Besides the framework's capability of handling multiple objectives and constraints simultaneously, the novelty is its applicability to all kinds of metaheuristics. Moreover, often multiple disciplines are involved in optimization, resulting in different types of software packages utilized for performance assessment. Therefore, the resulting optimization problem typically consists of multiple independently evaluable objectives and constraints with varying computational expenses. Besides providing a taxonomy describing different ways of independent evaluation calls, this thesis also proposes a methodology to handle inexpensive constraints with expensive objective functions and a more generic concept for any type of heterogeneously expensive optimization problem. Furthermore, two case studies of real-world optimization problems from the automobile industry are discussed, a blueprint for solving optimization problems in practice is provided, and a widely-used optimization framework focusing on multi-objective optimization (founded and maintained by the author of this thesis) is presented. Altogether, this thesis shall pave the way to solve (computationally expensive) real-world optimization more efficiently and bridge the gap between theory and practice.
Show less

Title: High-precision and Personalized Wearable Sensing Systems for Healthcare Applications
Creator: Tu, Linlin
Date: 2022
Collection: Electronic Theses & Dissertations
Description: The cyber-physical system (CPS) has been discussed and studied extensively since 2010. It provides various solutions for monitoring the user's physical and psychological health states, enhancing the user's experience, and improving the lifestyle. A variety of mobile internet devices with built-in sensors, such as accelerators, cameras, PPG sensors, pressure sensors, and the microphone, can be leveraged to build mobile cyber-physical applications that collected sensing data from the real world...
Show moreThe cyber-physical system (CPS) has been discussed and studied extensively since 2010. It provides various solutions for monitoring the user's physical and psychological health states, enhancing the user's experience, and improving the lifestyle. A variety of mobile internet devices with built-in sensors, such as accelerators, cameras, PPG sensors, pressure sensors, and the microphone, can be leveraged to build mobile cyber-physical applications that collected sensing data from the real world, had data processed, communicated to the internet services and transformed into behavioral and physiological models. The detected results can be used as feedback to help the user understand his/her behavior, improve the lifestyle, or avoid danger. They can also be delivered to therapists to facilitate their diagnose. Designing CPS for health monitoring is challenging due to multiple factors. First of all, high estimation accuracy is necessary for health monitoring. However, some systems suffer irregular noise. For example, PPG sensors for cardiac health state monitoring are extremely vulnerable to motion noise. Second, to include human in the loop, health monitoring systems are required to be user-friendly. However, some systems involve cumbersome equipment for a long time of data collection, which is not feasible for daily monitoring. Most importantly, large-scale high-level health-related monitoring systems, such as the systems for human activity recognition, require high accuracy and communication efficiency. However, with users' raw data uploading to the server, centralized learning fails to protect users' private information and is communication-inefficient. The research introduced in this dissertation addressed the above three significant challenges in developing health-related monitoring systems. We build a lightweight system for accurate heart rate measurement during exercise, design a smart in-home breathing training system with bio-Feedback via virtual reality (VR) game, and propose federated learning via dynamic layer sharing for human activity recognition.
Show less

Title: Gender-related effects of advanced placement computer science courses on self-efficacy, belongingness, and persistence
Creator: Good, Jonathon Andrew
Date: 2018
Collection: Electronic Theses & Dissertations
Description: The underrepresentation of women in computer science has been a concern of educators for multiple decades. The low representation of women in the computer science is a pattern from K-12 schools through the university level and profession. One of the purposes of the introduction of the Advanced Placement Computer Science Principles (APCS-P) course in 2016 was to help broaden participation in computer science at the high school level. The design of APCS-P allowed teachers to present computer...
Show moreThe underrepresentation of women in computer science has been a concern of educators for multiple decades. The low representation of women in the computer science is a pattern from K-12 schools through the university level and profession. One of the purposes of the introduction of the Advanced Placement Computer Science Principles (APCS-P) course in 2016 was to help broaden participation in computer science at the high school level. The design of APCS-P allowed teachers to present computer science from a broad perspective, allowing students to pursue problems of personal significance, and allowing for computing projects to take a variety of forms. The nationwide enrollment statistics for Advanced Placement Computer Science Principles in 2017 had a higher proportion of female students (30.7%) than Advanced Placement Computer Science A (23.6%) courses. However, it is unknown to what degree enrollment in these courses was related to students’ plans to enroll in future computer science courses. This correlational study examined how students’ enrollment in Advanced Placement Computer Science courses, along with student gender, predicted students’ sense of computing self-efficacy, belongingness, and expected persistence in computer science. A nationwide sample of 263 students from 10 APCS-P and 10 APCS-A courses participated in the study. Students completed pre and post surveys at the beginning and end of their Fall 2017 semester regarding their computing self-efficacy, belongingness, and plans to continue in computer science studies. Using hierarchical linear modeling analysis due to the nested nature of the data within class sections, the researcher found that the APCS course type was not predictive of self-efficacy, belongingness, or expectations to persist in computer science. The results suggested that female students’ self-efficacy declined over the course of the study. However, gender was not predictive of belongingness or expectations to persist in computer science. Students were found to have entered into both courses with high a sense of self-efficacy, belongingness, and expectation to persist in computer science.The results from this suggests that students enrolled in both Advanced Placement Computer Science courses are already likely to pursue computer science. I also found that the type of APCS course in which students enroll does not relate to students’ interest in computer science. This suggests that educators should look beyond AP courses as a method of exposing students to computer science, possibly through efforts such as computational thinking and cross-curricular uses of computer science concepts and practices. Educators and administrators should also continue to examine whether there are structural biases in how students are directed to computer science courses. As for the drop in self-efficacy related to gender, this in alignment with previous research suggesting that educators should carefully scaffold students’ initial experiences in the course to not negatively influence their self-efficacy. Further research should examine how specific pedagogical practices could influence students’ persistence, as the designation and curriculum of APCS-A or APCS-P alone may not capture the myriad of ways in which teachers may be addressing gender inequity in their classrooms. Research can also examine how student interest in computer science is affected at an earlier age, as the APCS courses may be reaching students after they have already formed their opinions about computer science as a field.
Show less

Title: Multiple kernel and multi-label learning for image categorization
Creator: Bucak, Serhat Selçuk
Date: 2014
Collection: Electronic Theses & Dissertations
Description: "One crucial step towards the goal of converting large image collections to useful information sources is image categorization. The goal of image categorization is to find the relevant labels for a given an image from a closed set of labels. Despite the huge interest and significant contributions by the research community, there remains much room for improvement in the image categorization task. In this dissertation, we develop efficient multiple kernel learning and multi-label learning...
Show more"One crucial step towards the goal of converting large image collections to useful information sources is image categorization. The goal of image categorization is to find the relevant labels for a given an image from a closed set of labels. Despite the huge interest and significant contributions by the research community, there remains much room for improvement in the image categorization task. In this dissertation, we develop efficient multiple kernel learning and multi-label learning algorithms with high prediction performance for image categorization... " -- Abstract.
Show less

Title: Evolution of distributed behavior
Creator: Knoester, David B.
Date: 2011
Collection: Electronic Theses & Dissertations
Description: In this dissertation, we describe a study in the evolution of distributed behavior, where evolutionary algorithms are used to discover behaviors for distributed computing systems. We define distributed behavior as that in which groups of individuals must both cooperate in working towards a common goal and coordinate their activities in a harmonious fashion. As such, communication among individuals is necessarily a key component of distributed behavior, and we have identified three classes of...
Show moreIn this dissertation, we describe a study in the evolution of distributed behavior, where evolutionary algorithms are used to discover behaviors for distributed computing systems. We define distributed behavior as that in which groups of individuals must both cooperate in working towards a common goal and coordinate their activities in a harmonious fashion. As such, communication among individuals is necessarily a key component of distributed behavior, and we have identified three classes of distributed behavior that require communication: data-driven behaviors, where semantically meaningful data is transmitted between individuals; temporal behaviors, which are based on the relative timing of individuals' actions; and structural behaviors, which are responsible for maintaining the underlying communication network connecting individuals. Our results demonstrate that evolutionary algorithms can discover groups of individuals that exhibit each of these different classes of distributed behavior, and that these behaviors can be discovered both in isolation (e.g., evolving a purely data-driven algorithm) and in concert (e.g., evolving an algorithm that includes both data-driven and structural behaviors). As part of this research, we show that evolutionary algorithms can discover novel heuristics for distributed computing, and hint at a new class of distributed algorithm enabled by such studies.The majority of this research was conducted with the Avida platform for digital evolution, a system that has been proven to aid researchers in understanding the biological process of evolution by natural selection. For this reason, the results presented in this dissertation provide the foundation for future studies that examine how distributed behaviors evolved in nature. The close relationship between evolutionary biology and evolutionary algorithms thus aids our study of evolving algorithms for the next generation of distributed computing systems.
Show less

Title: Algorithms for deep packet inspection
Creator: Patel, Jignesh
Date: 2012
Collection: Electronic Theses & Dissertations
Description: The core operation in network intrusion detection and prevention systems is Deep Packet Inspection (DPI), in which each security threat is represented as a signature, and the payload of each data packet is matched against the set of current security threat signatures. DPI is also used for other networking applications like advanced QoS mechanisms, protocol identification etc.. In the past, attack signatures were specified as strings, and a great deal of research has been done in string...
Show moreThe core operation in network intrusion detection and prevention systems is Deep Packet Inspection (DPI), in which each security threat is represented as a signature, and the payload of each data packet is matched against the set of current security threat signatures. DPI is also used for other networking applications like advanced QoS mechanisms, protocol identification etc.. In the past, attack signatures were specified as strings, and a great deal of research has been done in string matching for network applications. Today most DPI systems use Regular Expression (RE) to represent signatures. RE matching is more diffcult than string matching, and current string matching solutions don't work well for REs. RE matching for networking applications is diffcult for several reasons. First, the DPI application is usually implemented in network devices, which have limited computing resources. Second, as new threats are discovered, size of the signature set grows over time. Last, the matching needs to be done at network speeds, the growth of which out paces improvements in computing speed; so there is a need for novel solutions that can deliver higher throughput. So RE matching for DPI is a very important and active research area.In our research, we investigate the existing methods proposed for RE matching, identify their limitations, and propose new methods to overcome these limitations. RE matching remains a fundamentally challenging problem due to the diffculty in compactly encoding DFA. While the DFA for any one RE is typically small, the DFA that corresponds to the entire set of REs is usually too large to be constructed or deployed. To address this issue, many alternative automata implementations that compress the size of the final automaton have been proposed. However, previously proposed automata construction algorithms employ a “Union then Minimize” framework where the automata for each RE are first joined before minimization occurs. This leads to expensive minimization on a large automata, and a large intermediate memory footprint. We propose a “Minimize then Union” framework for constructing compact alternative automata, which minimizes smaller automata first before combining them. This approach required much less time and memory, allowing us to handle a much larger RE set. Prior hardware based RE matching algorithms typically use FPGA. The drawback of FPGA is that resynthesizing and updating FPGA circuitry to handle RE updates is slow and diffcult. We propose the first hardware-based RE matching approach that uses Ternary Content Addressable Memory (TCAM). TCAMs have already been widely used in modern networking devices for tasks such as packet classification, so our solutions can be easily deployed. Our methods support easy RE updates, and we show that we can achieve very high throughput. The main reason combined DFAs for multiple REs grow exponentially in size is because of replication of states. We developed a new overlay automata model which exploit this replication to compress the size of the DFA. The idea is to group together the replicated DFA structures instead of repeating them multiple times. The result is that we get a final automata size that is close to that of a NFA (which is linear in the size of the RE set), and simultaneously achieve fast deterministic matching speed of a DFA.
Show less

Title: Some contributions to semi-supervised learning
Creator: Mallapragada, Paven Kumar
Date: 2010
Collection: Electronic Theses & Dissertations

MSU Libraries

Digital Repository

Enabled Filters

Pages

Pages

MSU Libraries

Digital Repository

You are here

Search results

Pages

Pages