Search results

Title: Online innovization : towards knowledge discovery and achieving faster convergence in multi-objective optimization
Creator: Gaur, Abhinav
Date: 2020
Collection: Electronic Theses & Dissertations
Description: Ì0300nnovization'' is a task of learning common principles thatexist among some or all of the Pareto-optimal solutions in amulti-objective optimization problem. Except a few earlierstudies, most innovization related studies were performed onthe final non-dominated solutions found by an evolutionary multi-objective algorithm eithermanually or by using a machine learning method.Recent studies have shown that these principles can be learnedduring intermediate iterations of an optimization run...
Show moreÌ0300nnovization'' is a task of learning common principles thatexist among some or all of the Pareto-optimal solutions in amulti-objective optimization problem. Except a few earlierstudies, most innovization related studies were performed onthe final non-dominated solutions found by an evolutionary multi-objective algorithm eithermanually or by using a machine learning method.Recent studies have shown that these principles can be learnedduring intermediate iterations of an optimization run and simultaneously utilized in thesame optimization run to repair variables to achieve a fasterconvergence to the Pareto-optimal set. This is what we are calling as ò0300nline innovization'' as it is performed online during the run of an evolutionary multi-objective optimization algorithm. Special attention is paid to learning rules that are easier to interpret, such as short algebraic expressions, instead of complex decision trees or kernel based black box rules.We begin by showing how to learn fixed form rules that are encountered frequently in multi-objective optimization problems. We also show how can we learn free form rules, that are linear combination of non-linear terms, using a custom genetic programming algorithm. We show how can we use the concept of k0300nee' in PO set of solutions along with a custom dimensional penalty calculator to discard rules that may be overly complex, or inaccurate or just dimensionally incorrect. The results of rules learned using this custom genetic programming algorithm show that it is beneficial to let evolution learn the structure of rules while the constituent weights should be learned using some classical learning algorithm such as linear regression or linear support vector machines. When the rules are implicit functions of the problem variables, we use a computationally inexpensive way of repairing the variables by turning the problem of repairing the variable into a single variable golden section search.We show the proof of concept on test problems by learning fixed form rules among variables of the problem, which we then use during the same optimization run to repair variables. Different principleslearned during an optimization run can involve differentnumber of variables and/or variables that arecommon among a number of principles. Moreover, a preferenceorder for repairing variables may play an important role forproper convergence. Thus, when multiple principles exist, itis important to use a strategy that is most beneficial forrepairing evolving population of solutions.The above methods are applied to a mix of test problems and engineering design problems. The results are encouraging and strongly supportsthe use of innovization task in enhancing the convergence of an evolutionary multi-objective optimization algorithms. Moreover, the custom genetic program developed in this work can be a useful machine learning tool for practitioners to learn human interpretable rules in the form of algebraic expressions.
Show less

Title: Using evolutionary approach to optimize and model multi-scenario, multi-objective fault-tolerant problems
Creator: Zhu, Ling (Engineer)
Date: 2017
Collection: Electronic Theses & Dissertations
Description: Fault-tolerant design involves different scenarios, such as scenarios with no fault in the system, with faults occurring randomly, with different operation conditions, and with different loading conditions. For each scenario, there can be multiple requirements (objectives). To assess the performance of a design (solution), it needs to be evaluated over a number of different scenarios containing various requirements in each scenario. We consider this problem as a multi-scenario, multi...
Show moreFault-tolerant design involves different scenarios, such as scenarios with no fault in the system, with faults occurring randomly, with different operation conditions, and with different loading conditions. For each scenario, there can be multiple requirements (objectives). To assess the performance of a design (solution), it needs to be evaluated over a number of different scenarios containing various requirements in each scenario. We consider this problem as a multi-scenario, multi-objective (MSMO) problem.Despite its practical importance and prevalence in engineering application, there are not many studies which systematically solve the MSMO problem. In this dissertation, we focus on optimizing and modeling MSMO problems, and propose various approaches to solve different types of MSMO optimization problems, especially multi-objective fault-tolerant problems. We classify MSMO optimization problem into two categories: scenario-dependent and scenario-independent. For the scenario-dependent MSMO problem, we review existing methodologies and suggest two evolutionary-based methods for handling multiple scenarios and objectives: aggregated method and integrated method. The effectiveness of both methods are demonstrated on several case studies including numerical problems and engineering design problems. The engineering problems include cantilever-type welded beam design, truss bridge design, four-bar truss design. The experimental results show that both methods can find a set of widely distributed solutions that are compromised among the respective objective values under all scenarios. We also model fault-tolerant programs using the aggregated method. We synthesize three fault-tolerant distributed programs: Byzantine agreement program, token ring circulation program and consensus program with failure detector $S$. The results show that evolutionary-base MSMO approach, as a generic method, can effectively model fault-tolerant programs. For the scenario-independent MSMO problem, we apply evolutionary multi-objective approach. As a case study, we optimize a probabilistic self-stabilizing program, a special type of fault-tolerant program, and obtain several interesting counter-intuitive observations under different scenarios.
Show less

Title: Layout optimization of truss structures by fully stressed design evolution strategy
Creator: Ahrari, Ali
Date: 2016
Collection: Electronic Theses & Dissertations
Description: "The field of structural optimization has gained much academic interest in the recent decades. Different streams of optimization methods have been applied to this problem including analytical methods, optimality criteria-based method and gradient-based methods. During the recent decade, there has been a growing interest among researchers to apply stochastic population-based methods, the so-called meta-heuristics, to this class of optimization problems. The motivation is the robustness and...
Show more"The field of structural optimization has gained much academic interest in the recent decades. Different streams of optimization methods have been applied to this problem including analytical methods, optimality criteria-based method and gradient-based methods. During the recent decade, there has been a growing interest among researchers to apply stochastic population-based methods, the so-called meta-heuristics, to this class of optimization problems. The motivation is the robustness and capability of meta-heuristics to avoid local minima. On the downside, their required evaluation budget grows fast when the number of design variables is increased, which limits the complexity of problems to which they can be applied. Furthermore, majority of these methods are tailored to optimize only the cross-sectional areas of the members, the potential saving from which is highly limited. At the same time, several factors have diminished practitioners' interests in the academic research on this topic, including simplicity of conventional test problems compared to real structures, variety of design constraints in practice and the complexity of evaluation of the total cost. This dissertation aims at addressing some of the most critical shortcomings in the available truss optimization methods, both from academic and practical perspectives. It proposes a novel bi-level method for simultaneous optimization of topology, shape and size of truss structures. In the upper level, a specialized evolution strategy (ES) is proposed which follows the principles of contemporary evolution strategies (ESs), although the formulation is modified to handle mixed- variable highly constrained truss optimization problems. The concept of fully stressed design is employed in the lower level as an efficient method for resizing the sampled solution in the upper level. The concept of fully stressed design is also utilized to define a specialized penalty term based on the estimated required increase in the structural weight such that all constraints are satisfied. The proposed method, called fully stressed design evolution strategy (FSD-ES), is developed in four stages. It is tested on complicated problems, some of which are developed in this dissertation, as an attempt to reduce the gap between complexity of test problems and real structures. Empirical evaluation and comparison with the best available methods in the literature reveal superiority of FSD-ES, which intensifies for more complicated problems. Aside from academically interesting features of FSD-ES, it addresses some of the practicing engineers' critiques on applicability of truss optimization methods. FSD-ES can handle large-scale truss optimization problems with more than a thousand design parameters, in a reasonable amount of CPU time. Our numerical results demonstrate that the optimized design can hardly be guessed by engineering intuition, which demonstrates superiority of such design optimization methods. Besides, the amount of material saving is potentially huge, especially for more complicated problems, which justifies simulation cost of the design problem. FSD-ES does not require any user-dependent parameter tuning and the code is ready to use for an arbitrary truss design problem within the domain of the code."--Pages ii-iii.
Show less

Title: Metamodeling framework for simultaneous multi-objective optimization using efficient evolutionary algorithms
Creator: Roy, Proteek Chandan
Date: 2019
Collection: Electronic Theses & Dissertations
Description: Most real-world problems are comprised of multiple conflicting objectives and solutions to those problems are multiple Pareto-optimal trade-off solutions. The main challenge of these practical problems is that the objectives and constraints do not have any closed functional forms and they are expensive for computation as well. Objectives coming from finite element analysis, computational fluid dynamics software, network flow simulators, crop modeling, weather modeling or any other simulations...
Show moreMost real-world problems are comprised of multiple conflicting objectives and solutions to those problems are multiple Pareto-optimal trade-off solutions. The main challenge of these practical problems is that the objectives and constraints do not have any closed functional forms and they are expensive for computation as well. Objectives coming from finite element analysis, computational fluid dynamics software, network flow simulators, crop modeling, weather modeling or any other simulations which involve partial differential equations are good examples of expensive problems. These problems can also be regarded as l03000300ow-budget'' problems since only a few solution evaluations can be performed given limited time. Nevertheless, parameter estimation and optimization of objectives related to these simulations require a good number of solution evaluations to come up with better parameters or a reasonably good trade-off front. To provide an efficient search process within a limited number of exact evaluations, metamodel-assisted algorithms have been proposed in the literature. These algorithms attempt to construct a computationally inexpensive representative model of the problem, having the same global optima and thereby providing a way to carry out the optimization in metamodel space in an efficient way. Population-based methods like evolutionary algorithms have become standard for solving multi-objective problems and recently Metamodel-based evolutionary algorithms are being used for solving expensive problems. In this thesis, we would like to address a few challenges of metamodel-based optimization algorithms and propose some efficient and innovative ways to construct these algorithms. To approach efficient design of metamodel-based optimization algorithm, one needs to address the choice of metamodeling functions. The most trivial way is to build metamodels for each objective and constraint separately. But we can reduce the number of metamodel constructions by using some aggregated functions and target either single or multiple optima in each step. We propose a taxonomy of possible metamodel-based algorithmic frameworks which not only includes most algorithms from the literature but also suggests some new ones. We improve each of the frameworks by introducing trust region concepts in the multi-objective scenario and present two strategies for building trust regions. Apart from addressing the main bottleneck of the limited number of solution evaluations, we also propose efficient non-dominated sorting methods that further reduce computational time for a basic step of multi-objective optimization. We have carried out extensive experiments over all representative metamodeling frameworks and shown that each of them can solve a good number of test problems. We have not tried to tune the algorithmic parameters yet and it remains as our future work. Our theoretical analyses and extensive experiments suggest that we can achieve efficient metamodel-based multi-objective optimization algorithms for solving test as well as real-world expensive and low-budget problems.
Show less

Title: Evolutionary multi-objective bi-level optimization for efficient deep neural network architecture design
Creator: Lu, Zhichao
Date: 2020
Collection: Electronic Theses & Dissertations
Description: Deep convolutional neural networks (CNNs) are the backbones of deep learning (DL) paradigms for numerous vision tasks, including object recognition, detection, segmentation, etc. Early advancements in CNN architectures are primarily driven by human expertise and elaborate design. Recently, neural architecture search (NAS) was proposed with the aim of automating the network design process and generating task-dependent architectures. While existing approaches have achieved competitive...
Show moreDeep convolutional neural networks (CNNs) are the backbones of deep learning (DL) paradigms for numerous vision tasks, including object recognition, detection, segmentation, etc. Early advancements in CNN architectures are primarily driven by human expertise and elaborate design. Recently, neural architecture search (NAS) was proposed with the aim of automating the network design process and generating task-dependent architectures. While existing approaches have achieved competitive performance, they are still impractical to real-world deployment for three reasons: (1) the generated architectures are solely optimized for predictive performance, resulting in inefficiency in utilizing hardware resources---i.e. energy consumption, latency, memory size, etc.; (2) the search processes require vast computational resources in most approaches; (3) most existing approaches require one complete search for each deployment specification of hardware or requirement. In this dissertation, we propose an efficient evolutionary NAS algorithm to address the aforementioned limitations. In particular, we first introduce Pareto-optimization to NAS, leading to a diverse set of architectures, trading-off multiple objectives, being obtained simultaneously in one run. We then improve the algorithm's search efficiency through surrogate models. We finally integrate a transfer learning scheme to the algorithm that allows a new task to leverage previous search efforts that further improves both the performance of the obtained architectures and search efficiency. Therefore, the proposed algorithm enables an automated and streamlined process to efficiently generate task-specific custom neural network models that are competitive under multiple objectives.
Show less

Title: Balancing convergence and diversity in evolutionary single, multi and many objectives
Creator: Seada, Haitham
Date: 2017
Collection: Electronic Theses & Dissertations
Description: "Single objective optimization targets only one solution, that is usually the global optimum. On the other hand, the goal of multiobjective optimization is to represent the whole set of trade-off Pareto-optimal solutions to a problem. For over thirty years, researchers have been developing Evolutionary Multiobjective Optimization (EMO) algorithms for solving multiobjective optimization problems. Unfortunately, each of these algorithms were found to work well on a specific range of objective...
Show more"Single objective optimization targets only one solution, that is usually the global optimum. On the other hand, the goal of multiobjective optimization is to represent the whole set of trade-off Pareto-optimal solutions to a problem. For over thirty years, researchers have been developing Evolutionary Multiobjective Optimization (EMO) algorithms for solving multiobjective optimization problems. Unfortunately, each of these algorithms were found to work well on a specific range of objective dimensionality, i.e. number of objectives. Most researchers overlooked the idea of creating a cross-dimensional algorithm that can adapt its operation from one level of objective dimensionality to the other. One important aspect of creating such algorithm is achieving a careful balance between convergence and diversity. Researchers proposed several techniques aiming at dividing computational resources uniformly between these two goals. However, in many situations, only either of them is difficult to attain. Also for a new problem, it is difficult to tell beforehand if it will be challenging in terms of convergence, diversity or both. In this study, we propose several extensions to a state-of-the-art evolutionary many-objective optimization algorithm - NSGA-III. Our extensions collectively aim at (i) creating a unified optimization algorithm that dynamically adapts itself to single, multi- and many objectives, and (ii) enabling this algorithm to automatically focus on either convergence, diversity or both, according to the problem being considered. Our approach augments the already existing algorithm with a niching-based selection operator. It also utilizes the recently proposed Karush Kuhn Tucker Proximity Measure to identify ill-converged solutions, and finally, uses several combinations of point-to-point single objective local search procedures to remedy these solutions and enhance both convergence and diversity. Our extensions are shown to produce better results than state-of-the-art algorithms over a set of single, multi- and many-objective problems."--Pages ii-iii.
Show less

Title: OPTIMAL DESIGN OF MARINE PROPELLERS USING MULTI-OBJECTIVE EVOLUTIONARY ALGORITHMS
Creator: Vihtelic, Peter Joseph
Date: 2021
Collection: Electronic Theses & Dissertations
Description: Marine propeller design is defined by competing objectives. The transportation industry has a unique challenge in designing new equipment because of its constant use. Poor designs are paid for upfront and continue to cost the operator throughout the part's entire service life. Propeller design is a complex process attempting to maximize factors, minimizing others while operating within material and equipment constraints. With these considerations, marine propellers are an ideal candidate for...
Show moreMarine propeller design is defined by competing objectives. The transportation industry has a unique challenge in designing new equipment because of its constant use. Poor designs are paid for upfront and continue to cost the operator throughout the part's entire service life. Propeller design is a complex process attempting to maximize factors, minimizing others while operating within material and equipment constraints. With these considerations, marine propellers are an ideal candidate for advanced computerized optimization. The evolutionary algorithms chosen for this review are specifically designed to solve such problems. The high dimensionality of the input variables and multiple nonlinear constraints make finding feasible solutions complicated and finding an optimal set impractical without computerized methods to evaluate and compare results. The utility of evolutionary algorithms is demonstrated effectively with this analysis and review of the marine propeller design problem.
Show less

Title: INTERPRETABLE ARTIFICIAL INTELLIGENCE USING NONLINEAR DECISION TREES
Creator: Dhebar, Yashesh Deepakkumar
Date: 2020
Collection: Electronic Theses & Dissertations
Description: The recent times have observed a massive application of artificial intelligence (AI) to automate tasks across various domains. The back-end mechanism with which automation occurs is generally black-box. Some of the popular black-box AI methods used to solve an automation task include decision trees (DT), support vector machines (SVM), artificial neural networks (ANN), etc. In the past several years, these black-box AI methods have shown promising performance and have been widely applied and...
Show moreThe recent times have observed a massive application of artificial intelligence (AI) to automate tasks across various domains. The back-end mechanism with which automation occurs is generally black-box. Some of the popular black-box AI methods used to solve an automation task include decision trees (DT), support vector machines (SVM), artificial neural networks (ANN), etc. In the past several years, these black-box AI methods have shown promising performance and have been widely applied and researched across industries and academia. While the black-box AI models have been shown to achieve high performance, the inherent mechanism with which a decision is made is hard to comprehend. This lack of interpretability and transparency of black-box AI methods makes them less trustworthy. In addition to this, the black-box AI models lack in their ability to provide valuable insights regarding the task at hand. Following these limitations of black-box AI models, a natural research direction of developing interpretable and explainable AI models has emerged and has gained an active attention in the machine learning and AI community in the past three years. In this dissertation, we will be focusing on interpretable AI solutions which are being currently developed at the Computational Optimization and Innovation Laboratory (COIN Lab) at Michigan State University. We propose a nonlinear decision tree (NLDT) based framework to produce transparent AI solutions for automation tasks related to classification and control. The recent advancement in non-linear optimization enables us to efficiently derive interpretable AI solutions for various automation tasks. The interpretable and transparent AI models induced using customized optimization techniques show similar or better performance as compared to complex black-box AI models across most of the benchmarks. The results are promising and provide directions to launch future studies in developing efficient transparent AI models.
Show less

Title: Metamodeling in Evolutionary Multi-Objective Optimization for constrained and unconstrained Problems
Creator: Hussein, Rayan
Date: 2022
Collection: Electronic Theses & Dissertations
Description: One of the main difficulties in applying an optimization algorithm to a practical problem is that the evaluation of objectives and constraints often involve computationally expensive procedures. To handle such problems, a metamodel (or surrogate model, or response surface approximations) is first formed from a few exact (high-fidelity) solution evaluations, and then optimized by an algorithm in a progressive manner. However, there has been lukewarm interest in finding multiple trade-off...
Show moreOne of the main difficulties in applying an optimization algorithm to a practical problem is that the evaluation of objectives and constraints often involve computationally expensive procedures. To handle such problems, a metamodel (or surrogate model, or response surface approximations) is first formed from a few exact (high-fidelity) solution evaluations, and then optimized by an algorithm in a progressive manner. However, there has been lukewarm interest in finding multiple trade-off solutions for multi-objective optimization problems using surrogate models. The literature on surrogate modeling for constrained optimization problems is also rare. The difficulty lies in the requirement ofbuilding and solving multiple surrogate models, one for each Pareto-optimal solution. In this study, we propose a taxonomy of different possible metamodeling frameworks for multi-objective optimization and provide a comparative study by discussing advantages and disadvantages of each framework. Also, we argue that it is more efficient to use different metamodeling frameworks at different stages of the optimization process. Thereafter, we propose a novel adaptive method for switching among different metamodeling frameworks. Moreover, we observe the convergence behavior of the proposed approaches is better with a trust regions method applied within the metamodeling frameworks. The results presented in this study are obtained using the well-known Kriging metamodeling approach. Based on our extensive simulation studies on proposed frameworks, we report new and interesting observations about the behavior of each metamodeling framework, which may provide salient guidelines for further studies in this emerging area within evolutionary multi-objective optimization. Results of this study clearly show the efficacy and efficiency of the proposed adaptive switching approach compared to three recently-proposed other metamodeling algorithms on challenging multi-objective optimization problems using a limited budget of high-fidelity evaluations.
Show less

Title: Solving Computationally Expensive Problems Using Surrogate-Assisted Optimization : Methods and Applications
Creator: Blank, Julian
Date: 2022
Collection: Electronic Theses & Dissertations
Description: Optimization is omnipresent in many research areas and has become a critical component across industries. However, while researchers often focus on a theoretical analysis or convergence proof of an optimization algorithm, practitioners face various other challenges in real-world applications. This thesis focuses on one of the biggest challenges when applying optimization in practice: computational expense, often caused by the necessity of calling a third-party software package. To address the...
Show moreOptimization is omnipresent in many research areas and has become a critical component across industries. However, while researchers often focus on a theoretical analysis or convergence proof of an optimization algorithm, practitioners face various other challenges in real-world applications. This thesis focuses on one of the biggest challenges when applying optimization in practice: computational expense, often caused by the necessity of calling a third-party software package. To address the time-consuming evaluation, we propose a generalizable probabilistic surrogate-assisted framework that dynamically incorporates predictions of approximation models. Besides the framework's capability of handling multiple objectives and constraints simultaneously, the novelty is its applicability to all kinds of metaheuristics. Moreover, often multiple disciplines are involved in optimization, resulting in different types of software packages utilized for performance assessment. Therefore, the resulting optimization problem typically consists of multiple independently evaluable objectives and constraints with varying computational expenses. Besides providing a taxonomy describing different ways of independent evaluation calls, this thesis also proposes a methodology to handle inexpensive constraints with expensive objective functions and a more generic concept for any type of heterogeneously expensive optimization problem. Furthermore, two case studies of real-world optimization problems from the automobile industry are discussed, a blueprint for solving optimization problems in practice is provided, and a widely-used optimization framework focusing on multi-objective optimization (founded and maintained by the author of this thesis) is presented. Altogether, this thesis shall pave the way to solve (computationally expensive) real-world optimization more efficiently and bridge the gap between theory and practice.
Show less

MSU Libraries

Digital Repository

Enabled Filters

MSU Libraries

Digital Repository

You are here

Search results