Ì0300nnovization'' is a task of learning common principles thatexist among some or all of the Pareto-optimal solutions in amulti-objective optimization problem. Except a few earlierstudies, most innovization related studies were performed onthe final non-dominated solutions found by an evolutionary multi-objective algorithm eithermanually or by using a machine learning method.Recent studies have shown that these principles can be learnedduring intermediate iterations of an optimization run and simultaneously utilized in thesame optimization run to repair variables to achieve a fasterconvergence to the Pareto-optimal set. This is what we are calling as ò0300nline innovization'' as it is performed online during the run of an evolutionary multi-objective optimization algorithm. Special attention is paid to learning rules that are easier to interpret, such as short algebraic expressions, instead of complex decision trees or kernel based black box rules.We begin by showing how to learn fixed form rules that are encountered frequently in multi-objective optimization problems. We also show how can we learn free form rules, that are linear combination of non-linear terms, using a custom genetic programming algorithm. We show how can we use the concept of k0300nee' in PO set of solutions along with a custom dimensional penalty calculator to discard rules that may be overly complex, or inaccurate or just dimensionally incorrect. The results of rules learned using this custom genetic programming algorithm show that it is beneficial to let evolution learn the structure of rules while the constituent weights should be learned using some classical learning algorithm such as linear regression or linear support vector machines. When the rules are implicit functions of the problem variables, we use a computationally inexpensive way of repairing the variables by turning the problem of repairing the variable into a single variable golden section search.We show the proof of concept on test problems by learning fixed form rules among variables of the problem, which we then use during the same optimization run to repair variables. Different principleslearned during an optimization run can involve differentnumber of variables and/or variables that arecommon among a number of principles. Moreover, a preferenceorder for repairing variables may play an important role forproper convergence. Thus, when multiple principles exist, itis important to use a strategy that is most beneficial forrepairing evolving population of solutions.The above methods are applied to a mix of test problems and engineering design problems. The results are encouraging and strongly supportsthe use of innovization task in enhancing the convergence of an evolutionary multi-objective optimization algorithms. Moreover, the custom genetic program developed in this work can be a useful machine learning tool for practitioners to learn human interpretable rules in the form of algebraic expressions.
Read
