. ;lr.,.. .Jnta .. to; an... s .1- Ir. rift c: v: ‘ ,z I; V .11.! . .D 3'... .0. z . .. r......:|..b.)u..:a I.” 3,553! {c- F. ,U ¢ It’z give. all .‘p fftisof... 1116318 ’1 ‘mulllillfilllllllIlllllnmnmm " 3 1293 01771 8291 ‘ LIBRARY Michigan: State Ul'iivv'i 351"] This is to certify that the dissertation entitled . ,, . -31, (MW/752 fléch WM [M@ £42! in” ' J .571 A" 5’1 24W malt/”J ,‘anflém 7 by . presented by Zamei M. Ai-Zawei has been accepted towards fulfillment of the requirements for OkD degree in AQUAWM em“? O W - Major profelsor Date March 3 v, [616ch PLACE lN RETURN BOX to remove this checkout from your record. TO AVOID FINE return on or before date due. MAY BE RECALLED with earlier due date if requested. DATE DUE DATE DUE DATE DUE 1198 chlRC/Mnmu UNSTABLE PERIODIC ORBIT EXTRACTION ERROR AND ITs EFFECT ON NONLINEAR SYSTEM PARAMETRIC IDENTIFICATION By Zamel M. Al-Zamel A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Mechanical Engineering 1999 ABSTRACT UNSTABLE PERIODIC ORBIT EXTRACTION ERROR AND ITS EFFECT ON NONLINEAR SYSTEM PARAMETRIC IDENTIFICATION By Zamel M. Al-Zamel In this dissertation the recurrence method is applied to extract unstable periodic orbits in the tent, the horseshoe and the Henon maps, the Duffing equation and an experimental two-well oscillator. The extraction error is quantified, and a proposed theoretical bound for that error is characterized. Also, the least-squares estimation method is applied in order to improve the extracted orbits. Finally, the effects of the extraction error and its improvement on the nonlinear systems parametric identifica- tion are quantified. In order to quantify the extraction error, the true solutions are Obtained either analytically or numerically in the maps and Duffing equation. After quantifying the error, the exact theoretical bounds are calculated based on the jacobian matrices of the known maps, and also estimated from the time series of the four systems. The results show the applicability of the proposed theoretical bound to bound the error at the recurrence point, while it fails to bound the error at some interior points. Improvement of the extracted orbits is Obtained by estimating the linear or affine models for the studied systems around every cycle point of the extracted periodic orbit. Using the linear or affine model in the estimation method leads to almost the same results with only one exception. In case of using the linear model, changing variables to shift the origin to the locality of the periodic points requires a special treatment. This estimation method successfully improves the extracted periodic orbits in the studied maps. In case Of Duffing equation, the extraction error is on the order of 10"2 and is reduced to zero, for the estimated orbits of period-l for l S 4, while for higher periods the estimation method is not capable of improving the extracted orbits, except at the recurrence point. The least-squares method is used again to identify the parameters Of maps. For clean signals, the extraction error has no effect on the identified parameters. In case of Duffing equation, the harmonic balance method is used to identify its parameters. Here, the effect of the extraction error on the identified parameters is approximately on the same order of the error itself. . The effects of noise and phase space reconstruction on the extraction and esti- mation results, including the parametric identification, are discussed. The extraction results are less sensitive to noise, compared to the estimation results. While the effect of reconstructing the phase space from a single signal is dramatic and destructive. Finally, the results of an experimental time series are shown. The unstable periodic orbits are extracted from the reconstructed phase space, and improved by estimation. Then, the parameters of the system are identified. ACKNOWLEDGEMENTS I am very grateful and thankful to my Lord and the Lord of universe, Allah, Who bestowed me with uncounted grace and relief. No doubt that I am depending on Him and I hope He will keep me so. I acknowledge the prayers and supports of my parents, and I cannot return them what they did form me. I cannot forget the prayers and sacrifices of my wife and her parents, and I hope I will be able to match their love and expectations. This work would not be as it is without the great advice and guidance of my advisor Professor Brian Feeny. His present mind and unlimited effort shaped this thesis and opened the way for me to achieve the goals of this work. I thank Professor Hassan Khalil fOr his great personality that I keep as an example for me, and for his guidance and advice from the first year that I came to Michigan State University. I thank Professor Steve Shaw for his help in this thesis and for the information that he gave in his class that I took. Also, I thank Professor Sheldon Newhouse for accepting my request to have him in my committee, and for his help in the mathematical interpretations for many subjects that are related to this thesis. I thank Professor Ranjan Mukherjee for his service in my committee. Finally, I appreciate the efforts and the great help of all the people who helped my wife and me bear our lives during the years that we spent in the United States. I thank those who worked in the Islamic Center to provide all Muslims with the many things that they need. I ask the Almighty to enable me to serve my country and deliver a part of what I have received. iv TABLE OF CONTENTS LIST OF FIGURES LIST OF TABLES 1 INTRODUCTION 1.1 Background ................................ 1.2 Motivation ................................ 1.3 Contributions ............................... 2 DETERMINING THE EXTRACTION ERROR 2.1 Maps .................................... 2.1.1 The tent map ........................... 2.1.2 The horseshoe map ........................ 2.1.3 Henon map ............................ 2.2 Flows ................................... 2.2.1 Duffing Equation ......................... 2.3 Summary ................................. 3 CHARACTERIZING THE THEORETICAL BOUND 3.1 Determining the Theoretical Bound on Extraction Error ............................ 3.1.1 Maps ................................ 3.1.2 Flows ............................... 3.2 Evaluating the Estimated Bounds .................... 3.2.1 The tent map ........................... 3.2.2 The horseshoe map .............. i .......... 3.2.3 Henon map ............................ 3.2.4 Duffing equation ......................... 3.3 Summary ................................. vi xii <0me 10 11 12 18 22 32 33 35 42 43 44 56 57 58 58 63 72 80 4 IMPROVING THE EXTRACTED ORBITS 4.1 Improvement Results ........................... 4.1.1 The tent map ........................... 4.1.2 The horseshoe map ........................ 4.1.3 Henon map ............................ 4.1.4 Duffing equation ......................... 4.2 Summary ................................. 5 NONLINEAR SYSTEMS PARAMETRIC IDENTIFICATION 5.1 Theoretical Background ......................... 5.1.1 Identifying parameters in maps ................. 5.1.2 Identifying parameters in flows ................. 5.2 Results and Error Analysis ........................ 5.2.1 Maps ................................ 5.2.2 Flows ............................... 5.3 Effect of Noise and Phase Space Reconstruction ....................... 5.3.1 Effect of noise ........................... 5.3.2 Effect of phase space reconstruction ............... 5.3.3 Summary ............................. 6 EXPERIMENTAL ANALYSIS 6.1 Extracting and Improving the Periodic Orbits ............. 6.2 Applying the Theoretical Bound ..................... 6.3 Identifying the Parameters ........................ 6.4 Summary ................................. 7 CONCLUSION A Program 1 B Program 2 BIBLIOGRAPHY vi 81 83 83 84 85 89 90 96 97 97 99 102 102 106 132 133 142 148 150 152 162 170 180 181 186 188 190 2.1 2.2 2.3 2.4 2.5 . 2.6 2.7 2.8 2.9 2.10 2.11 2.12 3.1 3.2 3.3 3.4 3.5 3.6 3.7 LIST OF FIGURES Two actual selected iterations around period-k point, (Ba. ...... The horseshoe attractor. ........................ The Henon attractor. .......................... Two different real periodic orbits of period-9, which are close to one another. .................................. A real periodic orbit and a real representation of a complex orbit or unrelated periodic orbit of period-9, which are far from one another. A real periodic orbit and a real representation of a complex orbit or an unrelated periodic orbit of period-10, which are close to one another. An extracted period-1 orbit and its corrected orbit for Duffing equation. An extracted period-2 orbit and its corrected orbit for Duffing equation. An extracted period-3 orbit and its corrected orbit for Duffing equation. An extracted period-4 orbit and its corrected orbit for Duffing equation. An extracted period-5 orbit and its corrected orbit for Duffing equation. An extracted period-6 orbit and its corrected orbit for Duffing equation. Two actual selected iterations around period-k point, Ta. ...... Actual eigenvectors and singular vectors for a strongly coupled lin- earized matrix of Henon map. ...................... Actual eigenvectors and singular vectors for a weakly coupled linearized matrix of Henon map. .......................... Unity shapes Of the l-norm (o), the 2—norm (o) and the oo—norm(l:l). Neighboring points for a periodic point of the horseshoe map. Nonlinear (a) and linear (b), stretching and contracting factors in the locality of some extracted periodic point, and the nonlinear (c) and linear (d) ratio of the change in slopes (in the x-y plane) due to that motion. .................................. The nonlinear (a) and linear (b) shapes that result from iterating a set of points that form a circle around the same extracted periodic point that has the factors in Figure 3.6. ................... vii 30 31 36 37 38 39 40 41 53 _— *Ffl 3.8 3.9 3.10 3.11 3.12 3.13 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 5.1 A plot of an extracted, corrected and estimated (improved) period-4 orbit. This extracted period-4 orbit has its largest error at the recur- rence point. ................................ A plot of an extracted, corrected and estimated (improved) period- 6 orbit. This extracted period-6 orbit has its largest error at some interior point. .............................. The theoretical bounds on the recurrence error, as a function of the phase of the orbit, for the extracted period-4 orbit that has its largest error at the recurrence point. ...................... The theoretical bounds on the recurrence error, as a function of the phase of the orbit, for the extracted period-6 orbit that has its largest error at some interior point. ....................... The magnitudes of the eigenvalues Of the local matrices of the extracted period-4 orbit that has its largest error at the recurrence point. The magnitudes of the eigenvalues Of the local matrices Of the extracted period-6 orbit that has its largest error at some interior point. A geometric illustration for the reason behind having bad estimation results when the system exhibits the intermittency phenomenon around some extracted periodic point. ..................... An extracted period-9 orbit plotted with its corrected and estimated orbits, for Henon map. ......................... An enlargement, for one periodic point, to show the improvement result of Figure 4.2. ............................... A false period-2 orbit plotted with its extracted and corrected orbits. The false period-2 orbit plotted with its simulated orbit. ....... A period-4 estimated orbit plotted with its extracted and corrected orbits. ................................... A period-5 estimated orbit plotted with its extracted and corrected orbits. ................................... A period-6 estimated orbit plotted with its extracted and corrected orbits. ................................... The average real parameter error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using orbits of lower periods and large extraction errors. ....... viii 73 76 78 79 87 88 91 92 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 The average real parameter error, ('3, and the average residual, F, for different values for the number Of harmonics, n, in the Fourier series using orbits of higher periods and large extraction errors. ...... The average real parameter error, é, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using orbits of mixed periods and large extraction errors. ...... The average real parameter error, F, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using orbits of lower periods and small extraction errors. ....... The average real parameter error, (’2, and the average residual, F, for different values for the number of harmonics, n, in .the Fourier series using orbits of higher periods and small extraction errors. ...... The average real parameter error, ('2, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using orbits of mixed periods and small extraction errors. ...... The average real parameter error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using nine extracted orbits of periods 1 S 6. .............. The average real parameter error, E, and the average residual, F, for different values for the number Of harmonics, n, in the Fourier series using the estimated orbits of the extracted orbits that are used in Figure 5.7. ................................ The average real parameter error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using eight extracted orbits of periods I S 4. ............. The average real parameter error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using the estimated orbits Of the extracted orbits that are used in Figure 5.9. ................................ The average real parameter error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using four extracted orbits of periods I S 2. .............. The average real parameter error, é, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using the estimated orbits of the extracted orbits that are used in Figure 5.11. ................................ ix 111 113 114 115 117 122 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20 5.21 5.22 5.23 6.1 6.2 6.3 6.4 6.5 The stable period-1 orbit that is used to identify the parameters of the Duffing equation. ............................. The average real parameter error, é, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using a stable period-1 orbit. ...................... The average real parameter error, (2', and the average residual, F, for different values for the number of harmonics, n, in the Fourier series based on the same extracted orbits that produced Figure 5.6, but to identify the parameters Of a higher degree model. ........... The average residual, F, for different values for the number of harmon- ics, n, in the Fourier series based on the same extracted orbits that produced Figure 5.6, but to identify the parameters of a lower degree model. .................................. The average real parameter error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series based on the stable period-l orbit that produced Figure 5.14, but to identify the parameters of a higher degree model. . .. ......... The average residual, F, for different values for the number of harmon- ics, n, in the Fourier series based on the stable period-1 orbit that produced Figure 5.14, but to identify the parameters of a lower degree model. .................................. The average real parameter error, E, and the average residual, F, for different numbers of the removed orbits. ................ The horseshoe map that is contaminated by a 1% noise level. The Henon map that is contaminated by a 1% noise level. ...... A period—2 extracted orbit that is extracted from the contaminated time series. ................................ The reconstructed and normalized Henon map attractor, for time delay, 1' = 1, and embedding dimension, d, = 2. ............... The experimental setup. ......................... The number of recurrence points for different values of the number of samples in period-1 orbit,m(1). ..................... A period-l extracted orbit and its improved orbit. .......... A period-2 extracted orbit and its improved orbit. .......... A period-4 extracted orbit and its improved orbit. .......... 124 125 127 128 129 130 132 134 135 140 144 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17 6.18 6.19 6.20 6.21 6.22 A period-8 extracted orbit and its improved orbit. The recurrence point is labeled as *. ........................... The local dynamics of the period-1 extracted orbit that is plotted in Figure 6.3. ................................ The local dynamics of the period-2 extracted orbit that is plotted in Figure 6.4. ................................ The local dynamics of the period-4 extracted orbit that is plotted in Figure 6.5. ................................ The local dynamics Of the period-8 extracted orbit that is plotted in Figure 6.6. ................................ A recurrence error of a period-6 extracted orbit. ........... The theoretical bound on the recurrence error, as a function of the phase of the orbit, j, for the extracted period—6 orbit that has the recurrence error shown in Figure 6.11. ................. A period-2 extracted orbit that is extracted from a: — :i: space, and its improved orbit. .............................. A period-4 extracted orbit that is extracted from :r — 2': space, and its improved orbit. .............................. A pefiod-8 extracted orbit that is extracted from a: — 5r space, and its improved orbit. .............................. A recurrence error of a period-4 extracted orbit. ........... The theoretical bound on the recurrence error, .as a function of the phase of the orbit, j, for the extracted period-4 orbit that has the recurrence error shown in Figure 6.16. ................. The average residual and the average parameters artificial error for the third degree model. ........................... A period-4 extracted orbit, produced from the set of parameters listed in Table 6.1 for n = 19, that is similar to the period-4 extracted orbit of the experimental time series. ..................... Another period-4 extracted orbit, produced from the set of parameters listed in Table 6.2, that is similar to the period—4 extracted orbit of the experimental time series. ........................ The average residual and the average parameters artificial error for the fourth degree model. ........................... A period-4 extracted orbit that is extracted from the fourth degree model and it is similar to the period-4 extracted orbit Of the experi- mental time series. ............................ xi 157 158 159 160 161 163 164 165 166 167 168 174 6.23 A period-4 extracted orbit that is extracted from the phase space Of the differential equation that has the identified parameters based on seven extracted orbits. .......................... 177 6.24 A period-4 extracted orbit that is extracted from the phase space of the differential equation that has the identified parameters based on the seven improved orbits in :c-i‘ phase space. ............. 178 6.25 A period-4 extracted orbit that is extracted from the phase space of the differential equation that has the identified parameters based on the seven improved orbits in the delays phase space ........... 179 xii 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13 3.1 3.2 3.3 LIST OF TABLES Number of the analytical, extracted and distinct extracted orbits for the tent map time series .......................... 14 error",6, in the tent map extracted orbits that lead to the maximum error ratios, é. .............................. 16 Actual recurrence distances**,ca, that correspond to error in Table 2.2. 16 The maximum error ratios, (i in the extracted periodic orbits, for the tent map. .......... a ....................... 17 Number of the analytical, extracted and distinct extracted orbits for the horseshoe map time series ....................... 19 error“, 6, in the extracted periodic orbits, for the horseshoe map, that lead to the maximum ratios, 3:. .................... 20 Actual recurrence distances", ca, that correspond to error in Table 2.6. 21 The maximum error ratios, 25:, in the extracted periodic orbits, for the horseshoe map. .............................. 21 Number* of the corrected, the extracted and the distinct extracted orbits for the Henon map time series. .................. 24 error“, 6, in the extracted periodic orbits, for Henon map, that lead to the maximum ratios, 25: ......................... 25 Actual recurrence distances", ca, that correspond to error in Table 2.10. 26 The maximum error ratios, 25:, in the extracted orbits, for Henon map. 26 Some of the error ratios, &, in the beginning of the extracted orbits of Duffing equation. ............................. 36 The exact theoretical bounds for the tent map that are calculated by the direct method. ............................ 47 The exact theoretical bounds for the horseshoe map that are calculated by the direct method ............................ 48 The maximum exact theoretical bounds for Henon map that are cal- culated by the direct method. ...................... 48 xiii 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 5.1 5.2 5.3 5.4 5.5 5.6 The estimated theoretical bounds for the tent map that are estimated by the least squares method ....................... The estimated theoretical bounds for the horseshoe map that are esti- mated by the least squares method ................... The maximum estimated theoretical bounds for Henon map that are estimated by the least squares method. ................. Duffing equation theoretical bounds that are estimated by the least squares method, for the selected orbits whose error ratios are listed in Table 2.13 .................................. The maximum actual error ratios, ‘32, in the extracted periodic orbits of the horseshoe map, with using the l-norm. ............. The maximum actual error ratios, fi, in the extracted periodic orbits of the horseshoe map, with using the 2-norm. ............. The maximum actual error ratios, &, in the extracted orbits of Henon map, with using the l-norm ........................ The maximum actual error ratios, 5:, in the extracted orbits Of Henon map, with using the 2-norm ........................ The maximum percentages of the cases where the actual error ratios,26:, violate the estimated theoretical bound, in the extracted orbits of a Henon map time series of length, N = 50,000, and with using the 2-norm. .................................. 6 (0’ The maximum percentages of the cases where the actual error ratios, violate the estimated theoretical bound, in the extracted orbits of a Henon map time series of length, N = 200,000, and with using the 2-norm. ...................... I ............ The ratios of the interior large errors,61, compared to the errors in the beginning, 6, for the extracted periodic orbits of Duffing equation. . . The identification results for the different cases of the tent map. . . The identification results for the different cases of the horseshoe map. The identification results for the Henon map ............... The number of distinct extracted orbits, n.1,, for the Duffing equation. The identified parameters of the Duffing equation that are associated with the minimum average residual value in Figure 5.6 ......... The variation of Fourier series coefficients for two extracted orbits Of periods 1 and 2 due to the improvement in their estimated orbits. . . xiv 55 55 56 57 59 59 63 64 66 66 72 103 105 106 108 112 123 ' 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20 5.21 5.22 The identification results for the Henon map by using the extracted orbits, with 0.1% noise ........................... The identification results for the Henon map by using the points Of the time series, with 0.1% noise. ....................... The identification results for the Henon map by using the improved extracted orbits, with 0.1% noise. .................... The identification results for the Henon map by using the extracted orbits, with 1% noise ............................ The identification results for the Henon map by using the points Of the time series, with 1% noise. .................... . . . . . The identification results for the Henon map by using the improved extracted orbits, with 1% noise. ..................... The identified parameters, for the third order estimation model of the Duffing equation, with a 1% noise level .................. The identified parameters, for the fifth order estimation model of the Duffing equation, with a 1% noise level .................. The identified parameters that are resulting from using three contam- inated extracted orbits of periods I = 1, 2 and 6 ............. The identified parameters that are resulting from using the improved orbits of the three extracted orbits that give the parameters in Table 5.15. ..................................... The identification results for the normalized Henon map. ....... The identification results for the reconstructed and normalized Henon map,for'r=1andde=2. ....................... The identification results for the reconstructed and normalized Henon map,forT=-1anddc=2. ...................... The identified parameters for the Duffing equation based on the re- constructed orbits of periods I S 9 that are extracted for T = 6 and dc = 3 .................................... The identified parameters for the Duffing equation based on 26 ex- tracted orbits from data that are reconstructed from a clean and nor- malized signal for 'r = 6 and de = 3 .................... The identified parameters for the Duffing equation based on 26 ex- tracted orbits that are reconstructed from a clean and original signal forr=6andde=3. ........................... XV 137 137 138 138 139 139 140 140 141 144 146 146 146 5.23 The identified parameters for the Duffing equation based on 26 ex- 61 6.2 6.3 6.4 6.5 tracted orbits that are reconstructed from a contaminated and nor- malized signal for 1' = 6 and (1.3 = 3 .................... The identified parameters of the experimental third order model that are associated with the average residual plot in Figure 6.18. ..... Another set for the identified parameters of the experimental third order model ................................. The identified parameters of the experimental fourth order model that are associated with the minimum average residual value in Figure 6.21. The identified parameters Of the experimental third order model, for seven extracted orbits and their improvement in both kinds of recon- struction phase spaces. .......................... The variation of Fourier series coefficients for a period-3 extracted orbit due to its improvement in its estimated orbits that are estimated in both kinds of the reconstructed phase spaces. ............. xvi 147 170 174 174 176 CHAPTER 1 INTRODUCTION In this chapter, a historical background for nonlinear system characterization is pre- sented first. This includes a broad range Of techniques such as : 0 Phase space reconstruction through the method of delays, in which the entire dynamics will be reconstructed from a small number of observable states 0 Determination of the invariant measures such as fractal dimensions, Lyapunov exponents, density histogram and entropies 0 Determination of the number Of active states 0 Determination of the topological invariants such as unstable periodic orbits through the extraction theory, knot theory and template analysis 0 Nonlinear prediction, cell mapping, weighted maps, neural networks, etc 0 Noise reduction and signal improvement, and o nonlinear system parametric identification Second, the motivation for studying extracting the unstable periodic orbits is discussed. Finally, the contributions of this research to the problem of nonlinear system parametric identification are stated. 1 1 .1 Background Efforts to interpret chaotic time series began nearly a century ago. The statement of Poincare (1854-1912) on the importance of unstable periodic orbits as, “the only breach by which we might attempt to enter an area heretofore deemed inaccessible,” has proven to have been insightful, at least for some of the aims for which such interpretations are sought. P. Drazin and C. King [1], the editors of Physica D58 (1992), summarized these aims : 1. To detect useful and interesting patterns by exploring the data 2. TO construct a model by using the data and as much additional knowledge Of the process as possible; and 3. To verify that the model can both reproduce and predict the patterns, and if necessary, to improve the model further More aims can be added, such as reducing the noise with which the time series is contaminated and controlling the nonlinear system by which the time series was produced. It is possible that this is what the editors meant when they added, to the third aim, improving the model further. It is known that the traditional methods by which the linear systems are analyzed are useless in the nonlinear system analysis without modifications. For example, the Fourier transformation and filters, which are usually used efficiently in the linear system analysis and noise reduction, can not be used in the nonlinear systems without modifications. This is because of the shape of the chaotic spectrum for nonlinear systems, in the frequency domain, which is broad and covers the entire domain. Therefore, it is impossible to specify the frequency at which the system is Operating. Also, any filtering for the signal, to remove the noise effect, will affect its dynamics. 2 Some types of filters increase the dimension of the analyzed system while others decrease it [2,3]. There are some chaotic systems such as the Henon map (which will be studied here) which have remarkably flat spectra so that any filtering is sure to have serious effects on the dynamical information contained by the time series generated by the system [4]. In the late 708 and 808, a lot Of research was concentrated on analyzing nonlinear systems by Observing their time series. The analysis focused on extracting, from the time series, the various invariant quantities for the nonlinear system. By these invariants the system can be characterized and identified. System invariants define the system quantities that are not sensitive to the initial conditions, so that they are fixed for at least a set Of initial conditions in the phase space. Attractors are among the invariant sets Of the system. Some nonlinear systems have more than one attractor. Each attractor has different characteristics. Among the invariants of the nonlinear system are measures such as fractal di- mensions, entropies, global and local Lyapunov exponents, density histograms, and topological invariants such as unstable periodic orbits. Reconstructing the phase space was a revolution in the field of time series analysis. This step can be achieved through the method of delays, which was proposed inde— pendently by Ruelle, Packard et. a1. [5] and Takens [6], whereby a scalar time series, y(n), is used to ~generate a large-dimensional vector time series. This is achieved by constructing the d-vector C(n) = [y(n),y(n + 7'), ...,y(n + (d — I)T)], where 7' is the delaying time and d is the embedding dimension. Takens’s theory proves that under suitable hypotheses, this reconstructed attractor is equivalent in some sense to the original attractor if d is large enough [6]. The proper choice of these two parameters is essential in order to reconstruct an accurate phase space that resembles the original one in its invariant set. There are many criteria by which the optimal embedding parameters can be achieved. ,..,.. m t Phi Buzug and Pfister [7] compared algorithms calculating Optimal embedding para- meters for delay time coordinates. Among these algorithms was mutual information analysis that was derived by Fraser [8] to determine the time delay, 1'. Among the other methods that are used to determine the Optimal delaying time were the integral local deformation, simple spreading of trajectories and fill-factor methods [9]. In general, the idea behind choosing the delaying time is simple. If it is too short, the successive points in the reconstructed phase space will depend on one another too strongly, such that the sequence of points does not represent the dynamics of the system. Also, if the delay is too long successive points will be totally independent, and again they will not represent the dynamics of the system. Also, the effect of noise will be larger if the chosen delaying time is longer than what is required [10]. For the choice of the embedding dimension, d, the same issues will rise again. The dimension has to be large enough to express the dynamics of the system or the reconstructed phase space will collapse upon itself. And if d is larger than what is required the effect of noise will be high. Also, the possibility Of having nearly periodic points will be low because of the relation between the number of data points needed to occupy a ball of a fixed size, 6, and the dimension of the phase space. This number .of the needed points is proportional to (“it [11]. Broomhead and King [12] used a singular value decomposition of the trajectory matrix to Obtain the number of nonzero singular values, which yields a sufficiently large embedding dimension. They plotted the singular values of that matrix for different values of the embedding dimension, then the dimension where these values reach their floor value will be selected'as a sufficiently large embedding dimension. Grassberger and Procaccia [13] observed the change in the invariants of the re- constructed phase space with different embedding dimensions. Then, when these invariants reached to their steady values, they took the corresponding embedding di- mension as an Optimal one. This is not always reliable, as Ding et. al. [9] noted that the correlation dimension, d2, plateaus at the first recOnstruction dimension, dc > (12. Liebert et. al. [14] proposed a method that is guided by topological considerations. Their criterion relies on the position of the neighborhood points at the transition from d to d + 1.. They stated that it is a large enough embedding dimension, d, if the inner points remain inner points and the points in the boundary defining the neighborhood remain boundary points at the transition from d to d + 1. Kennel et. al. [15] implemented the false nearest neighbors method to determine the sufficient embedding dimension. In this method, the dimension d in which the neighboring points will remain also neighboring points in dimension d + 1, will be selected as a sufficient embedding dimension. L. Can [16] proposed a method to determine the minimum embedding dimension from a scalar time series. This method is a modification for the false neighbor method [15]. It is argued that it is not subjective, not strongly dependent on the length of the time series and able to distinguish deterministic signals from stochastic signals. E. Ott et. al. [17] has a survey on the methods that are used to determine the Optimal embedding dimension from a scalar time series based on Takens’ theorem [6]. After reconstructing the phase space (or using the original one, which is not prac- tical especially in the experimental time series), the analyst will be able to apply the different methods to analyze the time series. Among the most important analyzing steps is noise reduction, in order to have a cleaned signal that represents the system more precisely. Much research has been dedicated to cleaning the signal and separating it from the noise. Some of the methods require the prior knowledge of the system and other methods do not. There are also methods that can use the statistics of an Observed clean signal to distinguish it from noise [10]. Sauer [18] presented a method to separate additive noise. A low-pass embedding method and singular value decomposition were used to project the input signal along directions belonging to the signal of interest. Rabinovitch and Thieberger [19] prO— posed a method that used a special autoregressive moving-average filter to create the signal from a random shock generator. Rowlands and Sprott [20] described a method that extracts from a chaotic series a system equation whose solution reproduces the general features of the original data even when the time series is contaminated with noise. Cawley and Hsu [21] presented a local geometric method for noise reduction. Their method is related to the local singular-value analysis technique. Shaw and King [22] used cluster analysis to classify the time series. They also used the principal component analysis to filter the data by looking for a few linear combi- nations of the original variables that account for most of the variances in the data. Some research was dedicated to noise reduction using periodic orbits [21,23,24,25]. For nonrecurrent points, other methods were presented [26,27,28]. Pueyo [29] presented a new method that is proposed to uncover chaotic dynamics in a time series. This method is based on the analysis of the local divergence of trajectories. This method is assumed to easily distinguish the chaos from the linearly auto-correlated random time series. Finally, Schittenkopf and Deco [30] presented a method that can identify the deter- ministic chaos by using an information-theoretic measure of the sensitive dependence on the initial conditions. This method can determine whether the time series is pure noise, chaotic but distorted by noise, purely chaotic or a Markov process. It also gives an approximation for the noise level. The nonlinear system parametric identification is the underlying motivation Of this thesis. Among the many methods for nonlinear system parametric identification is the method that was introduced by Nayfeh [31] where the resonance frequencies were used. Stry and Mook [32] presented a method where the time series was used. The harmonic balance method was used by Yasuda et. al. [33,34]. Extracting the unstable periodic orbits from the time series (Auerback et. al. [35], Lathrop and Koslelich [36] and Tufillaro et. al. [37]) can be used in parametric identification. The orbits can be extracted from a time series produced by a map or a sampled vector field by watching any two points, say 2:,- and an”, which come very close to one another after I: iterations or sampling intervals. The length of time that separates the two points is an indication for the period of the extracted orbit, k. In maps, the points from x,- to $g+k_1 are the elements of the extracted period-k orbit. In vector fields, the number of the extracted samples has to be divided by the number Of samples in period-1 orbits. Parameters, which are related to and effective on the extraction process, will be discussed in this thesis. The extracted unstable periodic orbits were used in nonlinear system parametric identification by Hammel and Heagy [38], Kesaraju and Noah [39], Yuan and Feeny [40] and Van de Wouw et. al. [41]. The present research is a continuation for the work done in reference [40]. In that paper, an ultimate bound was derived for the error associated with extracting the unstable periodic orbits. This bound is related to the singular values of the linearized system matrices around these periodic orbits. 1.2 Motivation As a result of the important role of the unstable periodic orbits in nonlinear system parametric identification and because of the existence of uncertainties and error in the extraction process of these orbits, the unstable periodic orbits extraction is chosen to be studied. The extracted unstable periodic orbits are used in : 0 nonlinear system parametric identification [38 — 41] 7 o Computation of Lyapunov exponents [10,42 - 46] o Computation of fractal dimensions [10,35,47,48,49] 0 Template analysis and knot theory [10,50,51], and 0 Evaluation of a signal determinacy [10,21 - 25] To this end, this thesis will focus on the error associated with unstable periodic orbits extraction. Ultimately, the goal is to apply the quantification of the error to the nonlinear system parametric identification. Specifically, this thesis aims to : 0 Evaluate unstable periodic orbit extraction error and compare it to the theo— retical bound. This provides a verification of the theory and an understanding of how conservative and applicable it might be. 0 Improve the extracted periodic orbit’s accuracy from the dynamics of the data 0 Examine the effect of the extraction error on the parameters that are identified by the extracted orbits 0 Examine the effect of improving these extracted orbits on these identified pa- rameters, and 0 Apply all of the previous points on maps and flows, and compare the results of flows to those of real experimental data. We also encounter several interesting issues in this work. For example, showing the absence of some exact periodic orbits from the extracted orbit list will indicate the deficiencies of the extraction process. Also, showing the difficulty to verify the validity and distinction of the extracted periodic orbits will explain inaccurate results attained by some previous researches [35,52,53,54]. The effect of noise is an important issue that is discussed. This includes studying the sensitivity of the attained results toward the existence of noise. 8 (a... ~..— 1.3 Contributions This research will contribute to the nonlinear system parametric identification an essential element. The unstable periodic orbits extraction process, through which the first step of parameter identification by unstable periodic orbits can be completed, will be improved. The exact unstable periodic orbits are Obtained to characterize the proposed theoretical ultimate bound, for the extraction error, by comparing it to the actual extraction error ratio. The different kinds of applications, which normally are applied on the extracted unstable periodic orbits, will be applied on the exact ones, too, in order to quantify the error associated with the use of these extracted orbits. These applications will include nonlinear system parameter identification and modeling. In summary, this thesis will contribute to nonlinear system characterization by : o Quantifying the extraction error in maps and flows. This includes the theoretical bound verification and the effect of the error on the identified parameters. 0 Improving the extracted orbits and their applicability. This includes studying the effect of this improvement on the identified parameters. CHAPTER 2 DETERMINING THE EXTRACTION ERROR In this chapter the quantifying results for the error in the extracted unstable periodic orbits are stated, and remarks about these results are presented. First, a study is performed on the extraction error in the tent map, the horseshoe map and the Henon map. After that, the extraction error in flows will be. studied. The numerically produced time series Of the Duffing equation is the subject of this study. The importance of the extraction process and its many applications in nonlinear system analysis are the main driving motivations toward studying the error in the extracted unstable periodic orbits. As it was mentioned in the introduction, these extracted unstable periodic orbits are used in many nonlinear system characterization techniques. Furthermore, the existence of a theoretical bound for that error [40] and the need to characterize it, is another reason behind this study. The uncertainty in these extracted orbits and the general lack of knowledge about the true orbits can be found in many previous researches [35,52,53,54]. Next, quantifying the error in the extracted orbits of maps is presented. 10 2.1 Maps In this section, the error in the extracted unstable periodic orbits is quantified for the next three maps : o The tent map with an expansion factor, Ac, that is in the form 22;.“ = Act; If 0 S 1:; _<_ 0.5 (2.1) and :12,“ = ACT,- — 1 if 0.5 < 2:,- g 1 This map coincides with the known tent map under the coordinate transforma- tion 1:.- -) 323 — 1:,- for -;- S 9:,- g 1, and its dynamics are the same. 0 The horseshoe map with the same expansion factor as the tent map and a contraction factor, Ac, that is in the form (I); = Acit,‘ +1 If 0 g y,- S 0.5 yi = ’\e i +1 y (2.2) $i+1 = Acxi + 1 — Ac . and If 0.5 < y,- S 1 yi-i-l = Acyi — 1 Again, this map coincides with the known horseshoe map under the coordinate transformation y.- —> g —y,- for % S y, S 1, and its dynamics are the same. Also, it is a special case in that no points in the unit square are mapped out of it. o Henon map that is in the form at,“ = 1 — 1.42:? + y,- (2 3) 31544 = 0.32.7; As mentioned in the introduction, the unstable periodic orbits can be extracted from a time series produced by a map by watching any two points, say 2:.- and an“, 11 which come very close to one another after I: iterations. Then, the points from :0,- to 1:94-; are the points of the extracted period-k orbit. The procedures of calculating the extraction error involve obtaining the exact unstable periodic orbits of the desired maps and also their time series. Then, the extracted unstable periodic orbits will be compared to their exact solution in order to calculate their error. To compare the exact and extracted orbits, they have to be sorted in some consistent way. Next, the results of these three maps are presented. 2.1.1 The tent map This map is chosen because it is a simple one-dimensional map that exhibits chaotic time series and has an analytical solution for its unstable periodic orbits. The exact solution for the unstable period-k S 10 orbits of this map is Obtained analytically via Mathematica. This solution is reproduced via a Matlab program (see Appendix A). The time series is evaluated via a Matlab program, for A. = 1.9...9, and also via a Mathematica program, for A6 = 2. The reason behind using Mathematica, even though Matlab is the software that will be used in this thesis, is because of the behavior of the Matlab time series with the value of A, = 2. The iterated sequence will go to 1 after some number of iterations. To study the effect of using 1.9...9 instead of 2 on the results that are obtained from Matlab, these results are compared to the Mathematica program results. Also, the results are compared to the case where Ae is taken as 2.0...01. The difference in the unstable periodic orbits extraction error ratios of these three processes is in magnitude of 10'”, where n is the number of nines, or zeros, in Ac value. In order to reduce this effect, the number of nines will be maximized as long as the time series continues to be chaotic. Fourteen nines will maintain chaos and give a difference of magnitude 10‘” in 12 the value of the extraction error ratios (to be defined later). In fact, even though the ' tent map time series is called chaotic, its steady state can be exactly determined for any rational initial condition that is specified on the computer (without noise) [55]. Also, it is easy to calculate a valid initial condition value that takes the time series to a desired steady state periodic orbit (see Appendix B). However, this observation is attainable via Mathematica software. The number of the analytical periodic orbits for the tent map can be calculated, for any period-k orbits. It can be calculated by using mobius inversion [56]. In this thesis, this number is calculated by the next equation F q 2" — Z2" + :2" (7%)): = ”I k '=1 (2.4) where lg, for z' = l, 2, ..., p, are the p largest factors Of k that are needed to include all the smaller factors with taking 1 = 1 for the primary numbers, and r,-, for i = 1,2, ..., q, are the q repeated factors in these large ones that are greater than one. The number of the analytical, extracted and distinct extracted orbits, na, 12c and 124,, are shown in Table 2.1, for orbits of period-k S 10. From a time series whose length N = 15, 000, most of the tent map periodic orbits were extracted. The distance, 6 = 0.005, is used to define a recurrence. It can be _ any value that leads to a reasonable number of extracted orbits, but it gives more accurate results if it is smaller. The number Of extracted orbits, and which ones are extracted, depends on the initial conditions, 6 and the length of the time series. The method of distinguishing the extracted orbits of same period-k, is to associate them with the analytical orbits that yield the minimum amount of error. If the analytical solution is unknown, which is the realistic case, the orbits within a distance 61 = 3 x c = 0.015, will be assumed to represent the same distinct orbit [35]. In both cases, the same orbits are obtained. And, these orbits are sorted, for every period-k, 13 Table 2.1. Number of the analytical, extracted and distinct extracted orbits for the tent map time series. k 12,, II, Tide 1 2 149 2 2 1 108 1 3 2 113 2 4 3 112 3 5 6 109 6 6 9 125 9 7 18 129 17 8 30 100 21 9 56 159 44 10 99 105 42 such that the orbit with the smallest value comes first, and so on. Usually, the representative orbit Of every group of similar extracted orbits is the average of these orbits [35]. Another method for finding a representative orbit, is to choose the orbit that has a minimum recurrence distance [39]. The second method is the one that is applied in this thesis and it is more accurate. From Table 2.1, it is clear that the program is able to extract most of the lower- period unstable periodic orbits. The lowest percentage is for period-10 orbits, and it is 2:“- z 42%. All the exact orbits are real. The orbits that were not extracted may have a high expansion factor, or maybe the initial conditions were such that the orbits were not visited. With an example time series of length N = 50, 000, all the orbits of period-k S 9 and 84% of the orbits of period-10, were extracted. The error at any point of the extracted orbit, 6, that gives the maximum error ratio i for every distinct extracted orbit is listed in Table 2.2, for some orbits of period-k S 10 and index 2 S 8. The orbits indices, 2, distinguish the extracted orbits and are sorted according to the smallest value in the analytical solution of each periodic orbit. 6.. S c is the actual recurrence distance. It is used instead of 6 since 14 I I I I I T,- I I I .4' _029_.... ........................................................................ ;..,.° ..... .....- 6 5 33k _O.3L............: .......... 16237 .......... :yzfiii ................................. - . . , , : .’ _0............§31.. .............. .,..’.. ........................... , ................................... - [I] 3 1.3" Ea, 4.32:3: ........................................................................................... - -033- ..................................................... ''''' ' TheUnstableE.V. ——-— TheStableE.V. .°.34I--------....E ................ .c..... ............... . $0. ..... E....-I i L i i A i 0.28 0.29 0.3 0.31 0.32 0.33 Figure 2.1. Two actual selected iterations around period-k point, ma. To compare the error ratios to the proposed ultimate bound ratio, we have to in Table 2.3. The values of these ratios, fi, are shown in Table 2.4. In all of these In these results we can see that the actual bound for the maximum error ratios of the tent map depends only on I: an Ac. Thus for every I: there is only one bound for distances the oo-norm is used. Figure 2.1 shows the distances for a two dimensional case. these error ratios. This bound can be generalized as 15 it is the minimum possible choice for c that will extract the same extracted orbit. divide them by the actual recurrence distances of their orbits, ca, which are shown Table 2.2. error”, 6, in the tent map extracted orbits that lead to the maximum error ratios, 1 ca' (.) Question mark means that the orbit for these indices is not extracted. (..) The listed values are the error x103. k z = 1 2 3 4 5 6 7 8 1 0.198 9.88 2 0.482 3 0.526 0.05 4 0.828 0.226 0.734 5 1.296 2.322 0.819 4.211 1.241 0.026 6 1.23 0.815 1.972 0.401 0.26 0.167 4.258 2.165 7 2.539 1.932 3.641 1.011 0.294 1.652 3.908 1.032 8 3.732 ?" ? 2.253 ? 2.309 3.474 2.176 9 0.859 2.704 ? 4.451 2.166 0.25 ? ? 10 ? ? ? 1.087 ? 0.583 4.728 ? Table 2.3. Actual recurrence distances**, 6,, that correspond to error in Table 2.2. (..) Question mark means that the orbit for these indices is not extracted . (...) The listed values are the distances x103. k z = 1 2 3 4 5 6 7 8 1 0.099 4.94 2 0.362 3 0.46 0.044 4 0.776 0.212 0.69 5 1.256 2.249 0.794 4.08 1.203 0.025 6 1.209 0.802 1.942 0.395 0.256 0.165 4.192 2.122 7 2.519 1.917 3.613 1.004 0.292 1.639 3.877 1.024 8 3.718 ?" ? 2.244 ? 2.3 3.46 2.168 9 0.858 2.699 ? 4.443 2.162 0.251 ? ? 10 ? ? ? 1.086 ? 0.582 4.723 ? 16 Table 2.4. The maximum error ratios, tent map. (.) Question mark means that the orbit'for these indices is not extracted. k 2:1 2 3 4 5 6 7 8 1 2 2 2 1.333 3 1.143 1.143 4 1.067 1.067 1.067 5 1.032 1.032 1.032 1.032 1.032 1.032 6 1.016 1.016 1.016 1.016 1.016 1.016 1.016 1.016 7 1.008 1.008 1.008 1.008 1.008 1.008 1.008 1.008 8 1.004 7‘ ? 1.004 ? 1.004 1.004 1.004 9 1.002 1.002 ? 1.002 1.002 1.002 ? ? 10 ? ? ? 1.001 ? 1.001 1.001 ? 6 2" 1 26—, in the extracted periodic orbits, for the where A, = 2, which is the expansion factor. Also, as shown in Table 2.2, some of the error values, 6, that lead to the actual bound value, for some periodic orbits, have very small magnitudes. 17 , 4 ‘ §v 9“ G -'%~- ~A,-‘.o‘np‘ A‘. l 5 . I I ‘ I L l I I .1 0.4 0.5 0.6 1 Figure 2.2. The horseshoe attractor. 2.1.2 The horseshoe map The major difference between this map and the tent map is that the horseshoe map is a two-dimensional map. The same procedures and extraction parameters as in i the tent map are applied here (i. e. the analytical solution and the time series with A, = 2,N = 15,000, 6 = 0.005 and A, = % are obtained, where the new parameter A, is the contraction factor). The horseshoe attractor, approximated by iterates of the map, is shown in Figure 2.2. The number of the analytical, extracted and distinct extracted orbits, n,,n, and 71.1,, for the horseshoe map are shown in Table 2.5, for orbits of period-k S 10. The 18 Table 2.5. Number of the analytical, extracted and distinct extracted orbits for the horseshoe map time series. k "a ne nde 1 2 16 2 2 l 1 1 3 2 5 2 4 3 1 1 5 6 3 1 6 9 2 2 7 18 3 3 8 30 3 3 9 56 14 6 10 99 5 3 analytical solution for the periodic orbits of this map has the analytical solution of the tent map as its y—component, while its sit-component is calculated separately (see Appendix A). Comparing the numbers in Table 2.1 with these in Table 2.5 shows the reduction in the number of the extracted orbits. This is caused by the increase in the dimension, d, of the horseshoe map, which increases the number Of data points that are needed to preserve the probability of occupying a ball of a fixed size, 6. This number of data points is proportional to :1; [11]. The distance, cl, by which the distinct orbits of period-k can be distinguished, in the absence of the exact solution, is chosen to be 3 x c = 0.015. All the distinct orbits, by using the exact solution, were correctly distinguished by 61 criterion. The lowest percentage of distinct extracted orbits for the horseshoe map extraction process is for period-10 orbits again, and it is a? z 3%. If the spatial distance, 5, or the length of the time series, N, are increased, this will improve the percentage. For example, if N is increased to 50, 000, instead of 15, 000, the lowest percentage will be 1‘“ z 6%. no 19 Table 2.6. error", 5, in the extracted periodic orbits, for the horseshoe map, that lead to the maximum ratios, fi. (* ) Question mark means that the orbit for these indices is not extracted. l" ) The listed values are the error x103. 1c 2:1 2 3 4 6.33 9.88 6.046 3.114 3.193 4.057 27“ H . . . 1.643 3.634 . . ? ? 4.417 ? . . . ? ? ? 4.498 4.02 4.879 4.661 2 3 4 5 6 7 8 9 ...1 O The errors, (I, that lead to the maximum error ratios are shown in Table 2.6. Their corresponding recurrence distances, 6,, are shown in Table 2.7. The values of these ratios, £3 are listed in Table 2.8. Again, in all of these distances the OO—norm is used. The actual bound is the same as in case Of the tent map. Its general formula would be :13, if the contraction factor, A,, was greater than Agl. This means that the maximum ratio will be for an extracted orbit that is moving in the contraction direction instead of the expansion direction. The major difference is the existence Of some values that are less than the tent map actual bound, such as in orbits of period-k = 3 and 9. This is because the extracted orbit is not on the expansion vector yet. The general formula for the minimum value of the horseshoe error ratios is 171,75, and this is when the extracted orbit is located on the contraction vector. In conclusion, the similarity between the actual upper bounds for the tent and horseshoe maps is due to the linearized matrix of the horseshoe map, which is diagonal and has the same expansion factor as the tent map. 20 Table 2.7. Actual recurrence distances**, 5,, that correspond to error in Table 2.6. (* ) Question mark means that the orbit for these indices is not extracted. (** ) The listed values are the distances x103. 2:1 2 3 4 5 3.165 4.94 4.534 2.905 2.794 3.804 . 1.591 3.578 4.348 ? 4.665 ? ? ? 4.48 4.004 ? 4.879 4.657 ? 1 2 3 4 5 6 7 8 9 H C Table 2.8. The maximum error ratios, £, in the extracted periodic orbits, for the horseshoe map. (* ) Question mark means that the orbit for these indices is not extracted. 2 = 1 2 3 4 5 6 7 8 2 2 1.333 1.072 1.067 1.016 1.008 1.004 1.004 1.00005 1.001 ? CWNGMAOJMH H O 21 Again, if this matrix had a contraction factor greater than the reciprocal Of its expansion factor, the bound would be different, but still all the orbits of period-k would have the same bound, i. e. 1:11;. Also, the ability of the eigenvalues to bound the actual error ratios is a result of having a diagonal matrix, which means that its eignvalues and vectors are the same as its singular values and vectors. And because there are no nonlinearity distortions. 2.1.3 Henon map The Henon map is expected to act differently because of its linearized matrix that is not diagonal and also depends on the x-component of the linearization reference point. The first difference is enough to make different ways of multiplication of any orbit’s linearized matrices produce different matrices. This is expected to give different bounds for the extracted orbits of the same period-k or even index 2. The “true” solutions for the unstable periodic orbits Of the Henon map are treated differently, too. They are calculated numerically via Mathematica, up to period- k = 5. Some of these “true” orbits are complex. These complex solutions will not be extracted since they are not real periodic orbits, and have large imaginary parts. On the other hand, complex orbits with small imaginary parts can lead to recurrences, which can then be extracted and thought to represent true periodic orbits. For the orbits of period-k > 5, the true solution is unachievable via Mathematica. Thus, a Matlab program that uses an intensive search method is designed to correct the extracted orbits in order to approximate the corresponding true ones [53,66,67]. This is acheived by altering the existed initial conditions and simulate the map for all the neighboring initial conditions, then choosing the initial conditions that yeild the minimum recurrence distance, 6,. In the correction step, the extracted orbits are corrected in such a way that will keep the advantage of correcting every point to the next point correction by using 22 its iteration. In this way, the points will be corrected to the closest related periodic points. If the points are corrected separately without the use Of the previous correction iteration, the corrected orbit may have unrelated points, i. e. points with different 2 or even 1:. The accuracy of these corrected orbits can be verified by obtaining a very small recurrence distance that is on the order of 10’” after one full cycle of k iterations. If there is an initial error in some point of these orbits, its recurrence distance may equal the initial error multiplied by nearly 100, for some high period-k orbits. However, the correction method exploits the knowledge of the map, and it is not useful in case of unknown systems. And if all of the true solutions are needed, then the method have to be applied on the entire region Of attraction. The number of the true periodic orbits for Henon map, as indicated in all the orbits of period-k S 5 that are calculated via Mathematica, is the same as the number of the analytical periodic orbits for the horseshoe map but with many complex orbits among them. For example, both of the orbits of period-3, two of the three orbits Of period-4 and all of the six orbits of period-5, are complex orbits. Furthermore, some of the real periodic orbits are not part of the attractor. An example for a real orbit that is disjointed from the Henon attractor is the first orbit Of period-1 orbits, i..e. with index 2 = 1, which is a periodic point at (—l.131, —0.3394). The shape of the Henon attractor is shown in'Figure 2.3. The number of the corrected, extracted. and distinct extracted orbits, n,, n, and 114,, for'the Henon map are shown in Table 2.9, for orbits of period-k S 10. Comparing the number of extracted orbits in Table 2.9 with these in Table 2.5 shows some similarity. However, there are more extracted orbits, n,, for Henon map in case of periods-k = 7,8 and 9. This is expected since its dimension is lower than the dimension of the horseshoe map. The program did not extract any orbit of period-10 from the time series of length 23 °°“ - I I I ! I ,3. ............... ............. ....................................................... _ ,2- ............... ................ ................ ............................. - 0.1- ............... ................ ................ ............. i ............ _ y 0- ............................................................................................ - _,,,_ ......................................................................................... , 4,, ................ _ 4,3- ............................................. E ................................................. _ —O.4 1 4 l 1 1 -1.5 -1 —O.5 O 0.5 1 1 .5 Figure 2.3. The Henon attractor. Table 2.9. Number* of the corrected, the extracted and the distinct extracted orbits for the Henon map time series. (* ) The first number corresponds to N = 15,000; while the second one corresponds to N = 50,000. K n, n; n}, 1 2 16,37 1,1 2 1 1,14 1,1 3 0 0,0 0,0 4 1 1,8 1,1 5 0 0,0 0,0 6 2 2,16 1,2 7 4 15,64 3,4 8 7 13,45 2,6 9 7 21,67 4,5 10 10 0,28 0,5 24 Table 2.10. error**, 6, in the extracted periodic orbits, for Henon map, that lead to the maximum ratios, g. (* ) Question mark means that the orbit for these indices is not extracted. (** ) The listed values are the error x103. k 2:1 2 3 4 5 6 7 8 1 ?" 2.184 2 0.958 3 4 3.736 5 6 6.496 2.511 7 9.763 2.01 11.116 5.63 8 2.589 10.359 8.073 2.848 ? 3.778 2.869 9 21.912 1.135 8.278 ? 22.596 3.944 ? 10 3.929 ? 1.08 4.656 7.157 ? ? ? N = 15, 000. If the spatial distance, 6, or the length of the time series, N, is increased; the program will extract more orbits. For example, if N is increased to be 50,000, the lowest percentage Of distinct extracted orbits will be 35::- z 50% for orbits Of period-k = 10, as listed in Table 2.9. It is unfair to compare this percentage to the previous ones since the number of orbits available for the data to visit is dramatically changed. The fair comparison, which showed the similarity of the extraction process of the Henon and horseshoe maps, was the previous comparison between the number of the extracted orbits of these maps with same extraction parameters. For the Henon time series Of length N = 50,000, the error, 6, that leads to the maximum error ratios is listed in Table 2.10. Its corresponding actual recurrence distances, 6,, are listed in Table 2.11. The values of the maximum error ratios, 63;, are shown in Table 2.12. Again, in all of these distances the OO-norm is used. In these results we can see that the actual bound for the maximum error ratios of the Henon map differs from one orbit of period-k to another orbit of the same period. 25 Table 2.11. Actual recurrence distances**, 6,, that correspond to error in Table 2.10. (* ) Question mark means that the orbit for these indices is not extracted. (** ) The listed values are the distances x103. k 2:1 2 3 4 5 6 7 8 1 ?" 1.77 2 0.662 3 4 2.678 5 6 4.893 1.582 7 4.848 0.75 3.41 4.57 8 1.712 2.638 4.36 2.01 ? 3.81 2.376 9 1.642 0.275 3.585 ? 4.019 1.938 ? 10 0.534 ? 0.486 2.416 4.939 ? ? ? Table 2.12. The maximum error ratios, &, in the extracted orbits, for Henon map. (* ) Question mark means that the orbit for these indices is not extracted. k 221 2 3 4 5 6 7 8 1 ?" 1.234 2 1.449 3 4 1.395 5 6 1.328 1.587 7 2.014 2.679 3.26 1.232 8 1.512 3.927 1.852 1.416 ? 0.992 1.207 9 13.49 4.12 2.309 ? 5.623 2.035 ? 10 7.359 ? 2.215 1.927 1.45 ? ? ? 26 This is what was expected before, since the linearized matrix for the Henon map is not diagonal and not even constant; add to that the uncertainty in the true orbits. The excessive error, in Table 2.12, may result from the fact that there are some corrected orbits that are so close to one another that they cannot be distinguished to the resolution of this study. Then, some extracted orbits may be compared to the wrong true ones. Some of the true orbits that are not compared are complex, or unrelated, orbits which means that they will not maintain small recurrence distances if corrected. While others may be real and related. For example, the two points (—0.9063, —0.38338) and (—0.9039, -0.383316) be- long to two different period-9 real corrected orbits. As mentioned above, the acuracy of these two real orbits is guaranteed by obtaining a recurrence distance that is on the order of 10‘“ for both of them, so that their possibility to represent the same true orbit is null. The two orbits are plotted in Figure 2.4. Without the ability to distinguish between these two real orbits, in the correction step, the corrected orbits will have only one orbit Of them that is used in the com- parison to the extracted orbits (for the purpose of quantifying extraction error). In our data, this makes the error ratio, fi, jump to 112.74 if only the first orbit is used, in the corrected orbits, or 252.72 if only the second one is used. The maximum error ratio, if both of them are used, is equal to 13.49, as listed in Table 2.12. This is an indication of the good accuracy of these corrected orbits since such a small error will cause this high jump in the value of the error ratios. Beside the closeness of distinct orbits, the fact that these orbits are unstable makes the correction of their extracted orbits more difficult. This is because any error will increase exponentially if the orbit is on the expansion direction. The existence of some corrected orbits that have large recurrence distances for their periodic points is necessary in order to reduce the maximum value of the error ratios. These orbits are extracted as real representations of complex orbits, or ex- 27 0-4 I I I I I N——* 0113111 0——© Orbitz 4,2 _ .......... g ............ g ........... f .......... g ............ g .......... , _o.3 .— .................................................................... '0.‘ i L 1 I -1.5 -1 -o.5 o o 5 1 1 5 Figure 2.4. Two different real periodic orbits of period-9, which are close to one another. 28 tracted for unrelated period—k, orbits whose period-k, > 1:. These orbits are needed more when they are far from the real ones. Among the corrected orbits, there is one orbit Of period—9 and another orbit Of period-10 that still have recurrence distances, for their points, that are on the order of 10'3. The effect of removing these real representations of complex orbits or unrelated orbits depends on whether there is a real, and related, corrected orbit that is close to them or not. For example, the real representation Of a complex orbit or unrelated corrected orbit of period-9, which has a large recurrence distance, has the point (-0.0026, —0.2934) as its first point. The closest real and related periodic orbit to it has the point (—0.3452, —0.3312) as its first point, as shown in Figure 2.5. If this real representation of a complex orbit or unrelated corrected orbit is eliminated from the corrected periodic orbits, the error ratio will jump to 238.3 instead of 5.6, which is for the fifth orbit. If the other real representation of a complex orbit or unrelated corrected orbit, that is a period-10 orbit and has the point (0.5941, -—0.216135) as its first point, is eliminated, the error ratio will jump to only 161.6, instead of 1.2. The reduction in the jump value, even though the period is 10 instead of 9, is due to the existence of a closer real orbit to it. This real orbit has the point (0.5804, —0.221) as its first point, as shown in Figure 2.6. i There is no clear end for the correction step of some extracted orbits. Once some orbits with recurrence distances on the order Of 10’3 are accepted, the correction step may continue with using very small gridding steps until obtaining very small error ratios. This is nothing but taking the position of the closest local minimum value of the recurrence distance for an extracted periodic point to represent its corrected periodic point. Then, the corrected orbit will have unrelated periodic points. This is a good reason to not accept any corrected orbit whose periodic points have 29 0-4 I I I I I 0.3. ......... 3 .......... 3 ........:°; .......... g .......... g ........... 02. .......... g .......... g ..... ?‘..°..5 .......... 5. .......... 0 0 Theme! ; ; E 3 5 x x Theoomplex . . 01- .............................................................. U o I n ‘ e u . . . . .X . . . . . - . . . . _o p. ......... ' .......... '...........' .......... ' .......... '. .......... .2 . . . . . . . . . . o . . . . . . . 41.5 -1 4.5 o 0.5 1 1.5 Figure 2.5. A real periodic orbit and a real representation of a complex orbit or unrelated periodic orbit of period-9, which are far from one another. 30 0-4 I I I I 0 i ’0 1‘ : : : ,3- .......... ,9. ......... ............ _. ............ :x : : : _ ~ . _ 0 ’0 Thereal 02. ........... g ............ 3....0 ....... ............ x x TMOOMPIOX : : Q : 3 0.1... ........... : ............ '. ............ ............ SQ ........... o o :0 y 0.. ................................................ z ............. -O.1~ ....................... , ........................ ............. as“ .02.. ........... ........... ............ : : 36° : _0.3 I I l i -1 -0.5 0 0.5 1 1.5 Figure 2.6. A real periodic orbit and a real representation of a complex orbit or an unrelated periodic orbit of period-10, which are close to one another. 31 recurrence distances that are greater than 10‘“). But, from another respected view- point, it is unfair to compare (for the purpose of quantifying the extraction error) the extracted orbits to some far-away real true orbits, while they are extracted for closer complex, or unrelated, true orbits. Determining the right true orbit, to which an extracted orbit has to be compared, is not a trivial matter, and it has a-dramatic effect on the value of the extraction error. 2.2 Flows Unlike maps, in sampled flows the extracted orbits contain a number of points much greater than their periods. Therefore, the first and the most important difference between extracting the unstable periodic orbits in maps and flows will be in the number of points, k, that will be used in the extraction process. In this section, the parameter k is only equal to the number of points in the extracted orbit, while a new parameter, I, will be used as the period of the extracted periodic orbit. For nonautonomous systems, period-l orbits contain a number Of points, m(1) = l m(1), where m(1) is the number of points in period-1 orbits. This number, m(1), equals the period of the period-1 orbits, T1, divided by the sampling time Of the experiment, t,. If this number of points, m(1), is not an integer, which is the realistic case, then this will be the first source Of error in the extracted orbits. In computer experiments, sampling time is the time step, At, that is used in the numerical calculations of these periodic orbits. It is good to choose At in such a way that will guarantee that m(l), for any period-l orbit, will be an integer number. This will reduce the effect of one source of error. In summary, the extraction process in flows will be the same as in case of maps but with two differences : 0 Using the values Of k = m(1),m(2), ...,m(ll) instead Of k = 1,2, ...,l;, where I; 32 is the largest period that will be extracted 0 Using the summation of the orbit points as a criterion value to distinguish between the distinct orbits of the same period Using the summation of the orbit points is replacing the use Of the criterion distance, 51 z 36, that was used to distinguish between the distinct orbits of the same period in maps, 1. e. orbits with different values of z. This is done because of the difficulty and the uncertainty of sorting all the period-l orbits in some conventional way, in order to be compared correctly. As a consequence of that, quantifying the extraction error in flows is only achieved by plotting the extracted orbits in the same plot together with the corrected ones. Then, the 2-norm value of the worst extraction error, for some selected period-l orbits, ‘ will be calculated. 2.2.1 Duffing Equation As a numerical example for the extraction process in flows, a time series that is pro- duced numerically from the Duffing equation will be studied. This equation resembles, in some manner, the experimental process examined later. Quantifying its extraction error will help us to quantify the extraction error in that experimental process. The Duffing equation has the form 5: + ad: + x3 — :1: = 7 cos(wt) (2.6) Guckenheimer and Holmes [57] have shown that for a = 0.15 and 7 = 0.3 the sys- tem reaches a stable period-1 orbit with an initial condition such as (mo,io,to) = (0.1,1,0), while it reaches a chaotic set with another initial condition such as (zo,io,to) = (1,.15,0). They also have shown that for a = 0.21 and 7 = 0.3 the 33. system reaches a stable period-3 orbit with an initial condition such as (20,20, to) = (1, 1, 0). Equation 2.6 can be rewritten in the state space form and as a three dimensional autonomous system that has the form 551 = $2 $2 = -Ot~’€2 - I? + 2:; + 7 cos(w2:3) (2.7) .733 = I As it was mentioned above, the advantage of knowing the period of the periodic orbits and their sampling time will be exploited. For a forced nonlinear ordinary differential equation with constant coefficients, the period-l orbits have a period, T; = IT, where T is the period of the forcing term. Therefore, for the values a = 0.15, 7 = 0.3 and w = lrad/sec, and with an initial condition such as (To,io,to) = (1, .15,0), it is expected to produce a chaotic time series that has many embedded unstable periodic orbits. The period T;, for any period-l orbit, is equal to 2hr sec. In order to have m(1)=64 points in period-1 orbits, the time step, At, has to be equal to % = 0.09817477042468. This value of At will make the point 3,464 returns close to the point z,- in the time series, if they are part of a period-1 orbit. Since the system has three state variables, having two neighboring points, say 2:,- and 2,4,, in the phase space of only the first two state variables is not enough to qualify the segment from 2:,- to 2,4,4 to be a periodic orbit. Another condition that guarantees these two points to have also a third state variable value that affects the system similarly, has to be satisfied. This condition is satisfied if k = m(l), then the segment will be a period-l orbit. This is the reason behind using only the values of k = m(1), m(2), ..., m(l). In other words, this system has an R2 x S 1 state space, with 31 representing explicit periodic dependence on time. 34 By using c = 0.5 At 2 0.05, many unstable periodic orbits are extracted from a chaotic time series Of length 50,000. Some of these extracted orbits are corrected iteratively by altering their initial values and taking the initial values that yield the minimum recurrence distances. Many orbits maintain recurrence distances which are on the order of 10'”. The orbits that fail to maintain that small recurrence distance are expected to be a part of periodic orbits of periods greater than 1. Comparing the extracted orbits to their corrected ones shows that for some period- ! orbits, the maximum error, 6, occurs in some interior point that is far from the two ends of the extracted orbits. This is a violation for the assumption, in Chapter 3, that the maximum error, 6, is the recurrence point, see Figure 2.1. 6 The error ratios, 2;, are not calculated here for all the extracted orbits, since the error, 6, is not easy to be calculated as a result of the difficulty of sorting all the orbits in some consistence way. Nevertheless, for only some extracted orbits, these error ratios are calculated for 6 = ma;r(61, 62), and listed in Table 2.13. The 2—norm is used in these calculations. Figures 2.7-12 show some of these extracted orbits that are plotted together with their corrected ones; for different values of period-l. 2.3 Summary In this chapter, the extraction errors of the four studied systems were quantified. The analytical solutions of the unstable periodic orbits of the tent and the horseshoe maps were obtained. And the true solutions of the Henon map and the Duffing equation were approximated by correcting the extracted orbits by an intensive search method. The extracted orbits‘were compared to their corresponding true solutions, and the largest actual error ratio for every distinct extracted orbit was tabulated. In the next chapter, the theoretical bound that was proposed to bound this extraction error will be evaluated. 35 Table 2.13. Some Of the error ratios, 26:, in the beginning Of the extracted orbits Of Duffing equation. k A 1 0.7287 2 0.6985 3 1.0578 4 1.0745 5 1.0464 6 0.9489 0-3 I I I I ----- - The extracted orbit The corrected orbit Figure 2.7. An extracted period-1 orbit and its corrected orbit for Duffing equation. 36 0-6 I r I I n I T o4.............’ ............ '. ...... 02b ----------- ~ ........... :...........l. ............ l ............ : ...... .Iolol:oconcu- non-:5- . a n . . c o . . a o - C e a u I - u . o - o . . a o p ............................................................... a i . a o . n n n n . . u . , . a . . . . o . . . - o u e o . o o . o . . . . . o . . . . - . . . . . . . h ............................................................................................ a . . . . v . . u 4 . e l u - o . . . . . . . . I I I . O u '-.—..—' ------ Theextractedorbit —- The corrected orbit Figure 2.8. An extracted period-2 orbit and its corrected orbit for Duffing equation. 37 1 I I ----- - Theextractedorbit : : — Theoorrectedorbit 0.8” ............. Z ............. .............. . . . ....... - 0.61. .............. i................% .............. é.................§ ............... E .............. —I -05 .. ........................................................................................... - _o.8 .- .......................... E ................................................................ - _1 l I 1 1 l —1 .5 -1 -0.5 0 0.5 1 1 .5 331 Figure 2.9. An extracted period-3 orbit and its corrected orbit for Duffing equation. 38 ----~- Theextractedorbit —— Theoorrectedorbit ....................... ooooooooooooo 4.8 1 J l l #L —1.5 -1 -O.5 0 0.5 1 1.5 Figure 2.10. An extracted period-4 orbit and its corrected orbit for Duffing equation. 39 ----- - The xtracted orbit — The oorrectd orbit -1.5 , -1 —o.s o 0.5 1 1.5 '73 1 Figure 2.11. An extracted period-5 orbit and its corrected orbit for Duffing equation. 40 1 l i I f 03. ............. ..... O.6-~ ' ‘ ----- - Theextractedorbit —— Thecorrectedorbit O.4~ : f 02 ............................................................ x10] .............................................. 1 ! .. I 42" .......................................................... i .049. .. .................................. ‘0 \ .0.6....'\ ................................................ ...,“... ................ .\ \ ..................... I ' _0.3.. ..... ..\ ...... : .\ 3 _1 ‘ - 1 L J -1.5 -1 -05 O 05 1.5 Figure 2.12. An extracted period-6 orbit and its corrected orbit for Duffing equation. 41 CHAPTER 3 CHARACTERIZING THE THEORETICAL BOUND As it was mentioned in the introductory background, one aim of this thesis is to characterize the theoretical upper bound that has been suggested to bound the actual error ratios, &. After having calculating these actual ratios, for some Of the extracted orbits of the three maps, up to period-10, and Duffing equation up to period-6, in Chapter 2, the theoretical bound will now be calculated. In Section 3.2, the ability of the theoretical bound to bound the actual values will be characterized. If the assumed bound is violated dramatically, then this will declare the failure of this theory and the reason behind'this failure will be sought. Another possibility is to characterize the bound as a conservative bound, and this is when it is much greater \ than the actual values. In both cases, another bound will be suggested. It is possible that the bound will be violated but with an acceptable amount Of error. In this case, the reason of this violation will be sought, and this bound will be accepted. 42 3.1 Determining the Theoretical Bound on Extraction Error The calculations for the theoretical bound begin with determining the linearized ma- trices that represent the discussed system around its periodic points. After that, the singular values of the appropriate matrices are used. A quick review for these calculation steps is presented. If the system is linearized around the period-k point, :r, in Figure 3.1, then (2:), — 23,) = A"(:L'o — ma) & (2:0 —- (to) = (Ak)"1(:t;, — 23,) 6a = In '- xol = |(:c,, - as.) - (1‘0 - xall = KA" - I)(Ito - wall S 6 (3'1) 6a = Ix. - xol = Kn - m(1)-(Ive - xa)|=|(1 - (Ak)")(mk - 230)] S 6 where A" is the linearized period-k matrix around 3,; c, and c are the actual and the criterion distances for the close return points, i. e. recurrences. By using the property of matrix singular values P2l-To " 93a] S “Ah " I)(5'30 — xall S 101le - l‘al Mll‘k - italSl((/‘1kl'l - I)(ivk — 36)] S llllxo - ital (3.2) where p2 and 112 are the minimum singular values of (Ak — I ) and ((A")‘1 — I) and p1 and m are the maximum singular values. The norm that is used here has to be the 2-norm. This will give the next two bounds 61=|xo-:ra|Si & 62=|x,.—x.|gi (3.3) P2 #2 Then, with the assumption that the maximum value of error will be at one Of the 43 ending points, which is not guaranteed, the ultimate bound will be a. = ma:r(61,62) g maa:(i, i) (3.4) P2 #2 Or, in the ratio form, which does not depend on any extraction parameter 61 61 62 1 I c ma:c( c c ) _ ma:z:(p2 #2) ( ) If the above assumption is false, this quantity merely provides a bound on the error in the extracted orbit at the recurrence point. This error bound will be verified by comparing it to the actual error ratios that were calculated in Chapter 2. Figure 3.1 shows two actual points around a period-k point. The points are not in sequence except if k = 1. The position of these two points suggests that this periodic point has a negative expansion factor and a positive contraction factor. Nevertheless, the linear matrix has both of its factors as negative values. These two contradictory observations are due to the shape of the nonlinear manifolds of this periodic point. Next, Determining the linear matrices that are used in the calculations of the theoretical bound, is presented. 3.1.1 Maps In maps, the theoretical bound for any extracted period-k orbit is a function of the singular values of its orbital linearized matrices around the periodic points of that extracted orbit. Every distinct period-k orbit has 1:: different orbital linear matrices, Ah”, where the phase index j = 1, ...,k corresponds to its 1: periodic points, :rfi'2'j, and 2 refers to its index. These orbital linear matrices can be calculated by the next two different methods. 44 -0.29 -0.31 I z - .................................................................................... I ........... I .z . I - I i .’ . . . ’ . . .l , . - _/ I i I.’ . .. _/: p .................... .,.. ,- ........... y ....................................... d : (172,7: _, ' _wa ..II I .’3 . .z e I a :I' h .......................... ’3 ................................................................... - .r‘ : --------- The Unstable EN. The Stable E.V. . . ~ . - o . . . . . . . . n o . . . . . . - C h ................................................................................. . .......... —1 . . . n . n . . . . . n - . . - u l C I I . . . u . 0.28 0.29 0.3 0.31 0.33 Figure 3.1. Two actual selected iterations around period-k point, To. 45 l) Jacobian method This method is good for known maps where the period-k orbit linear matrix, Ak’z’j, can be calculated from the known local linearized matrices based at every point Of that period-k orbit. It is used here to calculate the exact theoretical bound since the required matrices are known. This exact theoretical bound will be used later to verify the accuracy of the estimated theoretical bound that will be calculated later. The local linearized matrices for the three maps are 0 -2.8.2: 1 1 Achon = (3°6) 2 0.3 0 ATcnt = 21 AHorseshoe = O CAI.— The period-k orbit linear matrix, Ak'2'j, can be calculated from the linear local ma- trices based at the points of its orbit. It has the form z,‘ k.z k,z k,z k,z k.z k,z Ak' , = A,_,...A, A, A, ...A,+,A,. (3.7) where Af’z is the local linear matrix that takes the points 2,, in the neighborhood of the period-k orbit point 235‘”. , to their next iteration points 2;.” , in the neighborhood of the period-k orbit point zfi'z’j“, if j < k. In case of j = k, At’z will take the points 2:5, in the neighborhood of the period-k orbit point 25"", to their next iteration points 25+}, in the neighborhood of the period-k orbit point zf'z'l. Note that the singular values of Ak'z'j depend on the phase index j. Therefore, the matrices (Ak - I) and ((A")‘1 — I) of Equation 3.2 will generate different theoretical bounds for different phase index j, for the same orbit. Hence, the k theoretical bounds for every pair of k and 2 values will be calculated. Then, every actual bound will be compared to its corresponding theoretical one. Table 3.1 shows the theoretical bounds for the tent map that are calculated by the direct method. And Table 3.2 shows the theoretical bounds for the horseshoe map. 46 Table 3.1. The exact theoretical bounds for the tent map that are calculated by the direct method. (.) Question mark means that the orbit for these indices is not extracted. k z = 1 2 3 4 5 6 7 8 1 2 2 2 1.333 3 1.143 1.143 4 1.067 1.067 1.067 5 1.032 1.032 1.032 1.032 1.032 1.032 6 1.016 1.016 1.016 1.016 1.016 1.016 1.016 1.016 7 1.008 1.008 1.008 1.008 1.008 1.008 1.008 1.008 8 1.004 ?" ? 1.004 ? 1.004 1.004 1.004 9 1.002 1.002 ? 1.002 1.002 1.002 ? ? 10 ? ? ? 1.001 ? 1.001 1.001 ? The theoretical bounds for the Henon map are not the same for all the points of a period-k and index 2 orbit. So that, only the maximum Of the k bounds for every period-k and index 2 orbit is shown in Table 3.3. Nevertheless, every theoretical bound is compared to its corresponding actual error bound. In all of these tables the 2—norm is used. In Section 3.2, the ability of these exact theoretical bounds will be evaluated by comparing them with the results Of the actual extraction error in Chapter 2. 47 Table 3.2. The exact theoretical bounds for the horseshoe map that are calculated by the direct method. (..) Question mark means that the orbit for these indices is not extracted. 2 =1 2 3 4 5 6 7 8 2 2 1.333 1.143 1.143 R l 2 3 4 5 6 7 8 9 p—a O Table 3.3. The maximum exact theoretical bounds for Henon map that are calculated by the direct method. (...) Question mark means that the orbit for these indices is not extracted. k 2:1 2 3 4 5 6 7 8 1 ?" 1.2211 2 1.2418 3 4 1.3266 5 6 1.3916 1.5742 7 1.899 2.4556 2.6462 1.797 8 5.1562 5.3704 2.4434 2.2611 ? 1.6031 1.6314 9 10.5618 10.355 2.0418 ? 1584 1.6398 ? 10 6.0414 ? 2.3292 2.1419 ? ? ? ? 48 2) Least-squares method In the second method, the least-squares method [58-61] will be used to estimate the linear and affine approximation models around every extracted periodic point. These approximation model can be estimated by two different ways. First way is to estimate, in one step, the desired orbital model that takes the points 2:,- to their k‘h-iteration points, 2.4),, in the neighborhood Of the periodic point 25’7"]. . See Figure 3.1. The second way is to estimate the k local models first. Then, the orbit-k model is calculated from these local models. This method is based on the data and does not exploit the knowledge of the system. The linear estimated model around a periodic point, say :cfi'z'j , has the form (4.4:. - sci-‘1? = A"""’ (x.- - 45"”) (3.8) where A""’j is the estimated linearized matrix that takes any relative point (2:,- —a:f‘z‘j) to its relative k‘h-iteration point (2,4,, — xffgj ), and the variables xf’z’j and 2:3? are representing the same periodic point, xfi'z'j, and they are used to shift the origin to the locality of the linearization area in the neighborhood of that periodic point. Since the models around all the periodic points are needed, this will require the estimation to be repeated around all the distinct extracted periodic points. Then, as mentioned before, the k different bounds, for every 11: and 2 pair, will be calculated. The estimation process begins with the selection of the spatial distance, 6,, that defines the neighborhood of the reference point, xf’z’j . After that, the neighboring points, 12,“, will be stored in matrix 2:1; while their lath-iteration points will be stored in matrix 2);. If the number of these neighboring point is n, the estimated parameters matrix that will be estimated by the least-squares method, i. e. A5”, will minimize 49 the square root Of the summation of the n squared error vector 62 = (92 - Ak'z‘jyn)T(yz - Ak’z'jyi) (3-9) k921j where y.- = x,- -— 2:,- is the relative state variables in the shifted coordinates. Then, the normal equation for matrix A""‘j will be Ak'z" = (yfyl)“yf3/2 (3°10) If the second way is used, the 16 linear local matrices, Af'z, have to be estimated first. Again, the local matrix, Ag”, takes the points in the time series, .25, that are in the neighborhood Of every periodic point, :rfi'z'j, to just their next iterated points, k,z,j+l :1 22,-4.1, that are in the neighborhood of the next periodic point, a: . After repeating this step around all the periodic points in the orbit, then the orbit matrix, Am”. , will be calculated by multiplying the matrices in the same order of iteration, i. e. Ak'z’j = Aff,”Aim?"At"...Af:,A§" (3.11) If the affine model is used, then the linear model sensitivity matrix, yl will be replaced by the affine model sensitivity matrix, y” = [y1 1], where l is a vector of ones with a proper length. And, the estimated matrix Am”. will be replaces by the combined matrix whose transpose is (Afi’z’jf = [(Ak'z'j)T 55”]. This combined matrix contains the required estimated matrices for affine model that has the form (2.4+. — mfg” = Am (a,- — 61"”) + b’W‘ (3.12) If the second way of estimation is used, then A’W”j will be calculated by the same 50 way, while the bk'z'j matrix will have the next form k,z,' k,z k,z k,z k,z k.z k,z k,z k,z k.2 where the index i has the value j if the subscript value will be S k 2' = (3.14) j — k if the subscript value will be > k with 2' = j Equation 3.13 is equivalent to the form _ i+k-2 1+1 k k 6W = 2 H Afi‘b,'z+b,fk_, (3.15) l=i m=i+k-2 If the model is an m-dimensional linear model that has p = m x m parameters, yl, which is the sensitivity coefficients matrix, has to be at least of rank 4:; = m, in order to Obtain a nonsingular matrix yfyl. In case Of affine model, the number of parameters, p = m x (m + 1), then the rank has to be at least (m + 1). and $1123, that are used to shift the origin to the locality of The variables 21"” the linearization area in the neighborhood of the periodic point, can be one of the next four choices 0' The analytical periodic point corresponding to this extracted point e The best extracted point and its lath-iteration point e The average of all the extracted points that are similar to this point and the average of their k‘h-iteration points 0 The average of only the neighboring points that are used in the estimation of this point and the average of their [cm-iteration points. 51 The distance 6, that defines the nearby points that will be selected has to be chosen small enough in order to not exceed the valid area for linearization, but not to the extent that will make the selected points less than the quantity, g. In fact, if the extracted orbits are extracted from a clean signal, the choice Of n = 5; will give the same if not better results. This is because the selected points will be closer to the linearization point, and the effect Of nonlinearity will be reduced. Table 3.4 shows the theoretical bounds for the tent map that are calculated by the least-squares method. Table 3.5 shows the theoretical bounds for the horseshoe map. Again, for Henon map, only the maximum theoretical bound, among the k different bounds for every period-k and index 2 orbit, is shown in Table 3.6. In all of these tables the 2—norm is used. For the tent and horseshoe maps, the estimated theoretical bounds are equal to the exact ones that are listed in Table 3.1 and Table 3.2. This indicates the high accuracy of the estimated matrices that are used in the calculations of the estimated theoretical bound. Also, it verifies the validity of using the estimated theoretical bound in the case when the system is unknown. For Henon map, these estimated theoretical bounds are very close to the exact ones that are listed in Table 3.3 except for the fifth period-9 orbit, which has 2:5. For this orbit, the theoretical bound exceeds the estimated one severely. This periodic point turns out to be associated with the shape of the singular vectors that are shown in Figure 3.2. However, this periodic point is not extracted first with any set of the extraction parameters that are used in this study. Hence, its theoretical bound is not compared to any actual error ratio. Therefore, in Section 3.2, the estimated bound is the only bound that will be evaluated. Figures 3.2 and 3.3 show actual eigenvectors and singular vectors for some periodic points of Henon map. Figure 3.2 shows the singular vectors and the eigenvectors of a strongly sheared linearized matrix. In the contrary of that situation, Figure 3.3 52 0 PP ------ USV1 ----- - ssv2 - - — SSV1 --- USV2 . . . O D o a . . I . b- .......... ' ' ' J ........................... _ . . ............ ... . ....... . ......... . . . . -1 _5 1 i 1 -1 -0.5 0 0.5 1 Figure 3.2. Actual eigenvectors and singular vectors for a strongly sheared linearized matrix of Henon map. shows the singular vectors that are very close to the eigenvectors. This means that the shear, i. e. the off-diagonal elements values, in the linearized matrix is weak. In conclusion, having a strongly sheared linear matrix, around any periodic point, may lead to an exact theoretical bound that is very high. For example, the point that has the vectors plotted in Figure 3.3, has an exact theoretical bound that is equal to 1584. This value is listed in Table 3.3 for the fifth period-9 extracted orbit since it is the maximum theoretical bound value for that orbit. And for this kind of points the estimated bound will be dramatically lower than the exact one. Next, the estimated theoretical bound in flows is determined. 53 0 pp ---—~- usw ----- - ssv2 --- SSV1 - - - USV2 Figure 3.3. Actual eigenvectors and singular vectors for a weakly sheared linearized matrix of Henon map. 54 Table 3.4. The estimated theoretical bounds for the tent map that are estimated by the least-squares method (.) Question mark means that the orbit for these indices is not extracted. k z = 1 2 3 4 5 6 7 8 1 2 2 2 1.333 3 1.143 1.143 4 1.067 1.067 1.067 5 1.032 1.032 1.032 1.032 1.032 1.032 6 1.016 1.016 1.016 1.016 1.016 1.016 1.016 1.016 7 1.008 1.008 1.008 1.008 1.008 1.008 1.008 1.008 8 1.004 7‘ ? 1.004 ? 1.004 1.004 1.004 9 1.002 1.002 ? 1.002 1.002 1.002 ? ? 10 ? ? ? 1.001 ? 1.001 1.001 ? Table 3.5. The estimated theoretical bounds for the horseshoe map that are estimated by the least-squares method (.) Question mark means that the orbit for these indices is not extracted. 2 6 7 8 2 2 =1 3 4 5 2 1.333 1.143 1.143 ?" ? 1.016 1 2 3 4 5 6 7 8 9 H O 55 Table 3.6. The maximum estimated theoretical bounds for Henon map that are estimated by the least-squares method. (.) Question mark means that the orbit for these indices is not extracted. k 2:1 2 3 4 5 6 7 8 1 ?"‘ 1.2213 2 1.2439 3 4 1.3285 5 6 1.3924 1.5709 7 1.9 2.487 2.6327 1.8015 8 5.3902 5.3467 2.4445 2.2548 ? 1.5919 1.6265 9 10.4809 10.1373 2.0371 ? 38.115 1.6447 ? 10 6.4584 ? 2.3534 2.1435 ? ? ? ? 3.1.2 Flows Since the error analysis is not achieved completely in flows, the theoretical bound will be determined for only the selected set of the extracted orbits whose error ratios are listed in Table 2.13. Only the least-squares estimation method is applied here, since the sampled flows do not have known algebraic matrices around their periodic points as in case of maps. They arise instead from differential equations that describe their motion in their phase space. There is no remarkable difference between using the least-squares estimation method in maps or flows. After obtaining the numerically produced time series from the ordinary differential equation, it is easy to apply the same procedures as in maps. The difference between applying the least-squares method in maps and flows can be observed in the estimation result. Unlike maps, the estimation result for flows is not acceptable if the period-l orbit approximation model is estimated by the first way (i. e. estimating it by using the neighboring points T,- and their lam-iterations). This is understandable since the orbit is very long, and it is dificult to have 56 Table 3.7. Duffing equation theoretical bounds that are estimated by the least-squares method, for the selected orbits whose error ratios are listed in Table 2.13. z = 1 0.9906 1.0073 1.1011 1.1707 1.1945 1.2357 QUQWMH’K‘ the required number of neighboring points with reasonable value of 6,. Having had estimated orbits undermines the accuracy of the theoretical bound that is calculated from the matrices of these estimated orbits. However the estimation method gives good estimated orbits if the second way Of estimation, in which the local models are estimated first, is used. Therefore, the local models are estimated first, then the period-l orbit matrix, or matrices in case of affine model, will be calculated. Table 3.7 shows these estimated theoretical bounds for the selected orbits that have their error ratios listed in Table 2.13. Next, evaluating the estimated theoretical bound is presented. 3.2 Evaluating the Estimated Bound After preparing the exact and the estimated theoretical bounds for all the extracted points and showing the similarity of them, the estimated theoretical bound will now be compared to the corresponding actual error ratios calculated in Chapter 2. The exact theoretical bounds are those bounds that are calculated from the known Jacobian matrices of the maps. The estimated theoretical bounds are those bounds that are calculated from the estimated matrices. Next, the results of evaluating the estimated theoretical bounds for all the three 57 maps and the Duffing equation are presented. The theoretical bound on the recurrence error is formulated from singular values based on local linearized dynamics. The 2- norm is used in the calculations of these singular values. Nonetheless, the effect of using other norms is studied in this thesis. 3.2.1 The tent map For the tent map, the actual extraction error ratios in Table 2.4 are bounded by the estimated theoretical bound in Table 3.4 exactly. As it was mentioned in Chapter 2, these actual extraction error ratios are calculated with using the oo-norm. However, they are not effected by the norm that is used. This is because the map is a one dimensional map, hence all the different kinds of norms, such as the l-norm, the 2—norm and the oo-norm, will give the same values. 3.2.2 The horseshoe map For the horseshoe map, the actual extraction error ratios in Table 2.8 are bounded by the estimated theoretical bound in Table 3.5. Unlike the tent map, the calculation of these error ratios is effected by the norm that is used. This is because the map is a two-dimensional map, hence using different norms will lead to different values. For example, if the l-norm is used in the calculations, instead of the oo-norm, the actual ratios will be less than the theoretical bound by a considerable amount, as shown in Table 3.8. While with using the 2—norm, the actual ratios will be closer to the theoretical bound but not equal to it, as shown in Table 3.9. If the oo-norm is used the actual ratios are equal to the theoretical bound in almost all the cases. This is clear from comparing the actual values in Table 2.8 to the estimated theoretical bound in Table 3.5. The effect of using different norms appears in the number of the extracted orbits, 58 Table 3.8. The maximum actual error ratios, 3:, in the extracted periodic orbits of the horseshoe map, with using the l-norm. (..) Question mark means that the orbit for these indices is not extracted. 2:1 2 3 4 5 6 7 8 1.8604 1.9277 1.0715 0.6548 0.7173 . ? ? . . 1.0285 0.6363 . ? . ? ? 0.5673 ? '7 0.8795 k 1 2 3 4 5 6 7 8 9 ? ? ? 0.8545 b—i C Table 3.9. The maximum actual error ratios, 36? in the extracted periodic orbits of the horseshoe map, with using the 2-norm. (..) Question mark means that the orbit for these indices is not extracted. 2 = 1 2 3 4 5 6 7 8 1.9902 1.9976 _ 1.2855 1.0536 1.1231 ? 0.9940 0.8894 0.7649 0.9866 {Dmflcfimbwwi—‘W p—a O 59 too. By using the OO-norm, the number Of the extracted orbits has its maximum value, while its minimum value is Obtained when the l-norm is used. This reduction in the number Of the extracted orbits, when the 1-norm is used, is easy to be justified in the light of the unity shapes of different norms. Figure 3.4 shows the unity shapes for all the three norms that are used in this section. It is clear that if a distance measures some value, d, when using the 2-norm, it will be less than d when using the oo-norm and greater than d when using the l-norm. Therefore, some of the orbits that are extracted using the oo-norm will not be extracted using the 2—norm or the 1-norm. This is because these orbits, which are extracted in case of using the OO-norm, have 6,, > 6 when the later two norms are used. These orbits, which are only extracted with using the oo-norm, are normally the orbits that are far from the periodic point. This is the first reason for the considerable drop in the actual error ratio when using the 1-norm. Furthermore, the linear model matrix is a diagonal matrix, for the horseshoe map, which means that the direction of the maximum error vector, 6, will be close to the direction of the stable eigenvector, in case Of 6 = 61, or the unstable eigenvector, in case of 6 = 62. Figure 3.5 shows actual neighboring points for a periodic point of the horseshoe map. It is clear that these points have relative positions, with respect to the periodic point, that are close to i90° for the points that are leaving the periodic point, and 180° or 0° for those that are coming toward it. In both cases, the angle of the error vector, 6, will be far from 21:45°. While the angle of the actual recurrence distance, 6,, will be closer. This will make the increase in the denominator value of these ratios, 6,, due to the use of the l-norm greater than the increase in the numerator value, 6. Hence the value of the ratios will drop. 60 15.. ........... ' ............ . ............. , ........................ , ............ _ ............ . ........... .- . I O n l O - g} 01......” Figure 3.4. Unity shapes Of the l-norm (o), the 2-norm (o) and the oo-norm(U). 61 “5 I I I I I I I I 0.7,. ......... .......... ......... _ ,7, ......... _ Q72- ......... i .......... i ..... , ..... .......... I ......... .. 0.7,- ......... .......... ......... _. 0,7. ......... .......... .......... .......... .......... .......... .......... i ......... _ u . . . . . - . . . . . . . - u u e . . . . . . 069b ................................................................................................ q . - a - . . . . . n I . . . n . . . . . . . n . 0.68 0.765 0.7655 0.766 0.7665 0.767 0.7675 0.768 0.7685 0.769 0.7695 £17 Figure 3.5. Neighboring points for a periodic point of the horseshoe map. 62 Table 3.10. The maximum actual error ratios, :5: map, with using the l-norm. (.) Question mark means that the orbit for these indices is not extracted. , in the extracted orbits of Henon k 2:1 2 3 4 5 6 7 8 1 ?“ 1.3934 2 0.8723 3 4 1.1461 5 6 1.1296 1.3271 7 1.1677 2.2417 2.152 ? 8 1.6197 2.7161 1.2078 1.1243 ? 1.0516 9 10.812 4.345 0.9573 ? 1.4293 ? 10 4.298 ? 1.7856 1.2708 ? ? ? 3.2.3 Henon map For Henon map, the actual extraction error ratios in Table 2.12 exceed the corre- sponding estimated theoretical bounds, for some periodic points. Nevertheless they are bounded by the maximum theoretical bounds that are listed in Table 3.6. This is because these actual error ratios have phase indices, j, that are not equal to the indices of these maximum theoretical bounds. As in case of the horseshoe map, using different norms effects the number of the extracted orbits. Here again, using the OO—norm gives the maximum value, while the minimum value is obtained when the l-norm is used. The effect of using different norms on the actual error ratios is not easy to be classified. The most important reason, which makes that effect hard to be classified, is the nature of the linear model matrix that is not diagonal in case of Henon map. As a result of that, the direction of the eigenvectors will not have the fixed angles 0°, 180° and :l:90°, for all the periodic points. Also, the singular vectors will not be the same as the eigenvectors. Having eigenvectors that have irregular angles will 63 Table 3.11. The maximum actual error ratios, £- map, with using the 2—norm. , in the extracted orbits of Henon (..) Question mark means that the orbit for these indices is not extracted. k 2:1 2 3 4 5 6 7 1 ?"' 1.2181 2 1.0566 3 4 1.0754 5 6 1.0054 1.471 7 1.4809 2.5645 2.6093 1.1759 8 1.5204 3.3914 1.6785 1.2598 ? 0.9715 1.1709 9 12.6816 4.1146 1.9015 ? 5.9457 1.626 ? 10 5.5049 ? 2.0857 1.5959 ? ? ? make the effect of using different norms irregular, even for the extracted points Of the same periodic point. This is because the effect Of using different norms on the values _Of 6 and c, is not regular. As it was mentioned in case of the horseshoe map, if c, has an angle that is closer to 21:45° than the angle of the numerator, 6, then the value of the ratios will drop, with using the 1-norm. This is again because the increase in the value of the denominator, 6,, will be greater than the increase in the value of the numerator, 6. But if the other case happens, then the value of the ratios will increase. In both cases, if the 2-norm is used, the value of the ratios will be between the two values of the other norms. Comparing the actual error ratios that are obtained by using different norms shows that both of the two mentioned cases occur. Therefore, the l-norm and the 2-norm error ratio values, as shown in Table 3.10 and Table 3.11, are less than the oo-norm values that are listed in Table 2.12, for some periodic orbits and greater for others. Furthermore, there are some periodic points, such as the period-1 periodic point, that do not have the 2—norm value of the actual ratios between their values with the 64 other two norms. This means that for the same exact periodic point, the extracted periodic point that gives the OO-norm maximum value is not the same as the extracted periodic point that gives the maximum value for the 1-norm. This is because the point that gives the oo-norm maximum value is not extracted with using the l-norm, or because of the existence Of a 1-norm maximum value that has a corresponding 00- norm value that is less than the existing OO—norm maximum value. The last part in this characterization Of the ability of the Henon map theoretical bound to bound the actual error ratios, is an effort to justify the violations in the values of these actual error ratios. The 2-norm is the only norm that is used here. This is because the theoretical bound uses the singular values that are calculated, in computer soft-wears, by using the 2-norm. Reviewing all the extracted periodic points shows that these violations depend strongly on the length Of the time series. With longer time series, more orbits will be extracted but with higher error ratios, in some of these extracted orbits. This is normally due to smaller 6., rather than larger 6. For example, a time series with length, N = 50, 000, has the first violation in the error ratios of its period-4 extracted orbit. The percentage value Of this violation if it is compared to the exact bound is W x 100 = 3.54%. If it is compared to 99966 the estimated bound, it has a percentage of Egg—9%“ x 100 = 3.531%. The violation occurs when the third periodic point is extracted first. The actual and the theoretical bounds for this periodic point are not the maximum actual and theoretical values for the period-4 orbit error ratios. Therefore, their values are not listed in Tables 2.12, 3.3 and 3.6. For this time series, the percentages of all Of its violations, compared to the estimation theoretical bound, are listed in Table 3.12. For a time series with length, N = 200, 000, the first violation occurs in the error ratio of the same extracted periodic point, but the percentage value of this violation if it is compared to the exact bound is 1W x 100 = 8.21%. If it is compared to 65 Table 3.12. The maximum percentages of the cases where the actual error ratios,£, violate the estimated theoretical bound, in the extracted orbits Of a Henon map time series of length, N = 50, 000, and with using the 2-norm. (..) Question mark means that the orbit for these indices is not extracted. k 2:1 2 3 4 5 6 7 8 1 ?" 0 2 0 3 4 3.531 5 6 0 0 7 47.951 47.272 142.088 0 8 45.045 53.549 16.089 15.862 ? 0 0 9 253.306 278.82 81.354 ? 470.979 23.833 ? 10 168.035 ? 51.138 50.67 ? ? ? ? Table 3.13. The maximum percentages of the cases where the actual error ratios,&, violate the estimated theoretical bound, in the extracted orbits of a Henon map time series of length, N = 200, 000, and with using the 2—norm. (..) Question mark means that the orbit for these indices is not extracted. k z=1 2 3 4 5 6 7 8 1 T“ 0 2 0 3 4 8.197 5 6 14.037 0.0544 7 67.89 47.272 142.088 42.391 8 186.922 97.596 32.867 95.088 ? 0 0 9 309.62 382.3 88.029 ? 470.979 30.953 ? 10 168.035 ? 51.138 86.273 ? ? ? ? 66 the estimated bound it has a percentage Of W x 100 = 8.197%. For this time series, the percentages of all Of its violations, compared to the estimation theoretical bound, are listed in Table 3.13. The worst value among all of the listed violations is in the error ratios of the fifth period-9 extracted orbit. If this is compared to the exact bound it will not be a violation, since its corresponding theoretical bound is 17.776. While if compared to the estimated bound it has a percentage of 471%, as listed in Tables 3.12 and 3.13. This is an indication that the estimated theoretical bound is not as accurate as the exact one. But this tends to occur only for the periodic points that have a very high shear in their linearized matrices. However, there are many points whose error ratios violate even the exact theoretical bound. Having many small violations is expected as a result of using the linearized model in the calculations of the theoretical bound. The nonlinearity of the stable and unstable manifolds reduces the ability of the theoretical bound specially in the case where the nonlinearity reduces the value of the denominator, 6,, and increases the value of the numerator, 6, in these actual error ratios. Figure 3.6 shows the linear and nonlinear stretching and contracting factors in the locality of some extracted period-k point, plotted in the range from 0° to 360° around that periodic point. This is produced by taking a set of initial condition points that" form a circle of radius 0.005 around the extracted periodic point, as shown in Figure 3.7, and iterating them 1: times. Then, the ratios of changing the magnitudes and slopes (in the x-y plane) of these points with respect to the extracted periodic point and its iteration are plotted. In the linear case, the maximum stretching factor is the same in the two directions that represent the large singular value vectors of the linear matrix, while the minimum contraction factor is also the same in the two directions that represent the small singular value vectors. The eigenvectors are in the direction of the angles at which 67 the ratios of the change in slopes have the unity value. The unstable eigenvector is the only one that is shown to have a unity value in the plot of the slopes ratio. For the stable eigenvector, the unity value is not shown in the plot due to the following three reasons : 1. The angular step that is used equals to 1°, hence it is not expected to get the exact angle. 2. Having the stable eigenvector in a direction close to the right angle gives an initial slope that is close to CO. 3. The resulting slope for that initial slope is close to zero, due to the direction Of the unstable eigenvector that has an angle close to zero. If the nonlinear maximum and minimum factors, which each one Of them occurs in one direction, are used, the theoretical bound will have higher values, and the violations will be less. This is because the effect of nonlinearity on the singular values is removed, but its effect on the shape of the manifolds still exists. Figure 3.7 shows the linear and nonlinear shapes that are resulting from iterating a cycle k-times around the same period-k point that has the factors plotted in Figure 3.6. It is clear that using the linear model changes the direction of the error vector, 62, dramatically. Whether this error vector has a 2-norm, I62I2, that is greater than I61]; or not, this will give a theoretical distance for c, that is much greater than its actual value. In this case, the theoretical bound will be less than the actual one. Figure 3.1 shows all of these error vectors. For the high violation percentages, another reason beside the nonlinearity has to be added, except if its effect is too strong in these cases. Since these high violations are only for the extracted orbits of period-k Z 7, the reason can be related to the uncertainty of the corrected solution that is assumed to be the exact solution (to 68 Factor 100 f 40 - 30 ................... 1 ................. .4 I. O ~08 . a 20 ................... .................. w 101......” ......., ................... o i 0 100 200 300 400 (b) ,Angle,9° 10 g : 10 ~, - 5 ........... ........................... ' ..... 5. .......... .......... g. ..................... .§. : . I .2 3 : . I : , '3 : : E OHH ’Ar' 4....4 g 0H”.- 4....4 0 ’ ' . e . 0 p : o . s . s e ' é - a _5; .......... ........................... ...... a _5 ........... ................................. 100 200 300 400 , 0 100 200 300 400 (c) Angle, 0° (d) Angle, 0° Figure 3.6. Nonlinear (a) and linear (b) , stretching and contracting factors in the locality of some extracted periodic point, and the nonlinear (c) and linear (d) ratio of the change in slopes due to that motion. 69 0.4 I T l I I T l l ' 0.4 0.39 0.38 0.37 0.36 0.35 i i j i i -1.5 -1.45 -1.4 -1.35 -1.3 -1.25 -1.2 -1.15 -1.1 -1.05 Figure 3.7. The resulted nonlinear (a) and linear (b) shapes that are resulting from iterating a cycle around the same extracted periodic point that has the factors in Figure 3.6. 70 which the extracted orbits are compared to quantify their errors). Having two large percentages in sequence, for the first two orbits of period-9, suggests strongly that there is a missing orbit in between. Finding a new corrected orbit close to the one that has a high violation percentage will reduce its percentage dramatically. This was the case when an unrelated orbit or a real approximation of a complex orbit was found just before the fifth real and related period-9 corrected orbit that had an error ratio of 212.7, using the 2-norm. The error ratio dropped to be only 5.95 for the new period-9 corrected orbit. This reduction, in the actual error ratio, means a reduction in the violation percentage from 21,110.6% to the existing one that equals 471%. Also, when the other unrelated orbit or real approximation of a complex orbit was found, just after the ninth real and related period-10 corrected orbit, that had an error ratio of 155.79, then the error ratio dropped to only 1.23 for the new period-10 corrected orbit. Again, this reduction, in the actual error ratio, means a reduction in the violation percentage from 15,458.8% to 22.48%. This great sensitivity of error ratios to finding any corrected orbit is an indication of the accuracy of the existing ones. In fact, all the efforts to find a new corrected period-9 orbit, by correcting the extracted orbit that has the worst error ratio, end with the same old corrected orbit. Even the extracted periodic point that has the worst error ratio goes to its corresponding point in the old corrected orbit. Since there are no closer minima to this extracted orbit, this is an indication that the existing corrected orbit is the orbit for which this orbit is extracted, and the violation is due to the nonlinearity or some other reasons. In fact, all the high violation percentages are due to an extraction error at an “interior” extracted point, i. e. a point other than the recurrence point, hence the theoretical bound cannot be applied on it. 71 Table 3.14. The ratios of the interior large errors,61, compared to the errors in the beginning, 6, for the extracted periodic orbits of Duffing equation. p-a QUAWMHW 3.2.4 Duffing equation For Duffing equation, the selected actual extraction error ratios in Table 2.13 are bounded by the theoretical bounds in Table 3.7. But it is not enough to bound the error in some high period-l extracted orbits. It has been shown, in Chapter 2, that some extracted orbits have their largest deviation from the true orbit at some “interior” point, i. e. a point other than the recurrence point. For the extracted orbits that have their maximum errors, 6;, in some interior point, the formulated bound is unable to bound the maximum errors in the extracted orbits. Table 3.14 shows the approximated ratios of these interior large errors compared to the extraction error, 6, in the beginning of each orbit. Having ratio of 1 means that there is no large interior error. While a periodic orbit is a saddle in its average dynamics, it is possible for its local dynamics to vary widely from source, to saddle, to sink. The large interior errors result from a long series of local dynamics that are sources or saddles. As in case of maps, the extracted periodic orbits in flows have orbital matrices that have invariant eigenvalues, i. e. they are the same for any value of the phase index, j. But, as in case of maps, too, the singular values differ from one point in the periodic orbit to another. This will lead to more than one theoretical bound for each 72 .03 _ ............ ....... . _. ..... ................ .......... ---- Theextracted orbit 3 3 3 — The estimated orbit The corrected orbit -1.5 -1 -0.5 0 0.5 1 1 .5 Figure 3.8. A plot of an extracted, corrected and estimated (improved) period-4 orbit. This extracted period-4 orbit has its largest error at the recurrence point. orbit. To illustrate the difficulty of determining whether the extracted periodic orbit has a large interior error or not, the two extracted periodic orbits plotted in Figures 3.8 g and 3.9 are studied. Figures 3.10 and 3.11 show the theoretical bounds for the two orbits plotted against the phase index, j . In Figure 3.10, the theoretical bounds of the extraction errors of the extracted period-4 orbit that has its large error at the recurrence point is shown, while in Figure 3.11 the theoretical bounds of the extracted period-6 orbit that has a large interior error is shown, too. In all of these plots, the recurrence point is corresponding to the phase index j = 1. 73 1 I l I l I /: f t 0.8" ...... , ..... I ............. , .... ....... I ........... _ 0.6.... ................ 04---- '-'--- Theextractedorblt ' The estimated orbit -—— Theoorrectedorblt 1.5 Figure 3.9. A plot of an extracted, corrected and estimated (improved) period-6 orbit. This extracted period-6 orbit has its largest error at some interior point. 74 The Theoretical Bound 4.5 r r r I r 1 1 I l O O I O O 0 . . . 4 .. ................................................................................................... q . . O O . . . . . . . . . 35p ........................................ . ......... ' .......... ‘ ..................................... q 0 e . . . e e o a— ............................................................................................. - - ................................................................... no ...................... d 25 . . . . . . . O a . I I O O I 2- IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII ~ The Phase, j Figure 3.10. The theoretical bounds on the recurrence error, as a function of the phase of the orbit, for the extracted period-4 orbit that has its largest error at the recurrence point. 75 The Theoretical Bound 25 r I ........................................... ------------------------------------------ 15- ............... ............ ................... 0 50 100 150 The Phase, j Figure 3.11. The theoretical bounds on the recurrence error, as a function of the phase of the orbit, for the extracted period-6 orbit that has its largest error at some interior point. 76 The similarity of the two plots of the theoretical bounds in Figures 3.10 and 3.11 undermines the possibility of determining whether the extracted periodic orbit has a large interior error or not, just from the theoretical bounds of the different phases. In fact, having a large theoretical bound value at j z 80, in Figure 3.11, corresponds to a very small extraction error. While the large interior error at j z 22 corresponds to a very low theoretical bound value. Another idea is to look at the local dynamics of the periodic orbit around all of its cycle points, and try to find a relation between their values and the existence of the large interior error. Figures 3.12 and 3.13 show the magnitudes of the eigenvalues of the local matrices for the two orbits. The clear difference is the existence of a long sequence of local saddle dynamics in Figure 3.13. The large interior error is in the interval that comes just after this long local saddle dynamics that have a flat peak from j=138 to j=8. It would be beneficial if the large interior error can be bounded by an estimated theoretical bound. The results concerning that issue are not clear, yet. Nevertheless, it is very clear that the error is due to the existence of the longest sequence of saddle local dynamics. The extracted orbits will have a very small error when there is long sequence of stable local dynamics. Furthermore, it is a good sign to have the recurrence point in a place just after the longest sequence of saddle or unstable dynamics. This will reduce the ratio of the large interior extraction error, 5;, compared to the extraction error at the recurrence point, 6. This reduction, in the ratio value, is mainly due to the increase in 6. For example, the extracted period-6 orbit in Figure 3.9 has a large interior error that is almost the same as in case of Figure 2.12. The recurrence point is now in the interval that has the longest sequence of saddle local dynamics. The ratio of the large interior error, 51, compared to the extraction error at the recurrence point, 5, is now 3. Its previous value, as listed in Table 3.14, is 8. 77 1.8 l I I I I : : : : : -— The first eigenvalue ‘—---- Theseoondeigenvalue , - . u o o u o . o a l o o e o o n o o e e 16.. .................................................................................................... ..I . o n o I . o u I o . o . . . . . . . e e o o o o e o o 1.4 a 0’ 5 . 3 12 fl 0 U o— [a G 1 u: ["i 0.8 0.3- ........ ......... g ......... .......... g .......... ........ _ 0.4 I i I I I I I J I 0 10 20 30 40 50 60 70 80 90 100 The Phase,j Figure 3.12. The eigenvalues of the local matrices of the extracted period-4 orbit that has its largest error at the recurrence point. 78 1.7 T - - - - - - The first eigenvalue -— The second eigenvalue 1.6 l. :5 .5 ...-L o) a in l The Eigenvalues l 3 K; 0.9 0.8 0.70 so 100 150 The Phase, j Figure 3.13. The eigenvalues of the local matrices of the extracted period-6 orbit that has its largest error at some interior point. 79 3.3 Summary In this chapter, the exact theoretical bounds on the extraction errors for the three studied maps were calculated. Also, the least-squares method was used to estimate these exact theoretical bounds and the theoretical bound of the Duffing equation. For the three maps, the estimated theoretical bounds were very close to the exact ones except in the case where Henon map has severe nonlinearity. However, the periodic point that was associated with this case was never extracted first, hence its theoretical bound was never used. The estimated theoretical bound was capable of bounding the extraction error in case of the tent and horseshoe maps, but, it was incapable to bound the large interior errors that were in the extracted orbits of the Henon map and the Duffing equation. However, the ratio of that interior error, 51, compared to the error in the recurrence point, 5,, was smaller for the case when the recurrence occurred just after the longest sequence of saddle local dynamics. This reduction in the value of the ratio was due to the increase in the recurrence error, 6,, and was not due to the reduction of the interior error itself. The large interior error, in this study, was observed in the extracted orbits of period-k Z 7, in case of the Henon map, and period-l Z 5, in case of the Duffing equation. For the Henon map, it has a maximum ratio of 5 compared to the error in the recurrence point. While in case of the Duffing equation, the extracted orbits that are studied have periods up to period-I = 6, and the maximum ratio is 8. Therefore, the bound is only applicable for the lower periodic orbits, otherwise it bounds the error in the recurrence point only. In the next chapter, the extracted orbits, for all the studied systems, will be improved by using the least-squares estimation method to estimate their exact orbits. 80 CHAPTER 4 IMPROVING THE EXTRACTED ORBITS Since the original system is normally unknown, using estimation methods to improve the extracted periodic orbits based on the knowledge of only its time series is a very important step. The error associated with every estimation method can be quantified in different ways. By quantifying the error, the best method will be determined. In this chapter, the local affine and linear models around the extracted periodic orbits of all the three maps and the Duffing equation will be used to estimate unstable periodic orbits. These orbits will then be compared to the corrected and the extracted ones. For maps, a comparison between the approximated periodic orbits, which are obtained by the following approximation methods, is made 1. Using the least-squares method to estimate the linear model 2. Using the least-squares method to estimate affine model 3. Using the average of the extracted orbits 81 4. Using the extracted orbit with minimum actual recurrence distance 5. Using the average of the close points that were used in the least-squares method. The least-squares estimation method that is used to improve the extracted orbits is explained throughly in Chapter 3. After having the desired model, i. e. the approx- imate local affine or linear maps , all that is needed is to solve the equations of these estimated models in order to obtain their periodic points. If the desired estimated model is the linear model, then its equation is Equation 3.8. It has the form (x... — min") = A’W (... — xi“) (4.1) Then, its period-1 point, which is the period-k point for the original system, is s = (1 — A’ijl (3321' — Ak'z'jrf’z'j) (4.2) If the desired estimated model is the affine model, then its equation is Equation 3.12. It has the form (13m: - 3:3.sz = Ak’z'j (:c; - Ii'z’j) + bk’z’j (43) Then its period-1 point, which is the period-k point for the original system, is :7: = (1 — Ak'w'r1 (x333 — Ak'z'jxf"’j + bkvm') (4.4) 82 4. 1 Improvement Results The results of improving the extracted periodic orbits for all of the three maps and Duffing equation are presented next. 4.1.1 The tent map By comparing the analytical periodic orbits to the best extracted ones, it is found that the error in these orbits, 6, is on the order of 10“. If the average of all the extracted orbits is used, the error is higher but on the same order of 10". When the average of all the neighboring points, which are used in the estimation method, is used, the error is on the order of 10‘3. When the estimated linear or affine model of the map is used in estimating the exact periodic orbits, all the estimated orbits are improved such that the error is on the order of 10"“. For tent map, the results are not affected by the choice of the number of selected neighboring points, as long as the neighborhood region does not contain'the discon- tinuity point. This is expected since this map is linear around its periodic points but changes its dynamics at the discontinuity point, i. e. x,- = l. 2 The very important remark here is related to the selection of the shifting variables k.z.j ’5'”. and 2:,- +1: , which are chosen to shift the origin to the locality of the linearization I: area in the neighborhood of the periodic point. They have to be chosen differently. This is because if they are chosen to have the same value, then the estimated periodic point will be exactly the same as that value. This is only the case when the linear model is used, while if the affine model is hm" ”J and 2:,- +1: , to shift the origin will not affect the used, choosing equal quantities, 1:,- results. This is clear from Equations 4.2 and 4.4. Equation 4.2 will give 56 = x if k k,2,j k,l,j at,- +1. = :2,- = :15. While Equation 4.4 will give :7: = :I: if xffgj + bk'm' = :c ’z’j = :c. 83 All of the different choices of the reference point, xfi'z'j , lead to the same estimation results. These choices are the same as the previous four choices of the variables xf‘z’j and 3:1] , that are used to shift the origin to the locality of the linearization area in the neighborhood of the periodic point. They are e The analytical periodic point corresponding to this extracted point e The best extracted point and its k‘h-iteration point e The average of all the extracted points that are similar to this point and the average of their k‘h-iteration points 0 The average of only the neighboring points that are used in the estimation of this point and the average of their k‘h-iteration points. 4.1.2 The horseshoe map Here again, the extracted orbits have an error on the order of 10“. Also, if the average of all the extracted orbits is used, the error is higher on the order of 10‘3. When the average of all the neighboring points, which are used in the estimation method, is used, the error is higher but on the same order of 10‘3. When the least-squares method is applied to estimate the exact orbits by esti- mating their linear or affine models, the error is reduced to be on the order of 10’”, which is the same as in case of the tent map. Also, for the horseshoe map, the results are not affected by the choice of the number of selected neighboring points, as long as the neighborhood reigion does not contain any point of the discontinuity line, i. e. y.- = %. This is for the same reason as the one was mentioned in case of the tent map. And the remarks that were mentioned about the selection of the reference and shifting variables, is nececessary here, too. 84 4.1.3 Henon map As it was mentioned in Chapter 2, the Henon map behaves differently. Comparing the best extracted orbits of period-k S 10 to their corrected ones shows that these orbits have an error of order 10’3. If the average of all the extracted orbits is used, the error is higher but on the same orders. When the average of all the neighboring points, which are. used in the estimation method, is used, the error is higher and on the order of 10'2. By using the first way of estimation, in which the least-sequares method is used to estimate, in one step, the orbital matrix, Ak'z'j , the error is reduced to be on the order of 10'3 ~ 10'5. In fact, there are some extracted periodic points whose estimated values have error that is worse than their extraction error. Figure 4.1 illustrates one reason behind having bad estimation results when the system exhibits the intermit- tency phenomenon around some extracted periodic point [62]. Mathematically, this means that the extracted fixed point is not really a fixed point, or it is a nonhyperbolic real saddle or a hyperbolic complex saddle. The failure to reach the previous accuracy is due to the nature of Henon map that exhibits the intermittency phenomenon. Also, the existence of quadratic term is making the manifolds of the map nonlinear, as it is clear from the shape of the Henon map attractor, which is the shape of the unstable manifolds of all of its periods. 80 that, the linearization will not match these manifolds like the way it did with the previous maps. Also, the large interior error will not be improved a lot. Furthermore, the very low contraction factors and very high expansion factors make the number of the neighboring points around these extracted periodic orbits very small. This will force the criterion distance, cc, to be larger, then the effect of nonlinearity will increase. Add to that the uncertainty of the corrected orbits that are used as an exact solution. 85 u u e . e e a u e o a e o e - e e e e o e . e e a e n 09- eeeeeeeeeeeeeeeeeeeeeeeeeee . eeeeeeeeee . oooooooooo I .......... ' eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee - . n u e o e e o e e I u e e e e e e e n e v e e e e a e O I I e a e e e e e u e u 'o e I e u u e o e e e a e e 08- eeeeeeee . oooooooooo ' eeeeeeeee . eeeeeeeeee . .......... . oooooooooo . nnnnnnnnnn I eeeeeeeeeeeeeeee . eeeeeeee q . e I a o e a e e e e u - o e I o o e . . - n e n e 0.7L. ........ ......... g .......... g ......... g .......... : .......... g ...... .; ......... g .......... g ........ q . a . a e o e - l I O I I O C 0 06- ............................ ' ---------- ' ......... ' ....... . ......... ' .......... ' oooooooooooooooooo - o . . . e o u e . . 0 . . u e - - . . . l I O . I . I 0 u . . . I I I I C I I D I o e u n . e e o 05— ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo - . o - . n e n o e e e . e I e , e u . I l U C I l I O D 0.4- ........ g ......... ....... : ......... g .......... g .......... ........ .. 0.3- ........ ......... g .......... 3"“3368t .......... g .......... g .......... g .......... g ........ _ 02- ........ g .......... : ........ g .......... g .......... 3 .......... g .......... g .......... g .......... g ........ a C O I I I I ‘ I I 0 I I O I I A I l C 01.. .................................... . .......... . .......... . ....................................... _ O u . o - e u e e - I I I O O I e O O I D 0 I I O Figure 4.1. A geometric illustration for the reason behind having bad estimation re- sults when the system exhibits the intermittency phenomenon around some extracted periodic point. 86 0.4 . 1 i x—x The extracted orbit 0—0 The corrected orbit y 0 - ------ G———0 The estimated orbit i Theperiod-1 point _0.1.. ............................................................................................... .. _02_ ............... ................ ................ ................ ............... _ _o.3.. ............................................................................................... .. _0.4 l ' 1 1 l l -1.5 -1 -O.5 0 0.5 1 1.5 Figure 4.2. An extracted period-9 orbit plotted with its corrected and estimated orbits, for Henon map. 87 Figure 4.3. An enlargement, for one periodic point, to show the improvement result of Figure 4.2. 88 When the second way of estimation, in which the 1: linear local matrices, Af‘z, are estimated first, is used to estimate the orbital matrix, Ak'z'j, the error is reduced to be on the order of 10"3 ~ 10'6. Figure 4.2 shows an extracted period-9 orbit, for Henon map, plotted with its corrected and estimated orbits. The improvement is not visible at this scale. Figure 4.3 is an enlarged plot, for one periodic point, to show the improvement result of Figure 4.2. The same thing happens when the affine model is used. The matrix Ah” is calculated by the same way, while the b""’j matrix is calculated as in Equation 3.13. It is important to emphasize that the number of the neighboring points that were used in the estimation has to be as small as possible, i. e. n = p/ m, in order to get the best results. This is, as it was mentioned before, to reduce the effect of nonlinearity, and it is only valid for a clean signal estimation. Also, the remarks about the selection of the reference points, i. e. shifting variables, in case of tent map, apply here, too. 4.1.4 Dufling equation In this section, the estimation method is applied on the extracted orbits whose ex— tracted error ratios are listed in Table 2.13. If these extracted orbits are compared to their corrected ones, their error is on the order of 10"”. When the estimated matrices, which are estimated by the second way of estimation that estimates the local dynamics first, are used to calculate the estimated periodic points, the error is reduced to 0, for many orbits of period-k _<_ 6. The bad news here that there is a possibility of having false periodic orbits. These false orbits are only representing a collection of points where every point is a periodic point for the estimated model, but not for the original system. Figure 4.4 shows an example for this kind of orbit that is estimated as a period-2 orbit. Verification that this orbit is not real can be obtained in two ways. First, by failing to correct the extracted orbit and produce a corrected orbit with small recurrence 89 distance, as it is clear in Figure 4.4. Second, by simulating the original system with using the suggested initial condition. Figure 4.5 shows the resulting trajectory. In the realstic case, there are three other ways that may indicate the falseness of the estimated orbits. First, by having a visual mismaching between the extracted orbit and its estimation. Second, by having a near nonhyperbolic periodic point. And third, by having a large violation for the theoretical bound at the recurrence point. Figures 4.6-8 show some extracted orbits that are improved by the estimation method. The improvement in the period-4 estimated orbit, in Figure 4.6, is very high. The period-5 and period-6 extracted orbits are improved at the reccurence point, but the interior error still exists. 4.2 ' Summary In this chapter, the extraction error in the extracted orbits of the tent and the horse- shoe maps was reduced, by the estimation method, to the order of 10‘”. While the error in many improved extracted orbits of the Heneon map was on the order of 10'6. For some periodic points, at which the Henon map exhibits the intermittency phenomenon, the error in the improved orbits was even worse than the error in the extracted orbits. The accuracy of the improved orbits of the Duffing equation of period-l < 5 was great and almost with zero error, but it was mentioned that these estimated orbits might be false and did not represent the original system. Furthermore, the improved extracted orbits of the Duffing equation of period-l Z 5 could not remove the large interior error, even though, the error at their recurrence points were nearly zero. In the next chapter, the effects of the extraction error and its improvement on the nonlinear systems parametric identification are studied. 90 0.8 - 0.6 - - --«--- Theextractedorbit $1 0.... -— Theestimatedorbit Figure 4.4. A false period-2 orbit plotted with its extracted and corrected orbits. 91 1'5 I T T I l T l The estimated orbit The simulated orbit _0.5_ ........................................................................................... _1...........;._ ......... '.......... .......... _15 L I 1 J J 1 t -2 -1 .5 -1 -O.5 0 0.5 1 1 .5 2 Figure 4.5. The false period-2 orbit plotted with its simulated orbit. 92 ~08? ............ “..’., .............................. '-'-" Theextractedorbit 3 —— The estimated orbit 3 The corrected orbit _1 l r 1 4 -1 .5 —1 -0.5 0 0.5 1 1 .5 Figure 4.6. A period-4 estimated orbit plotted with its extracted and corrected orbits. 93 1'5 I l I l l l h ' - ' .- ° ' The extracted orbit -—- The estimated orbit . . : 3 The corrected orbit _1.5 l l 1 l . -2 -1 .5 -1 -0.5 0 0.5 1 1 .5 2 93 1 Figure 4.7. A period-5 estimated orbit plotted with its extracted and corrected orbits. '94 -\ 5 o.a- ...... /§ 0.6 .. .......................................... 04» I ''''' - Theextractedorbit ' / The estimated orbit —- The corrected orbit 1.5 Figure 4.8. A period-6 estimated orbit plotted with its extracted and corrected orbits. 95 CHAPTER 5 NONLINEAR SYSTEMS PARAMETRIC IDENTIFICATION In this chapter, the effects of the extraction error and its improvement on the identified parameters of nonlinear systems are studied. First, a theoretical background about the identification methods that are used in maps and flows is represented. After that, these methods are applied in order to identify the parameters of the tent map, horseshoe map, Henon map and Duffing equation. In the identification of the Duffing equation parameters, the validity of using the residual as a measure for the real error in the identified parameters is checked. This enables us to' know the accuracy of using the residual as a measure of the identification error when the real parameters are unknown. Finally, the effects of noise and reconstructing the phase space on the extraction and estimation results are studied. First, the effect of having only noisy time series is discussed. Also, the effect of only reconstructing the phase space from a clean one dimensional observed signal is discussed, too. In the end, the extraction process and 96 the estimation method are applied on a time series that is reconstructed from only a single contaminated observed signal. 5.1 Theoretical Background From the shape of the extracted periodic orbits, it is easy to know whether their time series is produced by iterating a map or solving a vector field numerically. If the time series has a period-1 orbit that contains only one point, then it is produced from maps. In this case, all the higher period-k orbits of the time series will be just a group of points that jump around the period-1 point. This jumping motion obeys the period-1 point expansion and contraction factors, as in Figure 4.1. This is applicable to Poincare maps, too. On the other hand, if the time series is produced experimentally from a continuous time physical system or numerically from a differential equation, then the extracted periodic orbits will represent sampled continuous differentiable functions in the phase space. In this case, the periodic orbit with minimum number of points is a period-1 orbit. Then, all the higher period-l orbits will have number of points, m(l) = (m(1). 5.1.1 Identifying parameters in maps In the case of maps, it is possible to use the least-squares estimation method to identify the parameters of the corresponding map for that time series [58—61]. Here, the model is not taken as a linear or affine map. Instead, it is a polynomial model that has, in case of a two dimensional system, the form 2 $i+1 011 012 10.- b1 Cu 612 37; d1 = + xiyi+ + (5.1) yet-1 .021 022 y? b: 621 C22 yt’ d2 97 Changing the notation of the variables from their previous representation in Chap- ter 3, is necessary. In Chapter 3, the variables .1: and y represented points in the plane, while now they represent only the :c and y components of these points. This is needed in order to have a representation for the my term in the nonlinear model. The only difference, beside this change in the notation of the variables, is in the form of the nonlinear sensitivity matrix, Y, and its corresponding estimated para- meters matrix, fl. For the general formula in Equation 5.1, the nonlinear sensitivity matrix, Y, has the form Y = [xi 3!: $191 $1 311 1l(nx6) ‘ (53-?) while the transpose of the estimated parameters matrix, flT, has the form ,67‘ = [AT b CT d](2x6, (5.3) which means that fl is a (6 x 2) matrix. It combines all the matrices that are needed for the nonlinear general formula in Equation 5.1. This matrix is estimated by using the normal equation, which has the form fl = (YTYYIYT [x2 yzl (5-4) where 1:2 and y; are the a: and y components of the iteration of the neighboring points that have 2:1 and y1 as their a: and y components. In Equation 5.4, it is better to use the pseudo—inverse of Y instead of the matrix (YTY)‘1YT. This is important because in case of having a poorly conditioned matrix, YTY, the results will not be acceptable. In Matlab, the command “pinv” produces 98 the required matrix. The equation will have the form fl = Yl [502 ml (5-5) where 1 indicates a pseudo-inverse. 5.1.2 Identifying parameters in flows If the time series is produced experimentally from a physical system modeled by an ordinary differential equation or numerically from an ordinary differential equation, the harmonic balancing method will be used to identify the parameters of that or- dinary differential equation [33,34,40]. In this method, the terms of the ordinary differential equation are replaced by their truncated Fourier series. The model that will be assumed for the unknown differential equation is 3 . it} + (12': + 2,8,3? 2 7 cos(wt) (5.6) i=0 The zit-component value of the periodic orbits is the only value that will be used to calculate the returning terms of all of these truncated Fourier series. The equations of thetruncated Fourier series for x, :i: and :i} are = 3229.4. +;( a, cos (zwt) )+ b- sin(iwt)) (5.7) Then i: = Z(—iwa; sin(iwt) + iwb; cos(z'wt)) (5.8) i=1 And 5: = Z(—i2w2a, cos(iwt) — izwzbgsin(iwt)) (5.9) 5:1 The terms $2 and 1:3 are calculated by calculating the representative time series first, 99 then they will have their own equations as 3:2 = %+ :(c, cos(iwt) + d, sin(iwt)) (5.10) x3 = 329+ i:(e.-cos(iwt) + f,- sin(iwt)) (5.11) When these Fourier series are implemented in Equation 5.5, and the same harmonic terms are equated, the resulted algebraic equation has the form - 0 0 1 co co co. .0. —w2a1 wbl 0 al C; e; '7 —w2b1 —wa1 0 b, d, fl , 1 ‘ a < 3° >.—. ' (5.12) 31 32 L 33 J —(nw)2a,, nwbn 0 an cn en 0 _—(nw)2b,. —nwa,. 0 bn dn fn‘ .0. where the known quantities in this equation are the Fourier coefficients. They have the equations a,- =% f” :r(t)cos(iwt)dt T (5.13) b,- =% a” m(t)sin(iwt)dt c.- = % f+T 1:2(t) cos(iwt) (ii (5.14) d,- = %ff+T 32(t)sin(iwt)dt 100 e; = % f” x3(t)cos(iwt) dt (5 15) f, = -% f” 23(t) sin(iwt)dt By putting all the unknowns in the left side column, the equation has the form 0 0 1 (10 co co 0 —1 wbl 0 al c1 e1 wzal r ‘ 0 —wal 0 ()1 (11 f1 ’7 £0sz a ,6 . < o r: (5.16) [31 flz tfia J 0 nwbn 0 an en en (nw)2a,, _ 0 -nwa,. 0 bn dn fn‘ . (nw)2b,, ‘ which has the general formula Afl=b (5.17) This equation has a residual vector r = Afl — b (5.18) The minimum value of this vector will be obtained by using the least-squares method. In this method, the normal equation for the parameter vector, fl, is fl = (ATA)"AT b (5.19) 101 Here again, it is better to use the pseudo-inverse of matrix A, then s = Al b (5.20) 5.2 Results and Error Analysis In this section, the results of identifying the parameters of all the studied systems are presented. As it was mentioned above, for maps, the parameters will be identified by using the least-squares estimation method, while for flows, the harmonic balancing method will be applied on the extracted and estimated periodic orbits. Here, the original clean signals are studied. Studying the effects of noise and state space reconstruction will be in the next section. First, the results of identifying the parameters of the three maps will be presented. After that, the results of identifying the parameters of the Duffing equation will be presented, too. 5.2.1 Maps In maps, the cases that are covered in this section are different in 1. The way of using the estimation method. 2. The reference point around which the estimation is applied 3. The distance, 5,, which defines the neighborhood of the reference point 4. The length of the time series 5. The amount of prior knowledge about the identified system There are two different ways that are used to identify the parameters of the studied maps. Both of these ways use the least-squares method. The first way uses the time series to estimate the quadratic, or higher, model around a reference point on 102 Table 5.1. The identification results for the different cases of the tent map. Case No. a b c 1 0 2 0 2 0 2 -l 3 .34 .02 -.02 the phase space. The second way uses only the unstable periodic orbits that are extracted from the time series. In the second way, the effect of improving these extracted periodic orbits will be studied, too. Next, the results of the mentioned cases are presented, for all the discussed maps. The tent map The tent map is a one dimensional map. So that, the general quadratic form is 32,-“ = ax? + bx,- + c (5.21) This map has two different functions that represent it. Having two extracted period-1 points is an indicator of the existence of discontinuity in the function by which the time series is produced. However, this discontinuity can be characterized easily from the identified parameters that are obtained from the tent map time series. In the case of the tent map, the identified parameters have three different results depending on the selection of the reference point. These results are listed in Table 5.1. They represent the cases where the reference point is chosen to be 1. The first extracted period-1 point that is close to :1: = 0 2. The second extracted period-1 point that is close to a: = 1 3. The discontinuity point, a: = 0.5 103 It is clear from the identification results, in Table 5.1, that the first two cases are representing the true functions of the original system. While the third one is representing the situation where the neighboring points of the reference point are treated differently. This is in agreement with the fact that the first function is applied on some of these points while the second function is applied on others. As mentioned above, having two extracted period-1 points is the reason behind the early judgement that the time series is representing two differents. But, identifying the system around many points on its phase space will identify every function and determine the region where it is applied. The choice of the distance, 6,, which defines the neighborhood of the reference point has no effect on the estimation results as long as the neighborhood does not contain the discontinuity point, x = 0.5. On the other hand, adding a quadratic term to the estimated model, does not effect the identification results. There is no way to use the extracted orbits in the estimation method. This is because the identification results will not be acceptable since for any periodic orbit of period-k > 1, there are points that are in the first function domain while others are in the second function domain. The horseshoe map As in case of the tent map, this map has two different functions that represent it. It also have two extracted period-1 points that indicate the existence of discontinuity in the function by which the time series is produced. Furthermore, the identified parameters have also three different results depending on the selection of the reference point. These results are listed in Table 5.2. They are representing the cases where the reference point is chosen to be 1. The first extracted period-1 point that is close to (0,0) 104 Table 5.2. The identification results for the different cases of the horseshoe map. Case NO- 011 012 021 022 b1 52 011 612 C21 022 d1 d2 1 0 0 0 0 0 0 1/3 0 0 2 0 0 2 0 0 0 0 0 0 1/3 0 0 2 2/3 -1 3 .34 1.2 0 .14 -.01 .02 0 .02 0 .01 0 -.02 2. The second extracted period-1 point that is close to (1,1) 3. The point (.5,.5) that is on the discontinuity line, y = 0.5 It is clear from the identification results, in Table5.2, that the first two cases are representing the true functions of the original system. While the third one is representing the situation where the neighboring points of the reference point are treated differently. Again, having two extracted period-1 points is the reason behind the early judge- ment that the time series is representing two functions. But, identifying the system around many points on its phase space will identify every function and determine the region where it is applied. And the choice of the distance, 6,, which defines the neighborhood of the reference point has no effect on the estimation results as long as the neighborhood does not intersect the discontinuity line, y = 0.5. Also, adding a quadratic and zy terms, to the estimated model, does not effect the identification results. As in case of the tent map, there is no way to use the extracted orbits in the estimation method and for the same reason mentioned there. Henon map Henon map has only one extracted period-1 point. It, also, has only one continuous function that produces its time series. The identified parameters are not effected by 105 Table 5.3. The identification results for the Henon map. 011 012 021 022 bl b2 Cu 012 021 622 d1 d2 -1.400000010.3010 the position of the reference point or the size of its neighborhood. For any reference point and distance 6,, the identified parameters have the values that are listed in Table 5.3. The only condition that has to be satisfied is to have at least ‘5; = 6 neighboring points around the reference point, where p = 12 is the number of the identified parameters and m is the dimension of the estimated model. When the extracted periodic orbits are used in the estimation method, the same results are obtained. 5.2.2 Flows To identify the parameters in flows, orbits of different periods are used in the harmonic balancing method. The use of the periodic orbits is necessary here. This is because the truncated Fourier series is used as an approximation for the differential equation terms. If the segment of the time series that is used in the estimation process is not a periodic orbit, then its approximation by a small number of the truncated Fourier series harmonics will not represent it accurately. Furthermore, using a large number of harmonics will lead to an ill-conditioned matrix, which is another numerical problem. The cases that are studied in this section are different in 1. The number of the unstable periodic orbits that are used 2. The period-l of these unstable periodic orbits 3. The amount of error in the recurrence point 4. The number of harmonics, n, in the truncated Fourier series 106 5. The stability of the extracted periodic orbits 6. The amount of prior knowledge about the identified system 7. The harmonics that are used in the truncated Fourier series 8. The length of the time series The effect of having an incorrect model, or an approximate representation of a model, was investigated in examples by Yuan [63], and also in an experiment [64]. The improved orbits are used in the estimation method, in order to quantify the effect of using the extracted orbits and their improvement on the accuracy of the identified parameters. Also, the variation in the Fourier series coefficients due to the use of these different orbits is shown, in order to compare it to the variation in the identified parameters. This provides us with another measure, beside the residual, for the identification error when the real parameters are unknown. Next, the harmonic balancing method is applied to identify the parameters of different cases that include all the differences mentioned above. The time series is produced numerically by integrating the Duffing equation that was studied in the previous chapters. Duffing equation As it was mentioned in Chapter 2, the true values for the parameters of the estimation model, Equation 5.6, are a = 0.15,,30 = 6; = 0,31 = -1,fla = l and 7 = 0.3. Inte- grating the differential equation with an initial condition such as (1:0, 5:0, to)=(l,0.15,0) and a time step (At = g—g = 0.2513274) will produce a chaotic time series that has many unstable periodic orbits. The number of points in their period-l orbits is 25. The extracted periodic orbits that are used in the identification process have periods up to period-l = 11. The number of the distinct extracted orbits, 72.1,, for 107 Table 5.4. The number of distinct extracted orbits, ndc, for the Duffing equation. The period-l 10 11 N o. of orbits 14 14 16 12 a time series of length N = 58000 and c = 0.35 x At = 0.0879646 is shown in Table 5.4 for every period-l. This number of orbits depends on the initial conditions, the selection of distance 6 and the length of the time series. In the first case, extracted orbits of periods I S 6 that have large extraction errors at their recurrence points are used to identify the parameters of Duffing equation. The number of these extracted orbits is eighteen. Figure 5.1 shows the corresponding plots of the average real parameter error, 6, and the average residual, F, for different values of the number of harmonics, n, in the Fourier series. The average residual value, 1", is the average of the absolute values of the elements in the residual vector, 1', in Equation 5.18, while the average real parameter error, E, has the form a = '= (5.22) where e; is the absolute value of the error in the 1"" identified parameter. In Figure 5.1, the average real parameter error has relatively a large value for small n that is .003. This is because the Fourier series is not able to match the original function with one or two harmonics, due to the large error at the recurrence point. The error has a steady value z .002 for the range of n = 3, ..., 8. When extracted orbits of periods 7 S l S 9 that have large extraction errors at their recurrence points are used, the results will be better for the values of n up to 4, with minimum value .0018 for n = 1, but worse after that, as it is shown in Figure 5.2. The number of these extracted orbits is kept the same as the number of the lower 108 0.016 I I —— The average residual -— - -‘- - - The averag real error 0.01 4 0.012 0.01 0.008 0.006 0.004 0.002 Number of Harmonics, it Figure 5.1. The average real parameters error, é, and the average residual, 1", for different values for the number of harmonics, n, in the Fourier series using orbits of lower periods and large extraction errors. periods orbits. If all of the previous extracted orbits of periods 1 S l S 9 that have large extraction errors at their recurrence points are used, the results will be effected by the lower periods and will have a large error for n = 1. After that, the error will be smaller than the previous two errors for the range of n = 2, .., 5, before getting worse than the lower periods error for the remaining values of 12. Figure 5.3 shows the corresponding plots of the average real parameters error, E, and the average residual, 1", for different values of the number of harmonics, n, in the Fourier series. 109 0.015 I 0.01 0.005 0 J'— i o 5 1o 15 Number of Harmonics, n Figure 5.2. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using orbits of higher periods and large extraction errors. 110 0.015 , —— The average residual 0.01 0.005 o 5 1o 15 Number of Harmonics, n Figure 5.3. The average real parameters error, E, and the average residual, 7", for different values for the number of harmonics, n, in the Fourier series using orbits of mixed periods and large extraction errors. 111 Table 5.5. The identified parameters of the Duffing equation that are associated with the minimum average residual value in Figure 5.6. a 30 51 52 53 7 0.1479 -0.0001 -0.9984 0.0003 0.9979 0.2994 In the second case, extracted orbits of periods 2 S l S 6 that have small extrac- tion errors at their recurrence points are used to identify the parameters of Duffing equation. The number of these extracted orbits is eighteen. Figure 5.4 shows the corresponding plots of the average real parameter error, E, and the average residual, F, for different values of the number of harmonics, n, in the Fourier series. For the orbits with lower periods, the real error has reduced its large value for small n to 0.0014. This is because the Fourier series is able to match the original function with one harmonic. The error has a steady value x 0.0012 for the range of n = 3, ..., 17. When extracted orbits of periods 7 S l _<_ 9 that have small extraction errors at their recurrence points are used, the results will be better for n = 1, with a minimum value .00087, but worse after that, as it is shown in Figure 5.5. The number of these extracted orbits is kept the same as the number of the lower periods orbits. If all the previous extracted orbits of periods 2 S I _<_ 9 that have small extraction errors at their recurrence points are used, the results will have the minimum value of error with a steady value that is 0.00087. Figure 5.6 shows the corresponding plots of this case. The identified parameters of the Duffing equation that are associated with the minimum average residual value in Figure 5.6, are shown in Table 5.5. 112 0.03 0.025 0.02 0.015 0.01 0.005 o 2 4 s s 10 12 14 1s 1s 20 Number of Harmonics, it Figure 5.4. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using orbits of lower periods and small extraction errors. 113 0.010 0.01 4 0.012 0.01 0.008 0.006 0.004 0.002 Number of Harmonics, 11 Figure 5. 5. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using orbits of higher periods and small extraction errors. 114 0.025 r r I 1 I —— Theaveraoeresidual I I I I I ---- - Theaverag realerror 0.02 0.015 0.01 0.005 0 0 2 4 6 8 10 12 14 16 18 20 Number of Harmonics, 71 Figure 5.6. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using orbits of mixed periods and small extraction errors. 115 In the third case, nine extracted orbits of periods I = 1,1,2,2,3,3,3,4 and 6, which have mixed recurrence errors are used to identify the parameters of Duffing equation. Figure 5.7 shows the corresponding plots of the average real parameter error, E, and the average residual, F, for different values of the number of harmonics, n, in the Fourier series. Using the improved orbits of the extracted orbits that produced Figure 5.7 leads to the result shown in Figure 5.8. It is clear that the average real parameter error has a steady value that is 0.0021. If the period-6 orbit is removed the average real parameter error will increase by 0.0001. See Figures 5.9 and 5.10. If the orbits of periods I = 3 and 4 are removed, too, the error will be worse. It has a steady value of 0.0037 in case of using the improved orbits of periods 1 and 2, as shown in Figures 5.11 and 5.12. Table 5.6 shows the variation of Fourier series coefficients for two extracted orbits of periods 1 and 2 due to the improvement in their estimated orbits. It is clear that the percentages of variations for higher harmonics are much greater than those for lower ones. This is because the higher harmonics have very small amplitudes. Therefore, it is better to avoid using high harmonics. 116 0.” 0.025 0.02 0.015 0.01 0.005 o 5 1o 15 Number of Harmonics, 11 Figure 5.7. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using nine extracted orbits of periods I _<_ 6. 117 j 10 15 Number of Harmonics, n Figure 5.8. The average real parameters error, E, and the average residual, F, for dif- ferent values for the number of harmonics, n, in the Fourier series using the estimated orbits of the extracted orbits that are used in Figure 5.7. 118 0.03 0.025 0.02 0.015 0.01 0.005 o 5 1o . 15 Number of Harmonics, 11 Figure 5.9. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using eight extracted orbits of periods I S 4. 119 1.6 l l o 5 1o 15 Number of Harmonics, 11 Figure 5.10. The average real parameters error, E, and the average residual, F, for dif- ferent values for the number of harmonics, n, in the Fourier series using the estimated orbits of the extracted orbits that are used in Figure 5.9. 120 0.00 1 —— The average residual : ----- - The averag real error 0.05 0.04 0.03 0.02 0.01 Number of Harmonics, 11 Figure 5.11. The average real parameters error, E, and the average residual, F, for dif- ferent values for the number of harmonics, n, in the Fourier series using four extracted orbits of periods I S 2. 121 7.5 1 — Theaverageresidual ------ Theaveragrealerror 5 1o 15 Number of Harmonics, 11 Figure 5.12. The average real parameters error, E, and the average residual, F, for dif- ferent values for the number of harmonics, n, in the Fourier series using the estimated orbits of the extracted orbits that are used in Figure 5.11. 122 Table 5.6. The variation of Fourier series coefficients for two extracted orbits of periods 1 and 2 due to the improvement in their estimated orbits. Term Ex. P-l Es. P-l Ex. P-2 Es. P-2 a0 .5864 .5852 .6409 .631 a1 -.58 -.5871 -.1787 -.169 ()1 .234 .2016 .0891 .0846 a; .0758 .0783 -.0236 -.0306 02 -.081 -.0735 .034 .0409 a3 -.0021 -.0107 .0004 -.0009 b3 .0217 .0205 -.0024 -.0061 (1., .0048 .0003 -.004 -.0001 54 -.009 -.0049 .0006 -.001 a5 .0043 .0002 .004 .0004 b5 .0024 .0007 -.001 .0003 In the fourth case, the using of stable periodic orbits to identify the parameters of the Duffing equation is presented. In fact, only a period-1 stable orbit that has the plot that is shown in Figure 5.13 is used to identify the parameters of the general estimation model in Equation 5.6. It turns that using a stable period-1 orbit to identify the parameters of the Duffing equation gives good identification results. But this requires at least three harmonics in the Fourier series approximation. This is required in order to have the number of equations, ne 2 p, where p is the number of unknowns in Equation 5.16. Figure 5.14 shows the corresponding plots of the average real parameter error, F, and the average residual, F, for different values of the number of harmonics, n, in the Fourier series. The average real parameter error has a steady value 0.0004 and for a large range of n 2 3. 123 1-5 ! F f Y I ' r a . . . e I s o o o s a e s e . n o u 05- ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss - . e s c . u e n n n . u a e u o a o e e s s _0.5._ ................................................................................................. .- _1........... .5 ............ ........ .......... . -1.5 J I I I I I .L -2 -1.5 -1 -0.5 0 0.5 1 1 .5 2 Figure 5.13. The stable period-1 orbit that is used to identify the parameters of the Duffing equation. 124 1 I. : g —— The average residual 1 ' ------ Theaverag realerror 1 2 a 4 5 s 7 s 9 10 Number of Harmonics, 11 Figure 5.14. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series using a stable ’ period-1 orbit. 125 In the fifth case, the effect of adding more terms to the estimation model on the identification results is shown. Also, the effect of removing the third degree term from the polynomial series in Equation 5.6 is shown, too. If the estimation model has the polynomial function up to degree 5 instead of 3 and has the sine forcing term beside the existing cosine term, the identification results will have a larger error. By using all of the extracted orbits that have small error in their recurrence points leading to the identified parameters that produced Figure 5.6, the error becomes larger but with a reasonable steady value that is less than .004, as shown in Figure 5.15. While if the estimation model has polynomial terms up to degree 2 instead of 3, the identification results will have a very large average residual. By using all of the extracted orbits that have small error in their recurrence points leading to the identified parameters that produced Figure 5.6, the average residual becomes larger and unable to maintain the low values that were maintained with the high models. See Figure 5.16. If the period-l stable orbit is used to identify the parameters of the higher degree model, the average real parameter error and residual have the plots that are shown in Figure 5.17. It is clear that the one orbit is able to identify the parameters of the higher degree estimation model. But again, this requires at least 5 harmonics in the Fourier series. However, if the same stable orbit is used to identify the parameters of the lower degree model, the average residual is shown in Figure 5.18. It is clear that the average residual is unable to maintain the low values that are maintained with the higher degree models. This is because the estimation model is not sufficient, and higher degree terms are needed. 126 0.18 I l I I I -—-—’ Themnnmenmwum g f f f f ------ Theawmupnmmenm" I I I 3 1 < e e e n e s . s e e IT‘II‘IOVIVISI'VIInI—I—‘I’TITU—I'Tif llllllll i—l' ........................................................... a e n s u I e I Q o e - v u e e e . a a e e c o s e s l O 0 s u s e u o I . I e u o . e u e o e e e a a e n I s ' . o e 0 e o 0.1“. ........ ......... ........ ........ .......... .......... .......... .......... ........ ." . O I O U I . C 0 I l D . O C 0127' .................................................................................................. .. I O I I ' I 8 O I C O I I l O I O I 0 O I I U I I O . O1- OOOOOOOO .0 IIIIIIIII . IIIIIIIIII I OOOOOOOOOO . IIIIIIIIII . IIIIIIIIII . IIIIIIIIII . IIIIIIIIII . IIIIIIIIII . ........ - ° . . . . . . . . . I O 0 t I I . I 0, g I . I I I I . - 0 0 008'“ .................. - .......... - .......... - .......... - .......... - .......... - ......................... 1.... O C O I ‘ I . I l O I U I C ' e e n a - I u e o omh ................................................................................................ ' ----1 . 0 I I C I O l O I I O I 0 n I h C . . n C O C o . . . u a u a b s u . a - s e e e u u o u e I a n a . om- oooooooo .0 ......... s eeeeeeeeee . gggggggggg . uuuuuuuuuu ~ .......... . nnnnnnnnnn . eeeeeeeeee ‘ eeeeeeeeee ~ cccccccc - . s e e e n s s e a I s e a s v I o s s . . . . e o o s e . ' ’ ‘3' i ' 1 o 2 4 e s 10 12 14 1s 1s 20 Number of Harmonics, 11 Figure 5.15. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series with using the same extracted orbits that produced Figure 5. 6, but to identify the parameters of a higher degree model. 127 0-11 I I 1 —T I I I I The average residual 1 2 3 4 5 6 7 8 9 10 Number of Harmonics, 11 Figure 5.16. The average residual, F, for different values for the number of harmonics, n, in the Fourier series with using the same extracted orbits that produced Figure 5.6, but to identify the parameters of a lower degree model. 128 x10 f I I I 4.5 “weavemoeresldual .......................................... .. --~--- Theaveragerealerror I 4... .......................................... ! ..................................................... ..l l 3.5,. ......... , .......... l ..................................................... 1 : : I l l l 3.. .......................................... i ..................................................... .1 I . . . I . e . . 251.l ........... ........... ........... ......... J I 1 I I l I I I l 2b .......................................... i ..................................................... q l 15- .......... l ..................................................... q : : : | l l 1.. ..................................................... _ l I . . . l 05.. .......... !,._._..._._...._..,_._.,._._.,_....E...._,,,_., ..................... : : : : ‘- ;;;;; o ........; ................................................................ .. l J L l l l l l 1 2 3 4 5 6 7 8 9 Number of Harmonics, 11 Figure 5.17. The average real parameters error, E, and the average residual, F, for different values for the number of harmonics, n, in the Fourier series with using a stable period-1 orbit that produced Figure 5.14, but to identify the parameters of a higher degree model. 129 O I: .‘ —1 q i The average resldual 0.2 0.15 0.1 0.05 o i i j 1 1 1 1 1 2 3 4 5 8 7 8 9 10 Number of Harmonics, 11 Figure 5.18. The average residual, F, for different values for the number of harmonics, n, in the Fourier series with using a stable period-1 orbit that produced Figure 5.14, but to identify the parameters of a lower degree model. 130 In the sixth case, the possibility of improving the identified parameters by the selective reduction, in which the orbit that has the largest residual in its harmonic- balance equations is removed first, is studied. The extracted orbits that have small error in their recurrence points leading to the identified parameters that produced Figure 5.6 are the subject of this process. Figure 5.19 shows the corresponding plots of the average real parameter error, E, and the average residual, F, for different numbers of the removed orbits. It is clear that reducing the number of orbits by removing the orbit that has the largest residual, in every step, will guarantee the reduction in the average residual with every step, while the average real parameter error will not be consistent. However, it is good to choose the number of terms of which the average residual has its greatest absolute slope. The last subject that is studied in the identification of the Duffing equation para- meters is the effects of using frequencies and phase angles that are different from the true ones, on the accuracy of the identified parameters. Using w = r x 1.11,, where w is the identification frequency and w, is the true frequency, in the identification process makes the identified damping a '= r x 01,, while for the other parameters, the identified values are the true values multiplied by r”. This is because the column in the right hand side of Equation 5.16 is multiplied by 1'2 due to the change in 1.0. Also the column in the left hand side matrix that corresponds to :1': coefficient, a, is multiplied by r. To keep the balance of Equation 5.16, a is multiplied by r again, while the rest of parameters are multiplied by 1'2 in order to accommodate the change in the frequency. If the phase angle, <13, is unknown using only the cosine forcing term is not enough to identify the real parameters. This is because the values of these parameters depend on the value of 45. Hence, for unknown initial time of a time series, the sine and cosine forcing terms will be used together to make the values of the real parameters 131 Theaverageresldual g 3 f g l ----- - Theaveragrealerror ; g g g ! 1.8 : Z ......... . ............. ............... ............. -..... e e I I I 0 I l O l e I D e 1.6" ............ . .............. , ............................ . .......................................... ,. e I c I e I a 1.4,... ...... . ..... ............. . .............. .. . ......... ..T I o a l O l O 1.22 ............ .............. ............. .......... .............. ............. ............. :..1 0.8-...4'....\...:. ............. J,\” ..... ‘.....L.+ 0.6,...u' ....... 3 ..... . I ........ ............. .............. \l ............. '..- .\. 0.4.. ............ g .............. g ............ j ........... g .............. g ...... ' ....... g ............. g..- I O Q 0 I C O o I . u I I . . . n e I . 0.2- eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee d e . . - I c I I I I I s I O I I C . C Number of Harmonics, 11 Figure 5.19. The average real parameters error, E, and the average residual, F, for different numbers of the removed orbits. different phase angles except for the coefficients of the two forcing terms. Hence, for unknown initial time of a time series, the use of sine and cosine forcing terms will keep the values of the system parameters from being changed. 132 space from a clean single observed signal will be discussed, too. In the end, the performance of the extraction process and also the estimation methods to be applied on a time series that is reconstructed from a single contaminated observed signal will be categorized. 5.3.1 Effect of noise To study the effect of noise on the accuracy of the extraction and estimation results, the time series of all the undergoing maps are contaminated, prior to extraction, by 1% and 0.1% of the noise levels. This kind of noise represents the sensor noise, and does not influence the generation of data. Figures 5.28 and 5.29 show the horseshoe and the Henon map that are contaminated by a 1% noise level. Next, the results of the extraction process and the estimation method, in maps and flows, are discussed. 1 . Maps For all of the maps, the extracted orbits are effected by noise to the extent that they have an error that is on the order of the noise itself. Furthermore, some unrelated orbits are extracted incorrectly. The unrelated extracted orbits are those orbits that are extracted for periods not equal to their true periods. For example, the Henon map has only one periodic orbit of period-2. An extra orbit, which is extracted for period-2 and has a large extraction error, is not really a period-2 orbit. To calculate its extraction error, it has to be compared to the true values of the cycle points for which it is extracted, even if they are part of an orbit of a different period. 133 1.2 1.2 0.8-............ 0.3-............ ‘l 0.4-............ -O.2 -0.2 ed by a 1% noise level. 1: amina Figure 5.21. The horseshoe map that is cont 134 0.5 Figure 5.22. The Henon map that is contaminated by a 1% noise level. 135 1.5 In all of the three maps, most of the unrelated orbits have true periods that are less than the periods for which they are extracted. They are not extracted for their true periods because they, due to noise, have large recurrences. By increasing the distance, 6, which is used to define the recurrence in the extraction process, these unrelated extracted orbits will be extracted for their true periods. But other unrelated orbits that have periods higher than the periods for which they are extracted will be extracted incorrectly. When the noise level is reduced to 0.1%, the extraction errors are at that level, too. Concerning the periodic orbit estimation results, the method failed to improve many of the extracted orbits for all of the three maps. In case of the tent and horseshoe maps, the extracted periodic points that are close to the discontinuity region are more difficult to be improved. For the rest of points, using a shell neighborhood around the extracted point that includes the points 9:.- with distances r1 < r.- < 1'2, with a small radius, r1, that is larger than the noise level improves the results of estimation. For the Henon map, using a distance, 6. = 20 C, where C is the noise level, to define the neighboring points that are used in the estimation process, gives the best results. There is a trade off between increasing the distance, 6,, in order to reduce the effect of noise and decreasing it to reduce the effect of nonlinearity. Using a shell neighborhood with a small radius that is larger than the noise level does not improve the estimation results for the Henon map. In fact, if the small radius is greater than 0.02, the result is much worse. This is because the effect of nonlinearity is high. 136 Table 5.7. The identification results for the Henon map by using the extracted orbits, with 0.1% noise. 011 012 021 022 b1 b2 611 012 021 622 d1 d'2 -l.392 -.059 .001 0 .007 .002 -.009 .988 .297 -.006 1.005 -.001 Table 5.8. The identification results for the Henon map by using the points of the time series, with 0.1% noise. 611 012 C21 022 d1 d2 0 1.001 .299 0 1 0 011 012 021 022 bl -1.4 -.004 0 0 .001 CS‘ The effect of noise on the accuracy of the identified parameters for the three maps is approximately on the same order of noise. Table 5.7 shows the noise effect on the parameters of Henon map that are identified by using only the nineteen periodic points of the extracted orbits of periods I S 6. If all the time series is used, the results are better, as shown in Table 5.8. This is because there are more points in the second method that reduce the effect of noise more. 137 Table 5.9. The identification results for the Henon map by using the improved ex- tracted orbits, with 0.1% noise. an 012 021 022 bl bz Cu 012 621 022 d1 612 -1.4 -.001 0 .003 .001 0 0 .999 .3 0 l 0 Table 5.10. The identification results for the Henon map by using the extracted orbits, with 1% noise. an 012 021 022 bl 52 Cu 612 C21 022 d1 d2 -l.405 -.265 -.009 -.186 -.011 .005 .005 1.036 0.317 .056 1.003 0 It is clear from comparing the identified parameters of noisy time series to the parameters in Table 5.3, that are identified from a clean signal, that the effect of noise is not destructive. Furthermore, the results can be improved by using the improved extracted orbits. Table 5.9 shows that improvement of the parameters that are identified by using the improved extracted orbits. Tables 5.10-12 illustrate the effects of the noise and the improvement of the ex- tracted orbits in case of having a 1% noise level. 138 Table 5.11. The identification results for the Henon map by using the points of the time series, with 1% noise. Cu 012 621 C22 ‘11 612 -.005 .995 .299 0 1.005 0 011 012 021 022 bl -1.398 -.079 0 -.015 -.002 OS‘ Table 5.12. The identification results for the Henon map by using the improved extracted orbits, with 1% noise. 011 012 021 022 b1 52 011 612 621 022 611 dz -1.395 -.124 0 -.019 -.004 .001 .013 .983 .299 -.001 .993 .001 2. Flows For the harmonic balance method, the identification results are less sensitive toward the existence of noise. To study the effect of noise on the accuracy of the identified parameters the time series is contaminated by a 1% noise level. Figure 5.30 shows a period-2 extracted orbit that is extracted from this contaminated time series. Table 5.13 shows the effect of the 1% noise level on the identified parameters of Equation 5.6. These parameters are identified by using the small error extracted periodic orbits that are used to produce Figure 5.6 and have the identified parameters that are listed in Table 5.5, in case of clean signal. One of the period-3 extracted orbits, in case of clean signal, is not extracted here, due to noise. Table 5.14 shows the identified parameters of the fifth order estimation model that has also, beside the existing cosine forcing term, a sine forcing term, too. 139 °'5 3 Y r . . W. I ,4. ........... ............ ............ ............ ........... g ........... ........... . 0,3,. ........... .......... ............ ............ .......... g ........... g .......... - ,2- ........... ........... ............ ............ ........... ............ ......... . o,....... ............ ............ ........... ......... .. I I I I . I I I I I I I I I I I - O I I I I I I I I l o .. ......... . ..................................................................................... .. I I I I I I I I I I I I I I I I I I I I I I I I I I I I o I I I I I I I I I I I I I I I I I I I I 4 1 b IIIIIIIIII ' IIIIIIIIIIII I IIIIIIIIIIIIIIIIIIIIIIIII . IIIIIIIIIIIIIIIIIIIIIIIII . IIIIIIIIIIIIIIIIIIIII q . I I I I I I I I I I I I I I I I I I I I I I . . I I I O 0 t - I O I I I l U 0 O O "2 p IIIIIIIIIII . IIIIIIIIII . IIIIIIIIIIIII . ............ . ............ : ........... . IIIIIIIIIII I IIIIIIIIIII - I I D I l I O 0 I D O I . I I I . I I L02 0 02 0.4 0.6 0.8 1 12 1.4 Figure 5.23. A period-2 extracted orbit that is extracted from the contaminated time series. Table 5.13. The identified parameters, for the third order estimation model of the Duffing equation, with a 1% noise level. 0 .30 31 .32 .33 ’7 .1459 0 -.9974 .0002 .9984 .2985 Table 5.14. The identified parameters, for the fifth order estimation model of the Duffing equation, with a 1% noise level. a 30 51 fl: 53 34 .35 7c ’7. .148 -.0021 -1.0021 .0105 1.007 -.0057 -.0035 .2994 .001 140 Table 5.15. The identified parameters that are resulting from using three contami- nated extracted orbits of periods I = 1, 2 and 6. 01 flo 131 32 53 7 0.1480 0.0127 -1.0198 -0.0179 1.0118 0.3026 Table 5.16. The identified parameters that are resulting from using the improved orbits of the three extracted orbits that give the parameters in Table 5.15. 0’ 50 31 ,32 fls ’7 0.1400 0.0010 -0.9983 0.0010 1.0020 0.2959 Table 5.15 shows the identified parameters that are resulting from using three contaminated extracted orbits of periods I = 1, 2 and 6 in the identification process. When the improved orbits of these extracted orbits are used, the identified parameters have the values that are listed in Table 5.16. The average residual value drops from 0.006 to 0.002 and the average real parameters error drops from 0.01 to 0.003. 141 5.3.1 Efl‘ect of phase space reconstruction In this subsection, the effect of reconstructing the phase space from one observed sig- nal is studied. The phase spaces of the horseshoe and Henon maps are reconstructed from the :r-component of their time series. Phase space reconstruction, or method of delays, was introduced in the work of Takens [6] among others. The optimal choice of a time delay, 1', and an embedding dimension, (1., was the subject of many researches through the last fifteen years [7-17]. To determine the proper time delay, 1', and embedding dimension, (1., it is possible to select the values that lead to extracted orbits of different periods whose shapes are similar to the original ones [11]. Having the original phase space helps to determine the best embedding parameters by this method. However, this method is not safe when the original system is unknown. 1 In this thesis, the time delay, 1', that corresponds to the first minimum in the average mutual information plot verses 1' [65], is used. The embedding dimension, (1., is determined by the singular systems analysis [12]. Next, the results of identifying the parameters in maps are presented. The results of identifying the parameters in flows are presented later. 1. Maps For the horseshoe map, the delay reconstruction of the phase space is not guaranteed to produce an embedding since the map is not smooth and has a discontinuity in its function [6]. Trying to reconstruct the phase space will result in a totally unrelated phase space for small values of 1'. If 1' is taken large, the y-coordinate in the recon- structed phase space will be similar to the z-coordinate, and the reconstructed phase space resembled the original one. This is a good indication that the two coordinates 142 are absolutely independent [10], which is true. In the opposite to the horseshoe map case, the Henon map can be reconstructed easily by choosing T = —1. The only difference between the original phase space and the reconstructed one is the factor of 0.3, by which the y-coordinate in the reconstructed phase space needs to be multiplied. There is no method that will suggest a negative value of 1'. And there is no doubt that the knowledge of the system was the reason behind choosing 1' = —1. Taking T = 1 will interchange the a: and y coordinates. This value of 1', which gives the closest shape to the original one, indicates the strong dependence of y on a: [10], which is true again. Figure 5.31 shows the reconstructed and normalized Henon map attractor for 1' = 1 and d, = 2. Furthermore, there is no way to figure out the value of the factor between the two coordinates, even for the easiest case where the y-coordinate depends only on the previous z-coordinate value, as in case of the Henon map. But this problem can be handled, by normalizing the phase space to be in the interval from -1 to 1, in all of the system coordinates. The effect of this normalization on the parameters of Henon map is shown in Table 5.17. The attractor is normalized by dividing the z-component by its absolute maximum value 1.285 while the y-component by its absolute maximum value 0.3854. This means that $.41 and 2:,- are divided by 1.285 and y,“ and y,- are divided by 0.3854. In function 25+] of Henon map, this will effect the coefficient of x? to be multiplied by 1.285, the 0.3854 coefficient of y,- to be multiplied by "1'7235— = 0.3 and the constant term to be divided by 1.285. While in function y,“ the coefficient of 2:,- has to be multiplied by 61.53%, = -%°. The effect of reconstructing and normalizing the phase space on the accuracy of the identified parameters, for the Henon map, is shown in Tables 5.18 and 5.19. In both of them, the phase space is normalized to be in a range from -1 to 1. Table 5.18 shows the identified parameters with using the time delay, 1' = 1, while Table 5.19 143 I I I I I I I I I I I . I I - I I I I I I I o - nnnnnnnn I. ooooooooo . oooooooooo . ................................... ' .......... . oooooooooo . ........ q I . I . I - - I I . . I I I I I I I I I I I I I I I . I I I I I l I I I I I I I I I I I I I I I I I I I I I h .................................................................................... - . I I I I l ‘ I I I I I I I I I I I I I I I I I I I I I I I D I I C O I I I I I I I I I 4 - ........................... .l I I I I I I . .......... . IIIIIIIIII ' lllll I I I I I IIIIIIIIIIIIIIIIIIIIII - . I I I . I I I I J I I . . O I I I ‘ I I C I I I 4' I O I I I I I I I I I I I I I I _‘ I I I I I I I I I I l I I I I I 02 b IIIIIIIIIIIIIIIIIIIIIII I IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII q I I I I I I I I I O I I I I I I I I V I O o I I I I I I I 0 I I I I I I I I I l I I I I I I I I I I I I I I O I I I I I . I O O I I I I I I I I I I I I I I I I I I I I I I 4- IIIIIII .I IIIIIIII . IIIIIIIIII . IIIIIIIIII . IIIIIIIIII _ IIIIIIIIII . IIIIIIIIII - IIIIIIIIII . IIIIIIII I. IIIIIII q . I O I I I I l U I I I I I I I I I I I I I I I I I I I I I I l C I I I I I I I I U I I I I I I I I I p IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII a II.- . I I I I I I I I I I I I I I I I I I I I I I I I I I I I o x. A: 5 A alt-+7- Figure 5.24. The reconstructed and normalized Henon map attractor, for time delay, 7' = 1, and embedding dimension, d6 = 2. Table 5.17. The identification results for the normalized Henon map. 011 012 021 022 bl 5: Cu 012 621 022 d1 d2 -1.7990 0 0 00 0 .310.77840 144 Table 5.18. The identification results for the reconstructed and normalized Henon map,for7'=land (1.2:?- Cu 612 621 022 d1 d2 0 1 0.3000 0 0 0.7784 011 012 €121 €122 b1 0 0 0 -1.7990 0 OS‘ Table 5.19. The identification results for the reconstructed and normalized Henon map,for'r=—1 and dc=2 . an 012 021 022 b1 52 Cu 012 021 622 d1 d2 -1. 7990 0 0 0 0 0 0 0.3000 1 0 0. 7784 0 shows the identified parameters with using the time delay, T = —1. It is clear from comparing Table 5.18 to Table 5.17 that using 1' = 1 interchanges the two coordinates. While using 1' = —1 does not effect the parameters, as it is clear from the similarity of Tables 5.17 and 5.19. Using any other value of 'r leads to parameters that are not related to the original ones. 145 Table 5.20. The identified parameters for the Duffing equation based on the recon- structed orbits of periods I S 9 that are extracted for T = 6 and cl. = 3. [33 1.0015 [30 00005 fly. -00003 A 10033 0.1503 0.3005 Table 5.21. The identified parameters for the Duffing equation based on 26 extracted orbits from data that are reconstructed from a clean and normalized signal for 7' = 6 and dc = 3. 51 -l.0019 fig 2.4877 30 0.0001 fig 00019 7.9 -0.0018 7c 0.1919 0.1541 2. Flows Reconstructing the phase space for the Duffing equation is only used in determining the indices of the extracted periodic orbits that are used in the identification method. Using the reconstructed states 27;, 2:2 and 2:3 in the identification method leads to parameters that are not related to the original ones. Table 5.20 shows the identified parameters that result from using the extracted orbits of the reconstructed phase space. The values 7' = 6 and (IC = 3 are used in the reconstruction method. It is clear from comparing Table 5.20 and Table 5.5 that the identified parameters are not affected by the use of the reconstructed phase space in determining the indices Table 5.22. The identified parameters for the Duffing equation based on 26 extracted orbits that are reconstructed from a clean and original signal for 1' = 6 and (1.3 = 3. 30 31 162 fla 7c ’7: 0.1541 0.0002 -l.0019 -0.0012 0.9972 0.3031 -0.0003 146 Table 5.23. The identified parameters for the Duffing equation based on 26 extracted orbits that are reconstructed from a contaminated and normalized signal for 7" = 6 and d6 = 3. a 50 161 162 33 7c 73 0.1522 0.0006 -1.0032 —0.0035 2.539 0.1897 0.0002 of the periodic orbits. This is because the only state that is used in the identification method is the original one. However, if the original signal that is used in the identification method is normal- ized by dividing it by its maximum value, arm: = 1.59, this normalization will affect the identified parameters of the differential equation as follows : 1. The constant term and the coefficients of the cosine and sine forcing terms will be divided by mm". 2. The terms that have a degree, n > 1, will be multiplied by the quantity 19‘“ man: ° Table 5.21 shows the identified parameters of the Duffing equation that result from using 26 extracted orbits from a normalized clean signal. If the orbits of the original signal are used, the identified parameters have the values listed in Table 5.22. In the harmonic balance method, the effects of noise and phase space reconstruc- tion when they are applied coincidentally are the same as their effects when they are applied separately. In other words, they are not sensitive to each other, and their effects on the identified parameters are not really observable. Table 5.23 shows the identified parameters of the Duffing equation that result from using the extracted or- bits of a reconstructed and contaminated phase space. The same orbits that produced Tables 5.21 and 5.22 are used here. 147 5.3.3 Summary In this chapter, the least-squares method was used to identify the parameters of maps. In case of using clean extracted orbits, the extraction error has no effect on the accuracy of the identified parameters. In case of noise, the improvement in the identified parameters due to the improvement in the estimated orbits was shown. However using all the points in the time series gave better results. Furthermore, an indicator of whether the time series of maps was produced from one function or more was described. It was shown that having two period-1 periodic points is a good indication that the time series is a multi-functional time series, i. e. produced by more than one function. This was the case with the tent map and the horseshoe map time series. When the least-squares method was applied to identify the Henon map parame- ters, using a shell neighborhood with small radius that is much larger than the noise level did not improve the identification results. And the best results were obtained with using the value , 6., = 20 C, where C is the noise amplitude, as a distance to define the neighborhood. The effects of normalizing and reconstructing the phase space of maps were pre- I sented. For a normalized time series, the coefficients of the map have to be modified by multiplying them by the mentioned factors, while the reconstruction with using, the delay time, 1' = 1 interchanged :r and y coordinates. For the Duffing equation, the harmonic balance method was used to identify its parameters. It was shown that using bad extracted orbits, i. c. with large extraction error, gave less accurate identified parameters. Increasing the number of the extracted orbits gave more accurate identified parameters, specially if these orbits are good ones, i. e. with small extraction error. Using estimated orbits in the identification method improved the accuracy of the 148 identified parameters and made it steady, or robust, for a wide range of the number of harmonics, n, in the Fourier series. Also, using one stable periodic orbit was enough to maintain a great accuracy for the identified parameters. The effect of reducing the degree of the estimation model on the accuracy of the identified parameters was much worse when some terms in the original system were removed. This can be used as an indication of whether the degree of the estimation model is high enough or not. Using the selective reduction, in which the orbit that has the largest residual in its harmonic balance equations is removed first, to reduce the extracted orbits that were used in the identification method did not guarantee the reduction of the real parameter error. However, using the number of orbits where the average residual had its maximum absolute slope would give the minimum real parameter error. The effect of using w = not, where w; is the true frequency, was studied. The coefficient of :i: has to be multiplied by r, while the remaining parameters have to b multiplied by r2. If the true initial time for the extracted orbits is unknown, the cosine and sine forcing terms have to be used in order to remove the effect of having a phase, 43, that is not equal to the true one. Finally, the effects of noise and the phase space reconstruction on the identified parameters were very limited in the case of using the harmonic balance method. This was not the case when the least-squares method was used to identify the parameters of the Henon map. 149 CHAPTER 6 EXPERIMENTAL ANALYSIS ' In this chapter, an experimental time series that represents a nonlinear two-well potential oscillator is studied. The experimental setup is shown in Figure 6.1. It consists of a stiffened beam buckled by two magnets. The beam was rigidified by two steel bars to make it behave as a single degree system. Thanks to Cusumano and Kimble for providing the time series to be used in this thesis. The tools that were applied on the Duffing numerical time series are applied here, too. However, since the real parameters are unknown, the identification error is only measured by the amount of the average residual, F. First, the phase space is reconstructed from the single observed signal. Then the periodic orbits are extracted. After that, some of the extracted orbits are improved by estimation. The theoretical bound on the extraction error is evaluated by comparing the extracted orbits to their improved ones. In the end, the harmonic balance method that was applied to identify the parameters of the Duffing equation, is applied. 150 lgT O__|I O O .358 Egon—...ngo 2:. .fio 2:me w 1...__..T 0nd. H 86 8801-8111 :ofimfioxm omwm [13335 151 6.1 Extracting and Improving the Periodic Orbits The phase space of the experimental time series is reconstructed by the method of delays, described in the introduction, for a time delay, 1' = 6, and an embedding dimension, d6 = 4. The extraction process, described in Section 2.2, is used to extract the unstable periodic orbits of the reconstructed time series. To determine the number of samples in period-1 orbits, m(1), the number of the obtained recurrences using different values of k = 1,..., 100 is plotted. Figure 6.2 shows that the number of recurrences has nonzero values at k = 25,50, 75 and 100. This leads to the selection of m(1) = 25 samples. The distance 6 = 0.0175 is used to define a recurrence in the normalized phase space. The number of the extracted periodic orbits is reduced by extracting only the orbit with the smallest actual recurrence, ca, for any consecutive orbits. The extracted orbits are reduced to the best nineteen orbits of periods I S 11. The least-squares estimation method, which is used to estimate the local models that represent the system in the neighborhood of its extracted orbits, is explained throughly in Section 3.1, while, using the estimated models to calculate the estimated orbits (improved) is described in Chapter 4. Figures 6.2-5 show the two-dimensional projections for some extracted orbits plotted with their improved ones. The local dynamics around the extracted orbits suggest that the extracted orbits have no large interior errors. Figures 6.6-9 show the local dynamics around all the extracted orbits that are plotted in Figures 6.2-5. The value of the phase index, j = 1, corresponds to the recurrence point for each orbit. As mentioned in Section 3.2.4, there is a conjecture that the large interior error occurs after a long sequence of local saddle dynamics that takes the extracted orbit far from its exact periodic points. Scanning the local dynamics around all the extracted orbits shows that the ex- 152 The Period, lc | o . a . - . . . . . . . . . . n . 25 P ........ l» ............................ ' ................... I .......... i ................... ‘ ........ - . . . . . . . . . . . . . o . . . . . . . 20 — ........ ' .......... ‘ .......... ' .......... ' .......... ' ....... . . .' ...... .. . -‘. . . u? - o - u' .......... ' ........ — . u . u . . . . . 15 i. .................................................... . ....................... . ...................... .. . . . . . . . . . v o . . . . . . . » o o . . . . . a . . . . . . - . . . . . . . . v u . . u . . . . . a . 10p ....................................................................... n ---------------------- d . u . . . . u . o . . . . c u o a . s u 60 7o 80 90 100 Number of Recurrences Figure 6.2. The number of recurrence points for different values of the number of samples in period-1 orbit,m(1). 153 51: i+T ----- - The extracted orbit 3 --—— The estimated orbit : _0.1 l l l l -o.1 o 0.1 02 0.3 o 4 o 5 33 7. Figure 6.3. A period-1 extracted orbit and its improved orbit. 154 0.6 -o.4 Figure 6.4. A period-2 extracted orbit and its improved orbit. 155 53247 -0.2 -O.6 Figure 6.5. A period-4 extracted orbit and its improved orbit. 156 $247 3371 Figure 6.6, A period-8 extracted orbit and its improved orbit. 157 1.4 ' i — The first eigenvalue ‘- ' - - - The second eigenvalue 13,............. _ .................. _. ‘ I \ 1,2- ...................... ’..; ............. \ _ ... it \_ 8 I 3 \_ .314 a a) .9 (a 1 0 .: E 0.9 0.8 0.7 i i 4 i 5 10 15 20 25 ThePhase, j Figure 6.7. The local dynamics of the period-l extracted orbit that is plotted in Figure 6.3. tracted orbit of period-1 has a sequence of local saddle dynamics that has a flat peak, while the higher period-l extracted orbits have no flat peaks in their local dynam- ics plots. Whoever, the flat peak in period-1 plot is due to the large scaling of the z-coordinate and not due to a long sequence of local saddle dynamics. Hence, it is expected that all the orbits have no large interior errors. 158 1.4 l j l I I ‘ ' ' ' ' — The first eigenvalue --'--- Theseoondeigenvalue L3-.H.H.n.u.n.nt ......... a.“.”.n.n.n.ng .......... 3 ...... .,1,N.U.H{H.H.H.g ........ a .5 C N I I ~: I: K I d o d .......................................................................................... l The Eigenvalues‘ b . . t o o a o a o o a u o a a o e e a n o v u e n n u e u o u c o a o o o a o e e 4 a a u o e o e o o o e o e o a e . v u u e a a . a o a o u o o 08*- ............................ , .......... , .......... _ .......... , ..................................... .. I 0., i i i i 0 5 1O 15 20 25 3O 35 4O 45 50 The Phase, 5 Figure 6.8. The local dynamics of the period-2 extracted orbit that is plotted in Figure 6.4. 159 1.4 l I I ’ Thetirsteigenvalue ; 3 3 3 g - '- Theseoondeigenvalue 1.31. ........ ..‘ ........ -_ .................... . ........................................ -l cu. in I ~ nee. val tgen lThe E ......................................... . . . . . 08-.....H,. ...., O . 0.7 i i i o 100 The Phase,j Figure 6.9. The local dynamics of the period-4 extracted orbit that is plotted in Figure 6.5. 160 1.4 1.3 1.2 d e d l The Eigenvalues‘ O a, .. 0.8 l I 20 4O 60 80 100 120 140 160 180 200 0.7 i i 0 The Phase, j Figure 6.10. The local dynamics of the period-8 extracted orbit that is plotted in Figure 6.6. 161 6.2 Applying the Theoretical Bound The theoretical bound for the experimental system is calculated from its estimated linear period-l matrices that represent its dynamics local to its extracted periodic points. The calculation steps of this theoretical bound are presented in Section 3.1. As it was mentioned in the beginning of this chapter, the theoretical bound is applied by comparing the extracted orbits to their improved ones. If the improved orbits closely approximate the true orbits, the theoretical bound should be able to bound the extraction error at the recurrence points. Hence, it may be applied as a diagnostic of the improved orbits. Comparing the extracted orbits from the reconstructed phase space to their im- proved ones shows that the theoretical bound cannot bound even the extraction error at the recurrence point. For example, Figure 6.11 shows the extraction error at the recurrence point of the periodic orbit that has the theoretical bound shown in Figure 6.12. The violation is clear since the value of it w 10, in Figure 6.11, exceeds the theoretical bound value 4.75 at j=1, in Figure 6.12. However, it is possible that the improved orbits are the ones that deviate from the exact orbits. Then the extraction error does not reflect the real one. In fact, using the theoretical bound in case of orbits that are extracted from a reconstructed phase space of Duffing equation leads to the failure of the theoretical bound to bound their recurrence errors, too. This result suggests using another method to reconstruct the phase space instead of the method of delays or increasing the embedding dimension with which the estimation is performed. A valid candidate method is to obtain the second state, which is in fact the derivative of the observed state, by performing the numerical differentiation on the 162 -0915- ...... i . . . i . . . . _, -o.92-' 1 -0.925~ a L i i i i _l 1 i i -0.5 -0.495 —O.49 43.485 -O.48 -0.475 -0.47 -0.465 -O.46 -0.455 -0.45 371 Figure 6.11. A recurrence error of a period-6 extracted orbit. observed state. Then, the equation of the second state is $10 + 1) — 271(i — 1) 2At (6.1) 22(2) = This method is sensitive to noise, even though the time step, At, has no noise in its values. Reconstructing the phase space by the differentiation leads to nearly the same extracted orbits that are extracted from the phase space that is reconstructed by the method of delays, but with some new and missed extracted orbits. Figures 6.12-14 show some extracted orbits that are extracted from a: - :5: space plotted with their improved orbits. The same orbits are extracted from the delay space and are shown in Figures 6.3-5. Comparing the extracted orbits to their improved ones shows that the theoretical bound fails to bound the extraction error at the 163 The Theoretical Bound 7 , j 6— ................................. -I 5 ............................................................................................ d o i i 0 50 100 150 The Phased Figure 6.12. The theoretical bound on the recurrence error, as a function of the phase of the orbit, j, for the extracted period-6 orbit that has the recurrence error shown in Figure 6.11. 164 0.4 , T r r : : : -- - — - - The extracted orbit 3 3 — The estimated orbit 0.3... ...... ........... ......__, 002-......... ..’. ..... ; .............. ...... . ...... ........ \ ..... . ....... ~ ............ -1 . ‘ t : \.' 1 ‘.. . : : : 3‘. : 0.1.. ..... ~ . ._ .3..\...._ .. .. ..- . : ; : : : \_ : .’ \_ . l l w 0.. “I ................... I .I '/ _o.1... ...... . / .. .. :/ ’ 2‘ ./' _O.21_ ........................................................ ’...f. ............................ _4 .’ /' _0.3_ ........................ .. -o.4 i i i i i i -0.8 -0.7 -O.6 -O.5 —0.4 -0.3 -0.2 -0.1 113 Figure 6.13. A period-2 extracted orbit that is extracted from a: — 1': space, and its improved orbit. recurrence point, too. Figure 6.16 shows the extraction error at the recurrence point of the extracted orbit that has the theoretical bound plotted in Figure 6.17. The violation is clear since the value of g: z 2, in Figure 6.16, exceeds the theoretical bound value 1 at j=l, in Figure 6.17. Again, it is possible that the improved orbits are not accurate, and hence the extraction error is not true. 165 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 Figure 6.14. A period-4 extracted orbit that is extracted from :1: — if: space, and its improved orbit. 166 0.6 1 r r r r i f i I ? ------ Theextractedorbit — The estimated orbit -0.6 _. ............................................................................................. _ -0.8 1 1 1 L 1 l J l -1 -0.8 -0.6 -0.4 —0.2 0 0.2 0.4 0.6 0.8 . Figure 6.15. A period-8 extracted orbit that is extracted from a: — 1': space, and its improved orbit. 167 0278 T H '7 ' " l " W‘T ' "f """"""" . 0.276: {B 0.272: 0.27% 0.268 I 0266‘ i g i i i -0.206 -0.204 -0.202 -0.2 -0.1 98 -0.196 -0.1 94 33 Figure 6.16. A recurrence error of a period-4 extracted orbit. 168 The Theoretical Bound g 0 1o 20 30 4o 50 60 7o 80 90 100 The Phase, 5 Figure 6.17. The theoretical bound on the recurrence error, as a function of the phase of the orbit, j, for the extracted period-4 orbit that has the recurrence error shown in Figure 6.16. 169 Table 6.1. The identified parameters of the experimental third order model that are associated with the average residual plot in Figure 6.18. No. of harmonics a fig A 32 33 7c 7, 1 0.1810 -0.0157 -0.3917 0.3701 1.7740 -0.0074 0.0895 2 0.0582 -0.0037 -0.2401 0.2374 1.2156 -0.0500 0.1170 3 0.0561 -0.0017 -0.2180 0.2211 1.1485 —0.0494 0.1190 19 0.0544 -0.0013 -0.2147 0.2212 1.1493 -0.0489 0.1184 Artificial Parameters 0.0550 0 -0.2100 0.2200 1.1500 -0.0480 0.1200 6.3 Identifying the Parameters In this section, the parameters of the experimental system are identified by the har- monic balance method. The extracted orbits and their improved ones are used in this method. The model that is assumed for this system is 3 :13 + out + 25,1" = 76 cos(wt) + 7, sin(wt) i=0 (6.2) Using all the nineteen extracted orbits leads to the identified parameters that has the average residual shown in Figure 6.18. The identified parameters have steady values that are listed in Table 6.1, for n = 19. Simulating the differential equation that has these parameters results in a phase space has the extracted period-4 orbit shown in Figure 6.19. Comparing this orbit to the period-4 extracted orbit of the experimental system, Figure 6.5, shows the qualitative similarity of the two systems. If the artificial para- meters, listed in Table 6.1, are used in the differential equation, the similarity will be lost. This indicates the sensitivity of the dynamical behavior to the variations in the parameters of the system. However, using the values of the identified parameters that result from using a 170 0.1 8 I I : : — The average residual - — - - . - The average artificial error 0.16 r - -i - 0.14 0.12 0.1 0.06 0.06 0.04 0.02 o .-'-'—1'-'-'-'-'-.-l'—'-'_'-.-'-J-'-'-'-'-°—'.l 0 5 10 15 20 25 Number of Harmonics, n Figure 6.18. The average residual and the average parameters artificial error for the third degree model. 171 °-" 7 V T I r Y F 0.6 0.4 0.2 -O.8 -O.8 -O.6 -O.4 -O.2 0 0.2 0.4 0.6 0.8 Figure 6.19. A period-4 extracted orbit, produced from the set of parameters listed in Table 6.1 for n = 19, that is similar to the period-4 extracted orbit of the experimental time series. 172 0-8 I I I I I I I I -1 -O.8 -O.6 -O.4 -O.2 O 0.2 0.4 0.6 0.8 Figure 6.20. Another period-4 extracted orbit, produced from the set of. parameters listed in Table 6.2, that is similar to the period-4 extracted orbit of the experimental time series. different set of extracted orbits in the identification method will keep the similarity of the shape of the phase space. Figure 6.20 shows a period—4 extracted orbit that is extracted from the phase space of the differential equation that has the parameters listed in Table 6.2. In fact, even if the degree of the estimation model is increased to include the fourth order term, the similarity can be achieved as it is clear from Figure 6.21. The differential equation in this case has the parameters listed in Table 6.3. 173 Table 6.2. Another set for the identified parameters of the experimental third order model. a .30 31 52 33 7c 7. .0635 -.0004 -.2167 .2136 1.14 -.0512 .1197 0.1 1 T T ! : g -— The average residual ! ----- The average artificial error 0.09 - . . .| ........................................... ‘ i 0.08 .. . . .l ............................................................................................ .. I i 0.07 ._ . . . .i ............................................................................................ T i 0.06 l 0.05 0.04 0.03 0.02 0.01 Number of Harmonics, 11 Figure 6.21. The average residual and the average parameters artificial error for the fourth degree model. Table 6.3. The identified parameters of the experimental fourth order model that are associated with the minimum average residual value in Figure 6.20. 30 41 132 53 134 7c 70 .0802 -.0127 -.1978 .383 .8577 -.3209 —.0252 .1433 174 08 l l l T l T l l -1 -0.8 -0.6 -O.4 -0.2 0 0.2 0.4 0.6 0.8 332 Figure 6.22. A period-4 extracted orbit that is extracted from the fourth degree model and it is similar to the period-4 extracted orbit of the experimental time series. 175 Table 6.4. The identified parameters of the experimental third order model, for seven extracted orbits and their improvement in both kinds of reconstruction phase spaces. Method 0 50 .31 32 33 7c 73 Extracted 0.0819 0.0066 -0.2181 0.1776 1.0492 -0.0660 0.1430 Est. in :c-i‘ P. S. 0.0808 0.0056 -0.2256 0.1928 1.1009 -0.0595 0.1403 Est. in delays P. S. 0.0834 0.0065 -0.2193 0.1810 1.0721 -0.0648 0.1411 Table 6.5. The variation of Fourier series coefficients for a period-3 extracted orbit due to its improvement in its estimated orbits that are estimated in both kinds of the reconstructed phase spaces. 8 ' Term Extracted Est. in the delays P. S. Est. in :r-ri: P. S. 00 -0.2600 -0.2614 -0.2615 a1 0.0099 0.0056 0.0056 01 -0.1577 -0.1539 -0.1546 02 -0.0080 -0.0098 -0.0097 02 -0.0298 ~0.0286 -0.0289 03 -0.0027 0.0029 0.0027 03 -0.0003 0.0003 0.0004 When seven extracted orbits of periods I = 1,1,2, 3,4, 5 and 8 are estimated and used in the parametric identification, the identified parameters have the values listed in Table 6.4. The corresponding phase spaces of these parameters values have the period-4 orbits that are shown in Figures 6.23-25. It is clear that the improved orbits have shapes that are more similar to the experimental period-4 orbit in Figure 6.5. The variation of Fourier series coefficients for a period-3 extracted orbit due to its improvement in its estimated orbits that are estimated in both of the delays and x-i' reconstructed phase spaces is shown in Table 6.5. 176 0.8 0.6 0.4 0.2 $i+T . - . . . . . . o ............................................................................................ - .............................................................................................. d ............................................................................................. _ ............................................................................................ q _1 I I I I I I I I - -o.s -o.a -o.4 -02 o 0.2 0.4 0.6 0.8 Figure 6.23. A period-4 extracted orbit that is extracted from the phase space of the differential equation that has the identified parameters based on seven extracted orbits. 177 0'8 l l l l I l I l 0.6—......m: .......... 3 .......... ....... ......... .7 o4_ ...... ¥ ........ _. _1 1 1 1 i_ 1 1 L 1 -1 -0.8 -O.6 -0.4 —0.2 O 0.2 0.4 0.6 0.8 Figure 6.24. A period-4 extracted orbit that is extracted from the phase space of the differential equation that has the identified parameters based on the seven improved orbits in 33-5: phase space. 178 08 I l I I I T I l 0.2 0.4 0.6 0.8 Figure 6.25. A period-4 extracted orbit that is extracted from the phase space of the differential equation that has the identified parameters based on the seven improved orbits in the delays phase space. 179 6.4 Summary In this chapter, the extraction process was applied on the reconstructed phase space of the experimental time series. The unstable periodic orbits were extracted and the least-squares estimation method was used to improve the extracted orbits. After that, the theoretical bound was applied by comparing the extracted orbits to their improved ones. The estimated orbits were very close to the extracted orbits but this did not prevent the extraction error at the recurrence point to exceed the theoretical bound. It is possible that the estimated orbits are not accurate and deviate from the exact orbits. Hence, it is not clear whether the theoretical bound is violated or the estimated orbits are not accurate. However, the magnitudes of violations were smaller in 35-23 phase space. As men- tioned in Chapter 3, the value of the extraction error ratio, §,is very sensitive to any small error in the true orbit, hence having some violations would not mean that the estimated orbits are too bad. Using the extracted orbits in the identification method gave a phase space that has some similarity to the original phase space. This similarity was obtained for more than one estimation model, hence the identified parameters are not unique. When seven extracted orbits were improved by estimation, the resulting phase space was improved and had a shape that is closer to the original one. The variation of the Fourier series coefficients associated with the improvement of a period-3 extracted orbit, was shown. The similarity of the coefficients that result from the improvements in :r-i' and the delays phase spaces is a good indication. 180 CHAPTER 7 CONCLUSION In conclusion to this thesis, the next points summarize the contributions and the attained results of this research. 1. The exact periodic orbits for the tent and horseshoe maps were obtained ana- lytically. In fact, the number of the periodic orbits and a valid initial condition that leads to a desired periodic orbit can be obtained, for any period-k, see Ap— pendix B. For example, the number of the period-32 orbits is 134,215,680 orbits. For Henon map and Duffing equation, the exact solution was approximated by correcting the extracted orbits via an intensive search method. 2. The effect of increasing the distance that defines a recurrence, 6, or the time series length, N, on the extraction results was shown. The number of the extracted orbits increases with increasing these parameters, with the possibility to extract orbits for periods that are lees than their true periods. Increasing only N will increase the extraction error ratio, 5;, even though the error, 5, will decrease, where 6 is the ultimate bound for the error in the extracted orbits and 6., is the actual recurrence distance for these orbits. 181 3. The distinction of the extracted orbits without exploiting the knowledge of the map is an important step since in practice the systems will be unknown. The 61 criterion, in which the extracted orbits of period-k that are separated by 151 or less will be assumed to represent the same orbit, was used. 4. The error in the extraction process was calculated for the tent, horseshoe and Henon maps. Furthermore, a general formula for the actual error ratio, 25;, was developed for the tent and horseshoe maps. This general formula is -l_—l,;;, where A is the maximum of Ac, the contraction factor, and Ac“, the reciprocal of the expansion factor. 5. For the Henon map, there is no such a general formula since its linearized matrix is not diagonal or even constant. Another reason behind the unclear actual bound for the Henon map is the uncertainty about its exact orbits, which are so close to one another and have complex, or unrelated orbits among them. Furthermore, the largest extraction error in Henon map extracted orbits can be at some interior point rather than the recurrence point. 6. The actual bounds were compared to their theoretical bounds. This included the evaluation of the theoretical bound by two different methods. First, calcu- lating the exact one by using the known linearized matrices of the maps. And also estimating the theoretical bound by using the least-squares method. The estimated theoretical bound and the exact one had very close values, except in case of severe nonlinearity. The ability of the theoretical bounds to bound the actual error ratios was great for the tent and horseshoe maps, specially if the oo-norm was used. For Henon map, having an interior large error violated the assumption that the largest extraction error is at the reccurence point. Hence, the theoretical bound cannot be applied to obtain a bound on the largest error in the extracted orbit. 182 7. The extraction process was applied on time series produced numerically by the 10. Duffing equation. The estimated theoretical bound was checked again for some of the extracted orbits, and it was able to bound the error at the reccurence points of these extracted orbits. But this was not enough to bound the large interior error in some of the extracted orbits of period-l > 4. The extracted orbits were improved by using the least-squares method. And the best extracted orbits, i. e. with the smallest actual reccurence distances, were used instead of the average of all the extracted orbits, which are normally used. The estimation results were great for the tent and the horseshoe maps. For Henon map, the estimated orbits were improved but not to the level of the other two maps. However, for the Duffing equation, the estimated orbits maintained a zero reccurence distance, but this did not mean that all the estimated points had a zero error. Comparing the estimated orbits to the corrected ones showed the deviation of these orbits at many interior points. . The effect of extraction error and its improvement on the parameter identifica- tion in maps and flows were studied. This included identifying the parameters by using the extracted and the estimated orbits and compare their results. For maps, the least-squares method was used, and the error in the identified para- meters was zero, in the case of a clean signal. For the Duffing equation, the harmonic balance method was used to identify its parameters. There were many identification cases that were studied to show the effect of many factors. Among the studied effects are the effects of the extraction error, the number of the used orbits and their periods and stability. As it was shown, using good orbits, i. e. with small extraction error, gave more accurate identified parameters. Also, .using more orbits increased the accuracy of the identified parameters, specially with good orbits, and reduced the necessary 183 11. number of harmonics, n, in the Fourier series. Furthermore, by using one stable orbit, the identified parameters had a steady value with respect to n, with a great accuracy. The effect of using an estimation model whose degree was higher, or lower, than the original system was studied. It was shown that using a lower degree led to inaccurate parameters with higher residuals, while using a higher degree model led to a good estimation. Finally, the effects of using different frequency, w, or initial time, to, were studied. The identified parameters have to be multiplied by the proper functions of r = 5‘, where wt is the true frequency. And using wrong to was very effective on the identified parameters, in the case of having only a cosine forcing term. Hence, a sine forcing term was added to the identification model in order to remove the effect of using different to. The effects of noise and phase space reconstruction on the parameter identi- fication were studied, too. The effect of noise on the extraction results was correlated with the noise level, while the effect of noise on the estimation of periodic orbits was worse, specially with using small 6., to define the neighbor- hood of the estimated orbits. However, for maps, the identification results, by least-squares method, were more accurate when all the time series points were used, instead of using only the extracted orbits or even their improved ones. Reconstructing the phase space had a dramatic effect on the identified para- meters of the tent and the horseshoe maps. This is expected since these maps are discontinuous. For the Henon map, choosing the time delay, 7’ = l, inter- changed the :c and y coordinates and did not identify the factor by which the y-coordinate in the original system is multiplied. While if 'r = —1 was used the identification results were exactly the same except for the factor by which the y-coordinate in the original system is multiplied. This difficulty can be handled 184 12. by normalizing the phase space. For flows, reconstructing the phase space had no effect on the parameter identification, as long as only the measured signal was used. However, normalizing the phase space affected the identified parame- ters differently. If there is no information about the original system, the signal will lose a lot of its importance in the parameter identification method. Finally, the extraction process was applied on a single experimental time series, in order to extract its unstable periodic orbits. These orbits were extracted from a reconstructed phase space then improved by estimating them. After that, the extracted orbits and their improved ones were used to identify the parameters of the original system. The future studies for this thesis can be summarized in the next points. 1. Using the improved periodic orbits in calculating the different invariants of the system such as the Lyapunov exponents and the fractal dimensions Developing a theoretical bound that can bound the interior large error . Performing a probability and confidence study for the attained results . Using the bifurcation theory to determine the parameters values that lead to a chaotic response and whether the system is structurally stable or not Extending the extraction, estimation and identification methods to more com- plicated systems such as systems of higher dimensions or different nonlinearity . Reducing the effect of noise by using one of the many proposed methods, and studying the effect of any filtering on the identified parameters Driving a general singular values formula that can bound the extreme expansion and contraction factors for a matrix using any norm 185 APPENDICES APPENDIX A Program 1 This Matlab program is to calculate the analytical periodic orbits for the tent and the horseshoe maps. fik=input(’Enter the max. desired no. of orbits, fik < 11,’) ink=input(’Enter the min. desired no. of orbits, ink < 11,’) c(1)=2;c(2)=1;c(3)=2;c(4)=3;c(5)=6;c(6)=9;c(7)=18;c(8)=30;c(9)=56; c(10)=99;c(l1)=l86;cc=0;sc(1)=0;dx(1)=1;dy(l)=l; for i=1:fik sc(i+1)=sc(i)+c(i)*i; dx(i+1)=3*dx(i)+l; dy(i+1)=2*dy(i)+1; end n=3;k=2;a(2,:)=[1,1];a(3,:)=[3,1]; while 11 < sc(fik+1) while cc < C(k) for i=n+1zn+k-1 a(i,2)=2*a(i-1,2); if a(i,2) > dy(k) a(i,2)=a(i,2)-dy(k); end ay(i)=a(i.2); for j=1zk if (ay(i)-2("‘j)) Z 0 a(i,1)=a(i,1)+3(j"1); a)’(i)=ay(i)-2("“j); end, end, end cc=cc+1;n=i+1;a(n,2)=2+a(n-k,2); nn=1;ii(n)=0; z=1; 186 while z==1 2:0; for i=1:k for j=nn:nn+c(i *i-1 ifj < n & a(Ham/dy(k) == a(i,2)/WU) a(n,2)=2+a(n,2); z=1;ii(n)=ii(n)+1; end end nn=j+1; end nn=1; end ay(n)=a(n,2); for j=1:k if (ay(n)-2("'j)) Z 0 a(n,1)=a(n,1)+3(j‘1); ay(n)=aY(n)-2“"”; end, end, end k=k+1;cc=0;a(n,:)=[3“‘"l),1]; end n=1;k=1; while n < sc(fik+1) while cc < c(k) for i= n:n+k-1 xa(i,:)=[a(i,1)/dx(k),a(i,2)/dy(k)]; end cc=cc+1;n=i+1; end k=k+1;cc=0; end 187 APPENDIX B Program 2 This Matlab program is to calculate the initial condition required to take the output of the tent map, or the horse shoe map, to a desired periodic orbit. It is also to calculate the period of the orbit to which a given initial condition will take the output of the two systems. Finally the number of these unstable periodic orbits is calculated for a given period, k. clear a,clear dx,clear dy dx(1)=0;dy(1)=0, p(2)=1;p(3)=1;p(4)=1;p(5)=1;p(6)=2;p(7)=1;p(8)=1;p(9)=1;p(10)=2;p(11)=1; 1>(=12) 2;p(13)=1;p(14)=2;p(15)=2;p(16)=1;p(17)=1;p(18)=2;p(19)=1;p(20)=2; p2(1)=2;p(22)=2;p(23)=1;p(24)=2;p(25)=1;p(26)=2;p(27)=1;p(28)=2;p(29)=1; W(()=)3;P(31)=1;P(32)=1; l2,(1=) 1' ,,l(3 1)=1;l(4,l)=2;l(5,1)=1;l(6,1)=3;l(6,2)=2;l(7,1)=1;l(8,1)=4; 1(9, 1:) 3 ,l(10, 1): 5 ,,l(10 2): 2' ,,l(11 1)=1;l(12,1)=6;l(12,2)=4;l(13,1)=1; (14,1):7-1,(14,2)— 2,1(15,1)-_-5,,1(15 2): 3;l(16,1)=8;l(17,1)=1;l(18,1)=9; (18, 2)=6; 1(19, 1): 1,,1(20 1): 10 ,,l(20 2)=4;1(21,1)=7;1(21,2)=3;1(22,1)=11; (22, 2): 2' ,l(23,1)=1;l(24,1)=12;l(24,2)=8;l(25,1)=5;l(26,1)=13;l(26,2)=2; (27, 1): 9 ,l(28,1)=14;l(28,2)=4;l(29,1)=1;l(30,1)=15;l(30,2)=10;l(30,3)=6; 1(31,1)= 1 ,1(32,1)=16; q(6)=1;<1(10)=1;q(12)=1;q(14)=1;q(15)=1;q(18)=1;<1(20)=1;q(21)=1;q(22)=1; q(24)=1;q(26)=1;q(28)=1;q(30)=3;q(32)=0; r(6,1)=1;r(10,1)=1;r(12,1)=2;r(15,l)=1;r(18,1)=3;r(20,1)=2;r(21,1)=1; r(22,1)=1;r(24,1)=4;r(26,1)=1;r(28,1)=2;r(30,1)=5;r(30,2)=3;r(30,3)=2, format long e t=input(’please enter 1 to get the period or 2 to get the I. C. 1', t=’); if t == h=input(’please enter the denominator of your y initial condition, h=’); l l l l 188 a(1)=1; for i=1:h a(i+l,l)=a(i)*2; if a(i+l) > h a(i+l,1)=2*a(i)-h; end for j=lzi-1 if a(1)==a( 1) z=i-j; sprintf(’The periodic orbit has %d periods.’,z) return end end end elseif t==2 h=input(’please enter the desired no. of periods, h=’); for i=1:h dx(i+l)=3*dx(i)+1; dy(i+1)=2*dy(i)+1; end format rat sprintf(’A Valid I. C. is’) ix=3(h'1)/d:r(h +1),iy =1/dy(h +1) format long snl=0;snr=0; for i=1:p(h) snl=2'("") + snl; end for i=1:q(h) snr=2'("") + snr; end noo=(2" — snl + snr)/h; sprintf(’This period has else sprintf(’You have to choose t = 1 or 2.’) end 189 BIBLIOGRAPHY BIBLIOGRAPHY [l] P. G. Dranzin and GP. King, 1992, “Introduction,” Physic D 58, vii - xi. [2] R. Badii, C. Broggi, B. Derighetti, M. Ravani, S. ciliberto, A. Politi and M. A. Rubio, 1988, Phy. Rev. Lett 60, 979. [3] P. C. Lo and J. Principe, 1989, IEEE Eng. Biol. Soc. 89, 638. [4] A. M. Albano, A. Passamante, T. Hediger and M. E. Farrell, 1992, “Using Neural Nets to Look for Chaos,” Physica D 58, 1-9. [5] N. H. Packard, J. P. Crutchfield, J. D. Farmer and R. S. Shaw, 1980, Phys. Rev. Lett. 45, 712. [6] F. Takens, 1981, “Detecting Strange Attractors in Turbulence,” Lecture Notes in Mathematics 898, eds D. A. Rand and L. S. Young, (Spring, Berlin, 1981) 366. [7] Th. Buzug and G. Pfister, 1992, “Comparing of Algorithms Calculating Optimal Embedding Parameters for Delay Time Coordinates,” Physica D 58, 127-137. [8] A. M. Fraser, 1989, “Information and Entropy in Strange Attractors,” IEEE' Trans. Info. Theory 35, 245-262. [9] M. Ding, C. Crebogi, E. Ott, T. Sauer and J. A. Yorke, 1993, “Plateau onset for correlation dimension : when does it occur ?” Phy. Rev. Lett 70, 3872-3875. [10] H. Abarbanel, R. Brown, J. Sidorowich, and L. Sh. Tsimring, 1993, “The Analysis of Observed Data in Physical Systems,” Rev. of Mod. Phys. 65, No. 4, 1331-1392. [11] -E. Kostelich, 1992,“Problems in Estimating Dynamics from Data,” Physica D 58, 138-152. [12] D. S. Broomhead and G. P. King, 1986, “Extracting Qualitative Dynamics from Experimental Data,” Physica D 20, 217-236. 190 [13] P. Grassberger and I. Procaccia, 1983, “Characterization of Strange Attractors,” Phys. Rev. Lett. 50, 346-349. [14] W. Liebert, K. Pawelzik and H. Schuster, 1991, “Optimal Embeddings of Chaotic Attractor from Topological Considerations,” Europhs. Lett. 14, 521. [15] M. B. Kennel, R. Brown and H. Abarbanel, 1992, “Determining Embedding Dynamics for Phase Space Reconstruction Using Geometrical Construction,” Phys. Rev. A 45, 3403-3411. [16] Liangyue Cao, 1997, “Practical Method for Determining the Minimum Embed- ding Dimension of a Scalar Time Series,” Physica D 110, 43-50. [17] E. Ott, T. Sauer and J. A. Yorke, 1994, Coping with chaos, Wiley, New York. [18] T. Sauer, 1992, “A Noise Reduction Method for Signals from Nonlinear Systems,” Physica D 58, 193-201. [19] A. Rabinovitch and R. Thieberger, 1992, “Analysis of Noisy Signals,” Physica D 58, 284-287. [20] G. Rowlands and J. C. Sprott, 1992, “Extraction of Dynamical Equations from Chaotic Data,” Physica D 58, 251-259. [21] R. Cawley and G. Hsu, 1992, “Local-Geometric-Projection Method for Noise Reduction in Chaotic maps and flows,” Physica D 58, No. 6, 288-298. [22] G. T. Shaw and G. P. King, 1992, “Using Cluster Analysis to Classify Time Series,” Physica D 58, 288-298. [23] E. Kostelich and J. Yorke, 1990, Physica D 41, 183. [24] J. Farmer and J. Sidorowich, 1991, Physica D 47, 373. [25] M. Casdagli, 1989, Physica D 35, 335. [26] N. Tufillaro, H. Solari and R. Gilmore, 1990, “Relative Rotation Rates— Fin- gerprints for Strange Attractors,” Phys. Rev. A 41, 5717. [27] N. Tufillaro, H. Solari and R. Gilmore, 1989, Phys. Rev. Lett. 64, 2350. [28] D. Lathrop and E. Kostelich, 1989, “Characterization of Strange Attractors by Periodic Orbits,” Phys. Rev. A 40, 4028-4031. 191 [29] S. Pueyo, 1997, Physica D 106, 57-65. [30] C. Schiltenkopf and G. Deco, 1997, “Identification of Deterministic Chaos by an Information-theoretic Measure of the Sensitive Dependence on the Initial Conditions,” Physica D 110, 173-181. [31] A. Nayfeh, 1985, “Parametric Identification of Nonlinear Dynamic Systems,” Computers and Structures 20, 487 - 493. [32] G. Stry and D. Mook, 1992, “An Experimental Study of Nonlinear Dynamic System Identification,” Nonlinear Dynamics 3, 1-11. [33] K. Yasuda, S. Kawamura and K. Watanabe, 1988, “Identification of Nonlinear Multi-Degree-of-Freedom Systems (Proposition of an Technique),” JSME Inter- national Journal , Series III, 31, 8-14. [34] K. Yasuda, S. Kawamura and K. Watanabe, 1988, “Identification of Nonlinear Multi-Degree—of-Freedom Systems (Identification Under Noisy Measurements),” JSME International Journal , Series III, 31, 302-309. [35] D. Auerbach, P. Cvitanovic, J.-P. Eckmann, G. Gunaratne and I. Procaccia, 1987, “Exploring Chaotic Motion through Periodic Orbits,” Phys. Rev. Lett. 58, 2387-2389. [36] D. Lathrop, 1989, “Characterization of an Experimental Strange Attractor by Periodic Orbits,” Phys. Rev. A 40, 4028-4031. [37] N. Tufillaro, T. Abbott and J. Reilly, 1992, An Experimental Approach to Non- linear Dynamics and Chaos, Addison-Wesley. [38] S. Hammel and J. Heagy, Oct. 1992, “Chaotic System Identification Using Linked Periodic Orbits,” Presented at the SIAM Conference on Applications of Dynamic Systems, Snowbird, Utah, 15-19. [39] R. Kesaraju and S. Noah, 1994, “Characterization and Detection of Parameter Variations of Nonlinear Mechanical Systems,” Nonlinear Dynamics 6, 433-457. [40] C. M. Yuan and B. Feeny, 1997, “Parametric Identification of Chaotic Systems,” Journal of Vibration and Control 4, 405-426. [41] N. Van de Wouw, G. Verbeek and D. Van Campen, 1995, “Nonlinear Parametric Identification Using Chaotic Data,” Journal of Vibration and Control 1, 291-305. 192 [42] J.-P. Eckmann and D. Ruelle, 1985, Rev. Mod. Phys. 57, 617. [43] J.-P. Eckmann and S. Kamphorst, 1986, “Lyapunov Exponents From Time Se- ries,” Phys. Rev. A 34, No. 6, 4971-4977. [44] A. Wolf, J. Swift, H. Swinney and J. Vastons, 1985, “Lyapunov Exponents From Time Series,” Physica D 16, 285—317. [45] J. Greene and J.-S. Kim, 1986, “Calculation of Lyapunov Spectra,” Physica D 24, 213-225. [46] H. F. Bremen, F. E. waodia and W. Proskurowski, 1997, “An Efficient QR Based Method for the Computation of Lyapunov Exponents,” Physica D 101, 1-16. [47] J. Farmer, E. Ott and J. Yorke, 1983, Physica D 7, 153. [48] J. Theiler, 1990, J. Opt. Soc. Am. A 7, 1055. [49] G. H. Gunaratne, M. H. Jensen and I. Procaccia 1988, Nonlinearity 1, 157. [50] G. Mindlin, H. Solari, M. Natiello, R. Gilmore and X.-J. Hou, 1991, Nonlinear Sci. J. l, 147. ‘ [51] J. Birmnan and R. Williams, 1983a, Topology 22, 47; 1983b, Contemp. Math. 20, l. [52] S. P. Allie and A. I. Mees, 1997, “Cycles from Short Time Series,” Applied Nonlinear Dynamics and Stochastic Systems Near the Millennium, edited by J. B. Kadtke and A. Bulsara, 47-50. [53] P. Cvitanovic, G.H. Gunaratne and I. Procaccia, 1988, “Topological and Metric Properties of Henon-Type Strange Attractors,” Phys. Rev. A 38, No. 3, 1503- 1520. [54] P. Cvitanovic, 1988, “Invariant Measurement of Strange Sets in Terms of Cycles,” Phys. Rev. Lett. 61, No. 24, 2729-2736. [55] H. T. Davis, 1962,1ntroduction to Nonlinear Difierential and Integral Equations, Dover, New York. [56] R. Artuso, E. Aurell and P. Cvitanovic, 1990, “Recycling of strange sets: 1. Cycle expansions,” Nonlinearity 3, 325-259. 193 [57] J. Guckenheimer and P.J. Holmes, 1986, Nonlinear Oscillations, Dynamical Sys- tems and Bifurcations of Vector Fields, (Springer, New York). [58] J. Beck and K. Arnold, 1977, Parameter Estimation in Engineering 69' Science, John Wiley & Sons. [59] K. O. Dzhaparidze, 1986, Parameter Estimation and Hypothesis Testing in Spec- tral Analysis of Stationary Time Series, Springer-Verlag, New York, translated by S. Kotz. [60] E. Walter, L. Pronzato and J. Norton (Translator), 1997, Identification of Para- metric Models: From Experimental data, Springer-Verlag, New York. [61] R. W. Farebrother, 1988, Linear Least Squares Computations, Marcel Dekker Inc. [62] S. H. Strogatz, 1994, Nonlinear Dynamics and Chaos, 2nd printing, Addison- Wesley, Reading, MA. [63] C. M. Yuan, 1995, A Method of Parametric Identification for Chaotic Systems, PhD thesis, Michigan State University, East Lansing. [64] B. Feeny and C. M. Yuan, 1999, “Parametric Identification of an Experimen- tal Two-well Oscillator,” ASME Design Engineering Technical Conferences, Las Vegas. [65] A.M. Fraser and H.L. Swinney, 1986, “Independent Coordinates for Strange At- tractors from Mutual Information,” Phys. Rev. A 33, 1134-1140. [66] C. S. Hsu, 1981 “A Generalized Theory of Cell-to-Cell Mapping for Nonlinear Dynamical Systems,” J. of Appl. Mech. 48, 634-642. [67] C. S. Hsu, 1987, Cell-to-Cell Mapping—A Method of Global Analysis for Non- linearSystems, (Springer, New York). Additional Refrences [68] R. Artuso, E. Aurell and P. Cvitanovic, 1990, “Recycling of strange sets: 11. Applications,” Nonlinearity 3, 361-286. [69] A. S. Weigend and N. A. Gershenfeild, 1994, Time Series Prediction: Forecasting the Future and Understanding the Past, Addison-Wesley, Reading, MA. 194 [70] A. Potopov, 1997, “Distortions of reconstruction for chaotic attractors,” Physica D 101, 207-226. [71] T. Sauer, J. A. Yorke and M. Casdogli, 1991, J. Stat. Phys. 65, 579. 195