)V1531_J RETURNING MATERIALS: P1ace in book drop to LJBRARJES remove this checkout from .—:—. your record. FINES will be charged if book is returned after the date stamped below. THE PLANNING OF MOTION by~ Rupert Charles Bell A DISSERTATION Submitted»to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Psychology 1984 Copyright by RUPERT CHARLES BELL 1984 ABSTRACT THE PLANNING OF MOTION BY Rupert Charles Bell Existing theories of animal motion, such as S-R theory and Lewin's field theory, are explored in the two dimensional case and found to make unlikely predictions about motion in certain situations. In detour problems, which real animals can solve, the theoretical predictions lead to stalling at an intermediate point rather than going around the barrier. Animals are also predicted to stall at intermediate points of relative safety rather than escape between predators. Animals are predicted to be unable to go out of their way to make use of shelters such as tall grasslands or other cover. As an alternative, a theory is developed based upon the idea that animals choose optimal paths, minimizing their expected exposure to fear before engaging in motion. In the situations that are problematic for prior theories, this new theory makes predictions which match empirical observations. A neural circuit is constructed which can solve for the required optimal paths approximately. DEDICATION To Pretty Kitty and Mrs. Magoo, who through their patient c00peration with numerous observations and manipulations, made the idea of this research come alive. ii ACKNOWLEDGMENTS The author would like to thank his graduate committee, particularly John Hunter and Ray Denny, for many interesting and helpful discussions. The spirit of Stanley Ratner was also an important impetus to the present work. Last, but not least, I would like to acknowledge the patient and careful assistance of my typist, Geri Wilson. iii TABLE OF CONTENTS LIST OF FIGURES . . . . . . . . . . . . . . . INDEX OF NOTATIONS. . . . . . . . . . . . . . INTRODUCTION. . . . . . . . . . . . . . . . . Chapter 1. EVALUATION OF PSYCHOLOGICAL THEORIES OF MOT ION O O O O O O O O O O O O O O O 1.1 Review of the Theoretical Literature on Animal Motion in Psychology. . . 1.2 Failures of the Lewin-Miller Theory in Two Dimensions . . . . . . . . . OPTIMAL MOTION PLANNING . . . . . . . . 2.1 The Planning Process. . . . . . . . 2.2 Time Penalty. . . . . . . . . . . . 2.3 The Moore Algorithm and Dynamic Programming . . . . . . . . . . . . 2.4 Computation and Analysis in the Continuous Case . . . . . . . . . . MATHEMATICAL EXAMINATION OF THE PLANNING THEORY: TRAJECTORIES. . . . . . . . . . 3.1 The Variational Form of the Trajectories for Static Fear and Maximum Velocity. 3.2 Summary of a Variable Velocity Case . 3 Direct Consequences of the Principle of Optimality . . . . . . . . . . . 3.4 Behavior in Relation to Shelters of Passage 0 O O O O O O O O O O O O O NEURAL NETWORKS FOR SOLVING THE OPTIMAL PLANNING PROCESS. . . . . . . . . . . . iv Page vi ix 19 42 42 47 49 55 57 57 81 87 89 95 Chapter Page 4.1 An Arbitrary Network Solver in Abstract Form . . . . . . . . . . . . . . 96 4.2 The Dolson-Bell Network for Discrete Networks. . . . . . . . . . . . . . . . . 100 4.3 A Neural Network to Compute Optimal Paths . . . . . . . . . . . . . . . . . . 106 5. DIRECTIONS FOR FURTHER RESEARCH . . . . . . . 117 5.1 Experimental Tests of the Optimal Planning Theory . . . . . . . . . . . . . 117 5.2 Extensions of the Present Work. . . . . . 118 6. SUMMARY . . . . . . . . . . . . . . . . . . . 121 APPENDIX A . . . . . . . . . . . . . . . . . . . . 122 APPENDIX B . . . . . . . . . . . . . . . . . . . . 130 LIST OF REFERENCES. . . . . . . . . . . . . . . . . 131 LIST OF FIGURES Figure 1.1.1 Generation of Forces in Lewin's Theory. . . 1.1.2 Summation of Forces in Lewin's Theory . . . 1.1.3 Symmetric Force Field of Aversive Object. . 1.1.4 Lewinian Trajectory and Forces. . . . . . . 1.1.5 Approach-Avoidance Conflict in Straight Alley According to Lewinian Theory . . . . 1.1.6 Avoidance-Avoidance Conflict in Straight Alley According to Lewinian Theory . . . . 1.1.7 Two Dimensional Avoidance-Avoidance Conflict in Lewin's Theory . . . . . . . . 1.1.8 Approach-Approach Conflict in One Dimensional Lewinian Model . . . . . . . 1.2.1 Stable Equilibrium in One-Dimensional Lewinian Approach Avoidance Contingency. . 1.2.2 Configuration of Animals for Two Dimensional Approach-Avoidance Conflict. . 1.2.3 Lewinian Trajectories in Two Dimensional Approach-Avoidance Conflict. . . . . . . . 1.2.4 Entrapment in Two Dimensional Lewinian Approach-Avoidance Situation . . . . . . . 1.2.5 "Probable" Trajectory of Real Cat in Two Dimensional Approach-Avoidance Setting . . 1.2.6 Stable Equilibrium in Two Dimensional Avoidance-Avoidance Setting. . . . . . . . 1.2.7 Entrapment Region in Two Dimensional Avoidance-Avoidance Model. . . . . . . . . 1.2.8 "Probable" Escape Route for Real Cat in Avoidance-Avoidance Model. . . . . . . . . vi Page 10 12 14 16 17 18 20 21 22 24 25 30 31 33 Figure 1.2.9 1.2.10 1.2.11 3.1.1 3.1.2 3.1.4 3.1.5 3.1.6 3.1.7 vii Cat Entrapped in a Cul-de—Sac by a Dog-- Lewinian Model . . . . . . . . . . . . . . Summation of Psychological Forces for the Cat in the Cul-de-sac. . . . . . . . . . "Probable" Escape Route for Real Cat From the Cul-de-sac . . . . . . . . . . . . . . Rotational (a) and Irrotational (b) Fields of Force About a Predator. . . . . . . . . Circulatory Paths Can Occur in a Rotational Lewinian Force Field . . . . . . . Deflection of an Animal's Path by a Lewinian Force Field Can Cause Unnecessary Detours in a Two Dimensional Runway Setting. . . . . . . . . . . . . . . . . . Iso-Aversion Contours Generated by Two Predators. . . . . . . . . . . . . . Example of a Network. The Series of Nodes 1, 3, 4, 5 Constitutes a Path J13, J3“, Jigs. o o o o o o o o o o o o o . Re-entry of Nodes Into the FRONTIER is Caused by Certain Combinations of Link costs 0 O O O O O O O O O O O O O O 0 O O Variational Path, Showing Vectors U, T, K and \V£nf O O O O I O O O O O O O O O O O 0 Increasing Fear for the Goal Flattens an Mimal.s Path. 0 O O O O O O O O O O O O O Computed Optimal Paths Show Flattening With Increasing Fear for the Goal . . . . . . . Computed Optimal Paths. . . . . . . . . . . An Arbitrary Path Occurs in Regions of Zero Total Fear. . . . . . . . . . . . . . There Can be Two or More Minimal Paths. . . There Can be an Absolute Minimum Path and Relative Minima. . . . . . . . . . . . . . Page 34 35 36 37 39 41 45 51 54 65 68 71 72 77 78 80 Figure 3.2.1 viii Page Fear as a Function of Velocity at a Fixed POSition O O O O O O O O O O O C I O O O O 83 The Angle E is the Angle Between the Tangent to the Path and an Iso-Aversion Line I O O O O O O O O I O I O O O I O O O 88 Circulatory Paths are not Possible in an Optimal Planning Theory According to the Principle of Optimality. . . . . . . . . . 90 Circulatory Path. . . . . . . . . . . . . . 91 Animals May be Expected to use Shelters of Passage According to the Optimal Planning Theory . . . . . . . . . . . . . . . . . . 92 Computed Behavior in Relation to Shelter: Dynamic Programming Solution With f(r)=0.5+ r ..........94 0.25 + r2 A Non-Planar Maze Cannot be Embedded in the Plane: However, a Planar Hybrid Path Problem Can be Embedded. . . . . . . . . . 97 Two-Channel Lockout Circuit Neuron Threshold = 1. . . . . . . . . . . . . . . 103 Traceback & Lockout Circuits (Two- Channel. . . . . . . . . . . . . . . . . . 105 A Section of Two Dimensional Space "Tiled" by Hexagons. . . . . . . . . . . . . . . . 107 Triangulation Showing Interconnection of comp‘alting Elements 0 O O O O O O O O O O O 108 A Single Hexagonal Computing Element. . . . 110 The Interconnection of Hexagonal Computing Elements is From the Output of Each Neighbor to the Lockout Circuit of the Other. . . . . . . . . . . . . . . . . . . 112 Six-way Lockout Circuit . . . . . . . . . . 113 INDEX OF NOTAT IONS* FA - aversive force FG - attractive goal force FR - resultant force v - velocity E - equilibrium point curl F - V x F the vector curl of a vector field F FH - force of avoidance generated by a hole T - in figures--denotes a trajectory Es - in figures--denotes an escape trajectory x - ordinary Cartesian coordinate y - ordinary Cartesian coordinate J - total exposure to fear f - local (anticipated) fear function T - in equations-~the total time to execute a path t - time f1 - the Spatial component of fear fr - fear for the reward wp - the time premium weight .. - the total cost of a leg of a path between 13 positions 1 and j *in order of occurrence ix CUM BACK FRONTIER Grad Kn Projl W7 x the cumulative exposure to fear up to a point in space, while the animal pursues an optimal path to get there an array (in the Moore Algorithm) containing the best way to get to a node as an index of another node a recursively defined working set in the Moore Algorithm a small increment of cost a dynamic programming stage to parametrize paths a parameter used the x-coordinate of a starting position the x-coordinate of the goal the y—coordinate of a starting position the y-coordinate of the goal derivative by parameter s the tangent unit vector the unit normal to the path the vector gradient of a scalar the curvature vector the natural logarithm the projection operator that projects on the line perpendicular to the path at a given point the gradient operator: same as Grad net goal fear path length the radius coordinate in polar coordinates the angle coordinate in polar coordinates COS F(r.9.é) Co:C1:C2 STEP xi the radius coordinate of the Starting position the radius coordinate of the goal a constant of integration a constant of integration a constant of integration a constant of integration cosine the Lagrangian function of a variational problem constants involving energy expenditure total energy consumption in executing a path a constant giving energy expenditure in the same scale as fear expenditure a parameter of velocity-dependent fear-- see Eq. (3.2.4) a parameter of velocity-dependent fear-- see Eq. (3.2.4) the total Lagrangian function of the variable velocity case the equivalant constant velocity fear function for the variable velocity case the maximum running velocity of the animal the minimum velocity of an animal in a variable-velocity model a term in a special case of the variable- velocity model the angle between the trajectory and an iso-aversion line a minimum path increment in the Bell-Dolson network xii - a particular point on a path - the total number of nodes in a discrete network - absolute scalar length of a vector INTRODUCT ION This work begins with a review of the literature in psychology on animal motion. The review discusses the theories of Loeb and Lewin. Lewin's theories provide the most appropriate present psychological theory which might be applied to explain the motion behavior addressed in this paper. In a subsequent chapter, we present a critique of Lewin's vector psychology, showing cases where it predicts very unlikely behavior. We then assume an optimal choice of path for the minimization of anticipated fear, an optimal planning model. Some detailed predictions are derived from the optimal planning model, including solu- tions to the problems of Lewin's theory. This paper will present a theory of optimal path choice. This theory will assume that an animal chooses its path in such a way as to minimize its exposure to fear. The key to the theory is the idea of planning ahead. An animal must make some rather complex prelimi- nary calculations and comparisons before it starts its motion. Otherwise, it might start in a direction which would initially seem attractive, but would subsequently be much less attractive than some alternate path. The present paper ignores the problem of choice of goal and considers only the optimal path to that goal once the 1 2 goal has been chosen. In this theory the animal considers anticipated fear and chooses a path that will minimize its future exposure to fear. For reasons of mathematical simplicity, our model assumes optimal planning. Although suboptimal planning is a more realistic concept, it is beyond the sc0pe of the present paper. We will consider an animal in a two dimensional field of motion which has decided to go to a specific goal. The animal must choose its path in such a way as to avoid regions of elevated fear. The specific model is that the animal chooses its path in such a way that the total time weighted exposure to fear is minimized. We will present a neural network model which will show that an animal could compute the trajectories generated by the behavioral model. Our investigations of the consequences of the Optimal planning theory provide some results that agree well with intuition: for example, an animal that follows our theory will go in a much straighter path to an object if it is afraid that something might happen to the object; e.g., a cat that fears for its kittens. A key example is the predicted path a mother cat would choose to get to its kittens in the presence of a sleeping dog. We call this the sleeping dog problem. Our presentation deals only with static fields of fear such as may be experienced by an animal going by a sleeping predator. 3 This work was inspired by the work of Menzel [1973]. Menzel showed that chimpanzees will take a nearly Optimal path around a field to pick up pieces of food that had been left there. That is, the chimps choose the sequence of goals approximating optimality in terms of minimum distance travelled. This experiment suggests that animals may plan optimally and suggested the present approach to the avoidance of anticipated fear. CHAPTER 1 EVALUATION OF PSYCHOLOGICAL THEORIES OF MOTION Section 1.1 Review of the Theoretical Literature on Animal Motion in Psychology There is a large body of literature on elicited animal behavior involving locomotion. In an early book on the subject of animal motion, Loeb [1918] discussed motion caused by asymmetries in the surround of the animal. He gave particular attention to asymmetries of the physical characteristics of the environment, such as salinity, temperature, and hydrogen ion concentration. This line of work has since been extended by numerous authors to give rise to the study of tropisms, taxes, and compass reac- tions on a broad scale. For general review chapters, see the textbooks by Denny and Ratner [1970] and by Hinde [1970], as well as the encyclopaedic reference by Grzimek [1977]. There are also works, such as that of Schmidt- Koenig and Keeton [1978] on animal navigation, which deal with specialized subt0pics of the field of animal loco- motion. Loeb's work does not take consideration of goals that the animal may have, or of distant sources of attraction or aversion. In fact, it does not take con- sideration of motivation, in the psychological sense, 4 '- 1) other than immediate motivation due to bodily sensations. By contrast, we are interested in the planning of motion, relative to distant sources of fear and attraction. We are also interested in behavior that arises from the animal itself, rather than that behavior elicited by immediate environmental stimuli. For this reason we will turn elsewhere for a beginning point from which to present our theory. The present work concerns emitted or Operant behavior, rather than elicited behavior. Therefore, we have chosen as a point of reference the theoretical work of Kurt Lewin on the organization of psychological space. In particular, his Principles of Topological Psychology (Lewin [1936]) and Field Theory in Social Science (Lewin [1951]) provide the main theoretical framework which suggested the present approach. In the present paper, we will be investigating the internal image of space and time using a different mathematical formulation than that of Lewin; however, these models find their conceptual roots in Lewin's works mentioned above. Lewin considers that psychological space--that is, space as the animal perceives it, or life-space--depends on the motivation of the animal, as well as upon the objective structure of physical space. (Actually, Lewin takes life-space to be an abstract generalization of the perceptual image of the physical environment, of which the physical aspect--the quasi-physical space—-is only one facet. 6 Since motion is difficult to quantify along the non- physical dimensions of the life-space, we will specialize on the Spacelike- and timelike- coordinates of life- space.) In topological psychology, Lewin discusses how motivations warp the structure of space. The life-space is bounded by those objects which are most distant among those that possess a valance emotionally, for the animal at that particular time. A valance is a positive or negative goal-value attached to an object. Every impor- tant object in the life space has some valance. Goals to be sought-after and objects to be avoided have positive and negative valances respectively. In Lewin's work on topological psychology, the plasticity of space plays a key role, so that objects which are of intense interest appear nearer than they would otherwise be, and objects which are obstructions may appear directly between the actor and the goal. There are many instructive diagrams concerning these matters in Lewin [1936]. Goals and other animals are substantial deter- miners of spatial organization in Lewin's models. See Lewin [1936], Chapter 4. Lewin's conceptual basis is the primary antecedent for the present paper. 7 In his topological psychology, Lewin gave us no clear algorithm or formula for determining what an animal's motion would be in a physical space. Lewin later remedied this ix) his vector psychology. The reader should in particular see Lewin [1951], pages 238- 303. In this work he dealt with what he called his vector psychology. In the vector psychology space is Euclidean and there are assumed to be forces acting in the animal's mind which determine motion by their vector summation. The net force which determines the animal's instantaneous motion is the vectorial summation of forces directed away from each object of negative valence and forces directed toward each object of positive valence. SeeFigureslnl.l and 1.1.2. Lewin is specific that these forces are central forces: that is, that they act in the direction of a line connecting the acting animal with the object which possesses the valence. See Figure 1.1.3. Moreover, motion is assumed to be in the direction of the resulting force and of a velocity monotonic in its strength. What this means is that Lewin's vector psychology predicts motion on the basis of a vector field operating on the animal. Lewin's primary discussion of the vector psychology is contained in Section 4, pages 256-269 of Lewin [1951]. The Specific reference to central fields of force is contained in pages 256-257 of the same work. {goal ‘ \ \ \ \\ FR, . J AVemve Ob‘eci GA // / / / // - / r6! ~%> ------ -o 304' I F, l A; l l l t Aversive. 053“} FIGURE 1.1. 1 Generation of Forces in Lewin's Theory Avert} we O‘U‘CVI /, Goal FIGURE 1.1.2 Summation of Forces in Lewin's Theory 10 FIGURE 1.1.3 Symmetric Force Field of Aversive Object 11 In Lewin's vector psychology an animal can some- times circumnavigate an Object that is undesirable to it in order to get to a goal. An example of the vectorial structure of this problem is shown in Figure 1.1.4. This prOperty Of the theory is based purely upon the vector composition of forces generated by the valences of various objects. The vector psychology is used by personality theorists to explain certain phenomena related to conflict (Hall and Lindsey, 1970 , pages 232-234). In personality theory the spaces are frequently abstract or "meta- phorical," in that they are not ordinary three-dimensional space, but some analogous space in which emotions are assumed to Operate. Lewin [1931], Miller [1944], and Dollard and Miller [1950] have used Lewin's vector psychology to describe four basic types of emotional conflict in a situation of one-dimensional motion. The four basic types Of conflict are approach-approach, approach- avoidance, avoidance-avoidance, and double approach- avoidance. The approach-approach conflict is charac- terized by having two attractive goals. Approach- avoidance conflict is a problem posed by having an attractive goal with an Object to be avoided interposed between the actor and the goal. Avoidance-avoidance conflict occurs in a situation where the actor is forced to make a choice between two unattractive goals. The 12 FA ;7 V Ffl.’ / I / \ \\ F ‘ (r \ \ :5 Goal Sh't pnAmr FIGURE 1. 1.4 Lewinian Trajectory and Forces 13 double approach-avoidance conflict has in each direction of motion an object to be avoided and an attractive object. These types of conflict are summarized concisely in Mahl [1971] and Lewin [1951]. The approach-avoidance conflict is featured in the situation shown in Figure 1.1.5. Here the animal is in a one-dimensional tunnel and there is a goal at the end of the tunnel and a feared object intermediate between the animal and the goal. According to Lewin (e.g., 1951, page 263), the force of repulsion from the object which has negative valence dies—out more quickly with distance than does the force of approach toward the goal object. This results in an equilibrium at an intermediate point as shown at E in Figure 1.1.5. According to the Lewin-Miller theory, the animal will stall at E where the motion has a stable zero of velocity. Small deviations in the motion around the point B will lead the animal to return to the point B. It is not yet experimentally clear whether such equilibria exist for real animals. Lewin [1951, p. 264] says that children in such a situation will waver around the point of equilibrium until one force becomes dominant "as a result of changes in the circumstances or of a decision." Such statements seem to obfuscate the theory rather than amplify it as the constructs employed are not intrinsic to the vector psychology. In any case, an animal in the real world of survival would likely be in more danger, rather than less, if it tended to stall 14 $¥nrn§*£ of' Force A I l I I I l l l l | I I (026‘ I .5 N‘ I Q” I I I I I I I ‘ I i ‘ I I l I (Dirfismce . o 0 j E Fennel God Objcct FIGURE 1.1.5 Approach-Avoidance Conflict in Straight Alley According to Lewinian Theory 15 for long periods of time at fixed locations due merely to a balance of psychological forces. We will discuss this further in the next section. In a one dimensional setting an avoidance- avoidance conflict, as pictured in Figure 1.1.6, is also stable. However, the psychological reality of this model, where the animal is caught in a tunnel between two aversive Objects, is entirely based on the one dimensional constraint on motion imposed by the tunnel. Indeed, in a two dimensional space, as shown in Figure 1.1.7, it seems entirely likely that the animal would escape out the side. In fact this equilibrium is unstable in two dimensions, as shown by the trajectories in Figure 1.1.7. The first order differential equation of the Lewinian motion in Figure 1.1.7 has a saddlepoint at E and so describes an unstable equilibrium. The approach-approach conflict,pictured in Figure l.l.8,is always unstable,even in one dimension a small perturbation in either direction leading to approach to the goal in that direction. The animal, therefore, will be expected eventually to approach one of the goals and move all of the way to that goal. The double approach-avoidance conflict, although seemingly a mere superposition of two approach-avoidance conflicts, is of sufficient complexity that it cannot be analyzed even in the one dimensional case without specific choice of the mathematical force gradients l6 S‘fre My!“ 0% Fa 'ce L0 ' 0 Panel E FCQIQJ Obj ed‘ 0 hand FIGURE 1.1.6 Avoidance-Avoidance Conflict in Straight Alley According to Lewin's Theory 17 Fear!) Pen (0‘ 01:}: c} 0 bjed FIGURE 1.1.7 Two Dimensional Avoidance-Avoidance Conflict in Lewin's Theory 18 9H: “9 *5 31+“: "3 4“ ° ‘- F’o me of- Force ”I I ’96" W“ Io 1; c. .fi “a i <— l -—-) l Goa I E Goa. FIGURE 1.1.8 Approach-Approach Conflict in One Dimensional Lewinian Model 19 involved. Depending on the shape of the approach- tendency and avoidance-tendency curves, the equilibrium may be either stable or unstable in the one dimensional double approach-avoidance conflict. Section 1.2 Failures of the Lewin-Miller Theory in Two Dimensions In the approach-avoidance setting, the Lewin- Miller theory predicts stalling in a one dimensional tunnel. See Figure 1.2.1. This stalling is due to the fact that the avoidance force dies out more quickly than the attractive force. We will now generalize this situa- tion to the one shown in Figure 1.2.2 where we have a cat. trying to get past the dog to its kittens in a two dimensional space. In this case, the cat can circum- navigate the dog to reach the kittens. In fact, according to the Lewin-Miller theory, it should do so providing it can once leave the equilibrium point shown as B in Figure 1.2.3. The equilibrium at E is unstable and leads, in the case of any small perturbation, to trajectories that go toward the kittens. Thus, the animal is not really trapped at E by its own motivations, since it will leave by one of the side trajectories if there is any small deviation from its initial position. On the other hand, this basic configuration can be generalized to cases where, even in two dimensions, the Lewin-Miller theory will lead to stalling. For example, we may consider that, interposed between the cat and its kittens, 20 l I I l l I p--- . (¥- (3 c), E Feat-e01 Mch-I we 05,121 obj “1 FIGURE 1.2.1 Stable Equilibrium in One Dimensional Lewinian Approach-Avoidance Contingency 21 7.. Cat 4D03 K‘HT'SS FIGURE 1.2.2 Configuration of Animals for Two Dimensional Approach-Avoidance Conflict 22 Db KI‘H'Gn: FIGURE 1.2.3 Lewinian Trajectories in Two Dimensional Approach-Avoidance Conflict 23 there is a slit-shaped hole transverse to the direct path to the kittens, as shown in Figure 1.2.4. In this case, if the cat starts within the region subtended by the slit- hole with the kittens as the center (see Figure 1.2.4), the cat will reach a stable equilibrium at E where it is entrapped by its own motivations on the Opposite side of the hole from the kittens. There is ample room for the cat to move safely to its kittens, but the vectorial theory predicts that the cat will not do so. For a real animal, the path shown in Figure 1.2.5 seems more probable: that is, the animal should circumnavigate the barrier regardless of the exact geometrical configuration. In the experimental literature there is con- siderable evidence that even lower animals such as gold- fish can circumnavigate a transverse obstacle. Lorenz [1982, pp. 237-241] discusses this behavior in goldfish. If there is an obstacle between the goldfish and its goal, the goldfish will execute the detour path. There are species-specific and age-specific differences in these behaviors. For example, according to Sholes [1965], an adult domestic chicken will not learn the detour behavior even when guided through the ”correct" path by the experimenter. On the other hand, nine day old chicks will learn detour routing to a goal if they are coached along the way. According to Spigel [1964], turtles will accomplish the detour path behavior by a process that seems to involve much trial and error 24 ‘ t\\\\\\‘ \\\\\v ’ FIGURE 1.2.4 Entrapment in Two Dimensional Lewinian Approach-Avoidance Situation 25 Ki'He ns Shirl: /; “s ., 7 .__A FIGURE 1.2.5 "Probable" Trajectory of Real Cat in Two Dimensional Approach-Avoidance Setting 26 locomotion. In any case, they do not stall at the equilibrium point. Dogs of various breeds and ages vary in their ability to solve detour problems. AS reported by Scott and Fuller [1965, pp. 226-229], only 8 of 203 puppies solved the detour problem on the first test at six weeks of age. The breeds tested were Basenji, Beagle, Cocker, Sheltie, and Fox Terrier. Basenji's far outperformed the other breeds, with 18 out of 44 solving the second test on the first trial. The authors state that adult dogs have no problem whatsoever solving comparable detour problems. Thus there are learning and/or developmental factors Operative in barrier test performance. According to Scott [1958], there seem to be two varieties of behavior with respect to the detour problem. One involves frustration, much vocalization, exploratory behavior, and an occasional solution to the problem. In the other mode, the dog is quiet and the solution is A comprehended in one step and immediately executed there- after. Thus Scott [1958, page 152] Speaks of a primary adaptive response consisting of frustrated exploration, competing with the immediate gestalt solution mode. The survival importance of immediate solution of barrier problems is underlined by the work of Frank and Frank [1982] comparing the behavior of wild timber wolves (Canis lupis lycaon) to the results obtained by Scott and 27 Fuller. They tested four six week old wolf pups in a manner closely replicating the tests of dogs by Scott and Fuller [1965]. The initial solution by wolf pups was much superior to the performance of the domestic dog puppies. The error scores were 6.5 for the wolf pups as compared with 50.0 for the dogs. This result is for a U-shaped barrier where the animal is originally trapped inside of a large U-shaped wall and must reach food on the other Side of the wall to solve the problem. In the test, the animal can see the food through a window in the barrier from its starting position, but can't reach the food. Other tests were also heavily in favor of wolf pups as better solvers of barrier circumnavigation problems than domestic puppies. This is also consistent with the result on the puppies alone. The Basenji's solved the problem better than other domestic dog pups, and the Basenji's are much more recently domesticated. Frank and Frank [1982] again suggest that there are two problem solving modes at work: the primitive trial and error exploration mode and the immediate gestalt "con- templative" mode. In any case, none of the animals except domestic chickens were indefinitely trapped on the wrong side of the barrier by their own motivational system. Next we will consider the avoidance-avoidance conflict of the Lewin-Miller theory as generalized to 28 two dimensions. Here it helps to introduce some mathe- matical sophistication. In particular, we Shall ask the question of whether or not a Lewinian force-field is irrotational. If a central field of force is the same strength in all directions, at an equal distance from the aversive object, the field is irrotational: (1.2-l) curl (Force Vector) = 0 Any summation of irrotational force fields is again irrotational, so that if this property holds for all objects generating an emotional force, then it also holds for all of them together (Arfken, 1966, pp. 49-55). An irrotational field of force presents a particularly simple structure mathematically. The motion will always be represented by a scalar potential field with motion along the path of steepest descent. Thus, if we assume our Lewinian forces to be irrotational, we have the Simple problem of an animal following a gradient of a scalar potential to a relative minimum. This potential function will be called the "avoidance potential" or "attractive potential" in the cases of aversive and attractive objects reSpectively. In the case of the superposition of a large number of such irrotational fields of force, we will call the sum of the potentials the "motivational potential." In algebraic Sign, attractive potentials are counted as negative and repulsive potentials are positive. 29 Consider a cat surrounded by four dogs at the corners of a square centered on the origin (see Figure 1.2.6). Let us assume that the avoidance potential of each dog takes the simple form of one divided by the distance from that dog. In this case, the field of force of each dog is an inverse square field. The potential will have a relative minimum at the origin and the cat will be entrapped there by its own motivations. The region of entrapment is shown in Figure 1.2.7. The equilibrium point of the entrapment is at the origin and any starting position within the cusp-Shaped region shown in Figure 1.2.7 will result in the animal eventually ending up at the origin. In this example, and many that will follow, the mathematical argument leading to the entrapment of the Lewinian animal does not depend on the spatial scale of the configuration of objects to which the animal is responding. If the square in Figure 1.2.6 were four meters on a side, a real animal might indeed become trapped. However, if the square were one hundred meters on a side, entrapment is relatively unlikely. However, the Lewinian treatment of both is the same and leads to the same results. The same argument can be applied to most of the examples which we will discuss. Therefore, the reader is encouraged to envision the scale of diagrams flexibly. In the preSent example, for large Spatial 30 Dog Dog Dog, FIGURE 1.2.6 Stable Equilibrium in Two-Dimensional Avoidance-Avoidance Setting 31 D03 Dog E n+mf+8nt Rea I' an FIGURE 1.2.7 Entrapment Region in Two Dimensional Avoidance-Avoidance Model 32 scale, we would expect the cat to escape via the kind of route Shown in Figure 1.2.8. Again consider the problem of entrapment. Figure 1.2.9 shows a dog which has trapped a cat in the cul-de-sac formed by a curved trench. In this case, given that the potential of the hole is a simple function of distance from the hole, such as its reciprocal, and similarly for the dog, entrapment again is predicted. In this case, the cat is trapped at the equilibrium point shown as E in Figure 1.2.9, according to the Lewin-Miller theory. The mathematical reason for this is shown in Figure 1.2.10, where the composition of forces is analyzed. Indeed, the entrapment region is obtained by dropping perpendiculars from the dog to the curved trench, as shown by the dotted lines in Figure 1.2.9. And again, we would expect that any real animal, at least as long as the dimensions of the trench were large, would escape as Shown in Figure 1.2.11. However, this is not the predic- tion of the Lewin-Miller theory. Therefore, it is seen that the Lewin-Miller theory does not do well in the two dimensional avoidance-avoidance problem either. The use of potentials to represent Lewinian theory is valid only if the fields themselves are irrotational in the sense of Equation (1.2.1). A Lewinian force field can be rotational. In Figure 1.2.12, schematic dogs are shown with (a) rotational and (b) irrotational central fields of force around them. In Figure 1.2.12(a), the 33 Es Dqg Chg 0 - 0 Cal: 0 o Dog J 003 FIGURE 1.2.8 "Probable" Escape Route for Real Cat in Avoidance-Avoidance Model 34 FIGURE 1.2.9 Cat Entrapped in a Cul-de-sac By a Dog--Lewinian Model 35 FIGURE 1.2.10 Summation of Psychological Forces for the Cat in the Cul-de-sac 36 Dog Cat FIGURE 1.2.11 "Probable" Escape Route for Real Cat From the Cul-de-sac Es 37 curl F # 0 (a) FIGURE 1.2.12 Rotational (a) and Irrotational (b) Fields of Force About a Predator 38 force is stronger in front of the dog than behind the dog, and thus is not equal in all directions. As might be expected from analogous phenomena in physics, such a rotational field may give rise to circula- tory motion, that is, motion where the animal goes around and around a path indefinitely. Consider again four dogs surrounding a cat as shown in Figure 1.2.13. Let each dog's field of force be primarily in the forward direction and have each dog face counterclockwise around the square formed by the dogs. In this Situation of asymmetric forces, the cat can be entrapped in a looped path, such as the trajectory shown as T, where it will circulate indefinitely. Thus, we certainly do not save the Lewin- Miller theory by allowing the force fields to be radially asymmetric and hence rotational. In fact, this model shows an utterly nonadaptive behavior as a very simple consequence of an asymmetric field of force. Certainly, if the scale of the square were large, we would in reality expect the animal to escape along some path such as BS in Figure 1.2.13. However, according to the Lewin-Miller theory, it cannot escape from the circulatory path shown as T. Another problem with Lewin's theory relates to the existence of runways. A great many animals have run- ways in areas of cover, such as grasslands or forest. These are cleared pathways that they follow to get from one place to another. A simple set of runways is 39 RES \ Q T \\ \ # II," " _____\_ W FIGURE 1.2.13 Circulatory Paths Can Occur in a Rotational Lewinian Force Field 40 depicted in Figure 1.2.14. Envision a forest with low cover for the animals and a predator, such as an owl sitting in a tree, at the place marked in the diagram. The obvious interpretation of Lewin concerning such a case is that the psychological force vector is projected on the direction of each runway and the runway chosen at any choice point is that with the largest component of force in the direction away from the animal down the run- way. As shown in Figure 1.2.14, this process can result in the animal taking a longer, and perhaps more dangerous, path than it would have taken if it followed some other rule. In the theory which we will present, each of the problems illuminated in this section will be resolved by making a mathematical model of the animal's decision process that differentiates more clearly between goals and the surrounding field of aversions. 41 Predator 0 604’ F6- m . Start FIGURE 1.2.14 Deflection of an Animal's Path by a Lewinian Force Field Can Cause Unnecessary Detours in a Two Dimensional Runway Setting Ch CHAPTER 2 OPTIMAL MOTION PLANNING Section 2.1 The Planning Process Motion during instrumental behavior results from two cognitive processes: perception and action. According to classic behavioristic theories both processes are very crude. Perception consists of scanning the environment for positive objects which satisfy needs and negative objects which produce damage or pain. Action consists of either running from the object of greatest fear or moving toward the object of greatest need. Lewin greatly elaborated the model of perception. The animal is assumed to create an internal model of the environment which preserves critical features of the geometry such as angles and distance. However, Lewin's model of action is still crude, the animal moves in a direction determined by immediate needs and fears. There is no anticipation of changes in needs of fears. The theory to be developed uses Lewin's model of perception, but elaborates the model of action by intro- ducing planning. Planning is introduced at two levels: goal choice and optimal movement. The model for goal choice is contemporary decision theory: 42 43 l. The animal generates a list of potential goals 2. The animal evaluates each goal in terms of the costs and benefits which derive from the attainment of that goal 3. The animal chooses the Optimal goal, be that the best available good or the least of evils. The focus of the present paper is on the costs incurred in achieving a goal, namely those costs which arise from the motion required to reach the goal. There are two kinds of costs associated with motion. First, to the extent that the goal is urgent, long paths to the goal are more costly than Short ones. Second, travel often requires exposure to danger, eSpecially attack from nearby predators. The Optimal path is that which minimizes the exposure to fear while keeping the length within reasonable limits. The focus of the present theory is on the planning of motion. Thus the goal will always be taken as given. However, planning of motion enters into the goal selection process; the animal cannot know the cost of a goal until the optimal path to that goal has been planned. If the Optimal path to a goal is too expensive, then that goal may be rejected. In this case, the optimal path will never be followed. The theory will be presented in three parts: discussion of how to minimize exposure to danger, modifi— cation of the Optimization criterion to take account of 44 the urgency of the goal and consideration of how to compute the optimal path. Given the Lewin model of perception, the animal can minimize exposure to danger by minimizing anti- cipated exposure to fear. This leads to defining the opti- mal path as one which minimizes the integral of anticipated fear along the path. The urgency of the goal can be taken into account by modifying the fear function which is inte- grated and then minimizing the modified integral. Three methods of computing the optimal path will be discussed: the Moore algorithm, dynamic programming, and calculus of variations. The Moore algorithm is important because there is a neural network which could carry out the Moore algorithm very efficiently. Calculus of variations is important because it is the mathematical method of choice for obtain- ing the predictions of the theory. Dynamic programming provides the bridge between these two formalisms. Suppose that a wild animal lives in a field of fear created by the presence of predators. For example, Figure 2.1.1 shows the field of fear due to two predators in terms of contours of equal fear (iso-aversion contours). We symbolize the value of the fear at a point (x,y) in the plane as (2.1.1) f(x,y) = value of fear at point (x,y) This fear is an anticipated fear. The animal associates the value of f(x,y) with the fear that would apply if the animal were at the point (x,y). It needn't be at (x,y) / . / I. A OPIeJmL-or FIGURE 2.1.1 Iso-Aversion Contours Generated by Two Predators 46 to do so, and in fact usually will not be. We will refer to f(x,y) as the fear function. In considering various paths through a field of fear, we will define optimal paths. Optimality will be defined in terms of exposure to fear. Exposure to fear will be quantified as the integral of the fear function over the path of the animal with respect to time: T (2.1.2) J -..-: exposure 4'0 I'm? = $003) 4*- 0 Here the integral extends over the entire path and is the continuous analog of the following sum: (7-4.3) J = .cxposure. +0 {lear- g 2 £IMJ)At In the case of discrete pathways, such as in a problem involving runways in a forest, the sum will suffice. In fact, we will have the following sum as the exposure to fear: (2.1.4) I II M J.” D :1 Fear iS constant on each leg of the path, or may be con— sidered to be so by taking averages, so that fi represents th the (average) fear level on the i leg of the path. In Equation 2.1.4, the fear on the ith leg of the path is 47 weighted by the time ATi taken to traverse this leg of the path. The expressions 2.1.2, 2.1.3, and 2.1.4 all express exposure to fear which we symbolize by J. In each case, we have a time weighted sum of fear along the path. For a given beginning and ending point an optimal path will be a path which minimizes the exposure to fear. Thus in the continuous case we pose the minimization problem T (2.1.5) Mm J ='- M‘m {'(XL+),;(+))oIt 0 The minimization is to be carried out with respect to all paths with the prescribed beginning and ending points, i.e., P1 = starting point - (x(0), y(0)) and P2 = ending point = (x(T), y(T)). Section 2.2 Time Penalty In this section we will consider the time dimen- sion of the boundary of the life space and the corre- sponding time horizon. 'This will result in some adjust- ments to the value of the goal and to the fear function f(x,y). For example, if the planning is assumed to extend only to the point where the goal is reached and the animal has a fear fr for the reward's safety, then we may write: 48 £093) .2: {3. (by) -I- ac,- (2.2.1) This formula expresses total fear at (x,y) in terms of the anticipated fear of the moving animal for itself f1 and the fear for the reward fr' In other cases the animal may have some independently determined priority for getting to the goal in a timely manner. For example, it may also put a premium on time Spent getting to the present goal because it wishes to explore other goals. Then we will have: '7 (2.2.2) J -.-.- S [£(X)3)+¥r] 0H. 4' WPT Since the time weight wp expressing this premium on time is in the same units as fear, we can rearrange our expres— sion to get (2.2.3) 3c (’93) ‘-"- 1F. (’93) + Fr 7"”? At this point, we note that the general form of the integrand can be taken to be (2.2.4) f(bb) = £095” 4" ‘Fo where fo is a lumped constant expressing the net time premium in units of fear. We will call the term fo the 49 net goal fear Since it usually consists of fr and some other terms. For example, in the above discussion (2.2.5) '90 -'= {if + W.) Section 2.3 The Moore Algorithm and Dynamic Programming We will propose a method of solution of the minimization problems for finding the minimum path dis- cussed in Section 2.1 which is based on purely combina- torial process. The reason for our choice is that the combinatorial method proposed can be implemented approxi- mately in neural networks. Our solution method will be a variety of dynamic programming (see Bellman, 1957 and 1961: Larson and Casti, 1978; Wagner, 1975). To make our method clear, we will first present a variant known as the Moore Algorithm (Moore, 1957). The Moore algorithm allows us to find the minimum path through a network (formally a digraph) given the costs of getting from one node to another. At this point we should introduce a few terms from discrete optimization theory: a network is a group of points, called nodes, con- nected by means of transition from one node to another, called links. In optimization theory, each link is associated with the BREE 0f getting from its starting node to its terminal node. A path going from node A to 50 node B is a series of links, starting at A and concatena- ting in such a fashion that the end of each link is the start of the next, with the end of the last link in the chain being B. A network is said to be connected if there is some path in it leading from any node to any other. That is, for any two nodes A and B, there is a path which begins at A and ends at B. In the language of optimiza- tion, our concept of fear exposure is the analog of cost. Thus the problem is that of minimizing the total cost, (2.3.1) 3' : Z: 333' P‘J’k th node is the starting point of a particular h where the i link in the network and the jt node is its terminal point. The first value of i is the start node and the last value of j is the goal. Jij is the cost of getting from node i to node j. We will, in addition, assume that the i's and j's chosen represent a path (see Figure 2.3.1). The Moore Algorithm is best illustrated by pro- viding a schematic computer program for its implementation. In the following we will have the elemental costs of links in the array J(N,M) with infinity assigned if the points N and M are not directly connected. The array CUM will contain the provisional minimum cost of getting from the START to the node I. That is, the provisional cost of getting to node I will be CUM (I). The back node, being the best node to come from to reach node N, will be given 51 FIGURE 2.3.1 Example of a Network. The Series of Nodes l, 3, 4, 5 Constitutes a Path J13: J3“, Jus 52 by BACK (N). The FRONTIER will be a set of nodes that are being evaluated as jumping off places for the con- tinuation of the trajectories. The computer program, in purely symbolic form, follows: INITIALIZATION STEP: Make the FRONTIER set contain only FORWARD the START. Set BACK (START) = 0. Set CUM (START) = 0. Set CUM (I) = infinity or a very big number for all other nodes. RECURSION: DO UNTIL FRONTIER SET EMPTY: Take any node in the FRONTIER. Call it I. For each other node K, for which J(I,K) is less than infinity, compute CUM (I) + J(I,K). Compare this to CUM (K). If it is greater than CUM (K), then go to the next test K; otherwise assign CUM (K) = CUM (I) + J(I,K), add node K to the FRONTIER and assign BACK (K) = I. When you have evaluated all prospective nodes K for the node I, then remove I from the FRONTIER. Loop back. BACKWARD RECURSION: To get minimum paths the following END. will suffice: choose the desired GOAL node, G. Then compute the sequence: G, BACK (G), BACK(BACK (6)), BACK(BACK(BACK (G))); etc. until the START node is reached. Then the reverse sequence of nodes is optimal as a path to get from the START to the GOAL. 53 A computer program implementing the Moore Algorithm in BASIC is given in Appendix A. In the case that the process of path selection can be made into a one way process consisting of STAGES, however arbitrarily defined, then dynamic programming can be used. In dynamic programming we don't need to consider the FRONTIER as a dynamic entity and simply use the stages as frontiers, one at a time, without the possibility of a node re-entering the FRONTIER. This approach is of limited value in our calculations since Situations such as are represented in Figure 2.3.2 can exist which make it impossible. However, stages were used in the neural network of Dolson and Bell [1981] which can be used as the basis for a neural network solution of the minimum path problem. In the Dolson-Bell solution for the discrete case of a finite network, all link costs Jik are made multiples of some common increment A. That is, all link costs were approximated as integers. In this case, we may define a STAGE as being all points n units out from the origin, i.e., an equidistant set as measured in cost from the START. For a Stage defined this way, there is no possi- bility of nodes being reentered. This greatly simplifies neural networks, but would make a great burden on a sequential machine. The only reason why it is practical at all is that neural networks are parallel processors 54 Goal Notes: 1.) Cost shown by links. 2.) Node (*) enters the FRONTIER at computation stage 1 and re-enters at Stage 4. FIGURE 2.3.2 Re-entry of Node into the FRONTIER is Caused by Certain Combinations of Link Costs 55 so that all paths may be computed in this manner con- currently (See Dolson and Bell, 1981). Section 2.4 Computation and Analysis in the Continuous Case The Moore Algorithm and dynamic programming can both be made continuous in a variety of ways. Bellman [1961] shows how to make the dynamic programming method work in the continuous case in a number of ways. Some of these same methods are explained in more detail in other works (Bellman, 1957; Larson and Casti, 1978). These methods inherently make use of continuity of some of the functions involved and employ interpolative techniques. Moreover, they are highly dependent on the digital computer for both numerical and combinatorial features of the algorithms. We will use one such program employing continuous dynamic programming to get solutions to our continuous case problem of minimizing paths in the sense of the integral ITI (2-4-1) Min J :: Min 19(0510112 0 for the purpose of illustrating the consequences of our theory. The relevant BASIC computer program is given in Appendix A. However, most of the time, for the purposes of illustrating the consequences of our theory, the calculus 56 of variations will be used to find properties of the paths predicted by Equation 2.4.1. The connection between the dynamic programming type of solution and the calculus of variations is given by Bellman [1961, Chapter 4]. For the purpose of suggesting a neural solution process, we will again revert to the Moore Algorithm. In this case we will proceed by making the network so fine that it approximates the continuous case. The resulting approximation is never exceedingly close, but may be good enough as an approximation to be used by animals in their everyday activity planning. CHAPTER 3 MATHEMATICAL EXAMINATION OF THE PLANNING THEORY: TRAJECTORIES Section 3.1 The Variational Form of the Trajectories for Static Fear and Maximum Velocity We will now discuss the case of motion in a Static field of fear at maximum velocity. In this case the paths can easily be derived by the calculus of variations (Bellman, 1961, Chapter IV; Arthurs, 1975), providing that the fear fields and trajectories obey certain conditions of analytic regularity. First of all, the assumption that animals would move at maximum velocity in a field of fear Should be justified. Drawing data from many empirical sources (Brody, 1964; Oron et. al., 1981; Sonne and Galbo, 1980; Chassin et. al., 1976; Taylor et. a1, 1970; and Taylor et. al., 1974), we can state with some confidence that the energy needed to move a specified distance decreases with running speed for most four-legged animals. There- fore, it is not unreasonable to assume that an animal in a state of fear would run at its maximum velocity, pro- viding its fear level is independent of its Speed (the contrary assumption will be dealt with in the next sec- tion). The reason for maximum velocity is immediately 57 58 apparent in terms of the assumption (Section 2.1) that the animal is choosing trajectories so as to minimize its exposure to fear, given by the integral: 1- (3.1.1) J = §f(x,5)4t 0 where x(t) and y(t) describe the trajectory in Space in terms of the time t. In fact, if the trajectory is assumed to be fixed in Spatial form, then the fear expo- sure will be inversely proportional to velocity. Thus both energy expenditure considerations and fear exposure mitigate in the direction of increased velocity. As expressed by Equation 3.1.1, the fear minimiza- tion problem is not yet cast in a way that the calculus of variations can be used directly to determine the spatial path. This is because the endpoint in time, T, is not fixed in advance, although the spatial endpoints (x1,y1) = P1 = the starting position, and (xz,y2) = P2 = the position of the goal, are fixed. For this reason, we will re-parametrize the trajectory problem by introducing a "dummy" parameter, S, which is zero at P1 and equal to l at P2. Then the problem becomes that of minimizing the integral: L (3.1.2) J- : L .Foqs)’ 3(0). ii; . A: 59 The last equation is gotten from Equation 3.1.1 by a direct change of variable. Now, due to the fact that the animal is assumed to run at maximum velocity, which is a constant, we can express dt in terms of the length of arc traversed in infinitesimal time as follows: J.(pa4{ (enjdl) VMFIX (3.1.3) At = We can then exploit the relationship between arc length and the differentials of x(s) and y(s) to eliminate t entirely from our model formulation 3.1.2. We proceed as follows: oi (paJL (“th) = ”1’ 4. J3“ : \[vx'h‘f—I-g'“)... d5 (3.1.4) Then combining (3.1.3) and (3.1.4), we obtain: ”'15) a: r. l' ' [X’Mt-I- 3’8)" ‘ VMAx This expression can be substituted into (3.1.2) to yield the following: 1 (3.1.6) J = -—l-- IIXB); 3‘0) x’IcILI-g’u)‘ A: VMAX O 60 as the exposure to fear on the path between P; and P2. Then, omitting the constant v X' we have a classical MA variational problem of minimization: i __ (3.1.7) Min (10(0), fl“))'\(x7$)2+*3'1‘)"-JS 0 with the two independent variables x(s) and y(s) being varied between the fixed endpoints, (3-1-3) Pak: (0) X'J‘Z’I) anal Pz“ =(1,X.)g..) 1 Here it is understood that the endpoints are fixed, but that the shape of the path can be varied between them to minimize the expression in (3.1.7) In the variational problem (3.1.7), we have two dependent variables, x(s) and y(s). Therefore, in our calculus of variations solution, there will be two Euler equations. These are as follows: 3 J i I I- )1} (3.1.9) Jung“? 3:; =- 1-5{§(X25) x. 0" “a and (I iii-(w) 3', (‘X"‘*’J"‘i (3.1.10) (fluff a3 =1: m.or (uh- 61 The symmetry of these equations, with respect to inter- .change of x and y, guarantees that these equations and equations that follow are identical with x and y inter- changed throughout. Therefore, we will proceed to solve 3.1.9 and assume that the solution of 3.1.10 is the same, with x and y interchanged. Writing out the derivative indicated within the braces on the righthand side of 3.1.9, we have: KI fix’)‘+(3')‘ (3.1.11) 5%}, \fiX'V-I-(a'); = Multiplying 3.1.11 by f and differentiating by s to get the entire expression on the righthand side of 3.1.9, we obtain: 4 x’ac _ f(x’)‘ “if + I; (fl—K’I‘Wy)‘ WEE-Hy)" 3" (3.1.12) , i q I / II I I x ——1X x3 33- (3) U} )C ‘¥ —:5 “ a I t I I. ”4 IIx’IWa'I‘ 3' [W Hall The lefthand side of equation 3.1.9 can be expressed as follows: (x01. (,1)‘ fl {RUN (3'); 3x (3.1.13) 62 Substituting 3.1.13 and 3.1.12 into Equation 3.1.9, we get, with very minimal rearrangement, the following: (3!):- .—a-i * xlal ' 9—5 f —— e a WI)L+/fl.)t OX J(x')&+(no)1' 3' — ou- (3.1.14) 4’- (21’)1 x” — X 9'5" [(X’)‘+ (JI)L]3/2. Dividing 3.1.14 by y'f, and changing signs throughout, we obtain: I I 54 8 _I 34'- x I L. . . _. (WP-:43? I. 03, [(7’1‘+/9')’ 1° a). (3.1.15) . I I, x/a/I _ 5' x L- (XI); + (7,)1J 3/1. Now we introduce several simplifications: these are based on the following well known identities. Namely we have the following substitutions, I 2f. = 32.1.. 1- (3.1.16) a): ox 9) {— <33 a5 wI x' _ c9 (3.1.17) ’Tx : _ 2 WiUI’d-(n'y' u-I 63 ’ I (3.1.18) (,4, = _ ’9 ‘ ) uy ____ K {3012* [3,); fixa)1+(ao)z where the T vector is the unit tangent to the trajectory, in the direction of motion, and the U vector is the unit normal to the path. Moreover, the expression on the righthand side of 3.1.15 is in fact the curvature of the path of motion, in parametric form: II, I” *2 —a" (3.1.19) K, = 3/2. [(x’)‘+ (3’)? Therefore, we can write the following vectorial equation: (3.1-20) 5‘: I'X'GMJ (Lac):y-6na((0uf) using these Substitutions in 3.1.15. The x between T and the gradient of log f is the vector cross-product, and the dot between U and gradient of log f is the vector dot product (scalar product). This result can be summarized much more Simply as follows: the curvature of the path is the perpendicular projection of the gradient of the logarithm of f: (3.1.21) K : Pros; Vin? 64 That is, if the gradient of the logarithm of the fear function is obtained, then its component perpendicular to the path of motion is the curvature vector of the path, defined canonically as the reciprocal of the radius of curvature times a unit vector pointing in the direction of the center of curvature of the path. Our other varia- tional equation, 3.1.10, yields exactly the same result under the corresponding operations with x and y inter- changed. Therefore, it yields no additional information as to the nature of the path. Equation 3.1.21 is, in highly compact symbolic form, the equation of the path of the animal predicted by our model. In this form it is not too useful for obtaining the path (for example, by integration), but it is very useful for deducing the con- sequences of our theory of animal motion at constant velocity in a field of fear. First of all, we note that the curvature vector K points in the direction of the center of curvature and has a magnitude equal to the rate of curvature of the path. Thus, the path curves toward the upward gradient of fear. That is, since fear generally increases with closeness to the feared object, the path generally curves toward the feared object. See Figure 3.1.1 for an illustration of these concepts and relationships. The interpretation of Equation 3.1.21 is very straightforward in many respects. Since only the derivative of the logarithm of the fear appears in the equation, we can at once observe that this 65 Goal uK Fear-ea) Obj“. Shrg/ FIGURE 3.1.1 Variational Path, Showing Vectors U. T. K, and V£nf 66 equation expresses the relationships of fear in a unitless manner; that is, multiplying fear by a constant through- out does not make any difference. In other words, the level of fear is determined only up to a multiplicative constant by our model. In contrast to Lewin's theory discussed previously, Equation 3.1.21 implies that the animal that is running in a field of fear turns toward the feared object, rather than being pushed away from it. At first this might seem counterintuitive. However, what it means in terms of trajectories is rather direct: the animal must start out its path in a direction deflected from the source of fear and then curve around the source. Another thing we can investigate directly by means of Equation 3.1.21 is the effect of adding a con- stant fear for the goal (ex., for the kittens) to the fear the cat experiences for itself. Let fo be a con- stant fear added to the Spatially dependent fear f1(x,y). In other words, let the total fear be (3.1.22) (L (x,y) = f11((105)-'(- ‘90 Then the gradient of the logarithm of the fear will be reduced in magnitude: Grlwl FIU‘JQ <3.1.23> Grad AIIIIXIWM .1 13.3“ I. 67 The more the constant fear fo, the smaller the gradient, and hence the smaller the curvature of the path, according to Equation 3.1.21. In the limit, as the constant fear becomes very large, the curve becomes a straight line (its curvature is zero). See Figure 3.1.2. Later on in this section we will provide an explicit computational example of this flattening of the trajectory caused by the addition of a constant fear term. Although Equation 3.1.21 has been derived under general assumptions as the solution of the minimization problem in Equation 3.1.7, it is not convenient for com- putational purposes. It is similarly unsuited for detailed analytic investigation of solutions. For these two purposes we will rewrite Equation 3.1.7 in polar coordinates, assuming that the feared object is at the origin. In this manner we will not have to worry about Singular derivatives of the coordinate transformation at the origin, since the running animal will never actually go through the origin. Noting that the increment of path length can be written as follows, (3.1.24) 0H = \rx’a)‘+3'(s)‘ 04$ it can also be written in polar coordinates according to (3.1.25) AL 3 ‘( I+ r1 é: Ar 68 5)th 0 61ml FIGURE 3.1.2 Increasing Fear for the Goal Flattens an Animal's Path 11‘] 69 where we have taken r as the independent variable and theta as the dependent variable. In the case of a radially symmetric fear function, (3.1.26) {3 -..-. f(r) = {lb/TWIN? we can rewrite the basic minimization problem, 3.1.7, as the following minimization problem: [L (3.1.27) Min J = Iflr)f/.+ r‘é‘ J!" '1 . 616 out)!“ 6 = 7; We may have problems with the endpoints r; and r; in that the trajectory may pass through the same radial distance from the origin multiple times. In this case, we will simply splice pieces of curve together, as analytic splines, so that the trajectory remains an extremal of the variational problem (first derivative continuous). The Euler equation for 3.1.27, with theta as the dependent variable, takes the following very simple form: (3.1.28) .2. (Hr) [Hr-‘9' -_-. Coast. be 70 (since theta does not appear explicitly in the integrand). Doing the indicated differentiation, we get the following: {:(r) (“9. (3.1.29) e -.-.: Coast \f1-I-r‘étfi Solving for 6 and integrating formally, we obtain, r C.Clr r (Fr: - c: (3.1.30) 90') : expressing theta as an integral including in its expres- sion the fear function f(r). For the simple case, I (3.1.31) f-(VI = '7' + 1‘30 where the fear is dependent on r and contains an arbitrary static fear f0, such as a fear for the goal-object, we obtain a family of curves depending on fo, the static fear, which are illustrated in Figure 3.1.3 and in Figure 3.1.4. These are exact curves based on formal calcula- tions much facilitated by the Simple forms of Equation 3.1.30 and Equation 3.1.31. These curves clearly illus- trate the effect of a static fear, such as a cat's fear for its kittens, in flattening the trajectories. The curves, in this case, are given by the functions that 71 53.3... :‘\\\>. E... “1...... .I. .L... :: 1.3 FIGURE 3 Computed Optimal Paths Show Flattening With Increasing Fear for the Goal 72 n+3 l—r aJu = o ) “U f It ,/ FIGURE 3.1.4 Computed Optimal Paths 73 follow, which are integrals 3.1.30 for the substitution in 3.1.31: .( (a-e.) (10° =0) (3.1.32) r‘ : I"o 8 c"—-1 {—0 [1 + 6 cos _____.(‘:"' ((9-9,):I (AW) (3.1.33) f(9) = The constants 80 and alpha in Equation 3.1.32 are constants of integration, as is re. These constants must be deter- mined by the endpoint conditions. In 3.1.33, c and 60 are the constants to be determined by the endpoint conditions. The boundary conditions consisting of the requirement that the trajectory must go through the Specified endpoints, the first being the point where the motion begins and where the planning is carried out mentally by the animal and the second being the position of the goal, must be satisfied by the solution Specified by choice of these constants of integration. This choice represents a com- plex computational process, involves solving transcen- dental equations, and is best left to the computer. The results in Figures 3.1.3 and 3.1.4 were determined by exact computation by means of one of the computer programs in Appendix A. In very few cases is the integral 3.1.30 74 solvable in quadratures. In most cases, it must be inte- grated numerically. However, it does have the advantage of reducing the problem to a one-variable integration. Let us now turn to the question of whether or not we have a true absolute minimum when we solve the Euler differential equation (Solution of 3.1.21 or of the integral 3.1.30, for example). When we solve a minimiza- tion problem such as 3.1.27: (3.1.34) MI?) 3- : Min I {:(r) fi+r‘é: Jr- by means of the Euler equation we are not guaranteed a minimum, but may instead be a maximum or some complex "saddle" condition. For the purpose of determining that we have a local or relative minimum, we must use the Legendre conditions (Courant and Hilbert, 1937, pages 214- 216). Given the function of r, 8, and 6: (3.1.35) F“) 9)é) == IVE} [I‘W‘éz we must prove that, 51F 36‘ (3.1.36) while 75 l 1 L 2. (3.1.37) 3:.EE. EL_E:L -—» ( B I: L) E349 a") babe However, differentiating 3.1.35 twice gives the following result: .— = I10“) ‘- (3.1.38) , r+r'- .2. I. 1 r (3.1.39) 15.. :-_ ii”) .1‘3/1 >0 be" ['3' "9 J (1M F(r)>o, (“>0 Moreover, differentiating 3.1.35 by theta gives zero, so the other partial derivatives are zero as follows: a‘F as 39' 0! 1D 2: C? . 31F (3.1.40) :0 % fitzo and CD a We can conclude that in the region of the plane excluding the origin for radially symmetric fear functions, f(r), the condition for a strong relative minimum is satisfied as long as fear is greater than zero throughout the 76 region of the solution: (3.1.41) (Hr) )0, In 4 «jun conB‘InI'aJ Q CO’MJ"[0H. If net fear function is zero in the region of the trajec- tory, it is easy to verify heuristically that the trajec- tory is also indeterminant in this region (see Figure 3.1.5). This occurs because of the integrand in the varia- tional problem 3.1.7 or 3.1.34 will be zero identically, regardless of the choice of path in such a zero-fear region. Of course, a positive net goal fear will obviate this condition and will guarantee that the solution is always a relative minimum. These results carry over to the general case 3.1.7 as well, but the algebraic manipu- lations become too unwieldy to present here. A word of caution is appropriate here about calculus of variations terminology; a "strong minimum" (guaranteed by the Legendre conditions) is a relative minimum against all paths sufficiently close to the chosen path, but does not guarantee an absolute minimum against all other choices, as is required by our theory. Usually, if we use the cal- culus of variations, we must make subsidiary arguments to establish an absolute minimum. The crucial matter in such a case, for example, may be to sort-out two alternative minima, as illustrated in Figure 3.1.6. If both paths are absolute minima, then an arbitrary choice can be taken. 77 flornj):=49 4:C&i*7?C> CFOG( 8+Irt FIGURE 3.1.5 An Arbitrary Path Occurs in Regions of Zero Total Fear 78 Minimum VA’IA 1 Start Dog f Goal A4741numfli FAIA CL FIGURE 3.1.6 There Can be Two or More Minimal Paths I 79 In another case we may have two minima, one of which is absolute and the other of which is relative, as in Figure 3.1.7. In this case, the tOp curve is a relative minimum and the bottom curve is an absolute minimum. Such a case can be sorted-out only by explicitly computing the appro- priate integral, such as 3.1.7, and making an actual quantitative comparison of the integral for the two paths. In other cases where we put two dogs in the plane or otherwise make the topography of fear complex, there will be many multiple trajectories all of which are relative minima and only one of which is the absolute minimum. In such a case neither the Euler equation nor the Legendre conditions tell us which is the absolute minimum and we must use other techniques such as compari- son of integrands or actual computation of the integrals along the candidate paths to determine which is the true minimum. Our theory of the behavior of animals in a field of fear, however, assumes that an animal seeks the true minimum in planning its trajectory. The Dynamic Programming solution, which we shall prOpose as the neural solution, has the property of yielding an absolute minimum directly which the calculus of variations does not. There is another problem hidden in the calculus of variations as a method of solution. Implicit in its formulation, the calculus of variations allows only such alternate paths as have a continuous tangent. This means that paths with "kinks" are not considered. With this #0 09 S‘I’Af‘t L CroaI FIGURE 3.1.7 There Can be an Absolute Minimum Path and Relative Minima 81 remark we conclude our discussion of the mathematics of the model which supposes minimum exposure to fear in the case of constant-velocity motion in the plane. Section 3.2 Summary of a Variable Velocity Case There may be cases where an animal would vary its velocity. Consider the energy needed for locomotion. Chassin et al. [1976], Taylor et. a1. [1970], and Taylor et al. [19741 assert that the rate of energy consumption for locomotion is a linear function of velocity with a zero-velocity intercept higher than the resting metabolism: J8 C,-I-C..\/ (motion) (3.2.1) —v—- =3 A't Co (rcg‘I'ILIJ) with co < c1. Thus the energy needed to traverse a path is given by, fl" _ . C c v elf (3.2.2) 8 — ( "" " > 0 In the trajectory problem which follows, we will intro- duce a constant u, which expresses the proportionality of the cost associated with energy expenditure to the cost of fear exposure. Thus we will be seeking a minimum of 82 (3.2.3) 347448 Since we are considering variable velocity, we will also consider the dependence of fear on velocity at each point along the path. In particular, we will consider the case where the fear function is given by: ,C — (L I“) H ) f (3.2.4) (x,y) —- 4"] I XJj + 0 In this expression the first term expresses the fear of the moving animal for its own safety and the second term Y has been is the net goal fear. The expression b + v chosen for the model as a function that can be made to rise very Sharply from a minimum at zero velocity. This choice is consistent with the fact that most predators have much greater visual sensitivity for fast moving objects than for objects that move slowly (Grzimek, 1977, p. 97; Suthers and Gallant, 1973, p. 340; Braitenberg, 1977, Chapter 7; Thompson, 1975, p. 205-213; Wallace, 1973, p. 130; and Clements and Dunstone, 1984). Thus we will expect the value of gamma to be greater than zero. For gamma having the value of 2 the graph of fear versus velocity is shown in Figure 3.2.1, assuming a constant value of x and y. Reparametrizing in terms of s which ranges from zero to one, as in the last section, we obtain: 83 ”\I:EM1(' I I I / I / / I ,x’ I / I I I I | I” H.014) : II. I me FIGURE 3.2.1 Fear as a Function of Velocity at a Fixed Position <9 Fear ‘\