CHOICE OF PERFORMANCE mnex FOR THE
0mm CONTROL or MACROECONDMC SYSTEMS ,

Thesis for me Dégree mm o.
MICHIGAN STATE UNIVERSITY
JONG - GOO PARK
1973

. __ 4...._—‘~

 

 

This is to certify that the

thesis entitled

CHOICE OF PERFORMANCE INDEX
FOR THE OPTIMAL CONTROL
OF MACROECONOMIC SYSTEMS

presented by
Jong-goo Park

has been accepted towards fulfillment
of the requirements for

Ph. D. degree in Economics

 

 

Major professor

0-7639

 

 

 

 

basic r:
Problem:
to anal:
the pr0<
garding

Specifie

Performa
when a C

for whie

iRVerSe
FTOQeSS
state Va
formanCe
Case whe

ABSTRACT
CHOICE OF PERFORMANCE INDEX
FOR THE OPTIMAL CONTROL
OF MACROECONOMIC SYSTEMS
By

Jong-goo Park

The specification of a performance criterion is one of the
basic requirements for the formal analysis of quantitative decision
problems. In macroeconomic policy problems, in particular, in order
to analyze the quantitative impact of a given set of policy decisions,
the procedure should be based on the policy maker's preference re-
garding the developments in his economy. Such preferences are
specified by a performance index in the analysis.

The purpose of this thesis is to analyze the selection of the
performance index by solving the inverse optimal control problem.
When a closed-100p system with a known control policy is given, the
inverse optimal control problem is to determine performance indices
for which the given control rule is optimum.

A sufficient condition is developed for the solution of the
inverse problem for a linear discrete-time multi-control regulatory
process (the main concern of the regulator problem is to keep the
state variables near a fixed target zero), with a quadratic per-
formance index. An explicit solution is obtained for a Special
case where the performance functional does not contain the control

variables, and the result is further extended to a linear tracking

 

problc‘
the st

adapte:

unique
are fa:
conditi
inverse

tion f0

perform.
stabili;
Whose a:
from the
Planning
the dEte
3031 var
back Con
1
can be m.
the Humbe
deMonstrz
80613 in

the mEtI'IC

 

policy mg

 

Jong-goo Park

problem (an important feature of the tracking problem is to make
the state variables track the changing targets), which can be easily
adapted to the macroeconomic regulation problems.

It is shown that the solution of the inverse problem is not
unique in general, which implies that the optimal control policies
are fairly robust against different performance indices under the
conditions stated above. Also, it is found that the solution of the
inverse problem for a linear regulator holds true without any modifica-
tion for the linear tracking problem.

Finally, the developed techniques for the selection of the
performance criteria are illustrated through a macroeconomic
stabilization policy model with a quadratic performance functional
whose arguments consist of the deviations of the policy goal variables
from their target values during the discrete time period of a finite
planning horizon. The illustration highlights three main points:
the determination of the steady-state values of the trajectories of
goal variables which are under control, the computation of the feed-
back control policy, and the construction of a performance index.

It is shown that the number of the policy goal variables that
can be made to attain the prescribed steady-state values is equal to
the number of the control variables in the model. It is also
demonstrated that the relative weights given to the competing policy
goals in the performance index can be quantitatively determined by
the method of the inverse optimal control problem for a dynamic

policy model.

CHOICE OF PERFORMANCE INDEX
FOR THE OPTIMAL CONTROL

OF MACROECONOMIC SYSTEMS
By

Jong-goo Park

A THESIS
Submitted to
Michigan State University

in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY
Department of Economics

1973

tion res:
so many
the most
me. My
Dr. Robe

0f Michig

 

 

fig

.9; .‘s
a...)

ACKNOWLEDGMENTS

(5.“

During the course of my graduate studies and disserta-
tion research and writing, I received invaluable assistance from
so many people that I regret I can name here only those who were
the most generous in their time and energy to guide and encourage
me. My deepest appreciation goes, therefore, to Dr. Anthony Koo,
Dr. Robert Rasche, Dr. William Haley, and Dr. KwangYun Lee, all

of Michigan State University.

ii

II

III

IV

 

Chapter

I

II

III

IV

TABLE OF CONTENTS

LIST OF FIGURES

INTRODUCTION

ECONOMIC APPLICATIONS OF OPTIMAL CONTROL THEORY

Optimal Growth Models
Development Planning Models
Short-Run Economic Policy Models
Consumer Choice Problems

Theory of Firms

Portfolio Selection

Pollution Control Problems

Hr—IHHHHp—a
\loxmrwap—I

DESIGN OF CONTROL SYSTEMS AND THE INVERSE
OPTIMAL CONTROL PROBLEM

2.1 Performance Index
2.2 Feedback Control
2.3 Inverse Optimal Control Problem

THE INVERSE OPTIMAL CONTROL PROBLEMS --
LITERATURE SURVEY

3.1 Inverse Control Problem for the Continuous-
Time Systems

3.2 Inverse Control Problem for Discrete-Time
Systems

THE INVERSE CONTROL PROBLEM OF DISCRETE-TIME
'MULTIVARIATE CONTROL SYSTEMS

4.1 The Solution of a General Inverse Problem

4.2 The Solution of a Class of Inverse Problem

4.3 The Inverse Optimal Control Problem in
Macroeconomic Policy Models

iii

Page

16
16

18
21

27

27

43

47

47
52

57

VI

APPLICATIONS OF THE INVERSE OPTIMAL CONTROL
TO MACROECONOMIC MODELS

5.1 A Single-Control Model
5.2 Two-Control Model

CONCLUSIONS AND RESEARCH RECOMMENDATIONS

6.1 Summary of Conclusions
6.2 Recommendations for Further Studies

BIBLIOGRAPHY

APPENDIX

iv

62

64
73

79

79
80

82

94

Figure

10.

Figure

10.

LIST OF FIGURES

ReSponses of System (5.4) with Zero Control
Responses of System (5.4) under Control (5.17)

Optimal Feedback Control for Price
Stabilization

Responses of the System (5.4) under Control (5.21)

Optimal Feedback Control for Interest
Stabilization

Responses of System (5.3) with Zero Control
Responses of System (5.3) under Control (5.24)

Optimal Feedback Control for Price
Stabilization

Responses of System (5.3) under Control (5.27)

Optimal Feedback Control for Full Employment

Page
65

69

7O

72

72
74

76

76
78

78

.—
—_

understa
making.
and ofte

fall in

to Study
its perf
various .
ChOOS Ing
Outcome.
eCGnomet
rules an
the vari
further
erate Ce
mks ma.
am Some

approach

 

mEthad.

SPECify (

 

INTRODUCTION

Recent trends in control theory deal with the fundamental
understanding of large scale systems and decentralized decision
making. Economic processes, which are characterized by complex,
and often unknown relationships among their constituent components,
fall in this class of problems.

The research tools of optimal control theory can be used
to study the dynamic responses of the economic system and to evaluate
its performances. Most of the economic decisions consist of examining
various actions together with their associated consequences, and
choosing the particular action which would generate the most desirable
outcome. Such elementary decision making could be improved by using
econometric models for the purpose of examining various decision
rules and their associated results in terms of the trajectories of
the variables generated. The decision making process could be
further improved by specifying some performance functional to gen-
erate certain optimal decision rules because the ad hoc decision
rules may not be optimal for certain reasonable performance criterion
and some better rules might be discovered by the optimal control
approach which would otherwise remain unnoticed by the elementary
method.

The performance criterion, in essence, enables one to

Specify a desired response toward which the system is Optimized.

In order
only mus
system,

desired

which pe
between

difficul
problems
uniquely
can be d

fol‘mance

formance
freCluent
to be CD

called n

 

control

cmtrol .

indiceS
°f Pouc-
lem is t
we ights

asS'Ociat

In order that a performance index be generally applicable, it not
only must reveal the performance characteristics of the optimal
system, but also must enable the decision maker to choose what the
desired characteristics of the Optimum system should be.

In practice, however, the actual process of selecting
which performance measure is to be used to "measure the distance"
between a desired trajectory and its approximation is a major
difficulty. That is, any mathematical criterion in practical control
problems is rarely explicit enough to define the optimum system
uniquely, and consequently even when a realistic performance index
can be defined, it is often found that the basic concept of a per-
formance index is too restrictive.

In this case, it is hardly expected that a certain per-
formance index can be generally agreed upon. In fact, it is
frequently argued that the choice of the performance functional
to be optimized is subjective and arbitrary. This poses the so-
called "inverse Optimal control problem" -- instead of seeking a
control rule corresponding to a given performance index, one can
try to determine all performance criteria, if any, for which a given
control rule is optimal.

In macroeconomic regulation problems where the performance
indices are, in general, of the quadratic form in the deviations
of policy goal variables from their target values, the inverse prob-
lem is to find, given economic policies, the relative "welfare"
weights to be given to the competing policy goals and "costs"

associated with the control policies.

eral
if i
for
crit
the

Spec.

the c
trust

those
using

former

additi.
SYstemE
SOIUtic
eXplici

 

 

By solving the inverse problems, one may discover gen-
eral prOperties shared by all optimal control policies. Further,
if it can be discovered that there exist many performance indices
for which a single control policy is Optimal, then the preceding
criticism about the choice of performance index is irrelevant since
the important aSpects of optimality will hold independently of the
Specific choice of a performance index.

Also, as the inverse optimal problem is the Opposite to
the optimal control problem, the solution of the inverse problem
must distinguish between control policies which are optimal and
those which are not, and perhaps disclose practical advantages of
using specific control policies in combination with Specific per-
formance indices.

AS the inverse Optimal problem is a relatively new
addition to the theoretical and methodological repertories of the
systems science and is more so for the social sciences, no general
solution of the inverse problem is yet found. In particular, the
explicit solution of the inverse problem is not known for the
discrete-time, multi-control regulatory processes.

Considering the fact that the market framework of the
economic system is working through the myriad individual decisions,
and the economic problems are generally formulated in terms of
discrete-time difference equations, it is desirable for the economists
that the solution to the inverse problem be found for the discrete-
time, multi-control systems.

The purpose of this thesis is to solve this subclass of

the inverse problem and to demonstrate the applicability of the

solution

new insi

includes
to the C
that an
dynamic
Cussion
problem.
problems

linear d

 

Five 11}
t0 Sane
VOIVeS a

for fur:

 

solution to macroeconomic regulation models, and thereby to derive
new insights for the economic policy manipulations.

The outline of this study is as follows. Chapter One
includes some examples of the applications of optimal control theory
to the economic field. The main concern of the chapter is to Show
that an economic regulation process can be viewed as a multi-Stage
dynamic optimization process. Chapter Two contains a brief dis-
cussion on control system design and the significance of the inverse
problem. Chapter Three presents a literature survey of the inverse
problems. In Chapter Four, the solution of the inverse problem for
linear discrete-time multi-control systems is develOped. Chapter
Five illustrates the applications of the results in Chapter Four
to some simple macroeconomic regulation problems. Chapter Six in-
volves a summary and conclusions and suggests some recommendations

for further study.

applicat
is conce

time.

and micr

per

per
(3) Sim

Of
(4) Con
(5) dyp
(6) mu]

(7) dyn

output 0.

 

CHAPTER ONE

ECONOMIC APPLICATIONS OF OPTIMAL CONTROL THEORY

Recently optimal control theory has found substantial
applications in economics, because much of modern economic theory
is concerned with optimal behavior of economic decision units over
time.

Principal applications of control theory in macroeconomic
and microeconomic problems have been:

(1) growth models for analysis of expansion of economies over long
periods of time;

(2) planning models for sectoral allocation of resources over
periods of time;

(3) short-run economic models for the study of short-run effects
of economic policies on macroeconomic goals;

(4) consumer choice problems over the household life cycle;

(5) dynamic models of investment and pricing by firms;

(6) multiperiod portfolio analysis models; and

(7) dynamic models of a number of sectors including water

resources and banking.
1.1 Optimal Growth Models

The central problem of these models is the division of

output over time between consumption and investment so that some

measure

such a m

F(K) at
equipmen

ment I:

of capit

t0 furtt.

Posit10r

F and

equation

be the S

 

of the C

measure of social welfare is maximized. A simplified version of
such a model is as follows.

Suppose that in a simple economy, the stream of output
F(K) attainable from the services of a given stock K of capital
equipment are allocated into consumption flow C and gross invest-

ment I:

F(K) = C + I . (1.1)

In turn, after deduction of provision for replacement
of capital equipment at a rate of OK investment expenditure leads

to further accumulation of capital stock:

K=I-6K,(-=—'). (1.2)

Also, aggregate saving rate S(t) describes the com-

position of output at each moment:

I(t) = S(t) F[K(t)] . (1.3)

Physical considerations impose the constraints:

C 2 O , I 2 O , O s s s 1 . (1.4)

Equations (1.1) - (1.4) constitute, for given functions
F and s and given initial condition K(O) = KO, a differential
equation describing the economic system.

Let
N

J = g U[C(t)] exp(-a t)dt (1.5)

be the social welfare criterion accurately reflecting the desires

of the community, where U(°) is a Specified smooth, concave,

positive
social r
S(t) we
straints
capital
3 standa
(Stolerr

Burmeist

1.2 Dev

the choi

primary

Planning

 

 

positive welfare function and a, a non-negative constant, is the
social rate of time preference. If the society by choosing proper
S(t) wants to attain the maximum value of J, subject to the con-
straints, (1.1) - (1.4) and at the same time maintain the terminal
capital stock equal tots prescribed value KN’ then the solution of
a standard problem in the Optimal control theory is required
(Stoleru, 1965; Lele, et.al., 1971; Shell, 1967; Sakakibara, 1970;

Burmeister and Dobell, 1972).

1.2 Development Planning Models

The development planning models are also concerned with
the choice between consumption and investment over time, but the
primary focus is on the sectoral allocation of investment over the
planning period.

Since the development programs, in general, emphasize
the overall economic growth as well as the intersectoral consistency
of the projects, the planning models are usually formulated in
complex nonlinear constraints including sectoral production functions
and capital accumulations and the overall resource constraints to
each sector. Also, as the development plan involves the changes in
the composition of aggregate supply and demand in the economy over
the planning period, the performance indices optimized by the planning
models are customarily specified to be additive over both various
consumption goods and time.

1)

Here the numerical control theory methods are employed

to facilitate the solution of the complex dynamic models (Arrow,

 

l
) For example, the Conjugate Gradient Method (Lasdon, et.al., 1967).

et.al.,

and Tayl

1.3 Shci

policy t
that pol
an accep
which he
macroec;
0f, for

ment, pr

80a} var

i“Strume

multiph

(aggI'Ega

1’ andg

(inveStn.
littome C

 

et.al., 1970; Little, 1969; Levhari, 1969; Simon, 1956; Kendrick

and Taylor, 1970).

1.3 Short-Run Economic Policy Models

These models are concerned with the choice of economic
policy to best regulate and stabilize the economy. It is assumed
that policy maker has a model of the economy that he believes is
an acceptable representation of the structure of the economy and on
which he will base his policy decisions. With difference equation
macroeconomic models in general, focus of the study is on the use
of, for instance, fiscal and monetary policies to control unemploy-
ment, price fluctuations, and balance of payments.

The main idea of such a study is to steer the policy
goal variables close to the targets by choosing appropriate policy
instruments.

As an illustration (Turnovsky, 1973), consider a problem
of determining government expenditures for regulating a standard
multiplier-accelerator model.

The system comprises the aggregate demand equation:

Z = CY + I + G (1.6)
(aggregate demand Z is broken down into consumption cY, investment

I, and government expenditures G); the flexible accelerator:

I = a? - kI (1.7)

(investment is an exponentially declining weighted average of past
income changes); and the output adjustment equation for the aggregate

excess demand:

where

Y over

where

expendi

Where

level 0
q1(Y - I
these 0'
Positiv.
equally
Cost of
cOSts.

be ing at

may be S
1964; Ph
NQrman,

1971; Tu‘

I = r(Z -Y) , (1.8)

where c, a, k, r are positive constants with O s c s 1.
From (1.6) - (1.8), the evolution of national income

Y over time given values of G and G is derived as follows.

Y+blY+b2Y-b3G-rG=O, (1.9)

where b1, b2, b3 are constants expressed in terms of c, a. k, r.

Now the policy maker is assumed to control government

expenditures so as to minimize some objective functional,
.J-f(Y Y2+F(C 62+N (Y 172

q2(G - (5)2 + n(G)2}dt

where Y and G. are some desired level of income and associated
level of government expenditures respectively. The terms,

q1(Y - E)2 and q2(G - G)2 are losses incurred by being away from
these Optima. The fact that these are quadratic implies that
positive and negative deviations from desired levels are weighted
equally and are increasingly costly. The term n(G)2 denotes the
cost of changing fiscal policy and can be described as adjustment
costs. f1(YN - E)2 and f2(GN- G)2 measure the terminal costs of
being away from the targets.

In the above problem, it is possible that the government
may be subjected to limits on the amount of its expenditures (Theil,
1964; Phillips, 1954, 1957; Pindyck, 1972; Erickson, et.al., 1970;
Norman, 1971; Livesey, 1970; Chow, 1970, 1972; Fischer and Cooper,

1971; Turnovsky, 1973).

1.11 Cons

as choos
divided I
human ca

earnings

income 1
and the
during t
possesse
stock us
the Cons

(capital

where F
the On 1}

hUman Ca

value of

 

Individu

Problem

 

10

1.4 ConSumer Choice Problems

The consumer choice problems characterize the individual
as choosing a time path of his available time (or energy) to be
divided between earning (by renting human capital) and investment in
human capital so as to maximize the present value of lifetime
earnings.

Following Haley (1973), then the individual's disposible
income in any period is the difference between his earnings rE(t)
and the cost of input used for the investment in human capital rK(t)
during that period, where E(t) is total stock of human capital
possessed by the individual at time t, K(t) the human capital
stock used as input to the human capital investment at t, and r
the constant rental rate per unit of time on the human capital stock
(capital markets are assumed to be perfect).

The human capital stock grows at a rate governed by
S(t) = F[K(t)] - 5E<t> . (1.10)

where F(K) is the flow of investment at time t attainable from
the only input of human Capital stock and 6, a constant rate of
human capital depreciation.

Assume that the individual wants to maximize the present

value of his disposible earnings over the life cycle:

N
J = (I; r[E(t) - K(t)] exp(-at)dt , (1.11)
where N is the end of the earning life cycle and a, the constant
individual time preference rate.
Then the general form of the individual optimization

problem is to maximize the objective functional (1.11) Subject to

and
the

Staf

1.5

devel
integ
by Chc

1972).

in the

variabl
IS asSU
tion of
financir

(0<Os

dynamic

 

subject I

 

11

the constraint (1.10) as well as such physical constraints as

K(t) 2 O, E(t) > 0,

E(t) :» K(t) ,

and at the same time to keep the positive human capital stock at
the end of the earning period (Hakansson, 1969; Haley, 1973;

Stafford and Stephen, 1972; Yaari, 1964; Ben—Porath, 1967).

1.5 Theory of Firms

Various dynamic models for the theory of firm have been
developed. Let us consider a firm which seeks to maximize the
integral of the discounted profit flows over the planning period
by choosing optimal level of capital stock in each period (Leland,
1972).

If growth permits better achievement of the firm's goal
in the future, capital stock K is one of the key decision
variables for the firm. Following most dynamic formulation, it
is assumed that the rate of change of capital stock, K, is a func-
tion of current profits P . This formulation describes a self-
financing firm which reinvests all or a positive fraction
(0 < a s l) of its profits.

Then, the problem is given by the following. In a

dynamic environment, the firm will maximize

N -6t
[I e - P(K,L)dt

L.
II

subject to

K aP(K,L), O < a S 1;

K(O) = K0 ... a given initial capital stock;

whe
EJFCN
et.4

Zabe

1.6

selec
his u
retur
Samue
proble
invest
Zt is
Prob {2
$1 inve
period.

that p

wealth,

going in

 

SUbject .

HO agi

12

where t is a constant discount rate, L is labor input for the
production of the firm, and N is a fixed terminal time. (Arrow,
et.al., 1958; Hakansson, 1970; Lucas, 1967; Thompson, et.al., 1971;

Zabel, 1967).

1.6 Portfolio Selection

The individual and/or firm is faced with the problem of
selecting a portfolio of bonds, stocks, and cash that will maximize
his utility function defined over the probability distributions of
returns generated by the various portfolios. For instance,
Samuelson (1969) considers an individual's portfolio selection
problem, postulating the existence of a risky asset that makes $1
invested in, at time t, return $1 Zt after one period, where
Zt is a random variable subject to the probability distribution,
Prob {2t 5 z} = p(z), z 2 0, along with the safe asset that makes
$1 invested in it at time t return $l(l + r) at the end of the
period. Yields at different time are assumed to be independent so
that p(zo,zl,...,zN) = p(zo)p(zl)...p(zN).

The problem is to find the Optimal fraction of total
wealth, at, that should be put into the risky asset, with 1 - a

I:

going into the Safe assets, at each instant of time. Specifically,

Max J = E
{Ct,at} t

ntntz

(1 + p)-tU(Ct)
0

subject to the wealth constraint,

_ -1
ct — wt - wt+1 [(1 - at)(1 + r) + atzt] ,

WO 8 given value, and

= C
w5+1
where l

sumptior

individt

con5umpt

dynamic

17 PO:

sumptior

macroecr

PrOduces
(Sector

ment,

maximiz
tion Of
planflin
u1~9nts b
fUncm0
fro“, C0

to hem

fiCIEnt

alOng H

 

l3

WN+1 = 0 (no bequeating of wealth at death),
where U(Ct) is the individual utility function depending on con-
sumption C at time t, p a constant discount rate, Wt the
individual's wealth at t, and E an "expectation" operator.

This problem is solved simultaneously for optimal saving-

consumption and portfolio-selection decisions over time using a

dynamic programming method. (Hakansson, 1969; Merton, 1969).

1.7 Pollution Control Problems

Haurie, et.al. (1972) analyze Optimal policies of con-
sumption and of pollution control in an economy using a two-sector
macroeconomic planning model.

It is assumed that one sector (Sector 1) of the economy
produces good Y to be consumed or invested and another sector
(Sector 2) produces good Z used exclusively to purify the environ-
ment.

Since in this problem there are two conflicting goals --
maximization of consumption flow and minimization of the accumula-
tion of pollution generated in the production processes -- over the
planning period, the policy maker faces the allocation of invest-
ments between two sectors in each period such that the objective
function including the weighted sum of social benefits derived
from consumption and the social cost of pollution accumulation is
to be maximized.

Emphasis is given to the following. By taking a Suf-
ficiently long interval of time, an Optimum path may contain an are

along which the pollution and per capita consumption are maintained

constan
criteri
per car
policy

cost at

subject

polluti

Where

 

 

14

constant. If such an arc is uniquely defined for each performance
criterion which gives different relative weights (A, 1 - X) to
per capita consumption and cost of pollution, it is possible for
policy maker to choose a particular weighting based on a long-term
cost effectiveness,analysis.
Formally, the problem is to maximize
_ N -6t H '6t
J - k g e c dt + (1 - x) $ e p(x3)dt , O S k S 1,
subject to the constraints of capital accumulations in two sectors,

pollution accumulation and the per capita consumption equations:

x.
I

1 — uly — (d +-n)x1
x2 = u2y - (d +'n)x2
x3 = G(Ly, Lz) - r x3

x
_ _ __1. ___ .
c - (1 - u1 u2)u3f1(u3), 0 5 U1 S 1, 1 1,2,3,

111 + u2 S 1,

where

xi (1 = 1,2) is the capital stock in sector 1 (measured

in terms of total labor (L) unit),

x3 is quantity of pollution agents accumulated,

ui (i = 1,2) is the gross investment in sector 1 (measured
in terms of good Y),

u3 is the prOportion of labor engaged in Sector One,

y is production of Y (measured in total labor unit),
2 is production of 2 (measured in total labor unit),
c is per capita consumption,

n is a constant rate of labor force increase,

f(

G(

economi
Optimiz
inverse
inverse

to have

to dISCI

ChHpter

 

15

d is a constant rate of capital depreciation in both sectors,
f(°) is production function of Sector One (homogeneous of
degree one),
G(') is pollution generating function with §%-> O, §%'< O
for all Y 2 0 and Z 2 O, I
r is a constant rate of natural elimination of pollution,
p(x3) is the social cost of pollution.

The main concern of this chapter was to Show that an
economic regulation process can be viewed as a multi-stage dynamic
optimization process. This permits one to discuss meaningfully the
inverse Optimum control problem in the economic field, because the
inverse problem starts with a given control rule which iS assumed
to have already been obtained in the optimization process.

Before examining the inverse problem formally, we need
to discuss some aspects of the control system design in the next

chapter (Chapter Two).

2.1

an ad
perty
the e

tions

the Sp
to the
ment 0.
This 3:

PErform

w'Ill'ch t

 

CHAPTER TWO

DESIGN OF CONTROL SYSTEMS AND THE INVERSE OPTIMAL
CONTROL PROBLEM

2.1 Performance Index

In modern control system design, it is emphasized that
an admissible control must have, in addition to the stability pro-
perty, an optimizing prOperty in some sense, for example, minimizing
the error of the system under control or satisfying certain Specifica-
tions of accuracy and speed of performance of the system.

In the design of control systems, the starting point is
the System Specification. This includes a description of the input
to the system and the desired response. Also included is a state-
ment of the basis on which the system performance will be judged.

This statement is in the form of a performance index. That is, the
performance index enables one to Specify a desired response towards
which the system is optimized.

In order that a performance index be generally applicable,
it not only must reveal the performance characteristics of the Optimal
system, but also must enable the designer to choose what the desired
characteristics of the optimum system should be.

The actual process of selecting which performance measure
is to be used to "measure the distance" between a desired output

time-function and an approximation to the desired function is the

16

major
that 1
the SC
obtair
a part
a clos
than n
Often
choice

realis

Often
restri
eXplic

that a

aSagE
Systems
are: (
therEfo
System
it a1w3

Of the

 

Can be
and TUt
Shifts,

Obtain;

 

 

17

major difficulty. If a realistic performance index is defined

that represents most of the design requirements of a problem, then
the solution for the optimal control function can usually be

obtained only by numerical methods which yield solutions to only

a particular problem. On the other hand, if it is desired to obtain
a closed-form solution for the control and, thereby, to solve more
than numerical problem, simple performance indices must be used which
often do not specify many of the design requirements. Thus the
choice of a performance index is generally a compromise between a
realistic criterion and one that is mathematically tractable.

Even when a realistic criterion can be defined, it is
often found that the basic concept of a performance index is too
restrictive. In practice, any mathematical criterion is rarely
explicit enough to define the optimum system uniquely. It is here
that a certain amount of personal Opinion is found.

Very often, the quadratic performance index is considered
as a generalized criterion for designing linear multivariable
systems. The advantages of using this particular quadratic index
are: (1) it results in a closed-form solution for the control and,
therefore, the properties of the control as well as the Optimal
system can be determined; (2) under a reasonable set of restrictions,
it always produces a stable system; (3) once the numerical elements
of the performance index are Specified, the optimal feedback gains
can be determined by a straightforward computer solution (Tyler
and Tuteur, 1966); (4) when the dynamic systems are under the random
Shifts, it enables one to disregard the uncertainty elements for

obtaining the control rule. (Simon, 1956; Theil, 1957).

quadra
resour
polici
1972;

as a g
theory

produc

and st

elemen
Contro
lens,

the soc
Variabj

element

2.2 Fe

which I
ProcesS
featurq
aCtUatE

Operat].

 

Cemmanc

trol OI

tions 5

 

18

The quadratic functional which is often times called a
quadratic social disutility function in economic problems on optimum
resource allocation over time and on macroeconomic stabilization
policies, has been used quite extensively (Sengupta, 1970; Chow,
1972; Simon, 1956; Theil, 1965). Also, it has been frequently used
as a generalized measure of the system performance in the control
theory (Kalman, 1964; Kuo, 1970); information theory (Adorno, 1962);
production, employment and inventory scheduling (Holt, et.al., 1956),
and statistical quality control theory (Barnard, 1959).

The quadratic index contains weighting matrices whose
elements can be Specified to Satisfy the requirements of specific
control problems -- in the short-run macroeconomic regulation prob-
lems, the elements of the weighting matrix are designed to measure
the social disutility associated with the deviations of the economic
variables from their specified targets -- and, in effect, these

elements become the design parameters of the Optimal system.

2.2 Feedback Control

A feedback control system is a combination of elements
which automatically COOperate to maintain a physical quantity or
process in accordance with a given command. It has three predominant
features; (1) it is a closed-100p system in which the control is
actuated by a quantity that is affected by the result of the control
operation; (2) it can establish control throughout a wide range of
command that may vary in a random manner, and (3) it permits the con-
trol of high-power operations at a remote point by low-power Opera-

tions at a local point.

are;

signé
opera
to be
feedb
the e
with

be ac‘
in the
of the
effect
feedba

compen

Controj
the p01
the Sys

weighte

{10118 0

Changs

there e
matrix)

lineal.)F

 

19

Some of the major reasons for employing feedback control
are; (1) the process or actuator which supplies the output may have
Signal transmission characteristics that make accurate Open-100p
operation (open-loop Operation yields an entire sequence of controls
to be followed from initial conditions) very difficult, (2) with
feedback, the precision of control can be made to depend largely on
the equipment used to measure the system output and to compare it
with its "ideal" value. This fact may enable accurate control to
be achieved in Spite of inaccuracies and variable characteristics
in the process. That is, the feedback will reduce the sensitivity
of the System characteristics to changes in parameters, (3) the
effects of disturbances on the output may be suppressed by employing
feedback, thereby eliminating the need for the elaborate disturbance
compensators that would be needed with open-loop control.

In the economic stabilization models, a linear feedback
control policy (the control policy is set to reSpond linearly on
the policy goal variables) is generally chosen in such a way that
the system under control (closed-loop system) will have a small
weighted sum of variances.

It can be seen that feedback is used to overcome limita-
tions of the physical components and is introduced to effect Specific
changes in the characteristics of the system (Ku, 1962).

In connection with the latter application of feedback,
there exists a problem of arbitrary pole (eigenvalue of the system
matrix) assignment (Willner, et.al., 1972). Given a controllable,

linear, time—invariant system,

x(t) = Ax(t) + Bu(t) , (2.1)

where
of C(N

back 1

yield

whose

vect01

where
eigenv

associ

Where

0f prot‘

Vhe re

 

Vaer,

for a1:

20

where x(t) is an n-vector of state variables, u(t) is an m-vector
of control variables, and A,B, are constant matrices, find a feed-

back gain matrix G(m X n) Such that the feedback control

u(t) = Gx(t) (2.2)

yields the closed-loop system
x(t) = (A + BG)x(t) (2.3)

whose eigenvalues can be assigned arbitrary.
For a single-control system (B in (2.1) becomes an n-
vector), the problem of the pole assignment is solved as follows.
A nonsingular matrix T is found such that

TFT'1 = A'+'Bh'T-1 = A.+'BG , (2.4)

where F is the Jordan canonical matrix of desired closed-100p
eigenvalues (xi, 1 = 1,2,...,n), with only a single Jordan block

associated with each multiple eigenvalue, and
G = h'T , (2.5)

where h' = (l, l,..., 1), an n-vector.
For T, we solve TF - AT = Bh' by solving a sequence

of problems of the form

II
on

(x11 - A):i (2.6)

where t1 is the 1th column of T. If Pi is a repeated eigen-

value, then (2.6) becomes

(111 - A)ti - B - t1-1 (2.7)

for all but the first column of T associated with xi.

for E
possi

CODII’

so th

where

define

then (

Which ;

Pattern

Paths,

  
  
  
  

0f the
enables

Charact

2'3 In

Underst
making,

map 16};

 

21

For multi-control System, the problem of finding G
for a given set of eigenvalues is generally nonlinear and has many
possible solutions. In order to obtain a linear solution, the

control u in (2.2) is restricted to the form
u(t) = 026 x(t) (2.8)

so that the closed-loop system (2.3) becomes
x(t) = (A + BaG)x(t) , (2.9)

where a is an m-vector with all elements equal to unity. If we

define
Ba = b , (2.10)
then (2.9) becomes
x(t) = (A‘+ bG)x(t) , (2.11)

which is the closed-loop system equation of a single-control system.
Since the eigenvalues of the system matrix govern the

pattern of time paths (in particular, the convergence of the time

paths, the Speed of the convergence, and the damping ratio, etc.)

of the state variables, the solution of the pole assignment problem

enables the designers of control systems to effect the "desired"

characterization of the system by prOper choice of feedback controls.

2.3 Inverse Optimal Control Problem

Recent trends in control theory deal with the fundamental
understanding of large scale systems and decentralized decision
making. Social and economic processes, which are characterized by

complex and often unknown relationships among their constitutent

22

components, have such problems amenable to control theory applica-
tion.

The research tools of optimal control theory can be
used to study the dynamic responses of the social and economic
systems and to evaluate their performance. Most of the social and
economic decisions usually practiced at their basic levels consist
of examining various actions together with their associated con-
sequences and choosing the particular action which would generate
the most desirable outcome. Such elementary decision making could
be improved upon by using dynamic models for the purpose of
examining various decision rules and their associated results in
terms of the time paths of the variables generated.

The decision making process could be further improved
by specifying some performance indices to generate certain optimal
decision rules because the ad hoc decision rules may not be Optimal
for certain reasonable performance criteria and some better rules
might be discovered by the Optimal control approach which would
otherwise remain unnoticed by the elementary method.

In this case, it is hardly expected that certain per-
formance indices can be generally agreed upon. In fact, it is
frequently argued that the choice of the performance index to be
Optimized is arbitrary and subjective, perhaps only a matter of taste.
The argument is even greater in the social and economic regulatory
systems compared to the physical and technological systems where
the relative merits among various components of trajectories are
better understood and clearer, and sometimes they are measured

Specifically in terms of energy and cost expenses. This suggests

23

that the design of socio-economic systems as well as physical and
technological systems involves the so-called "inverse Optimal con-
trol problem" -- instead of asking for a control policy correSpond-
ing to a given performance index, one might seek to determine all
performance criteria, if any, for which a given control policy is
optimal.

By solving this problem, one might be able to discover
general prOperties Shared by all optimal control policies. Further-
more, if it can be discovered that there exist many performance
indices for a single optimal control policy, then the preceding
criticism about the choice of performance index will be irrelevant
since the important aspects of optimality will hold independently
of the specific Choice of a performance index.

Also, as the inverse optimal control problem is the
opposite of the optimal control problem, the solution must dis-
tinguish between control policies which are optimal and those which
are not, and perhaps disclose practical advantages of using Specific
control policies in combination with specific performance indices.
For instance, if the given control policy has some undesirable
effects on the closed-loop system, the policy maker may want to
find another control policy. But once the system equation and the
specific performance index are given, the control policy is uniquely
determined and consequently the properties of the closed-100p System
cannot be changed. This means that given a system equation, difv
ferent performance indices should be Specified in order to get
different control policies. Here the solution of the inverse optimal

control problem may help to find out the appropriate performance

24

index together with the control policy by which the closed-100p
system can achieve the desired characteristics of "goodness" (e.g.,
moderate overshoot, high lOOp gain, and flat frequency response).
One important class of this problem is how to design
optimal systems with prescribed closed-100p eigenvalues. We know
that if the system is controllable, it is always possible to find
a feedback gain matrix which will assign an arbitrary set of eigen-
values to the closed-loop system matrix (cf. Section 2.2). That is,
if some prescribed set of eigenvalues is assigned to the closed-
1oop system matrix such that the closed-100p system can reveal the
"desirable" characteristics, the corresponding feedback gain matrix
can always be found. Once this ”desirable" feedback gain matrix
is obtained, the corresponding performance index can be found by
the method of inverse Optimal control.
As an example for the inverse Optimal control problem,

consider a Simple dynamic macroeconomic model.

9 = u(c + 1 - y) (2.12)
é = 8(ay - c)
where
y = Y - YO
c = C - CO
1 — I - I0

0:3 ... dynamic adjustment coefficients

Y = GNP
C = consumption
I = investment

25

a = marginal prOpensity to consume.
Subscript "0" denotes the equilibrium values.

The dynamic system (2.12) can be written as

-a a. a
2': = x + u , (2.13)
3a -6 0
where x = (y,c)', a vector of state variables, and u = i, a con-

trol variable. Here the prime denotes the transpose.

Let the performance index for the system (2.13) be
T

2
= I
J A (xtht + kut)dt (2.14)
where Q is symmetric and positive semidefinite and k is a
positive constant. Then the Optimal control policy in the feedback

form is given byl)

u = g'x (2.15)

where g is the time-varying feedback gain vector which can be
uniquely determined by the method of Optimal control theory.

The inverse Optimal control problem for the example is:
given an optimal control policy (2.15), find all performance func-

tional of the form (2.14), if any.2)

Specifically, determine the
weighting matrices, Q and k in (2.14).
In the above example, if the given control policy for

the investment has some undesirable effects on the closed-loop

system, e.g., too drastic change in consumption level over the

 

1)

2) We would like to find only one, but this is not always the
case.

A * indicates the control is optimal with respect to (2.14).

26

planning period, the policy maker may want to find another control
policy. But once the system equation (2.13), and weighting matrices
Q and k in (2.14) are given, the closed-loop system cannot be
changed.

Here, the solution of the inverse Optimal control prob-
lem may help to find out the appropriate weighting matrices Q and
k, and the control policy by which the closed-100p system can

achieve the desired characteristics. The solution to the problem

described above will be the subject of Chapter Four below.

CHAPTER THREE

THE INVERSE OPTIMAL CONTROL PROBLEMS -- LITERATURE SURVEY

Since the inverse optimal problem is to find the per-
formance indices (if any), given an optimal control policy, solu—
tion of the problem, in general, starts with the assumption that
there exists a solution for the Hamilton-Jacobi equation (which
provides a sufficient condition for Optimality) or for the matrix
Riccati equation which can be derived from the Hamilton-Jacobi
equation.

The main purpose of this survey is of a rather technical
nature, i.e., to examine the solution process for the inverse prob-
lems, which could be utilized for a broader class of the inverse

problem.

3.1 Inverse Control Problem for the Continuous-Time Systems

Kalman (1964) considers, for the first time, the inverse
Optimal control problem for the following case:
(a) The system is governed by a linear differential equation with
constant coefficients (linear, time-invariant, continuous-

time system).
x(t) = A x(t) + b u(t) , (3.1)

where x is a real n-vector, the state of the system, u(t) is a

continuous, real-valued function of time, the control function,

27

28

A is a real constant n X n matrix, and b is a real constant

n-vector .

(b) The control policy is linear and constant,

110:) = -g'X(t) , (3-2)
where g is a real, constant n-vector of feedback coefficients.
(c) The performance index is a quadratic form with constant co-

efficients in the state and control variables.

t1 t1

2
J = 1im L(x(t),u(t))dt = lim a g (x'H'Hx + u )dt

t—K'JO t—ooo

1 l

where H is a 1 X n matrix with rank = l.

1) (3.3)

(d) There is only one control variable (or single-input system).

The basic assumptions employed are

(i) The system (3.1) is completely controllable; rank(b,Ab,...,An-1b) = n,

i.e., all the state variables can be affected by some suitable choice
of the control function u(t).

(ii) The pair (A,H) is completely observable;
rank(H',A'H',...,(A')n-1H') = n. This assures y = Hx must not
vanish identically along any free motion of the system unless the

initial state x0 = 0.

Then the inverse Optimal control problem in this case

is as follows:

 

1) This particular form of the quadratic functional L(x,u) can be
said to represent a more general form. Consider L(x,u) =

£{x'Qx.+ 2(r'x)u +'ou2} «2,r,o are constants, Q ==Q'). Without
loss of generality, set a = 1. Then

2L(x,u) = x'Qx + 2(rlx)u + u2 = x'(Q - rr')x + (u + r'x)2. _lzet
Q - rr' = H'H and u = u + r'x. Then 2L(x,u) = x'H'Hx + u
the system. x = Ax +-bu must be changed to x = Ex +1bu where
K's A - br', and g in u(t) = -g'x(t) is to be replaced by

g = s - r-

and

29

Given a completely controllable constant linear system (3.1) and
constant linear control law (3.2), determine all loss functions
L in (3.3) such that the control law minimizes the performance
index (3.3).

A necessary and sufficient condition for u(t) = -g'x(t)
to be a stable optimal control law is that there exists a matrix
P which satisfies the following algebraic relations (Kalman, 1964)
(a) P = P' is positive definite
(b) Pb ... g (3.4)
(c) -PAg - Aé P = H'H + gg' (Riccati equation),
where A8 = A - bg'.

Since P in the above relations is unknown, it is desirable
to eliminate P and to get a Simple relation connecting the control
law g and the "representative" performance weighting matrix H.
For this, Kalman shows that a necessary and sufficient condition for
g to be an Optimal control law is that g be a stable control law

and that the condition

‘1 +g'§(iw)bI2 = l + IH@(iw)bI2 holds for all real w,1) (3.5)
-1 .2 . .
where 6(3) = (SI - A) , 1 = -1, and s 18 a complex variable.
If the condition (3.5) holds, H may be obtained by
factorizing the non-negative polynomial
. . 2 ._ . 2
I1 +'g T(1W)b\ - l — \H§(iw)bI . (3.6)

Since g is assumed to be stable, the rational function H@(iw)b

 

1) It is assumed that H is a matrix consisting of one row only.
Otherwise, the condition (A,H) is completely observable is not
sufficient to guarantee that the optimal control law is completely
observable.

30

must not possess any common cancelable factor which has a zero
with non-negative real part. Using the canonical forms on matrices
A and b (Wonham and Johnson, 1964), it is possible to identify
the components of an n-vector H = (h1,h2,...,hn) with the
numerator coefficients of the rational function

h (1w)n'1 +...+ h

n 1

H§(iw)b =
(iw)n + an(iw) +...+ a

1
where ais, (i = 1,...,n) are constant scalars.

Then the matrix H so constructed is the solution for
the inverse Optimal control problem.

Thau (1967) extends the Kalman's inverse problem to the
multiple input system and a class of non-linear control system:

(a) The dynamic system equation considered is
x = f(x) + Bu , (3.7)

where x is an n-vector state variables and u(t) is an m-vector
control variable continuous in t. B is an n X m constant matrix.
(b) The integrand of the performance criterion is a sum of a func-
tional of the state variables and a functional of the control.

That is, the performance index is given by

V(x(O),u) = (I, [q(x) + h(u)]dt , (3.8)

where q(x) and h(u) are smooth functions of their arguments.

(c) It is assumed that

dh . .
n(u) 5‘3: 13 a one-to-one mapping,

2
h(O) =0 and £1mtg->0.
du

(d) The origin x = 0 is considered the target set and the

31

feedback control law given by

*
u (t) = ¢[x(t)] (3.

is assumed to be Such that the resulting closed-100p system,

*
x = f(x) +'Bu (t) a F(x) + B¢(x), is asymptotically stable.

9)

*
Then assuming an optimal control law u (t) exists, the

inverse problem is as follows:

Given a control law (3.9) with the above-mentioned prOperties,

find the most general performance functional (if any) of the form

(3.8) which is minimized by the control law (3.9).

For the control problem (a) - (d), the necessary and
sufficient condition for the optimality is that the value of the
Optimum performance index (3.8) Vo(x) satisfies the Hamilton-

Jacobi equation

0
max {-(q(x) + h(u)) - 3%— (f(x) + 311)} = O, v°(O) = O. (3

11

From (3.10), the optimal control ¢(x) satisfies

aV_O_
“(p(X)) = -B ax (3
and
v0
q(X) = -h(¢(X)) - g;—'[f(X) + B¢(x)] . (3-

To obtain more explicit results, Thau considers a com-

pletely controllable multiple input linear time-invariant system

a = Ax +’Bu , (3

.10)

.11)

12)

.13)

where A is an (n X n) matrix and B an (n X m) matrix, and

the performance index in the form of

V = h g (x'H'Hx + u'u)dt . (3.

14)

32

The given control law which drives the system toward the
origin is
u = -Gx , (3.15)

where G is a known constant matrix. It follows from (3.11) and

(3.12) that
G = B'P , (3.16)
where P satisfies the algebraic Riccati equation,

H'H + G'G = -PA - A'P + PBG + G'BP ,
or

-PA -A'P=H'H+G'G, A =A-BG. (3.17)
g g g

Following Kalman (1964), Thau derives the frequency-
domain characterization of optimality for the multiple-input system.

Using (3.16) and (3.17),

T'(iw)T(iw) = I + F'(iw)F(iw) for all real w, (3.18)
where T(iw) = I +'G§(iw)B and H§(iw)B = F(iw)

There is, however, no general way at the present to find

the explicit expression for H from the equation (3.18).

 

Thau further considers a class of nonlinear systemsl)
given as
x = Ax + be(u) (3-19)
u = g'x , (3.20)
1)

The development here incorporates Panda's (1971) corrected
version of Thau's solution.

33

where x,b, and g are n-vectors and A is an n X n matrix.
Here G(u) is considered to be a known scalar function, defined

and continuous for all u, 9(0) = 0, ue(u) > O for all u # O and
+1”

I 9(u)du diverges.
0

\

With the additional assumptions that (i) the value of the
optimum performance index is given by V0(x) = %x'Px, where P is
positive definite, (ii) 9(u) in (3.19) can be expressed as a

power series in odd powers of u with all positive coefficients,

)

°° 1 + 1
i.e., 9(u) = X a,u , i E (I ),

i=11 0
function of n in (3.11) is also expressed as a power series

all ai > 0, and (iii) the inverse

-1 m
n (U)=}:
i=1

criteria of the form (3.8) for the system (3.19) and the control

ciul, i 6 (1;); explicit expressions for all performance

law (3.20) can be obtained as follows (Panda, 1971).

9(g'x) = n'1<-b'Px>
a co a

___._ _ I I __ __L c
q(x) % x (PA + A P)x + Cl iil 1+1 (g x)

i+1, i 6 (1;). (3.21)

In order to have non-negative q(x) in (3.21), the con-
dition, ‘1 - g'§(iw)b\2 2 1, should hold for all real w, where
um) = (iwI - A)-1.

It is seen above that Thau gets the explicit algebraic
conditions for the solution of the inverse optimal control problem
by introducing the assumption that the optimum performance is of
the form Vo(x) = % x'Px, where P is a (positive definite)

symmetric matrix.

 

1)
odd.

+
IO represents the set of all integers that are positive and

34

Yokoyama and Kinnen (1972) show the necessary and suf-
ficient conditions for Optimized performance indices of a general
class of controllable and uncontrollable systems with weakened
assumption about the form of the optimum performance.

Yokoyama and Kinnen have the following problem:

(a) The system,
)2 = G(X) + Bu 5 Ax + F(x) + Bu . (3.22)

(b) A feedback control law, u(x). (3.23)
(c) The functional form of the performance indices is restricted

to the general structure,

g {L(x) + u'Ru}dt . (3.24)

The assumptions for the problem are
(i) x is an n-vector of state variables. G(x) is an n-
dimensional vector valued function of class C2 satisfying
G(O) = O, and A is an n X n matrix such that Ax is the
first-degree homogeneous term of G(x).
(ii) u is an m-vector of control variables and B is an
n x m matrix of rank r such that O < r s m 5 n.
(iii) u(x) is an m-dimensional vector valued function of
class C2 such that u(O) = O and the origin of the
synthesized control system (3.22) is asymptotically stable in
the large.

(iv) R is restricted to an m X m matrix, symmetric and

35

2
positive definite and L(x) to class C such that
L(O) = O and
g {L(¢(t,x) + u[¢(t,x)]'Ru[¢(t,x)]}dt (3.25)
is well defined, where ¢(t,x) is a solution of (3.22)
from x C R“.

Given a system equation (3.22) and a feedback control

law (3.23) a priori, the problem is to seek L(x) in the performance

indices (3.24) optimized in the synthesized feedback control system.

It is assumed that the inverse problem is considered for

the control equivalent canonical form (Luenberger, 1967). Thus

A =FA1W

 

 

A21

n

L

and B =

where the

 

 

 

n-r = A(e) O 0 ..... O '1 Le
O O A(1,2) ..... 0 L1
r
O
O ................... O A(r-l,r) Lr-l
LA(r,e) A(r,l) ........ (r,r) .4 Lr
Le L1
F 3 _
0 I 0 n {r
L.0 1 I Lr
0‘L L

following is true:

(i) Le’Ll""’Lr-l’ and Lr are integers determined by A and B
r

such that

Le + igibi = n, O < L1 3 L2 g...s Lt = r,

= 0; if A,B is a controllable pair

L

e

f 0; if A,B is not a completely controllable pair,

36

(ii) A(i,i+1) is an Li x {3+1 matrix such that

A(i,r+1) = [0: IL 1, i = 1,2,...,r-1,

i

's are unspecified.

A(uni)
For convenience, define the following:
= r - =
(i) x x1.) n r _[rx(e)l

x(l)

LXZJ r L3(rLJ

 

 

 

 

 

the components of x2 = x(t) are directly dependent on u from
the structure of B. Those of x1 are indirectly controlled state
variables.
(ii) G1(x) n-r IF1(x) n-r
G(X) E . F(X) =
G2(X) r F2(X) 1'
(iii) u(x) = u1(x) m-r
(3.26)
u2(x) r
(iv) R11 R12 m-r
R = (3.27)
!
R12 Rzzj r
1

- ‘ -
and R0 R22 R12R11 R12
For the optimal performance index V(x), the Hamilton-

Jacobi equation

min {L(x) + u'Ru + (a§£51)'[6(x) + Bu]} = 0 (3.28)

U

must be uniquely realized at each x E Rn by u such that

u:

—a R'13'(5§£§1) . (3.29)

37

Identifying (3.29) with the specified u(x), it follows
from (3.26), (3.27) that

1

 

 

 

 

u1(x) = -R11 R12 u2(x) (3.30)
and
M = -2
8X2 R0u2(x) . (3.31)
2
With the symmetry prOperty of §:§é£l , the following is
derived.
r— r‘ X .
avg ) _ avgx)‘1 2 5”2(x) l
= -2 -——-—- R dx + W(x )
ax 5x1 0 5x1 0 2 1
a (3.32)
M _2 R0U2(X) ,
L 5x2 J L 4
where W(x1) is an (n-r)-dimensional vector-valued function of
class C1 (i.e., with continuous first derivatives) to provide
2V x2 au (x)
the symmetry for a—-££l, and the integral I -;-- dx is defined
axlax 5x
1 0 l
au2(X)
(for an r-dimensional row vector function -;;—- a E(x)) as
1
x2 xn-r-l-l
g E(x)dx = g e1(x1,x2,...,xn_r,r1,0,...,0)dr1
x
n-r+2
+3; e2(x1,x2,...,xn_r+1,r2,0,...,O)dr2 +...
x
n

+ & er(x1,x2,...,xn;1,rr)drr .
The optimal performance index has an expression

X o
V(x) =f (dz-£51m): . (3.33)
o

Yokoyama and Kinnen (1972) show that a performance index

(3.24) can be optimized by the specified u(x) if and only if

38

the following conditions are satisfied:

(a) u1(x) = -R11 R12 u2(x),
au2(x)
(b) R0(-;;--) is symmetric,

2

(c) there exists an (n-r)-dimensiona1 vector-valued function

W(x1) of class C1 insuring the symmetry of

x2 au2(x) . 3W(x1)
-2 . .r<--;.-—> +-——
5 1 0 a 3x1

(d) L(x) and R are related by

2 5112(X) . .
L(x) = ué(x)ROu2(x) + Zué (X)ROGZ(X) + 2 g [:-g;I—;] Rodx2 Gl(x)

- W'(x1)G1(x) . (3.34)

The above procedure does not guarantee that the solution
is unique and the resulting performance index V(x) is positive-
semi definite. However, Yokoyama and Kinnen develop a method
which insures the positive definiteness of V(x) as well as
-9(x) by adjusting V(x) and L(x) that are obtained as (3.33)
and (3.34).

All the inverse optimal problems mentioned so far con-
sider the performance index of a special form, i.e., the integrand
of the index does not explicitly depend on time.

Kurz (1969) and Bellman (1970) consider the inverse
problem which involves the performance index with time-dependent
integrand.

Kurz's system equation is

(a) x = f(x) ~ u, x(0) = x (3.35)

O 3

39

with performance criterion of the form

(I)

J = g e'5th(u)dt . (3.36)
(c) The control law is given by

u = ¢(x) . (3.37)

It is assumed that f(x) in (3.35) is strictly concave
with f'(x) > O, f"(x) < O for all x(t); h(u) in (3.36) is strictly
concave belonging to the class C2, and 5 > O is constant. The
control law u is assumed to be monotonic, continuously differenti-
able function with ¢'(x) > 0.

Then, the inverse problem is to find the function h(u)
and 6 in the performance index such that the given u = ¢(x) is
the optimal control law for the system (3.35).

For the Hamiltonian defined by

H(x.u,t.p>e“ = h(u) + p[f(x) - u] ,

where p(t)'e-6t is a "costate" variable, the optimality condition
is given by
fi(t) = P(t)(6 - f'(X)) (3-38)
h'(u) = p(t) . (3.39)

Assuming that x f(x) - ¢(x) has at most one stationary

* * * *
solution x , f'(x ) > O, and f'(x ) - ¢'(x ) < 0, then the inverse
problem of u = ¢(x) has a solution. In fact, since the system
is a simple scalar equation, an explicit solution can be obtained

analytically as follows.

40

From (3.38) and (3.39),

= f' x - 6

 

u
.3 - h" u .u (3.40)
h'(U)
Moreover, since u = $(x),
33:27.?” (3.41)
Thus from (3.40) and (3.41),
11322 _ f'(xl ' 5 (3.42)

- h'(U) — ¢'(X)[f(><) - c1500]

With x(u) = ¢-l(u), the equation (3.42) can be considered as an
equation in u only. The general solution of (3.42) may be

formally written as
f'x(u) - 6
¢'[X(U)][f(X(U)) - @(X(U))]

* C
At the stationary point x , it must be that x = O,

h'(u) = M exp{-f an}, (M > 0).(3.43)

* * * .
which implies that u = ¢(x ) = f(x ) is constant and p(t) = O

in (3.39), and therefore (3.38) gives

*
f'(x) =6 . (3.44)

The common features of the inverse problems discussed so
far are:
(1) the system matrices are constant (time-invariant systems),
(2) the optimization period is infinite.

The above-type of the inverse problem is of particular
interest, mainly because the problems involve a constant feedback

gain matrix for the control law.

41

More interesting optimization problems, however, are
formulated in terms of the time-varying system matrices and involve
a finite time period of optimization.

Jameson and Kreindler (1973) consider the following dynamic

system
(a) x = Ax + Bu , x(tO) = x0 , (3.45)
(b) a given control law

u = -Gx , (3.46)
and a performance index

N
(C) J = x'(N)FX(N) +'{ (xWQx + u'Ru)dt , (3.47)

0

where x is an n-vector of state variables, u an m-vector of
controls, N is a fixed terminal time. The matrices, A,B,G,Q, and
R are time-varying and assumed to be uniformly bounded and con-
tinuous on [toy]. In addition, Q =Q' and F = F' are positive
semi-definite and R = R' is positive definite.

The explicit expression for G in (3.46) in this problem
is given by

1

G = R- B'P (3.48)

where P is the positive semi-definite solution of the Riccati

equation

-I'> = PA + A'P - PBR-LB'P +Q
P(N) = F . (3.49)

*
The minimum value J of the performance index (3.47)

is

42
3* = x(t0)'P(tO)x(tO) . ' (3.50)

The inverse problem here is to find F, Q, and R in the
performance index (3.47), given the system equation (3.45) and the
control law (3.46).

Note that the existence of symmetric P, R, Q, and F
satisfying (3.48) and (3.49) is a necessary condition for a closed-
loop system x = (A - BG)x to be optimal with respect to the per-
formance index (3.47).

The solution of inverse problem is obtained by consider-

ing (3.48) , i.e. ,

RC = B'P (3.51)
or equivalently

RGB = B'PB , (3.52)

G'RG = G'B'P . (3.53)

Writing R = L'L, (3.52) implies

1 1 1

(L'1)'B'PBL' = (L'1)'RGBL' =LGBL' . (3.54)

If R = R' is positive definite and P = P' is positive

semi-definite, then (3.52), (3.53) and (3.54) imply

RGB = B'G'R , (3.55)
rank (BG) = rank (G) , (3.56)
GB has non-negative real eigenvalues, (3.57)

respectively. Therefore, if R = R' (positive definite) and
P = P' (positive semidefinite) could be constructed such that

(3.55) - (3.57) hold, then the chosen R and P satisfy (3.51).

43

Jameson and Kreindler (1973) developed a procedure for constructing
such R and P as follows.
By suitably choosing P, which is a positive definite,

real, symmetric matrix such that PA = AF, R can be chosen as
R =vr'v' , (3.58)

where V is a matrix of eigenvectors of B'G' and A is the
diagonal matrix of corresponding eigenvalues.
The matrix P can be constructed, by choosing a symmetric,

positive semidefinite matrix Y, as

P = G'R(RGB)+RG +-Y , (3.59)

where (RGB)+ denotes the Penrose generalized inverse of RGB,
i.e., (RGB)(RGB)+(RGB) = (RGB).

Once R and P are constructed as (3.58) and (3.59),
F and Q can be found from the Riccati equation (3.49).

It should be noted that Q so determined may not be non-
negative definite. However, the performance index of the form (3.47)
with the chosen P, Q, R, and P attains its absolute minimum J*
over all square-integrable controls for all x(to) and all

tO<NSoo.

3.2 Inverse Control Problem for Discrete-Time Systems

All the above inverse optimal control problems are con-
cerned with the continuous-time system equations. But there exist
the problems, eSpecially in such fields as biology and economics,

etc., for which discrete-time models are the natural ones to assume.

44

For instance, many of the functions of time representing
responses with time-lags that are met with in economics do not
correspond with continuous-time equations of known and simple form,
as analogous relationships usually do in, say, engineering fields.
The economic reSponses are not given "experimentally" and must,
mathematically, be considered to be of arbitrary form, and so are
conveniently Specified by the discrete-time models.

In other problems, where computation must be done on a
digital computer, the discrete-time model often results as suitable
approximation to a continuous-time system. A final important class
of discrete-time systems comes from sampled-data system. In sampled-
data systems, a continuous system is driven by an input specified
at discrete-time points and has state and output variables available
only at discrete-time points.

Wu and Schroeder (1968) solve the inverse problem for time-

invariant, single-input, discrete-time system. They consider the

 

dynamic system given by

= + = .
xt+1 Axt But , xto X0 , (3 60)
with performance index,
°° 2
J(x.) = 2 (x'H'Hx +'u ) , (3.61)
J _ t t t
t-J
and the control law given by
= - ' . .
ut g xt (3 62)

It is assumed that the given control law has the stability

property, i.e., xt+1 = (A - bg')xt is asymptotically stable.

45

The system equation (3.61) and the performance index
(3.63) are assumed to be in the canonical form (Wonham and Johnson,

1964; Tuel, 1967). That is,

 

 

 

 

 

 

A = '0 1 o 0‘1 , b = V o)
o o o
O O l L1J
L—al “'82 ... -anJ
H'H = ' qlnj . (3.63)
C) an
qun q2n ... qnn_,)

Then, the inverse problem is to determine n elements of H'H,

q , i = 1,2,...,n, given the system equation (3.60) and the con-

in
trol law (3.62).

The Riccati difference equation for this problem is given
by
P = H'H + A'P(A - bg') , (3.64)

where g' = (b 'Pb + 1)-1b'PA. (3.65)
Considering the canonical forms in (3.63), (3.65) can

be written as

' = F = ___—- ..
g g1) P +_1 F a1 Pnn 1 , (3.66)
82 P n -8 P

 

 

 

 

LgnJ gpn'lan -811 PnnJ

46

where P = [pij]’ i,j = 1,2,...,n. This gives an explicit solution

for the last column of the matrix P,

= + + :23 ..., , 3.
pi-lm (pnn 1)gi aipnn ’ 1 ’ ’ n ( 67)
where
81 3
p = “‘_—T:—“ - ( .68)
nn a1 g1

The elements in the matrix H'H can be easily obtained by sub-
stituting the canonical forms (3.63) into the Riccati equation
(3.64), and by using (3.67), (3.68).

It is seen from the above literature survey that the gen-
eral solution of the inverse problems has not yet been found. In
particular, no explicit solution of the inverse problem is known
for the discrete-time multi-control systems.

Considering the fact that the market framework of the
economic system is working through the myriad individual decisions
and the economic problems are generally formulated in terms of
discrete-time difference equations, it is desirable for the
economists that the solution to the inverse problem be found for
the discrete-time multi-control systems. The solution to this
subclass of the inverse problem is the subject of the following

chapter.

CHAPTER FOUR

THE INVERSE CONTROL PROBLEM OF DISCRETE-TIME MULTIVARIATE
CONTROL SYSTEMS

4.1 The Solution of a General Inverse Problem

In this section a general inverse control problem will
be formulated for an autonomous discrete-time linear system with
an g_priori linear feedback control law, and then a sufficient
condition will be develOped for the solution of the inverse prob-
lem. More Specifically, the inverse problem is considered for
the system

_ _ 0
— Axt + But , x — x (4.1)

x o

t+1

a feedback control law
u = -Gx , (4.2)

and with the performance indices restricted to the general quadratic
structure
on
J = Z [xéth + uéRut] . (4.3)
t=O
Here, x is an n-vector of state variables; u is an m-vector of

control variables; A and B are constant n X n and n X m

matrices; and Q and R are constant symmetric positive

47.

48

1)

semidefinite and definite matrices. The following assumptions
will be made:
(A.l) The matrix A is nonsingular.
(A.2) The matrix B is of full rank, m.
(A.3) The closed-100p system (4.1) with (4.2) is asymptotically
stable in large.2)
(A.4) The pair (A,B) is stabilizable and the pair (D,A) is
observable, where Q = D'D. ‘

The assumptions (A.l) and (A.2) are made mainly for
simplicity, and (A.3) and (A.4) are for the existence of positive
semi-definite solution of matrix Riccati equation (4.9) below.

(Aoki, 1973).

By introducing a new control variable
6 = u - u , (4.4)

the problem (4.1) - (4.3) can be transformed into an equivalent

 

problem:
xt+1 = Axt +But , (4.5)
St = -E§t , (4.6)
and
J = 2 x'Qx (4.7)
t=0 t t
1)

Optimal control of this problem involves maintaining the state
variables close to the zero state (origin) after some initial dis-
turbance away from the zero state. This problem is commonly re-
ferred to as a regulator problem (Dorato and Levis, 1971).

2) "Asymptotically stable in large" means that any initial points
away from origin approaches origin as time goes to infinity (La
Salle, 1962-63).

 

 

r W

_ ‘— A

Where x = xt , A = B ,

LUQJ 0 Im

_ 3 _

BJO , Q=[Q 0] , ....

I »0 R

L WJ

6-= [6, Im]'A. Note that the assumptions (A.l), (A.2), (A.3),
and (A.4) are preserved for the equivalent problem (4.5) - (4.7).

It will be seen that the inverse problem can be solved easily for

l
the equivalent system compared to the original system (4.1) - (4.3). )
Since the control policy (4.6) is optimal, the feedback
gain matrix is characterized by (Kleinman and Athans, 1966)
_ _u _ '1_u '—
G = (B PB) B PA , (4.8)
. . . . . . . 2)
where P satisfies the folloWing matrix Riccati equation
F =6+ Ema - BE) . (4.9)
In order to express 6' explicitly in terms of P, let P be
partitioned as
f 7
P11 P12
P = , (4.10)
.' P
L112 ZZJ

 

 

where P22 is an m X m nonsingular matrix. Then 6 in (4.8)

can be written as follows.

 

1)

Note that the performance index (4.7) for the transformed
system does not contain a quadratic term involving the newly de-
fined control vector (4.6).

2) Refer to Kleinman and Athans (1966) or Kuo (1970) for deriva-
tion of discrete Riccati equation.

50

E = P22 [P12 ’ P223K
= LP; P1'2 . Imj-K (4.11)
= [_G, Im1A ,
which shows that G = P5; P12’ the feedback gain matrix for the

system (4.1) - (4.3).
The closed-100p optimal system has the following property.

t
Lemma 1: The (n+m) h-order closed-100p system matrix

A - BE. for (4.5) - (4.7) is singular, and if A. is non-singular,

then rank (A - BG) = n.

Proof: From (4.8)

X - 1'3— = X - E(B'PB)-LB-'P_A = EX, (4.12)
where E e: In+m - B-(B'PB-YIB'P. Since EE = E, the idempotent
matrix,

rank (E) = trace (E)
= trace (In+m) - trace {(B'PB) 1B'PB}

n + m - m = n .

Therefore, rank.(EA) s n and thus EA) is singular. Furthermore

if 'X is nonsingular, then rank.(EA) = n, completing the proof.
The solution of the inverse problem for (4.5) - (4.7)
can be obtained as follows. Equations (4.8), (4.11), and (4.12)

implyl)

E'P(X - 13—5) = o , (4.136)

 

1

) Due to the transformation of the problem carried out in (4.4) -
(4.7), we have the following homogeneous equation (4.13) to solve
for P.

51

or

r"I o 7

n

3'1 X'= o . (4.13b)
L‘G 0.1

 

 

Since A is non-singular, A is also non-singular. Thus (4.13)

is equivalent to

B'P = o . (4.14)
L_-G OJ

By inspection, if P is chosen to be

 

 

P = w[c, 1m) , (4.15)

where W is an (n+m) X m arbitrary non-zero matrix still to be

1)

decided, then
In 0 In 0
P = w[c, I ] = o . (4.16)
-G o m -G 0
‘_ 0
On the other hand, since B = , P may be chosen to be
I
m

M
P =[ J , (4.17)
0

where M is an n x (n+m) non-zero matrix yet to be determined.

Then

_ M
B P -- [0, 1m] [0] = 0 . (4.18)

 

1) In fact, the matrix [6, Im] is an em X_(n+m)_ complete left
annihilator (with rank m) of the matrix (A - BC) in (4.13a),
i.e. [6, Im](A - BG) = O (Bodewig, 1959).

There

The'
opti

of t

the

H

and

is

apj

SpE

 

 

52

Therefore, the solution P of (4.13) is, from (4.15) and (4.17),

M
P {J +w[c, 1m] . (4.19)
o

The matrix P obtained by (4.19) still needs to satisfy (4.9) for
optimality, and thus the following theorem leads to the solution
of the inverse problem.

Theorem 1: A sufficient condition for the solution of
the inverse problem for (4.1) - (4.3) is that there exist M f 0,

w # o in (4.19) such that

and
6'= P - K‘P(K - 85)
is symmetric and positive semidefinite.
As it is intractable at the moment to find the

apprOpriate M and W in (4.19), a solution is sought for a

special case in the next section more explicitly.

4.2 The Solution of a Class of Inverse Problem

For the economic system, in contrast to physical and
technological systems, the relative weight on the state vari-
ables is more significant and heavier than the weight on the
control efforts to be implemented. In fact, the existing
government structure, for example, requires a fixed (basic)
operational budget and does not require additional expenses in
implementing a designed Optimal control policy, because it is quite
possible that the only costs associated with government policy are

associated with levels and not with any changes in the policy.

ing U

is ze

afe

and

whe

lem

Can

can

wit

1)
the

 

we)

53

Thus this section is devoted to the case where the weight-
ing matrix R for the control variables in the performance index
is zero. This leads to an explicit formula for the solution of
the inverse problem.

The inverse problem is considered for the system

_ _ O
xt+1 - Axt +‘But , x0 - x (4.20)
a feedback control law
= .2
ut -Gtxt (4 l)
and with a modified performance indices of the forml)
N
= c
J E xtht (4.22)
t=O

where rank 62) = rank (B).

Since the system (4.20) is controllable, the above prob-
lem (4.20) - (4.22) can be transformed into the control equivalent
canonical form by applying a nonsingular transformation T to the
state variable x. We assume that (4.20) - (4.22) are in the

2
canonical form (Tuel, 1967) ); that is

A = , B = , (4.23)

with the following properties:

(i) B2 is a nonsingular (m x m) triangular matrix,

 

1) Now problem involves a finite-time Optimization in contrast to
the previous section 4.1.

2) This transformed form is convenient for simplifying subsequent
work and obtaining compact results. See (4.31) - (4.33) below.

54

(ii) the matrices A21 and A22 are m X (n-m) and
m X m respectively but are otherwise arbitrary,

(iii) there exists a set of m positive integers {Li}

depending on the structure of the system such that

 

 

m
2 L. = n,
i=1 1
(iV)
M ‘
A =:
11 0 W2 0
O O
L'0 ......... O me
rel O ..... 0 2
A12 = 0 e2 0 . O
0
L0 ......... O emj

 

 

where A11 is (n-m) X (n-m), A12 is (n-m) X m; W1 is a

(Li - 1) X (Li - 1) matrix of the form

F0 1 o .... 05

 

 

W = O O l O ... 0
I 1 O
O ......... O 1

L0 ........... OJ

and ei is a (L.

1 - 1) column vector of the form

F0)
0

 

 

L13

55

Since the control policy (4.21) is optimal, the feedback

gain matrix is characterized by (Kleinman and Athans, 1966)
= I " I
Gt (6 Pt+l B) 113 Pt+lA . (4.24)
where Pt satisfies the following matrix Riccati equation

= ' - = '-
Pt Q+APt+1 (A BGt) ,t O,l,...,N 1,

IN =Q, (4.25)

It is always possible to factorize a positive semi-

definite matrix Q into the product DD' where D is a matrix
Of full rank corresponding to the rank of Q. Thus (4.24) and

(4.25) lead to

PN_1 = Q + A'Q[A - B(B'QB) lB'QA]

= Q + A'DD'[A - B(B'DD'B)'lB'DD'A]

_1 _ (4.26)
= Q + A'DD’A - ADD'B(D'B) (B 'D) lB'DD'A
=Q,
provided that the indicated inverse exists. Note that (4.26)
implies, for all t = O,l,...,N,
Pt =Q, (4.27)
ct = c = (D'B)-]'D'A , (4.28)

which implies that the admissible control law (4.21) must be a

constant feedback control. Moreover, (4.28) implies
D'(A - BG) = 0, (41-29)

which is analogous to (4.13) for the general inverse problem.
Since A - BC is singular by Lemma 1, (4.29) has a non-

trivial solution for D'. The matrix solution D can be chosen as

Iti

Mo rt

Sin

“OH

fro

 

 

 

The

 

W

US

 

 

56

0' = 23 (4.30)

where S is a complete left annihilator of (A - BG), i.e.,
S(A - BG) = O, and Z is an arbitrary m X m nonsingular matrix.

Note that, for P defined in (4.10)

A - B6 A - B(B'PB) -1B'PA

[In - B(B'PB)-1B 'P]A (4.31)

I l O
n-m

ll
3,

-P’1P' 0

22 12

It is easy to see from (4.31) that

_. " I
Moreover,
-1 O
' = = ' =
2

Since both Z and B2 are nonsingular in (4.33), D'B is also
nonsingular. Thus, (D'B)--1 exists as is necessary in (4.26).
The solution of the inverse problem for the canonical

from (4.21) - (4.23) is therefore

Q = DD' = S'Z'ZS . (4.34)
The solution for the original system is then simply

6 = T'QT = T'S'Z'ZST , (4.35)

where T is a non-singular matrix for the canonical transformation
leading to (4.21) - (4.23).
The rank of Q and Q can be determined as follows by

using the Sylvester's inequality

57

rank (S'Z') + rank (ZS) - m 5 rank (S'Z'ZS)
(4.36)
5 {rank (S'Z'), rank (ZS)}.

Since rank (S'Z') = rank (ZS) = m, (4.36) implies that
rank (0) = rank (Q) = m = rank-(B).
It has been observed that the solution of the inverse

problem can be obtained by determining P22 P12 from the known
matrices A, B and G in (4.31). The computational procedure
will be illustrated through examples in Chapter Five. One must
also note that the solution of the inverse problem is not unique

because it depends upon the choice of a nonsingular matrix Z as

is shown in (4.34).

4.3 The Inverse Optimal Control Problem in Macroeconomic Policy

Models

The basic approach develOped in the previous sections
will be extended to a macroeconomic model. Although the system
is in a more complicated form than what was discussed in the pre-
vious sections, it will be seen that the solution of the inverse
problem developed in Section 4.2 also holds true for this economic
system.

For many economic policy problems, linear macroeconomic

models can be reduced to the following equation (Pindyck, 1972)

= + .
yt+1 Ayt But + Czt + VC (4 37)

where yt ... n-vector of endogenous variables,
u ... m-vector of control variables,
2 ... r-vector of other exogenous variables outside the

policy consideration,

58

v ... n-vector of random disturbances with zero mean
vector and a constant variance-covariance matrix,
A,B,C, ... constant matrices.
The deterministic policy question is formulated by

examining the conditional behavior of y in (4.37) under the

t+l

influence of ut given that the policy-maker sets the disturbance

vt equal to zero and utilizes a projection of values of z

t,
t = O,l,...,N. Then, for the deterministic policy problem, (4.37)

can be written as

= + + . .
yt+1 Ayt But Czt (4 38)

The performance index is given by

N-l
J = E(YN - 9N) 'Q(yN - 9N) + A: E (yt - it) 'Q(yt - it) + ut'Rut. (4-39)

t=0

where yt is an n-vector of target values of yt, which is

Specified for the entire planning period.1) The assumptions (A.l) -

(A.3) in Section 4.1 will also be made here for simplicity of the

analysis.

Using the discrete-time maximum principle and the matrix
identity, I - (Y +-X)-1X = (Y +-X)-1Y, the Optimal control policy

for the system (4.38) - (4.39) can be expressed as (See Appendix):

u*=-(R+B'K B)-1B'I( A -(R+B'K B)-]'B'
t t+1 t+l yt t+1 gt+1

I " I
-(R + B K B) 13 x“, Czt , (4.40)

t+l

where Kt and gt satisfy the following equations

 

1) Optimal control problem for (4.38), (4.39) is Often referred
to as a linear tracking problem (Athans and Falb, 1966).

59

= ' - ' '
Kt Q + A KtflEA E(R + B KH 1B) 113 Kt+1A , (4.41)
= _ I I ' I _
gt A [Kt+1 B(R + B Kt+1 B) :1]; 13g”1
l _ I _ A
+ A (Kt+1 Kt+1 B(R + B Kt+1 B) 113' ')CI<t+1 Qyt . (4.42)

Now, consider the case where R = O in the above model. Let the
positive semi-definite matrix, Q be factorized into the product
DD' where rank (D) = rank «2) and (D'B) is nonsingular. Then,

the matrix Riccati equation (4.41) becomes

_ I _ I ___

— Q + A Kt+1[A B(B Kt+1 B) 13' 'AKt+1 , KN Q, (4.43)
and as in Section 4.2,

KN_1 = Q + A'DD'[A - B(B'DD'B)-]‘B'DD'A] (4.44)

= Q + A'DD'A - A'DD'B(D'B)-1(B'D)-lB'DD'A =Q,

which implies Kt =‘Q for all t = 0,1,2,...,N. Similarly, (4.42)

is reduced to

_ _ I I ' I _
gt - A [Kt-+1 B<B 1<t+1 B) ’B I]gt+1 (4.45)

I _ I " I _ A
+ A [Kt+1 Kt+1 B(B Kt+1 B) 18 Kt+1]Czt Qyt,

and

-A'[DD' B(B 'DD '3) '18' - I] (4233,)

gN-l

+ A'[DD' - DD 'B(B 'DD 'B)'1B 'DD']CzN - Q9},

A'[D(B 'D) '18' - 13m) 'yN

+ A'[DD' - DD'B(D'B)-1(B'D)-LB'DD']CZN ' Q9101

.QYN‘I, (4'46)
which implies that

8,; = 'Q’S't . t = O,l,...,N . (4,47)

6O

*
Again, consider the Optimal control policy ut in (4.40)

for the period N-l, i.e.,

-(B'DD'B)'IB'DD'AyN_1 - (B'DD'B)'1B'gN - (B'DD'B)-1B'DD'CzN_1

'7"
H"

1 l

-(D'B)' D'AyN_1 + (D 'B)'1(B 'D)-LB'DD 'yN - (D 'B)- 0%sz

-(D'B)']‘D'AyN_1 + (D'B)'1D'yN - (D'B)'lD'CzN_1 , (4.48)

which implies that

1

C.‘
II

-(D'B)-1D'Ayt + (D'B)- D'§t+1 - (D'B)-1D'Czt (4.49)

III

.ey +13) -Fz ,t=o.1....,N-1.

t t+l t

Thus the optimal feedback gains are

(D'B)-1D'A

=G’
(D'B)-1D' - E, (4.50)
(D'B)-1D'C = F .
The equations in (4.50) are equivalent to
D'(A - B6) = 0,
D'(I -BE) =0, (10-51)
D'(C - BF) = 0,
which implies that
D'(A - B6) = D'[A - B(D'B)-1D'A] = 0'[1 - B(D'B)-1D']A = 0
D'(I - BE) = D'[I - B(D'B)-1D'] = 0 (4.52)
D'(C - BF) = D'[C - B(D'B)-1D'C] = 0'[1 - B(D'B)-1D']C = 0

Equations in (4.52) show that a solution for D' for
any equation will also Satisfy remaining two equations. This

implies that the solution of the inverse problem for a linear

61

regulator model in Section 4.2 Still holds true for this linear
tracking problem. Using the Similar arguments that were made in

Section 4.2, the solution for D' in (4.52) is

D' I 28 , (4.53)

where s[1 - B(D'B)‘1

D'] = 0, and Z is an arbitrary m X m non-
singular matrix. Thus the required solution for the weighting

Inatrix Q is given by
Q = S'Z'ZS . (4.54)

The Steps Obtaining the solution of the inverse problem
were studied in Section 4.2.

Recall that the inverse problem (4.38) - (4.40) in this
section was set up for the macroeconomic policy problems. Thus,
the solution of the problem provides the explicit procedure to
quantify the performance index (or preference function) for the
policy making processes, enabling the decision maker to get the
numerical measure of the relative "welfare" weights to be given to
the competing policy goals.

Note, however, the solution (4.54) is not unique in this
problem where the performance index does not contain the costs of
adjusting the control policies. This implies that the Optimal con-
trol policies are fairly robust against different performance indices
under the conditions stated in this inverse problem.

The develOped techniques so far will be illustrated

through examples in the following chapter.

CHAPTER FIVE

APPLICATIONS OF THE INVERSE OPTIMAL CONTROL TO
MACROECONOMIC MODELS

The dynamic economy under study is characterized by

the following short-run macroeconomic model.1)
pt-I-1 = alpt + aZwt ' azyt + aBmt
rr+1 = blpt + b2rt + b3yt - b4mt (5.1)
ut-I-l = C1px: + Czut + c3”: ' C43,1:

where
p; price
r; interest rates
U; unemployment rate
m; money supply
y; real production

w; money wage rate

a., b., c ; constant coefficients (positive),

(all the variables are in terms of quarterly change rates.)
The price equation in (5.1) is basically an inflationary
model which involves the cost-push effects of wages and the

aggregate demand-pull effect. That is, price increases are

 

1) This is a submodel of Pindyck's short-run macroeconomic policy
model of the U.S. (1972), Slightly modified by Shupp's inflationary
model of the U.S. (1972).

62

63

positively related to increases in average unit labor cost and
to increases in money supply.

The second equation in (5.1) expresses the interest
adjustments to the disequilibrium in the markets. Since demand
for bonds changes together with changes in real value of bonds and
money stock (hence, inversely to the price changes) and the supply
of bonds (investments) are positively related to the changes in
income, interest rate variations are positively related to price
and income changes, and inversely to the money supply.

The last equation of the model explains the changes
in unemployment rates in the labor market.

Now, define a performance index for the system (5.1)

as follows:

J= 1:20: -x*)'Q(x -x*) (5 2)
t=0 t t t t ’ '
where xt = (pt, rt, ut)' and x: is the target trajectory of
xt, and Q ==Q' is positive semi-definite.
For the purpose of illustration, we assign numerical

values to the coefficients of the system (5.1) based on the models

of Pindyck (1972) and Shupp (1972).

pH,1 0.77 O 0 pt 0.02 0.23 -0.23
rt+l = 0.48 0.37 0 rt +' -0.16 0 mt +‘ 0.03 yt
ut+1 -0.01 0 0.8 ut 0 0.002 wt -0.0004
(5.3)
Eigenvalues of the system (5.3) are 11 = 0.77, 12 = 0.37,
and k3 = 0.8. Since all the eigenvalues are real and less than

unity in absolute value, the basic homogeneous solution for the

64

time paths of state variables, pt’ rt, u , apart from the influence

t
of exogenous variables and of initial conditions, are damped, non-
oscillatory movements, i.e., all reSponses of the open-100p economic

system converge to their equilibrium values without oscillations

(Kuo, 1970).

5.1 A Single-Control Model

We consider the first two equation submodel of the system
(5.3), assuming that the labor market is always in equilibrium and
also noting that unemployment rate ut influences neither the price

changes pt nor the interest rate variations rt.

Pt+1 0.77 0 pt 0.02 0.23 -0.23 wt
= + mt+

rt+1 0.48 0.37 rt -0.16 0 0.03 t

. (5.4)

In the submodel (5.4), wt and yt are treated as vari-
ables outside the policy considerations. For the purpose of
simplicity, it is assumed that the goal Of policy is to maintain
a stable price and interest rate, i.e., p* = r* = 0, and at the
beginning of the planning period, both price and interest rates
have been stabilized; p0 = r0 = 0. In addition, as the model is
of the short-run nature, constant values are given to wt and yt;
wt = 0.9%, yt = 0.38%.

For the case with zero control, mt = 0, i.e., keeping

money supply constant, the behavior of pt and rt over the

period is shown in Figure l.

 

65

pt = - 0.52(0.77)t + 0.52
rt a - 0.62(0.77)t+ 0.21(0.37)t+ 0.41

 

 

 

_,..___.pt
——————— -l--'----¥rt
l l
5 10 15 20 t
Figure 1. Responses of System (5.4) with Zero Control

It is seen from Figure 1 that when the money supply is
kept constant, the policy goal, pt = rt = 0 (t > 0), is not
attained.

Now, assume that policy makers want to achieve price
stability using a feedback control of the form (4.49) in Chapter
Four:

*
mt = glpt + 82rt +'k , (5.5)

where g1, g2, and k are constant scalars.
Specifically, it is desired that zero steady-state value

of p be attained by using a control of the form (5.5).

t
A digression is in order at this point. Let the system

(5.4) be written as

x = Axt +Bmt+Czt , (5.6)

t+1

= ' _-_—_ ' = I
where xt (pt , rt) , zt (w , yt) (0.9, 0.38) ,

t

66

0.77 0 0.02 0.23 -0.23
0.48 0.37 -0.16 0 0.03

Note in this example, Czt is a constant vector.

From (4.49) in Chapter Four, the feedback control

policy (5.5) is also written as

-1
= + 5.
mt gxt gA CZt , ( 7)

where 8 = (81,82)
Then the system under the feedback control (or
closed-loop system) is governed by the relation

_ -1
xt+1 - (A + Bg)xt + (BgA +I)Czt . (5.8)

The steady-state values of xt in the closed-loop system (5.8)

is given byl)

X
II

(A + Bg)xe + (BgA’1 +1)Czt

013'

:4
ll

(1 - A - Bg)-1(BgA-1 + neet . (5.9)

From (5.9), we derive
-1 e e
Bg(A Czt + x ) = (I - A)x - Czt . (5.10)

By solving (5.10) for g, it is possible to find the required feed-
back control rule for a prescribed steady-state value vector xe
of the state variables.

Note, however, that g(A-1c2t + xe) in (5.10) is a

scalar. Define

801*th +xe) = k, (5.11)

where k is a scalar.

 

l
) Since the closed-loop system (5.8) is assumed to be stable,
the indicated inverse in (5.9) exists.

67

Then it is seen that in order to solve (5.10) for g,
xe should be chosen Such that

xe = (1 - A)'1(Bk + Czt) (5.12)

holds.1)

If xe in (5.12) is considered as an arbitrarily chosen
known vector, (5.12) becomes a two-equation system with only one
unknown k, and in general, cannot be solved. One element of x8
should be set "free" to be determined together with k by the
system. This implies that only one element of xe can be

2)

specified arbitrary if k is a scalar. That is, if there is

only one control variable in the model, only one state variable Of
the closed-100p system can be steered to have the prescribed steady-
state value. This can be generalized that the number of the policy
goal variables which can be controlled to attain the steady-state
values is equal to the number of the control variables in the model
(cf. Tinbergen, 1967).

As the state variable xt = (pt , rt)' in (5.8) are
required to have the steady-state values, the closed-loop system
must be stable, and eigenvalues of the closed-100p system should be
restricted to be less than unity in absolute magnitude. This implies
that the feedback gain vector of the required control for the price

Stabilization should be determined so as to make the closed-100p

system be stable ("arbitrary pole assignment problem" in Chapter Two).

 

1

) Since all the eigenvalues of A are less than unity in absolute
value, (I - A)‘1 exists.
2
) k in (5.11) is a scalar since the system (5.6) has one control,

m o
t

68

That is, from (2.5) in Chapter Two, the feedback gain

vector 3 needs to satisfy

g = h'T.1 or gT = h' , (5,13)

where h' = (l, 1) and a nonsingular matrix T = (t1 , t2). The
columns of matrix T, t1 and t2 are determined for the pre-

scribed closed-IOOp eigenvalues )1 as follows.

(111 - A)ti B . i = 1,2 . (5.14)

Equations (5.13) and (5 14) imply

 

gti = 1 . (5.15)

Since the solution for the inverse problem requires
that the closed-loop system matrix, A + Bg, is singular (Lemma 1,
Chapter Four), we assign k1 = 0 for (5.14) and find the
corresponding t1.

Then, from (5.11) and (5.15), the following relations
should be satisfied simultaneously for the required feedback gain

vector g:

-0.026535 g1 - 0.215176 g2 = 0.9428

-0.026 g1 + 0.4661 g2 = l . (5.16)

Solving (5.16) for g1 and 82’ we find the following

"desired" feedback control policy:

*
mt = -36.4 pt + 0.11 rt - 5.7 . (5.17)

The behavior of the resulting closed-loop system is shown in

Figure 2.

69

 

 

a 12
pt = - 0.01(0.39) + 0.01
rt = - 1.52(o.39)t + 1.52
2.0—
r
1.0—
0.
O J l IJJI' t

 

5 10 15
Figure 2. Responses of System (5.4) under Control (5.17) 4

 

Given the chosen feedback control policy (5.17), the 1
weighting matrix Q in the performance index can be determined
by the method of inverse Optimal control problem discussed in
the previous chapter.
Recall (4.52) - (4.54) in Chapter Four:
D'(A + Bg) = 0,
Q = DD'
For the given system (5.4) and the feedback gain vector
8 (5-17).
0.04 0.00224 1 0
A-+ Bg = = E , (5.18)

6.31 0.352 157 0

where E is the inverse of the product of the elementary (column)

operation matrices. Thus,
D' = z(-157 l) , (5.19)

where z is a non-zero, otherwise arbitrary, scalar, Choosing

z = 1,

70

24649 -157

Q = . (5.20)
-157 1

From Figure 2, it is seen that one of the targets,
price stability, is now attained by the prescribed feedback con-
trol policy. Also, Since the dominant eigenvalue of the closed-
lOOp system, 0.39, is smaller than that 0.77 of the zero-control
system, trajectories of both price and interest rate converge to
their steady-state values in shorter time in this case.

Note, however, that the steady-state value of the
interest rate is higher in the closed-100p system with the chosen
control rule (5.17). This can be explained by the fact that the
optimum control policy involves the continuously decreasing money
supply (Figure 3), effecting the price stability, but on the other

hand, causing the interest rate to increase continuously.

 

 

 

Z
l l l
O 5 10 15 t
WV? _AA A.— __‘v—v W
W A::W'A AV;A WW
-4.0b
mt = - 36.4 pt + 0.11 rt - 5.7
-500-
N ’: mt

 

 

Figure 3. Optimal Feedback;Control for Price Stabilization

 

71

It is to be noted that the weighting matrix Q in
(5.20) gives the very heavy weight to the price variable compared
to the interest rate. Negative sign of the Off-diagonal elements
of Q indicates that there exists "trade-off" between the monetary
policy for price regulation and that for_interest rate control.

Anologously, another feedback control policy is found

for interest rate stabilization:

*
"‘t - 313': + 2.31 rt + 0.06 . (5.21)

 

Time paths of price and interest changes and of the
money supply changes under this policy (5.21), are Shown in
Figure 4 and Figure 5, respectively.

In this case, the closed-loop system matrix is

A + bg = 0.83 0.05

-0.001 -0.00006
and the weighting matrix Q is found to be

Q = 1 0

0 594441 . (5 .22)

AS expected, in (5.22) the interest rate variable is
given larger weight compared to the price variable.

One remark concerning the chosen control policies (5.17)
and (5.21): the continuously decreasing or continuously increasing
money supply policy would not be feasible in reality, for, say,
political reasons, if the change rates are big all over the period.
It is important, however, to observe that in the examples, the

control is not constrained and no cost for conducting control policy

72

pt . - 0.71(0.83)t + 0.71
rt - - 0.001(0.83)t + 0.001

N

 

 

 

 

 

 

t
It:
0.5L-
0.
L 11 1 .L I rt
0 5 10 15 20 H t
Figure 4. Responses of the System (5.4) under Control (5.21).
2
mt = 3 pt + 2.31 rt + 0.06
3.0“—
mt
r—1I>-
l l
15 20 t

 

Figure 5. Optimal Feedback Control for Interest Stabilization

73

is included in the performance criterion, and accordingly the con-
trol is free to vary in the control Space (i.e., the real line in
this case). Obviously, further research needs to be done when the
controls have some constraints and/or the costs of conducting the

policies are incOrporated in the performance index.

5.2 Two-Control Model I

Dropping the perfectly competitive, full employment t

assumption for the labor market, we will consider the overall

 

system given by (5.3).

In this model,we choose wt as a control variable,
since in an assumed imprefectly competitive market, the discre-
tionary pricing power of unions can be used to demand monetary
wage increases in excess of productivity gains. TO the extent
that price expectations are based upon recent price changes, these
demands for larger monetary wage increases will persist in the
absence of any continuing excess demand. Furthermore, these large
increases in money wage rates induce a higher rate of price in-
flation which confirms the original expectations. In such
circumstances, temporary wage-price controls by curbing these
expectations may prove effective. Since it is generally agreed
that wage controls are the more easily administered, wt is chosen
to be a control variable.

The time paths of state variables for the system with
zero controls, mt = 0 = w , i.e., when both money supply and money

I:

wage rate are kept constant, are shown in Figure 6.

74

 

 

 

 

 

 

 

 

 

 

pt - 0.38(0.77)t - 0.38
0.1 - rt - O.456(0.77)t- O.186(0.37)t- 0.27
ut - 0.127(0.77)t- 0.147(0.8)t + 0.02 u
0 1 '1 --l>--T—" t
15 20 t
—001
-0.2
~‘..“‘————--->————t;
.3
f p1
-0.4 (-
Figure 6. ReSponses Of System (5.3) with Zero Control
In order to have price stability with a linear feed-
back control, the chosen feedback gain vector g = (g1, 82, g3)'
should satisfy the following relations simultaneously [(5.11),
(5.15) above]:
-0.028125 g1 + 0.027015 g2 + 0.000348 g3 = 0.08763
-0.3247 g1 + 0.8536 g2 - 0.00656 g3 = l
-O.5 g1 +,4 g2 - 0.01321 g3 = l . (5.23)
From (5.23), the following feedback control policy is
Obtained:
= a(-3-44 pt - 0.21 rt - 9.76 ut + 0.37) (5.24)

*
m
t
*
w
t

75

1)

where a = [l] .
l

The resulting time paths of state variables and of the
control variable are shown in Figure 7 and Figure 8, respectively.
With the selected control policy (5.24), the closed-

100p system matrix is

A‘+ Bag = -0.09 -0.053 -2.44
1.03 0.404 1.56
-0.02 -0.0004 0.78
= 1 0 03
0 1 0 E ,
-0.35 -0.05 oj

 

where E is a nonsingular matrix.

Applying the inverse Optimal control method as dis-

cussed in (4.52) - (4.54),

D'(A+ Bag) = 0,
DD' “<2 .

where D' = z(0.35, 0.05, l) , (5.25)

the weighting matrix Q in the performance index is found by

setting 2 = l in (5.25) to be

 

(0.12 0.02 0.35‘
Q = 0.02 0.002 0.05 (5.26)
10.35 0.05 1 J

 

 

1
) For the computational convenience, the system is converted to

a single-control model [(2.8) - (2.11), Chapter Two].

76

 

 

 

 

 

 

 

 

 

 

z
0.01- ‘1t
0 - ‘—--T—-—4-_——_—_l=.:——-.—-+-_--pt
0 ‘” ‘ ‘
| 5 t 10 t 15 t
-0.01‘ pt - -0.008(O.27) +0.007(0.824) +0.001
-0.02*I; rt = 0.063(0.27)t+0.007(O.824)t- 0.07.
\ ut = -0.0002(0.27)t-0.0028(0.824)t+0.003
\
\
\
—0.051- \
\
\
\\
_________________ _F — -..—rt
Figure 7. Responses of System (5.3) under Control (5.24).
2
mt - - 3.44 pt- 0.21 rt - 9.76 ut + 0.37
0.36-
ml:
0.35.— "
J l
O 5 10 15 t

Figure 8. Optimal Feedback Control for Price Stabilization.

77

Similarly, the control policy which would achieve full
employment is:
*
mt
* =a(-7.7 pt+0.07 rt+237.1 ut+0.5)

W
t

where (y = (l , l)' . (5.27)
The correSponding time paths of the state and control
variables are shown in Figure 9 and Figure 10.

The weighting matrix for the performance index, in this

 

case is
2
r(0.021) 0 -0.021‘
2
Q = 0 (0.0007) 0.0007 (5.28)
L-0.021 0.0007 1 J

 

Comparing the two cases considered so far for the system
(5.3) (price stabilization policy and full employment policy), we
observed the following.
(i) The model reveals the familiar Phillips Curve relation for the
price changes and unemployment variations. That is, when the
steady-state value of ut is reduced from 0.003% to 0.001%, that
of pt increases from 0.001% to 0.045% (Figure 7 and Figure 9).1)
(ii) Relative weight for the price variable is higher for the price
stabilization policy compared to the full employment policy (vice

versa), even though the major weight is found to be given to the

unemployment variable in both cases.

 

1) Magnitudes of figures may be too small to claim the Phillips
curve relation. However, the figures show the essential char-
acteristics of the curve.

78

 

 

 

 

 

 

 

 

 

z ..
0.05 —
t t ' t
=0.068(0.27) -0.1l3(0.21) +0.045
rt = —0.S44(0.27)t+0.592(0.21)t-0.048
0'02 u =0.0018(0.27)t—0.0028(0.21)t+0.001
t
0.01 n
4.1 I t
0 5 31V 15 i t
-0.01
—0.02
I
i
‘ r
—0 05 ‘ , —-- ——————————————— -.r — — — —E—
. \ /
-0.06 \ I/
\_,r
Figure 9. Responses of System (5.3) under Control (5.27).
Z”
mt = - 7.7 pt-l- 0.07 rt + 237.1 ut + 0.5
0.40—
0.39—
.__ .13.:_. m~
0.38—

 

Figure 10. Optimal Feedback Control for Full Employment.

CHAPTER SIX

CONCLUSIONS AND RESEARCH RECOMMENDATIONS

6.1 ‘Summary of Conclusions

By solving the inverse optimal control problem, this
study has demonstrated the feasibility of quantifying performance
criteria for the economic decision making problems. Economic
processes, which are characterized by complex and Often unknown
relationships among their constituent components, require one to
formulate the decision making problems in general, in terms of dis-
crete-time difference equation systems. This demands the solution
of the inverse problem for the discrete-time multi-control systems
for the choice Of the performance indices.

A sufficient condition was developed for the solution
of the inverse problem for a linear discrete-time multi-control
regulatory process with a quadratic performance index. An explicit
solution was obtained for a special case where the performance
functional does not contain the control variables, and the result
was further extended to a linear tracking problem, which could be
easily adapted to the macroeconomic regulation problems.

It has been shown that the solution of the inverse
problem is not unique in general, which implies that the optimal
control policies are fairly robust against different performance

indices under the conditions stated above. Also, it was found that

79

80

the solution of the inverse problem for a linear regulator holds
true without any modification for the linear tracking problem.

In Chapter Five, the illustrations about the applica-
tions of the developed techniques for the selection of the per-
formance criteria emphasize three points; the determination of the
steady-state values of the trajectories of goal variables which
are under control, the computation of the feedback control policy,
and the construction of a performance index. It was shown that
the number of the policy goal variables that can be made to attain
the prescribed steady-state values is equal to the number of the
control variables in the model. It was also demonstrated that the
relative weights given to the competing policy goals in the per-
formance index can be quantitatively determined by the method Of

the inverse Optimal control problem for a dynamic policy model.

6.2 Recommendations for Further Studies

Further research is needed to find a general solution
of the inverse problem for the performance indices which involve
the state variables as well as the control variables. This will
enable one not only to quantify a more general class of performance
functional but also to compare quantitatively the relative merits
of different control policies.

Another line of research can be conducted for defining
the performance functional of a more general type, which could
even reflect the sociological, group dynamic, and gametheoretical
aSpects of the administrative interactions involved in the determina-

tion Of policy objectives.

81

Considering the fact that the economic decisions are
typically made under uncertainty and decision makers increase their
knowledge by the cumulative past experiences, the study of the
adaptive control problems would greatly improve our knowledge of
the economic decision processes.

Since the adaptive systems include a performance index
as an essential function which permits correction of system dynamic
reSponse during actual operation, the performance index takes on
much greater significance in the multistage adaptive decision pro-

cesses (Murphy, 1965).

B IBLIOGRAPHY

BIBLIOGRAPHY

Adorno, D.S. (1962), "Optimal Control of Certain Linear Systems
with Quadratic Loss," Information and Control, Vol. 5, pp. 1-12.

 

Advani, R. and R. Mukundan (1970), "Optimal Control Methods Applied
to Economic Systems," International J. of Systems Science,
Vol. 1, NO. 2, pp. 153-72.

Allen, R.G.D. (1966), Mathematical Economics, St. Martin's Press,
New‘York.

 

Anderson, B.D.O. and J.B. Moore (1971), Linear Optimal Control,
Prentice-Hall, Englewood Cliffs.

 

Aoki, M. (1967), Optimization of Stochastic System, Academic Press,
New‘York.

 

(1973), "Sufficient Conditions for Optimal Stabilization
Policies," Review of Economic Studies, Vol. XL(1), NO. 121,
Jan., pp. 131-8.

Arrow, K.J . S. Karlin, and H. Scarf (eds.) (1958), Studies in the
Mathematical Theory of Inventory and Production, Stanford
Univ. Press, Stanford.

 

Arrow, K.J., M. Kurz (1970), Public Investment, the Rate of Return,
and Optimal Fiscal Policy, Baltimore.

 

Athans, M. and P. Falb (1966), Optimal Control -- An Introduction
to the Theory and Its Applications, McGraw Hill, New York.

 

Barnard, G.A. (1959), "Control Charts and Stochastic Processes,"
J. of Royal Statistical Society, Series B., Vol. 21, pp. 239-71.

Bellman, R.E. (1957), Dynamic Programming, Princeton Univ. Press,
Princeton.

Bellman, R. and R. Kalaba (1961), "Dynamic Programming and Feedback
Control," 1960 Proc. IFAC, Butterworths, pp. 460-76.

(1963), "An Inverse Problem in Dynamic Programming and
Automatic Control," Journal of Math. Anal. and Appl., Vol. 7,
NO. 2, Oct., pp. 322-35.

82

83

Bellman, R. (1970), "Dynamic Programming and Inverse Problems in
Mathematical Economics,‘I Journal of Math. Anal. and Appl.,
Vol. 29, No. 2, Feb., pp. 424-28.

Benavie, A. (1970), "Economics of the Maximum Principle," Western
Economic Journal, Vol. VIII, NO. 4, Dec., pp. 426-30.

Ben-Porath, Y. (1967), "The Production of Human Capital and the
Life Cycle of Earnings," J. of Political Economy, Vol. 75,
NO. 4, Aug., Part I, pp. 353-65.

Bodewig, E. (1959), Matrix Calculus, North-Holland, Amsterdam.

 

Boltyanski, V.G., R.V. Gamkrelidze, E.F. Mishchenko, and
L.S. Pontryagin (1961), "The MaximquPrinciple in the Theory
of Optimal Processes of Control," 1960 Proc. IFAC, London,
Butterworths, pp. 454-59.

 

Brockett, R.W. (1970), Finite Dimensional Linear Systems, John Wiley,
New York, London, Sydney, Toronto.

Brown, J.F. and B.A. Kalymon (1971), "A Simulation Programming
Approach to Bank Asset Management in an Uncertain Environment,"
Working Paper NO. 171, Western Management Science Institute,
UCLA, Los Angeles, March.

Buchanan, L.F. (1969), "Problems in Optimal Control of Macroeconomic
Systems," Computing_Methods in Optimization Problems, (eds.)
M. Beckmann and H.P. Kfinzi, Springer-Verlag, Berlin, Heidelberg,
New York, pp. 30-42.

Bullock, T.E. and J.M. Elder (1971), Wluadratic Performance Index
Generation for Optimal Regulator Design," Paper No. W8-3, Proc.
1971 IEEE Conference on Decision and Control, Univ. of Florida,
pp. 123-4.

Burmeister, E. and A.R. Dobell (1972), "Guidance and Optimal Control
of Free-Market Economics: A New Interpretation," IEEE Trans.
Systems, Man, and Cybernetics, Vol. SMC-Z, No. 1, Jan., pp. 9-15.

 

Butkovskii, A.G. (1964), "The Necessary and Sufficient Conditions
for Optimality of Discrete Control Systems," Automation and
Remote Control, Jan., pp. 963-70.

 

Canon, M.D., C.D. Cullum, Jr., and E. Polak (1970), Theory of
Optimal Control and Mathematical Programming, McGraw-Hill,
New‘York.

Chen, R.T.N. and D.W.C. Shen (1968), "Sensitivity Analysis and
Design of Multivariable Regulators Using aIQuadratic Performance
Criterion," Proc. 1968 Joint-Automatic Control Conference of
the AACC. The Univ. of Michigan, pp. 229-37.

84

Chow, G. (1970), "Optimal Stochastic Control of Linear Economic
Systems," J. of Money, Credit, and Banking, 11., Aug., pp. 291-302.

(1972), "Optimal Control of Linear Econometric Systems

with Finite Time Horizon," International Economic Review, Feb.,
pp. 16-25.

Dobell, A.R. and Y.C. Ho (1967), "Optimal Investment Policy: An
Example of Control Problem in Economic Theory," IEEE Trans.
on Automat. Contr., Vol. AC-12, No. 1, Feb., pp. 4-14.

 

 

Dorato, P. and A.H. Levis (1971), "Optimal Linear Regulators: The
Discrete-Time Case," IEEE Trans. Automat. Contr., Vol. AC-l6,
No. 6, Dec., pp. 613-20.

 

Dorfman, R. (1969), "An Economic Interpretation of Optimal Control
Theory," American Economic Review, Vol. LIX, Dec., pp. 817-31.

Dreyfus, S.E. (1968), "Introduction to Stochastic Optimization
and Control," H. Karreman (ed.) Stochastic Optimization and
Control, Wiley, New York, pp. 3-24.

 

Elder, J.M. (1971), ”The Inverse Optimal Linear Regulator Problem,"
Ph.D. Dissertation, Univ. of Florida.

Erickson, D.L., C.T. Leondes, and F.E. Norton (1970), "Optimal
Decision and Control Policies in the National Economy," Proc.
1970 IEEE Symposium on Adaptive Processe§,_Decision,_and
Control, Univ. of Texas, Austin.

Erickson, D.L. (1968), "Sensitivity Constrained Optimal Contr01
Policies for a Dynamic Model of the U.S. National Economy,"
Ph.D. Dissertation in Engineering and Applied Sciences, UCLA.

Fischer, S. and J.P. Cooper (1971), "Stabilization Policy and Lags
in a Closed Economy," (Mimeo) Department of Economics, The Univ.
of Chicago, Chicago, Aug.

Fox, K., J. Sengupta, and E. Thorbecke (1966), The Theory of
Quantitative Economic Policy, North-Holland, Amsterdam.

 

Friedlaender, A.F. (1973), ”Macroeconomic Policy Goals and Revealed
Preference,"(Quarterlpr. of Economics, Vol. LXXXLII, No. 1.,
Feb., pp. 25-43.

Friedman, B.M. (1972), "Optimal Economic Stabilization Policy: An
Extended Framework," J. of Political Economy, Vol. 80, No. 5,
Sept./Oct., pp. 1002-22.

 

Fromm, G. and P. Taubman (1968), Policy Simulations with An
Econometric Model, The Brookings Institution, Washington, D.C.

 

85

Gardner, M.F. and J.L. Barnes (1942), Transients in Linear Systems,
Vol. I, John Wiley and Sons, New York.

Gel'fand, I.M. and S.V. Fomin (1963), Calculus of Variations,
Prentice-Hall, Englewood Cliffs.

Gilbert, E.G. (1962-63), "Controllability in Multivariable Control
Systems," SIAM J. of Control, Ser. A, Vol. 1, No. 1, pp. 128-51.

Hadley, G. and M.C. Kemp (1971), Variational Methods in Economics,
North-Holland, Amsterdam, London.

Hakansson, N.H. (1969), "Optimal Investment and Consumption
Strategies under Risk, an Uncertain Lifetime, and Insurance,"
International Economic Review, Vol. 10, No. 3, Oct., pp. 443-466.

 

(1970), "An Induced Theory of the Firm under Risk," J. of
Finance and Quantitative Analysis, Jun.

 

Haley, W.J. (1973), "Human Capital: The Choice between Investment
and Income," Forthcoming American Economic Review.

Halkin, H. (1964), "Optimal Control for Systems Described by Dif-
ference Equations," Advances in Controlg§ystems, Theory, and
Applications. (ed.) C.T. Leondes, Vol. 1, Academic Press,
New York, London, pp. 173-96.

Hamada, K. (1969), "Optimum.Capital Accwmulation by an Economy
facing an International Capital Market," Journal of Political
Economy, Vol. 77, pp. 684-97.

Haurie, A., M. Polis, and P. Yansouni (1972), "On Optimal Pollution
and Consumption Control in a Macroeconomic System," Paper No.
WP 3-5, Proc. of 1972 IEEE Conference on Decision and Control,
New Orleans, pp. 144-8.

Hestenes, M.R. (1965), "On Variational Theory and Optimal Control
Theory," SIAM J. on Control, Series A, Vol. 3, No. 1, pp. 23-48.

Hickman, B.G. (ed.) (1965), Quantitative Planning of Economic Policy,
The Brookings Institution, Washington, D.C.

Holbrook, R. (1972), "Instrument Instability Problem," American
Economic Review, March.

Holt, C.C., F. Modigliani, and J.F. Muth (1956), "Derivation of a
Linear Decision Rule for Production and Employment," Management
Science, Vol. 2, Jan., pp. 15-77.

 

Intriligator, M.D. (1971), Mathematical Optimization and Economic
Theory, Prentice-Hall, Englewood Cliffs.

86

Iyoha, M.A. (1971), "Optimal Balance of Payments Strategy by
Stochastic Dynamic Programming," Western Economic Journal,
Vol. IX, No. 4, Dec., pp. 429-40.

Jameson, J. and E. Kreindler (1973), "Inverse Problem of Linear
Optimal Control," SIAM J. Control, Vol. 11, Feb., pp. 1-19.

 

Jordan, B.W. and E. Polak (1964), "Theory of a Class of Discrete
Optimal Control Systems," Journal of Electronics and Control,
Vol. 17, pp. 697-713.

Jury, E.I. and F.W. Semelka (1958), "Time Domain Synthesis of
Sampled-Data Control Systems," Trans. ASME, Vol. 80, Nov.,
pp. 1827-38.

Kalman, R.E. (1960), "Contributions to the Theory of Optimal Control,"
Boletin de la Sociedad Matematics Mexicana, pp. 102-19.

 

(1961), ”On the General Theory Of Control," 1960 Proc.
IFAC, London, Butterworths, Vol. 1, pp. 481-92.

(1962-63), "Mathematical Description of Linear Dynamical
Systems," SIAM J. on Control, Series A, Vol. 1, pp. 152-223.

 

(1963), "The Theory of Optimal Control and the Calculus
of Variations," Mathematical Optimization Techniques, (ed.)
R. Bellman, Berkeley, Univ. of California Press, pp. 309-31.

(1964), "When is a Linear Control System Optimal?,"
Trans. of ASME Journal of Basic Engineering, Vol. 86, Series D,
No. 1, Mar., pp. 51-60.

Katz, S. (1962), "A Discrete Version of Pontryaginfs Maximum
Principle," J. of Electronics and Control, Vol. 13, pp. 179-84.

Keeler, E., M. Spencer, and R. Zeckhauser (1972), "The Optimal
Control of Pollution," Journal Of Economic Theory, Vol. 4,
pp. 19-34.

Kendrick, D.A. and L. Taylor (1970), "Numerical Solution of Non-
linear Planning Models," Econometrics, Vol. 38, No. 3, May,
pp. 453-67.

Kendrick, D.A. (1971), ”Control Theory in Economic Models: Survey,"
Paper No. W7-l, Proc. of 197111§3E Conference on Decision and
Control, Univ. of Florida, pp. 80-4.

Kleinman, D.L. and M. Athans (1966), "The Discrete Minimum Principle
with Application to the Linear Regulator Problem," Report
ESL-R-260, Feb., Electronic Systems Laboratory, Dept. of
Electrical Engineering, MIT, Cambridge.

87

Koenig, H.E., Y. Tokad, H.K. Kesavan, and H.G. Hedges (1967),
Analysis of Discrete Physical Systems, McGraw Hill, New York.

 

Koo, A.Y.C. (1963), "An Empirical Test of Revealed Preference
Theory," Econometrica, Vol. 31, Oct., pp. 646-64.

 

Kozhinskaya, L.I. (1972), "Performance of Linear Controllable
Systems," Automation and Remote Control, Vol. 33, No. 1,
Pt. 1, Jan., pp. 17-25.

Kreindler, E. and A. Jameson (1972), "Optimality of Linear Control
Systems," IEEE Transactions on Automatic Control, Vol. AC-17,
No. 3, Jun., pp. 349-51.

Ku, Y.H. (1962), Analysis and Control of Linear Systems, Inter-
national Textbook.Co., Scranton.

 

Kuo, B.C. (1970), Discrete Data Control Systems, Prentice-Hall,
Englewood Cliffs.

 

Kurz, M. (1968), "The General Instability of a Class of Competitive
Growth Processes," Review of Economic Studies, Apr., pp. 155-74.

(1969), "On the Inverse Optimal Problem," Mathematical
Systems Theogy and Economics I, (eds.) H.W. Kuhn and G.P. SzegB,
Springer-Verlag, Berlin, Heidelberg, New York.

 

Kushner, H.J. (1965), "Near-Optimal Control in the Presence of
Small Stochastic Perturbations," Journal of Basis Engineering,
Vol. 87, No. 1, March, pp. 103-8.

 

Lancaster, K. (1965), "The Theory of Qualitative Linear Systems,"
Econometrica, Vol. 33, April, pp. 395-408.

Lange, 0. (1970), Introduction to Economic Cybernetics, Pergamon
Press, PWN, Warszawa.

La Salle, J.P. (1962-63), "Stability and Control," SIAM J. on
Control, Series A, NO. 1, pp. 3-15.

 

Lasdon, E.S., S.K. Mitter, and A.D. Warren (1967), "The Conjugate
Gradient Method for Optimal Control Problems," IEEE Trans.
Auto. Contr1., April, pp. 132-38.

 

Lee, E.B. and L. Markus (1967), Foundations of Optimal Control
Theory, Wiley, New York.

Lee, K.Y., S. Chow, and R.O. Barr (1972), "On the Control of Dis-
crete-Time Distributed Parameter Systems," SIAM J. on Control,
Vol. 10, No. 2, May, pp. 361-76.

Lefschetz, S. (1965), Stability of Nonlinear Control Systems,
Academic Press, New York.

88

Leland, H.E. (1972), I'Why Profit Maximization.May Be Better Assump-
tion than You Think,‘' Technical Report NO. 80, The Economics
Series, Institute for Mathematical Studies in the Social
Sciences, Stanford Univ.

Lele, M.M., D.H. Jacobson, and J.L. McCabe (1971), Wlualitative
Application of a Result in Control Theory to Problems of
Economic Growth," International Economic Review, Jun., pp. 209-26.

Levhari, D. and T.N. Srinivasan (1969), "Optimal Savings under
Uncertainty," Review of Economics and Statistics, Vol. 36,
No. 106, Apr., pp. 153-64.

Levis, A.H., M. Athans, and R. Schlueter (1971), "On the Behavior
of Optimal Linear Sampled-Data Regulators," International

Journal Of Control, Vol. 13, Feb., pp. 343-61.

Little, I.M.D. and J.A. Mirrless (1969), Manual of Industrial
Project Analysis in Developing Countries, Vol. 11, Social
Cost Benefit Analysis, Paris.

 

 

Livesey, D.A. (1970), "The Modeling and Control of U.K. Economy,"
Presented at the Second World Congress of the Econometric
Society, Cambridge, England, Sept.

(1971), "Control Theory and Input-Output Analysis,"
International Journal of Systems Science, Vol. 2, No. 3, Nov.,
pp. 307-18.

Lucas, R.E., Jr. (1967), "Optimal Investment Policy and the Flexible
Accelerator," International Economic Review, Vol. 8, No. 1,
Feb., pp. 78-85.

Luenberger, D.G. (1967), "Canonical Forms for Linear Multivariable
Systems," IEEE Transactions on Automatic Control, Vol. AC-12,
No. 3, Jun., pp. 290-93.

Markus, L. (1969), "Dynamic Keynesian Economic Systems: Control
and Identification," Mathematical Systems Theory and Economics I,
(eds.) H.W. Kuhn and G.P. SzegO, Springer-Verlag, Berlin,
Heidelberg, New York, pp. 203-220.

Martens, A. and R.S. Pindyck (1972), "An Application of Optimal
Control to Investment Allocation for Development Planning,"
Paper NO..WA 2-4, Proc. of 1972 IEEE Conference on Decision

and Contrgl, New Orleans, pp. 31-6.

Meditch, J.S. (1964), "The Pontryagin Maximum Principle and Some
of Its Applications," C.T. Leondes (ed.) Advances in Control
Systems, Academic Press, New'York, London, pp. 55-74.

Merton, R.C. (1969), "Lifetime Portfolio Selection under Uncertainty:
The Continuous Time Case," Review of Economics and Statistics,
Vol. LI, No. 3, Aug., pp. 247-57.

89

Morgan, B.S. (1964), "The Synthesis of Linear Multivariable Systems
by State-Variable Feedback," IEEE Trans. Automat. Contr., Vol.
A0-9, No. 4, Oct., pp. 405-11.

Moylan, P.J. and B.D.O. Anderson (1971), "Nonlinear Regulator
Theory and an Inverse Optimal Control Problem," Proceedings
of 1971 Joint Automatic Control Conference, pp. 462-67.

 

Mulholland, R.J. and R.P. Rhoten (1972), "Optimal Inner-Product
Angular Momentum Controllers," IEEE Transactions on Automatic
Control, Vol. AC-17, NO. 1, Feb., pp. 95-7.

Mundell, R.A. (1965), "Growth, Stability, and Inflationary Finance,"
J. of Political Economy, Vol. 73, April, pp. 97-109.

Murphy, R.E. (1965), Adaptive Processes in Economic Systems,
Academic Press, New'York.

Nelson, W.C. (1964), "On the Use of Optimization Theory for
Practical Control System Design," IEEE Trans. Automat. Contr.,
Vol. AC-9, No. 4, Oct., pp. 469-76.

Norman, A.L. (1971), "Optimal Economic Policy and Econometric
Models," Paper No. W7-2, Proc. of 1971 IEEE Conference on
Decision and Control, Univ. of Florida, pp. 85-95.

Panda, S.P. (1971), "Comments on "On the Inverse Optimum Control
Problem for a Class of Nonlinear Autonomous Systems," IEEE
Trans. Automat. Contr., Oct., pp. 509-11.

Paryani, K. (1972), "Optimal Control of Linear Discrete Macro-
economic Systems," Ph.D. Dissertation in Systems Science,
Michigan State Univ., East Lansing.

Pearson, J.B. and B. Sridhar (1966), "A Discrete Optimal Control
Problem," IEEE Trans. Automat. Contr., Vol. AC-11, April,
pp. 171-4.

Phillips, A.W. (1954), "Stabilization Policy in a Closed Economy,"
Economic Journal, Vol. 64, Jun., pp. 290-323.

(1957), "Stabilization Policy and Time Form of the Lagged
Reaponses," Economic Journal, Vol. 67, Jun., pp. 265-77.

Pindyck, R.S. (1972), "An Application of the Lineariluadratic
Tracking Problem to Economic Stabilization Policy," IEEE
Trans. Automat. Contr., Vol. AC-17, Jun., pp. 287-300.

Rasche, R.H. (1973), "Criteria for the Choice of Operating
Instruments for Open Market Policy," GMimeo) Department of
Economics, Michigan State Univ., East Lansing.

90

Rekasius, Z.V. and T.C. Hsia (1964), "On an Inverse Problem in
Optimal Control," IEEE Trans. Auto. Contrl., Vol. AC-9, NO. 4,
Oct., pp. 370-75.

Rozonoer, L.I. (1959), "Pontryagin Maximum Principle in the Theory
Of Optimum Systems, I, II, III," Automation and Remote Control,
Vol.20, Part 2, Ju1.-Dec., pp. 1288-1352, 1405-1421, 1517-32.

(1961), "On the Variational Methods of Analysis of Automatic
Control Systems Performance," 1960 Proc. of IFAC, London,
Butterworths, pp. 477-80.

Rugh, W.J. (1971), "On an Inverse Optimal Control Problem," IEEE
Transactions on Automatic Control, Vol. AC-l6, No. 1, Feb.,
pp. 87-8.

Sakakibara, E. (1970), ”Dynamic Optimization and Economic Policy,"
American Economic Review, Vol. LX, No. 5, Dec., pp. 826-36.

 

Sage, A.F. (1968), Optimum Systems Control, Prentice-Hall, Engle-
wood Cliffs.

Samuelson, P.A. (1960), "Efficient Paths Of Capital Accumulation
in Terms of the Calculus of Variations," Mathematical Methods
in the Social Sciences, (eds.) K.J. Arrow, et.al., Stanford.

(1969), "Lifetime Portfolio Selection by Dynamic Stochastic
Programming," Review of Economics and Statistics, Vol. LI,
No. 3, Aug., pp. 239-46.

(1972), "Maximum Principles in Analytical Economics,"
American Economic Review, Vol. LXII, NO. 3, Jun., pp, 249-62.

 

Sarachik, P.E. and K. Kreindler (1965), "Controllability and
Observability of Linear Discrete-Time Systems," International
Journal of Control, Vol. I, NO. 5, May, pp. 419-32.

Schultz, W.C. and V.C. Rideout (1961), "Control System Performance
Measures: Past, Present, and Future," IRE Trans. Automat.
Control, Vol. AC-6, No. 1, Feb., pp. 22-34.

Sengupta, J.K. (1970), "Optimal Stabilization Policy with a
Quadratic Criterion Function," The Review of Economic Studies,
Vol. XXXVII(1), NO. 109, Jan., pp. 127-46.

Sengupta, J.K., G. Tintner, and B. Morrison (1963), "Stochastic
Linear Programming with Applications to Economic Models,"
Economica, Vol. 30, August, pp. 262-76.

Shell, K. (ed.) (1967), Essays on the Theory of Optimal Economic
Growth, MIT Press, Cambridge.

91

Shupp, F.R. (1972), "Optimal Control, Uncertainty and a Temporary
Income Policy," Paper No. WA-2-2, Proc. 1972 IEEE Conference
on Decision and Control, New Orleans, pp. 21-25.

 

Simon, H.A. (1956), "Dynamic Programming under Uncertainty with a
Quadratic Criterion Function," Econometrica, Vol. 24, Jan.,
pp. 74-81.

 

Simon, J.D. and S.K. Mitter (1968), "A Theory of Modal Control,"
Information and Control, Vol. 13, No. 4, Oct., pp. 316-53.

Stafford, F.P. and P.E. Stephen (1972), "Labor, Leisure, and
Training over the Life Cycle," (Mimeo), Presented at 1972
Meetings of the Econometric Society in Toronto, Ontario, Canada.

Stoleru, L.G. (1965), "An Optimal Policy for Economic Growth,"
Econometrica, Vol. 33, No. 2, April, pp. 321-48.

Taylor, L. (1970), "The Existence of Optimal Distributed Lags,"
The Review of Economic Studies, Vol. XXXVII(1), No. 109, Jan.,
pp. 95-106.

Thau, F.E. (1967), "On the Inverse Optimal Control Problem for a
Class of Nonlinear Autonomous Systems," IEEE Transactions on
Automatic Control, Vol. AC-12, Dec., pp. 674-81.

Theil, H. (1957), "A Note on Certainty Equivalence in Dynamic
Planning," Econometrica, Vol. 25, April, pp. 346-49.

 

(1958), Economic Forecasts and Policy, North-Holland,
Amsterdam.

(1964), Optimal Decision Rules for Government and Industry,
North-Holland, Amsterdam.

(1965), "Linear Decision Rules for Macrodynamic Policy
Problems," Ch. 2, pp. 18-42, Quantitative Planning of Economic
Policy, The Brookings Institution, Washington, D.C. (ed.)

B .G. Hickmsn .

Thompson, R.C., M. George, P. Brown, and M.S. Proctor (1971),
"Optimal Production, Investment, and Output Price Controls
for a MonOpoly Firm of the Evan's Type," Econometrica, Vol. 39,
No. 1, Jan., pp. 119-30.

Tinbergen, J. (1967), Economic Policy: Principles and Design,
North-Holland, Amsterdam.

 

 

Tou, J.T. (1963), Optimal Design of Digital Control Systems,
Academic Press, New York, London.

(1964), Modern Control Theqpy, McGraw-Hill, New York,
San Francisco, Toronto, London.

92

Tuel, W.C., Jr. (1967), "An Improved Algorithm for the Solution
of Discrete Regulation Problems," IEEE Transactions on
Automatic Control, Vol. AC-12, No. 5, Oct., pp. 522-28.

Turnovsky, S.J. (1973), "Optimal Stabilization Policies and
Stochastic Linear Economic Systems," Review of Economic
Studies, Vol. XL(1), No. 121, Jan., pp. 79-95.

Tustin, A. (1953), The Mechanism of Economic Systems, Harvard
Univ. Press, Cambridge.

Tyler, T.S. and F.B. Tuteur (1966), "The Use of a Quadratic Per-
formance Index to Design Multivariable Control Systems,"
IEEE Trans. Automat. Control, Vol. AC-11, No. 1, Jan.,
pp. 84-92.

Weiss, L. and R.E. Kalman (1964), Contributions to Linear System
Theory, RIAS Technical Report 64-9, A Division of the Martin
Co., Baltimore, April.

Whittle, P. (1969), "A View of Stochastic Control Theory,"
Journal of Royal Statistical Society, Series A, pp. 320-34.

Widodo, R.J. (1972), "Design of Optimal Control Systems with
Prescribed Closed-LOOp Poles," ElectronicS:LetterS, Vol. 8,
NO. 13, Jun., pp. 339-40.

Willner, L., R.H. Ash, and R.J. Roy (1972), "An Improved General
Algorithm for Arbitrary Pole Assignment," IEEE Trans.
Automat. Contr., Vol. AC-17, Oct., pp. 734-36.

Wishart, D.M.G. (1969), "A Survey of Control Theory," Journal of
Royal Statistical Society, Series A, pp. 293-319.

Wonham, W.M. and C.D. Johnson.(l964), "Optimal Bang-Bang Control
with.Quadratic Performance Index," J. of Basic Engineering,
Trans. of the ASME, Ser. D, March, pp. 107-115.

Wonham,iIJ4(1967), "On Pole Assignment in Multi-input Controllable
Linear Systems," IEEE Transactions on Automatic Control,
Vol. AC-12, Dec., pp. 660-65.

(1969), "Optimal Stochastic Control," Automatica, Vol. 5,
No. 1, Jan., pp. 113-6.

Wolovich, W.A. (1972), "The Use of State Feedback for Exact Model
Matching," SIAM J. on Control, Vol. 10, No. 2, Aug., pp. 512-23.

Wu, S.H. and W.J. Schroeder (1968), "Inverse Optimal Control Problem
for Discrete Systems," Conference Record of Second Asilomar

Conference on Circuits and Systems, Pacific Grove, U.S.A.,
pp. 561-5.

93

Wymer, C.R. (1972), "Econometric Estimation of Stochastic Dif-
ferential Equation Systems," Econometrica, Vol. 40, May,
pp. 565-77.

Yaari, M.E. (1964), "On the Consumer's Lifetime Allocation
Process," International Economic Review, Vol. 5, No. 3,
Sept., pp. 304-17.

Yokoyama, R. and E. Kinnen (1972), "Inverse Problem of Optimal
Regulator," IEEE Transactions on Automatic Control, Vol. AC-l7,
Aug., pp. 497-503.

Zabel, E. (1967), "A Dynamic Model of the Competitive Firm,"
International Economic Review, VIII, Jun., pp. 199-208.

Zemach, R. (1968), "A State Space Model for Resource Allocation
in Higher Education," IEEE Transactions on Systems Science
and Cybernetics, Vol. SSC-4, NO. 2, Jul., pp. 108-18.

APPENDIX

APPENDIX

THE SOLUTION OF A LINEAR TRACKING PROBLEM

The dynamic system is governed by

+ But + Czt , y = y0 , (A.l)

Ay 0

yt+1 = t

where yt is an n-vector of state variables, ut an m-vector of
control variables, zt an r-vector of other exogenous variables
outside the control consideration. The matrices, A (non-singular),
B (with rank m), and C are constant matrices.

The performance functional is

N-l
= - A I _ A - A g - A
J 3567,, yN) Q(yN yN) + ’1 z {(yt yt) (Myt yt)
t=0
+ uéRut} , (A.2)
where Q ==Q' is a positive semi-definite matrix with rank m,

R = R' a positive-definite matrix, and 9C a vector of target
values of the state yt assumed to be Specified for the entire
Optimization period.

The problem is to determine the control sequence
{u:, t = 0,1,2,...,N-l} such that the corresponding state vari-
able sequence {y:, t = O,l,...,N} satisfies the given initial
condition, y; = y0 such that the performance functional (A.2)
is minimized.

In order to get the necessary conditions for the solu-

tion, construct the Hamiltonian,

94

95

H(yt. Pt+1’ ut) = 5(yt - 9t) 'Q()'t - it)

+ ' + ' +
5 utRut Pt+1(Fyt + But Czt)

[Here, the system equation (A.l) is considered to be yt+1 - yt =
Fyt +But + Czt. Thus F + I = A], where PC is the vector of

"co-states"

The minimization of the Hamiltonian is written

13* 'k = * ' * =
§;-'(y:P , t+ +1, ut) Rut +-B Pt+1 0 . (A.3)

The canonical equations for the problem are

* x P* x * *
- = 5— = + + A.
yt+1 yt 3Pt+1 (yt’P +1’ ut) Fyt But Czt ( 48)
* * P* * x
- = -aH—- = - c - I
* 0
with the "Split" boundary conditions y0 = y and (A.5)
* = 3.. x - A . * _ A = * - A
W 5553,10,, yN) Q(yN yN)} Q(yN yN) - (A-6)
We know that (Lee, et.al., 1972)
* * + A 7
Pt — Ktyt gt . ( .p)
Substituting (A.7) into (A.3),
6* = -R‘lwx y + g ) (A.8)
t t+l t+1
and with (A.7), (A.8), (A.4a), and (A.4b),
* *
yt+1 - yt = - BR NIB K t+1y:+1 BR NIB gt+1+ (A.9)
*

to
I

to
II

* * *
- A - '

96

From (A.9),
(I + BR-IB'Kt+1)y:+1 = (I + F)y: - BR-lB'gt+1 + Czt
= Ay: - BR'hs'gtH + Czt . (A.11)
Substituting (A.7) into (A.10),
A'K y* +Qy* - K y* = -A'g + g + Q3) . (A.12)
t+1 t+1 t t t t+1 t t

Define W = I + BR-IB'Kt+ Then (A.11) becomes

1.

= -1 * _ - - g '1
yt+1 W Ayt w 1BR 13 gt+1 + w Czt . (A.13)

Substituting (A.13) into (A.12),

-1 * -1 - -1 * *
I w _ I _
A Kt+1{ Ayt W BR 1B gt+1 + W Czt} +Qyt Ktyt

=-' + + " . -
Agt+1 8t Qyt (A14)

Rearranging (A.14),

1

Q*+A'1< w'A*-A'K w‘LBR’LB' +A'K w Cz -Q" +A'g
y yt t+1 gt+1 t+1 t yt: t+1

t t+l

*
— Ktyt + gt (A.15)

*
for any initial value y0 and for all yt, which implies that

-1
= I
Kt Q + A Kt+1W A (A.16)

- - -1 A
gt A Kt+1W l‘BR 113 g”1 + A gt+1 + A Kt+1W ozt Qyt . (A.17)

By the transversality condition (A.6) and (A.7),

* * " - *+ f * A18
PN-Qin-YN)-KNyN 3N orany yN. (- )

Then,

and

and

00
H
II

97

KN = Q (A.19)
gN = _QSIN . ‘ (A.20)
Recall w"1 = (1 + BR-lB'Kt+1)-1
= 1 — B(R + B'KtflBYIB'Kt+1 . (A.21)
Substitution (A.21) into (A.16) and (A.17),
Kt = Q + A'Kt+1[A - B(R + B'Kt+1B)-LB'Kt+1A] (A.22)
-A'[1<t+1 - Kt+1B (R + B'KHlels'1<t_|_1]BR'IB'gt+1 + A'gt+1
+ A'[Kt+1 - Kt+1B(R + B'Kt+1B)-LB'KC+1]Czt - Qyt
0t + A'[Kt+1 - Kt+1B(R + B'Kt+1B)-LB'Kt+1]Czt - Qyt . (A.23)
-A'[Kt+1 - Kt+1B(R + B'Kt+lB)-1B'Kt+1]BR-IB'gt+1 + A'8t+1
-A'[Kt+1BR-1B'- Kt+1B(R +-B'Kt+lB)-1B'Kt+lBR-1B' - I'jgt+1
-A'[Kt+1B[I - (R + B'Kt+1B)-lB'Kt+1B]R-]‘B' - I}gt+1
-A'[Kt+1B[I - R'1(1 + B'KfilBR-1)-lB'Kt+1B]R-lB' - I}gt+1
_A'{Kt+1B[I + R-IB'Kt+1B]-1‘R-1B' - Ijgt+1
-A'{Kt+lB(R + B'Kt+lB)’1RR'IB' - i}gt+1
-A'{Kt+1B(R + B'Kt+1B)-]‘B' - I}gt+1 . (A-24)
-A'{Kt+1B(R + B'Kt+1B)-1B' - I}gt+1
+A'{Kt+1 - Kt+1B(R + B'Kt+1B)-1B'Kt+1]Czt - Qyt . (A.25)

Substituting (A.13) and (A.21) into (A.8),

98

-1 , -1
“r e -R (I -B'Kt+1B(R+BK B) ]B'K

t+1 t+lAy

‘1 I I
+R [I BKt+1B(R+BK+

'1 I " I
t 13) 33 Kt+1BR 13 gt+1

'1 I I
R [1 BKt+1B(R+BKt+B

-1 I - I
1 ) 38 Kt+lczt ‘ R 13 gt+1 ‘

By making use of the matrix identity

I — X(Y +X)'1 =Y(Y + X)'1 ,

C
II

.. ' - . * ‘1 ' " I - I
-R 1R(R +'B Kt+1B) 1B Kt+lAyt + R R(R + B Kt+lB) 1B Kt+1BR 1B gt+1

_ '1 I ' I _ ' I
R R(R + B Kt+1B) 1B Kt:102t R 1B gt+1

I
-(R+B K B) 113' 'AthH*

t+1

B} R 1B' g

, -1B
-{_1 -(R+BKt+B

_ I ' I
1 ) Kt+1 (R + B Kt+1B) 18 Kt+1CZ

t+1
Again, using the matrix identity
I - (Y + X)'1x = (Y + X)'1Y

I I " 'II
-(R + B K +1B) 1B' 'RC+IM -(R + B Kt+1B) 1RR B g

C
II

t+1

I ' I
-(R +IB Kt+1B) 1B Kt+1Cz

I I " I
-(R+BK B)-'Ay:]'BKt+1-(R+BK B)1Bgt+1

t+1 t+1

_ I " I
(R +'B K B) 1B Kt+ICzt . (A.26)

t+1

Now consider the case where R = 0 in the above model.
Let the positive semidefinite matrix: Q be factorized into the
product DD' where rank (D) = rank 62) and (D'B) is non-singular.

Then the matrix Riccati equation (A.22) becomes

99

= I _ I ' I
Kt Q + A Kt+1[A B(B Kt+lB) 113 Kt+lA]

1%, =Q
RN_1 = Q + A'DD'[A - B(B'DD'B)']'B'DD'A]
= Q + A'DD'A - A'DD'B(D'B)-1(B'D)-]‘B'DD'A
=Q
This implies
Kt=Q for all t=0,1,2,...,N. (A.27)
The equation (A.25) becomes

_ I I ‘ I _
at — -A th+1B<B Km» 13 133cm

I _ I ' I _ A
+A [Kt+1 Kt+lB(B Kt+1B) 13 Kt+1]Czt Qyt .

Then

-A'[DD'B(B'DD'B)-IB' - I](-Q’$'N)

gN-l

+A'[DD' - DD 'B(B 'BB'B)'1B'BB']cZN - Q9N

+A'[D(B 'D) "113' - I]DD'§IN

+A'[DD' - DD'B(D'B)-1'(B'D)-]B'DD']CzN - Q§N_1 = -Q$rN_1.
(A.28)

Equation (A.28) implies that

gt=-Qyt , t=0,l,...,N. (A.29)

*
Again, consider the optimal control policy ut (A.26) for the

period N-l.

100

a“
P}

.(B'DD'B)'lB'DD'Ay;_1 - (B'DD'BYIB'gN - (B'DD'B)'IB'DD'CzN_1

' -1 ' * I '1 I ‘ I IA _ ' - '
-(D B) D AyN-l + (D B) (B D) 1B DD yN (D 3) 1D CZN_1

1

-(D'B)'11)'Ay;_1 + (D'B)'ID"yN - (D'B)- D'CzN_1 , (A.30)

which implies that

* ..
“t — -(D'B) 1

l

D'Ay* + (D'B)- D'? - (D'B)-1D'Cz
t t+1 t

*
-Gyt +Eyt+1 - th , t = O,l,...,N-l . (A 31)

mm

BI"
L|l
YI'III
Tnlll

3
7
4
7
2
4
1
3
o
3

rmmuIlumlnm

5“
19

 

HHIHHJIHU