COMPUTATIONALLY EFFICIENT NONLINEAR OPTIMAL CONTROL USING
NEIGHBORING EXTREMAL ADAPTATIONS

By

Amin Vahidi-Moghaddam

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

Mechanical Engineering – Doctor of Philosophy

2025

ABSTRACT

Nonlinear optimal control schemes have achieved remarkable performance in numerous engi-

neering applications; however, they typically require high computational time, which has limited

their use in real-world systems with fast dynamics and/or limited computation power. To address

this challenge, neighboring extremal (NE) has been developed as an efficient optimal adaption

strategy to adapt a pre-computed nominal control solution to perturbations from the nominal tra-

jectory. The resulting control law is a time-varying feedback gain that can be pre-computed along

with the original optimization problem, which makes negligible online computation. This thesis

focuses on reducing the computational time of the nonlinear optimal control problems using the

NE in two parts.

In Part I, we tackle model-based nonlinear optimal control and propose an

extended neighboring extremal (ENE) to handle model uncertainties and reduce computational

time (Chapter 3). Nonlinear Model predictive control (NMPC), which explicitly deals with system

constraints, is considered as the case study due to its popularity, but ENE can be easily extended to

other model-based nonlinear optimal control schemes. In Part II, we address data-driven nonlinear

optimal control and introduce a data-enabled neighboring extremal (DeeNE) to remove parametric

model requirement and reduce the computational time (Chapter 4). Data-enabled predictive control

(DeePC), which makes a transition from the model-based optimal control to a data-driven one using

raw input/output (I/O) data, is considered as the case study due to the attention it has received, but

DeeNE can be easily extended to other data-driven nonlinear optimal control approaches. We also

compare the control performance of DeeNE and DeePC for KINOVA Gen3 (7-DoF Arm Robot).

Moreover, we introduce an adaptive DeePC framework, which can be easily transformed into an

adaptive DeeNE, to use real-time informative data and handle time-varying systems (Chapter 5).

Finally, we conclude the thesis and discuss the future works in Conclusion (Chapter 6).

TABLE OF CONTENTS

CHAPTER 1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1 Background .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Thesis Outline and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . .

INTRODUCTION .
.
.

.

.

.

.

CHAPTER 2 PRELIMINARIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
2.1 Model-based Nonlinear Optimal Control . . . . . . . . . . . . . . . . . . . . . .
2.2 Data-Driven Nonlinear Optimal Control
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Data-Enabled Predictive Control
. . . . . . . . . . . . . . . . . . . . . . .
2.4 Neighboring Extremal Optimal Control

1
2
4

6
6
7
8
9

.

.

.

.

.
.
.

3.1 Background .
.
3.2 Problem Formulation .
.
3.3 Main Result

CHAPTER 3 EXTENDED NEIGHBORING EXTREMAL OPTIMAL CONTROL . . . . 12
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
.
3.3.1 Nominal Optimal Solution . . . . . . . . . . . . . . . . . . . . . . . . . . 16
. . . . . . . . . . . . . . . . . . . . . . . 18
3.3.2 Extended Neighboring Extremal
. . . . . . . . . . 24
3.3.3 Nominal Non-Optimal Solution and Large Perturbations
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.4 Simulation Results
3.5 Chapter Summary .

. .

.
.

.
.

.
.

.
.

.

.

.

.

.

.

.

.

.
.

. .

4.3 Main Result

4.1 Background .
.
4.2 Problem Formulation .

CHAPTER 4 DATA-ENABLED NEIGHBORING EXTREMAL CONTROL . . . . . . . 39
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
.
4.2.1 Non-Parametric Representation of Unknown Systems . . . . . . . . . . . . 42
4.2.2 Data-Enabled Predictive Control
. . . . . . . . . . . . . . . . . . . . . . . 43
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
.
4.3.1 Nominal Lagrange Multipliers . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3.2 Data-Enabled Neighboring Extremal . . . . . . . . . . . . . . . . . . . . . 45
4.3.3 Nominal Non-Optimal Solution . . . . . . . . . . . . . . . . . . . . . . . 48
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.4.1 Cart-Inverted Pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.4.2 KINOVA Gen3 .
4.5 Experimental Verifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.6 Chapter Summary .

4.4 Simulation Results

.

.

.

.

.

.

.

.

.

.

.

.

.

.

CHAPTER 5 ADAPTIVE DATA-ENABLED PREDICTIVE CONTROL . . . . . . . .

5.1 Background .
.
5.2 Problem Formulation .
.
5.3 Main Result

. 63
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
.
5.3.1 Adaptive Data-Enabled Predictive Control . . . . . . . . . . . . . . . . . . 69
. . . . . . . . . 71
5.3.2 Adaptive Reduced-Order Data-Enabled Predictive Control

. .

.
.
.

.

.

.

.

.

iii

5.4 Simulation Results

.

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.4.1 Linear Time-Varying System . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.4.2 Vehicle Rollover Avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . 78
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

.

.

.

5.5 Chapter Summary .

6.1 Contributions .
.
6.2 Future Works

CHAPTER 6 CONCLUSION .
.
.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
.
6.2.1 Nonlinear Fundamental Lemma
. . . . . . . . . . . . . . . . . . . . . . . 84
6.2.2 Data-Enabled Control Barrier Function . . . . . . . . . . . . . . . . . . . 84

.
.
.

.
.

.
.

CHAPTER 7 PUBLICATIONS .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

BIBLIOGRAPHY .

.

.

.

.

.

.

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. 87

iv

CHAPTER 1

INTRODUCTION

Constraint-aware optimal control schemes can explicitly handle system constraints while

achieving optimal closed-loop performance [1, 2]. However, such controllers typically involve

solving an optimization problem at each time step and are thus computationally expensive, es-

pecially for nonlinear systems. This has hindered their wider adoption in applications with fast

dynamics and/or limited computation resources [3]. As such, several frameworks have been de-

veloped to improve computational efficiency of nonlinear optimal controllers. One approach is to

simplify the system dynamics with model-reduction techniques [4, 5]. However, these techniques

require a trade-off between system performance and computational complexity, and it is often still

computationally expensive after the model reduction. Another approach is to use function approx-

imators, where functions such as neural networks [6, 7], Gaussian process regression [8, 9], and

spatial temporal filters [10, 11] are exploited to learn the optimal control policy, after which the

learned policy is employed online to achieve efficient onboard computations. However, extensive

data collection is required to ensure a comprehensive coverage of operating conditions. Another

sound approach is to use cloud computing for moving on-board computations from the plant to

a cloud, which employs computer system resources to provide on-demand computing power and

data storage services to users [12]. However, the cloud computing has given rise to a set of new

challenges related to request-response communication delays between the plant and the cloud [13].

Neighboring extremal (NE) [14, 15] is another promising paradigm to attain efficient computations

by proposing a time-varying feedback gain on the state deviations. Despite promising performance,

the NE rely on accurate parametric representations of real systems, but this can be challenging for

complex systems. With this challenge in mind, this thesis focuses on three main objectives:

(i) Developing an extended NE (ENE) for model-based optimal control under model uncertainties;

(ii) Developing a data-enabled NE (DeeNE) for data-driven optimal control;

(iii) Developing an adaptive data-enabled optimal control for time-varying systems.

1

1.1 Background

For decades, the design of autonomous systems has critically relied on mathematical models

describing how these systems behave. Models allow scientists and engineers to make predictions

about the system’s behavior and plan future decisions. Model-based optimal control tries to attain

peak performance while guaranteeing system safety, i.e., ensuring that control actions respect

physical limits and safety considerations. As a promising model-based optimal control framework,

model predictive control (MPC) arises to accomplish this task by solving a constrained optimization

problem with future state predictions [16, 17, 18]. However, obtaining the required mathematical

model is often very time consuming and expensive for nonlinear and complex systems [19].

This shortcoming of the model-based optimal control motivates researches on various data-driven

methods that make this controller more practically viable for the nonlinear and complex systems;

however, the existing approaches cause model uncertainties, which necessitates robust optimization

techniques or adaptive strategies to maintain reliable control performance.

As systems and data become increasingly complex and more widely available, respectively,

scientists and practitioners are turning to data-driven methods instead of model-based techniques

[20]. While the model-based optimal controllers rely on plant modeling, data-driven optimal

control involves synthesizing a controller from input/output (I/O) data collected on the real system

[21, 22]. There are two paradigms of the data-driven control: i) indirect data-driven control that first

identifies a model using the I/O data and then conducts control design based on the identified model

[10], and ii) direct data-driven control that circumvents the step of system identification and obtains

control policy directly from the I/O data [23]. A central promise is that the direct data-driven

control may have higher flexibility and better performance than the indirect data-driven control

thanks to the data-centric representation that avoids using a specific model from identification [24].

Moreover, it is generally difficult to map uncertainty specifications from system identification over

to robust control in the indirect data-driven control, while, this may become easier in the direct

data-driven control.

2

Recently, a result in the context of behavioral system theory [25], known as Fundamental Lemma

[26], has received renewed attention in the direct data-driven control. Rather than attempting to

learn a parametric system model, this result enables us to learn the system’s behaviour such that

the subspace of the I/O trajectories of a linear time invariant (LTI) system can be obtained from the

column span of a raw data Hankel matrix of time series trajectories. A direct data-driven optimal

control, called data-enabled predictive control (DeePC) [27], has recently been proposed in the

spirit of the Fundamental Lemma. The DeePC algorithm relies only on the raw I/O data to develop

a non-parametric predictive model, learn the behavior of the unknown system, and perform safe

and optimal control policies to drive the system along a desired trajectory using real-time feedback.

In comparison with the machine learning-based controllers, the DeePC is more computationally

efficient, less data hungry, and more suitable to rigorous stability and robustness analysis [28]. The

DeePC algorithm has been successfully applied in many scenarios, including quadcopters [29] and

power systems [30].

The NE [14] is a promising paradigm to attain (sub-)optimal performance with efficient compu-

tations suitable for the systems with fast dynamics and limited onboard computations. Specifically,

given a pre-computed nominal solution, the NE provides an optimal correction law (to the first

order) to the deviations from the nominal trajectory. The nominal control sequence can be obtained

from a remote powerful controller (e.g., a cloud) or can be computed ahead of time based on an

approximated initial state. The resulting NE control law is a time-varying feedback gain on the

state deviations which is pre-computed along with the original optimal control problem. Therefore,

this adaptation makes negligible online computation and is used towards nonlinear optimal control

problems that are computationally too expensive. The NE has been employed in several engineer-

ing systems, including ship maneuvering control [31], power management [32], full bridge DC/DC

converter [33], and spacecraft relative motion maneuvers [34]. However, the NE framework does

not deal with the model uncertainties and the data-driven controllers (e.g. DeePC).

3

1.2 Thesis Outline and Contributions

Chapter 2 reviews the model-based nonlinear optimal control, the data-driven nonlinear optimal

control, the DeePC framework, and the NE framework. Chapter 3 studies a robustification of the

NE, called ENE, against the model uncertainties for the model-based nonlinear optimal control

such that the model uncertainties are approximated and incorporated into the optimization problem

as preview information. Chapter 4 studies a data-driven NE, called DeeNE, for the data-driven

nonlinear optimal control using the DeePC framework such that the required parametric model is

removed using Fundamental Lemma. Chapter 5 studies an adaptive DeePC strategy, which can

be easily transformed into an adaptive DeeNE, for time-varying systems such that data matrix is

updated using real-time informative data. The conclusions and the future works are provided in

Chapter 6. Below are the detailed contributions of the main chapters.

Chapter 3: We study the problem of optimal trajectory tracking for the nonlinear systems with

the model uncertainties. In modern applications, the optimal controllers frequently incorporate the

model uncertainties as the preview information (e.g., using a preview prediction model [35]) while

the actual model uncertainties are measured or approximated online. For the NE’s control law, a

time-varying feedback gain is pre-computed along with the original model-based nonlinear optimal

control problem. However, the NE framework only deal with the state perturbations; therefore, the

ENE is developed to consider the preview deviations in the NE adaptations. The derived ENE law

is two time-varying feedback gains on the state perturbations and the preview perturbations.

Chapter 4: We study the problem of data-driven optimal trajectory tracking for the nonlinear

systems with a non-parametric model. Given an initial I/O trajectory and a desired reference

trajectory, the DeePC predicts the behavior of the real system and provides an optimal control

sequence using raw I/O data; however, this approach has shown high computational cost due to

dimension of decision variable. Several approaches have been proposed to reduce the computational

cost of the DeePC for linear time-invariant (LTI) systems. However, finding a computationally

efficient method to implement the DeePC on the nonlinear systems is still an open challenge. We

4

propose the DeeNE to approximate the DeePC policy and reduce its computational cost for the

constrained nonlinear systems. The DeeNE adapts a pre-computed nominal DeePC solution to the

perturbations of the initial I/O trajectory and the reference trajectory from the nominal ones.

Chapter 5: We study the problem of data-driven optimal control for the time-varying systems.

DeePC uses pre-collected input/output (I/O) data to construct a data matrix for online predictive

control. However, in systems with evolving dynamics, incorporating real-time data into the DeePC

framework becomes crucial to enhance control performance. We propose an adaptive DeePC

framework for the time-varying systems, which enables the algorithm to update the data matrix

online by using real-time informative data. By exploiting the minimum non-zero singular value of

the data matrix, the developed adaptive DeePC selectively integrates informative data and effec-

tively captures evolving system dynamics. Additionally, a numerical singular value decomposition

technique is introduced to reduce the computational complexity for updating a reduced-order data

matrix.

5

CHAPTER 2

PRELIMINARIES

In this chapter, we review preliminaries of model-based nonlinear optimal control, data-driven

nonlinear optimal control, data-enabled predictive control (DeePC), and neighboring extremal

(NE), to provide contexts for later chapters.

2.1 Model-based Nonlinear Optimal Control

Consider the following discrete-time nonlinear system as:

𝑥(𝑘 + 1) = 𝑓 (𝑥(𝑘), 𝑢(𝑘)),

𝑦(𝑘) = ℎ(𝑥(𝑘), 𝑢(𝑘)),

(2.1)

where 𝑘 ∈ N+ represents the time step, 𝑥 ∈ R𝑛 denotes the state vector of the system, 𝑢 ∈ R𝑚 is
the control input, and 𝑦 ∈ R𝑝 denotes the output of the system. Moreover, 𝑓 : R𝑛 × R𝑚 → R𝑛 is
the system dynamics with 𝑓 (0, 0) = 0, and ℎ : R𝑛 × R𝑚 → R𝑝 represents the output dynamics.

Now, consider the following safety constraints for the system (2.1):

𝐶 (𝑦(𝑘), 𝑢(𝑘)) ≤ 0,

(2.2)

where 𝐶 : R𝑝 × R𝑚 → R𝑙.

Definition 1 (Closed-Loop Performance) Consider the nonlinear system (2.1) and a control prob-

lem of tracking a desired time-varying reference 𝑟 by the output of the system 𝑦. Starting from the

initial state 𝑥0, the closed-loop system performance over 𝑁 steps is characterized by the following

cost function:

𝐽𝑁 (y, u, r) =

𝑁
∑︁

𝑘=0

𝜙(𝑦(𝑘), 𝑢(𝑘), 𝑟 (𝑘)),

(2.3)

where y = [𝑦(0), 𝑦(1), · · · , 𝑦(𝑁)], u = [𝑢(0), 𝑢(1), · · · , 𝑢(𝑁)], and 𝜙(𝑦, 𝑢, 𝑟) denotes stage

cost.

With the defined closed-loop performance metric, the control goal is to minimize the cost func-

tion (2.3) while adhering to the constraints in (2.1)-(2.2). The optimal control aims at optimizing

6

the system performance over 𝑁 future steps for the system (2.1), which is expressed as the following

constrained nonlinear optimization problem:

(y∗, u∗) = arg min

y,u

𝐽𝑁 (y, u, r)

s.t.

𝑥(𝑘 + 1) = 𝑓 (𝑥(𝑘), 𝑢(𝑘)),

𝑦(𝑘) = ℎ(𝑥(𝑘), 𝑢(𝑘)),

𝐶 (𝑦(𝑘), 𝑢(𝑘)) ≤ 0,

𝑥(0) = 𝑥0.

(2.4)

where the optimal control sequence (y∗(0 : 𝑁), u∗(0 : 𝑁)) is the solution of the above model-based

nonlinear optimal control.

2.2 Data-Driven Nonlinear Optimal Control

In practice, the real nonlinear system (2.1) may not be available; thus, system identification

algorithms are typically used to identify the system model. We denote the identified model as:

ˆ𝑥(𝑘 + 1) = ˆ𝑓 ( ˆ𝑥(𝑘), 𝑢(𝑘)),

ˆ𝑦(𝑘) = ℎ( ˆ𝑥(𝑘), 𝑢(𝑘)),

(2.5)

where ˆ𝑥, ˆ𝑦, and ˆ𝑓 denote the states, the outputs, and the dynamics of the identified model,

respectively. It is worth noting that ˆ𝑓 is identified using the system identification algorithms by

collecting sufficient data samples from the real system (2.1).

Now, using the nominal model (2.5), the data-driven nonlinear optimal control is presented as

follows:

( ˆy∗, u∗) = arg min

ˆy,u

𝐽𝑁 ( ˆy, u, r)

s.t.

ˆ𝑥(𝑘 + 1) = ˆ𝑓 ( ˆ𝑥(𝑘), 𝑢(𝑘)),

ˆ𝑦(𝑘) = ℎ( ˆ𝑥(𝑘), 𝑢(𝑘)),

𝐶 ( ˆ𝑦(𝑘), 𝑢(𝑘)) ≤ 0,

ˆ𝑥(0) = 𝑥0.

7

(2.6)

where this control framework represents the indirect data-driven optimal control and requires the

system identification process. However, the direct data-driven optimal control circumvents the step

of the system identification and obtains control policy directly from collected data samples.

2.3 Data-Enabled Predictive Control

As a direct data-driven optimal control, the DeePC [27] makes a transition from model-based

optimal control strategies (e.g. model predictive control (MPC)) to a data-driven one such that

it seeks an optimal control policy from raw input/output (I/O) data without encoding them into

a parametric model and requiring system identification prior to control deployment.

Inspired

by Fundamental Lemma [26], the system model (2.1) is replaced by an algebraic constraint that

enables us to predict the length-𝑁 future input-output (I/O) trajectory for a given length-𝑇𝑖𝑛𝑖 past

(I/O) trajectory.

The Hankel matrices H(𝑢𝑑) and H(𝑦𝑑) are built from the offline collected I/O samples 𝑢𝑑 and

𝑦𝑑 as:

H(𝑢𝑑) =

𝑢1

𝑢2

𝑢2
...












𝑢𝑇𝑖𝑛𝑖+𝑁 𝑢𝑇𝑖𝑛𝑖+𝑁+1



𝑢3
...

· · · 𝑢𝑇−𝑇𝑖𝑛𝑖−𝑁+1
· · · 𝑢𝑇−𝑇𝑖𝑛𝑖−𝑁+2
. . .

...

· · ·

𝑢𝑇















,

(2.7)

where H(𝑢𝑑) ∈ R𝑚(𝑇𝑖𝑛𝑖+𝑁)×𝐿 needs to have full row rank of order 𝑚(𝑇𝑖𝑛𝑖 + 𝑁) + 𝑙, where 𝑙 ≤ 𝑛

represents the observability index, to satisfy the persistency of excitation requirement, and the

number of its columns is denoted as 𝐿 = 𝑇 − 𝑇𝑖𝑛𝑖 − 𝑁 + 1. The Hankel matrix of outputs
H(𝑦𝑑) ∈ R𝑝(𝑇𝑖𝑛𝑖+𝑁)×𝐿 is built in an analogous way from the collected samples 𝑦𝑑. Then, the

Hankel matrices are partitioned in Past and Future subblocks as:








where 𝑈𝑃 ∈ R𝑚𝑇𝑖𝑛𝑖×𝐿, 𝑈𝐹 ∈ R𝑚𝑁×𝐿, 𝑌𝑃 ∈ R𝑝𝑇𝑖𝑛𝑖×𝐿, and 𝑌𝐹 ∈ R𝑝𝑁×𝐿.


𝑈𝑃



𝑈𝐹





𝑌𝑃



𝑌𝐹




=: H(𝑢𝑑),

=: H(𝑦𝑑),









(2.8)

8

Lemma 1 (Fundamental Lemma [26]) Consider a controllable linear time-invariant (LTI) sys-
tem, there is a unique 𝑔 ∈ R𝐿 such that any length-𝑇𝑖𝑛𝑖 + 𝑁 trajectory of the system satisfies the
following linear equation under a full row rank H(𝑢𝑑) as:


𝑢𝑖𝑛𝑖












where 𝑈𝑃, 𝑌𝑃, 𝑈𝐹, and 𝑌𝐹 are fixed data matrices obtained from the offline collected I/O data,


𝑈𝑃






𝑈𝐹



































(2.9)

𝑔 =

𝑦𝑖𝑛𝑖

𝑌𝐹

𝑌𝑃

𝑢

𝑦

,

(𝑢𝑖𝑛𝑖, 𝑦𝑖𝑛𝑖) is a given length-𝑇𝑖𝑛𝑖 initial trajectory, and (𝑢, 𝑦) is a length-𝑁 future trajectory which is

predicted online.

□

For a given initial trajectory (𝑢𝑖𝑛𝑖, 𝑦𝑖𝑛𝑖) collected from the real system (2.1), one can replace

the optimization problem (2.4) with the DeePC as [27, 36]:

(y∗, u∗, 𝜎y

∗, 𝜎u

∗, g∗) = arg min
y,u,𝜎y,𝜎u,g

𝐽𝑁 (y, u, 𝜎y, 𝜎u, g, r)

𝑠.𝑡.

𝑌𝑃

𝑦𝑖𝑛𝑖

𝑔 =




𝑈𝑃
𝑢𝑖𝑛𝑖


















𝑈𝐹


















𝐶 (𝑦, 𝑢) ≤ 0,

𝑌𝐹

𝑢

𝑦

+















,

𝜎𝑢

𝜎𝑦

0

0





























(2.10)

where 𝐽𝑁 (y, u, 𝜎y, 𝜎u, g, r) is the modified cost function for the DeePC with noisy data and
nonlinearities, 𝜎𝑢 ∈ R𝑚𝑇𝑖𝑛𝑖 is an auxiliary slack variable to cover process noises, and 𝜎𝑦 ∈ R𝑝𝑇𝑖𝑛𝑖

is an auxiliary slack variable to cover measurement noises and nonlinearities.

2.4 Neighboring Extremal Optimal Control

Solving a nonlinear optimal control at each time step causes a high computational cost for the

control framework. To address this challenge, the NE [15] adapts a pre-computed nominal control

9

solution to perturbations from the nominal states. The resulting control law is a time-varying

feedback gain that can be pre-computed along with the original optimal control problem, which

takes negligible online computation.

Consider the closed-loop performance (2.3) for the control objective of regulating the state 𝑥

as follows:

𝐽𝑁 (x, u) =

𝑁−1
∑︁

𝑘=0

𝜙(𝑥(𝑘), 𝑢(𝑘)) + 𝜓(𝑥(𝑁)),

(2.11)

where x = [𝑥(0), 𝑥(1), · · · , 𝑥(𝑁)], u = [𝑢(0), 𝑢(1), · · · , 𝑢(𝑁 − 1)], and 𝜓(𝑥) denotes terminal

cost.

Now, the model-based nonlinear optimal control (2.4) is expressed in the following form:

(x∗, u∗) = arg min

x,u

𝐽𝑁 (x, u)

s.t.

𝑥(𝑘 + 1) = 𝑓 (𝑥(𝑘), 𝑢(𝑘)),

𝐶 (𝑥(𝑘), 𝑢(𝑘)) ≤ 0,

𝑥(0) = 𝑥0.

(2.12)

where the Hamiltonian function and the augmented cost function are defined for the above nonlinear

optimization problem as:

𝐻 (𝑘) = 𝜙(𝑥(𝑘), 𝑢(𝑘)) + 𝜆𝑇

(𝑘 + 1) 𝑓 (𝑥(𝑘), 𝑢(𝑘)) + 𝜇𝑇

(𝑘)𝐶𝑎

(𝑥(𝑘), 𝑢(𝑘)),

¯𝐽𝑁 (𝑘) =

𝑁−1
∑︁

𝑘=0

(𝐻 (𝑘) − 𝜆𝑇

(𝑘 + 1)𝑥(𝑘 + 1)) + 𝜓(𝑥(𝑁)),

(2.13)

(2.14)

where 𝐶𝑎 (𝑥(𝑘), 𝑢(𝑘)) represents the active constraints at the time step 𝑘. 𝜇(𝑘) ∈ R𝑙𝑎
Lagrange multiplier for the active constraints, and 𝜆(𝑘 + 1) ∈ R𝑛 represents the Lagrange multiplier

is the

for the system dynamics (2.1).

Using variational analysis, the NE minimizes the second order variation of the augmented cost

10

function (2.14). Consequently, the NE solves the following nonlinear optimization problem as:

(𝛿x∗, 𝛿u∗) = arg min
𝛿x,𝛿u

𝛿2 ¯𝐽𝑁 (𝑘)

𝑠.𝑡.

𝛿𝑥(𝑘 + 1) = 𝑓𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑓𝑢 (𝑘)𝛿𝑢(𝑘),

(2.15)

𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐶𝑎
𝐶𝑎

𝑢 (𝑘)𝛿𝑢(𝑘) = 0,

𝛿𝑥(0) = 𝛿𝑥0,

where the solution of the above optimization problem is:

𝛿𝑢(𝑘) = 𝐾∗(𝑘)𝛿𝑥(𝑘),

𝐾∗(𝑘) = −

(cid:104)

(cid:105)

𝐼 0

𝐾 𝑜 (𝑘)

𝐾 𝑜 (𝑘) =























with

𝑍𝑢𝑢 (𝑘) 𝐶𝑎
𝑢

𝐶𝑎

𝑢 (𝑘)

0

𝑍−1
𝑢𝑢 (𝑘) 0

0

0










𝑍𝑢𝑥 (𝑘)

𝐶𝑎








𝑇 (𝑘)

𝑥 (𝑘)
−1










,









𝑖 𝑓 𝑘 ∈ K𝑎

,

𝑖 𝑓 𝑘 ∈ K𝑖

𝑍𝑢𝑥 (𝑘) = 𝐻𝑢𝑥 (𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑆(𝑘 + 1) 𝑓𝑥 (𝑘),

𝑍𝑢𝑢 (𝑘) = 𝐻𝑢𝑢 (𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑆(𝑘 + 1) 𝑓𝑢 (𝑘),

𝑍𝑥𝑥 (𝑘) = 𝐻𝑥𝑥 (𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝑆(𝑘 + 1) 𝑓𝑥 (𝑘),

𝑍𝑥𝑢 (𝑘) = 𝐻𝑥𝑢 (𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝑆(𝑘 + 1) 𝑓𝑢 (𝑘),

𝑆(𝑘) = 𝑍𝑥𝑥 (𝑘) −

(cid:104)

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇 (𝑘)

(cid:105)

𝐾 𝑜 (𝑘)

𝑍𝑢𝑥 (𝑘)

𝐶𝑎

𝑥 (𝑘)

















(2.16)

(2.17)

.

(2.18)

where K𝑎 and K𝑖 are the sets of time steps at which the constraints are active (i.e., 𝐶 (𝑥(𝑘), 𝑢(𝑘)) = 0)

and inactive (i.e., 𝐶 (𝑥(𝑘), 𝑢(𝑘)) < 0), respectively.

11

CHAPTER 3

EXTENDED NEIGHBORING EXTREMAL OPTIMAL CONTROL

In this chapter, we focus on incorporating model uncertainties into Neighboring Extremal

(NE). The NE framework only deal with state perturbations while we can incorporate the model

uncertainties as preview information (e.g., using a preview prediction model), where the actual

model uncertainties are measured or estimated online. We develop an extended NE (ENE) frame-

work to consider the preview deviations in the NE adaptations such that the control policy is two

time-varying feedback gains on the state and preview perturbations. We also develop a constraint

activity-based criteria for the ENE framework to handle large perturbations. Promising simulation

results on cart inverted pendulum problem demonstrate the efficacy of the ENE algorithm.1

3.1 Background

Due to the vast success in optimal control and advancement in sensing, modern control ap-

plications can incorporate the model uncertainties into the control design as preview information.

For example, the road profile preview obtained from vehicle crowdsourcing is exploited for si-

multaneous suspension control and energy harvesting, demonstrating a significant performance

enhancement using the preview information despite noises in the preview [39]. Another example

is thermal management for cabin and battery of hybrid electric vehicles, where traffic preview is

employed in hierarchical model predictive control to improve energy efficiency [40]. Moreover,

light detection and ranging systems are used to provide wind disturbance preview to enhance the

controls of turbine blades in [41]. We develop an extended neighboring extremal (ENE) framework

that can adapt a nominal control law to state and preview perturbations simultaneously. This setup

is applicable when a nominal preview is available, and the preview signal is estimated online.

1The material of this chapter is from “Extended Neighboring Extremal Optimal Control with
State and Preview Perturbations,” IEEE Transactions on Automation Science and Engineering, 2023
[37] and “Event-Triggered Cloud-based Nonlinear Model Predictive Control with Neighboring
Extremal Adaptations,” IEEE 61st Conference on Decision and Control, 2022 [38].

12

Neighboring extremal (NE) [42, 43, 44, 45] is a promising paradigm to attain efficient com-

putations by proposing a time-varying feedback gain on the state deviations. Specifically, given

a pre-computed nominal solution based on a nominal initial state, the NE yields a control policy

(to the first order) that adapts the nominal control to deviations from the nominal state. The nom-

inal solution can be computed offline and stored online, can be performed on a remote powerful

controller (e.g., cloud), or computed ahead of time by utilizing the idling time of the processor.

The NE framework has been employed in several engineering systems, including ship maneuvering

control [31], power management [32], full bridge DC/DC converter [33], and spacecraft relative

motion maneuvers [34]. Using a parameter estimation for the unknown systems, parameter pertur-

bations are considered in the NE, where the estimated parameters are considered constant during

the predictions of the optimal control problem [46]. In [14], disturbance perturbations have been

considered for the NE in the nonlinear optimal control problems; however, the formulation derived

is limited to a constant disturbance.

In this chapter, we develop the ENE framework for the nonlinear optimal control problems to

adapt a pre-computed nominal solution to both state perturbation and preview perturbation. This is

a generalization of the NE framework [15] where only considers the state perturbation. Moreover,

we treat the ENE problem when nominal non-optimal solution and large perturbations are appeared,

and a multi-segment strategy is employed to guarantee constraint satisfaction in the presence of

large perturbations. Furthermore, promising results are demonstrated by applying the developed

control strategy to the cart inverted pendulum problem. Compared to Chapter 3, we incorporate the

model uncertainties into the NE framework so that we do not need to return the NMPC at several

time steps to handle the model uncertainties, which significantly reduces the computational cost.

This chapter is outlined as: Section II describes the problem formulation and the preliminaries of

the nonlinear optimal control problems. The proposed ENE framework is presented in Section III.

Simulation on the cart-inverted pendulum is presented in Section IV. Finally, Section V discusses

conclusions.

13

3.2 Problem Formulation

In this section, preliminaries on nonlinear optimal control problems are reviewed, and pertur-

bation analysis problem on the optimal solution is presented for the nonlinear systems with state and

preview perturbations. Specifically, the following discrete-time nonlinear system, that incorporates

a system preview, is considered as:

𝑥(𝑘 + 1) = 𝑓 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)),

(3.1)

where 𝑘 ∈ N+ represents the time step, 𝑥 ∈ R𝑛 denotes the measurable/observable states, and
𝑢 ∈ R𝑚 is the control input. Here 𝑤 ∈ R𝑛 represents the preview information that can be road

profile preview in suspension controls [39], wind preview for turbine controls [41], and traffic

preview in vehicle power management [40]. Furthermore, 𝑓

: R𝑛 × R𝑚 × R𝑛 → R𝑛 represents

the system dynamics with 𝑓 (0, 0, 0) = 0. Moreover, we consider the following general nominal

preview model:

𝑤(𝑘 + 1) = 𝑔(𝑥(𝑘), 𝑤(𝑘)),

where 𝑔 : R𝑛 × R𝑛 → R𝑛 represents the nominal preview dynamics.

We consider the following safety constraints for the system:

𝐶 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) ≤ 0,

where 𝐶 : R𝑛 × R𝑚 × R𝑛 → R𝑙.

(3.2)

(3.3)

Definition 2 (Closed-Loop Performance) Consider the nonlinear system (3.1) and the control

objective of regulating the state 𝑥. Starting from the initial conditions 𝑥0 and 𝑤0, the closed-loop

system performance over 𝑁 steps is characterized by the following cost function:

𝐽𝑁 (x, u, w) =

𝑁−1
∑︁

𝑘=0

𝜙(𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) + 𝜓(𝑥(𝑁), 𝑤(𝑁)),

(3.4)

where x = [𝑥(0), 𝑥(1), · · · , 𝑥(𝑁)], u = [𝑢(0), 𝑢(1), · · · , 𝑢(𝑁 − 1)], w = [𝑤(0), 𝑤(1), · · · , 𝑤(𝑁)],

and 𝜙(𝑥, 𝑢, 𝑤) and 𝜓(𝑥, 𝑤) denote the stage and terminal costs, respectively.

14

Assumption 1 (Twice Differentiable Functions) The functions 𝑓 , 𝑔, 𝐶, 𝜙, and 𝜓 are twice con-

tinuously differentiable.

With the defined closed-loop performance metric, the control goal is to minimize the cost

function (3.4) while adhering to the constraints (3.1) and (3.3). The optimal control aims at

optimizing the system performance over 𝑁 future steps for the system (3.1) using the nominal

preview model (3.2), which is expressed as the following constrained optimization problem:

𝑜, u

𝑜, w

(x

𝑜) = arg min
x,u,w

𝐽𝑁 (x, u, w)

s.t.

𝑥(𝑘 + 1) = 𝑓 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)),

𝑤(𝑘 + 1) = 𝑔(𝑥(𝑘), 𝑤(𝑘)),

𝐶 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) ≤ 0,

𝑥(0) = 𝑥0, 𝑤(0) = 𝑤0.

(3.5)

Consider a nominal trajectory x𝑜, u𝑜, and w𝑜 obtained by solving (3.5) with w𝑜 being the

nominal preview. This computation can be performed on a remote powerful controller (e.g., cloud

computing or edge computing) or can be computed ahead of time based on an approximated initial

state. During implementation, the actual state 𝑥(𝑘) and the preview information 𝑤(𝑘) will likely
deviate from the nominal trajectory. Let 𝛿𝑥(𝑘) = 𝑥(𝑘) − 𝑥𝑜 (𝑘) and 𝛿𝑤(𝑘) = 𝑤(𝑘) − 𝑤𝑜 (𝑘) denote

the state perturbation and the preview perturbation, respectively. Now, to solve the nonlinear optimal

control problem (3.5) for the actual values at each time step 𝑘, we seek a (sub-)optimal control policy

𝑢∗(𝑘) = 𝑢𝑜 (𝑘) + 𝛿𝑢(𝑘) to efficiently adapt to the perturbations of the nominal trajectory. As such,

using the nominal trajectory and the perturbation analysis, we develop an extended neighboring

extremal (ENE) framework to account for both state and preview perturbations through two time-

varying feedback gains. Moreover, to handle large perturbations, we modify the ENE algorithm

to preserve constraint satisfaction and retain optimal control performance. The details of each

algorithm and their benefits for nonlinear optimal control will be presented in the next part.

15

Figure 3.1: Schematic of Extended Neighboring Extremal Optimal Control with State and
Preview Perturbations.

3.3 Main Result

In this section, we present the ENE framework for the optimal control problem (3.5) subject

to state and preview perturbations. As shown in Fig. 3.1, a nominal trajectory is first computed

based on system specifications (e.g., nominal model, nominal preview model, constraints, and cost

function) along with a nominal initial state and preview. Then, the ENE exploits time-varying

feedback gains to adapt to state and preview perturbations to retain optimal control performance.

3.3.1 Nominal Optimal Solution

In this subsection, we analyze the nominal optimal solution using the Karush-Kuhn-Tucker

(KKT) conditions. Specifically, define K𝑎 and K𝑖 as the sets of time steps at which the constrains

are active (i.e., 𝐶 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) = 0 in (3.3)) and inactive (i.e., 𝐶 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) < 0),

respectively. From (3.5), the Hamiltonian function and the augmented cost function are defined as:

𝐻 (𝑘) = 𝜙(𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) + 𝜆𝑇

(𝑘 + 1) 𝑓 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) + ¯𝜆𝑇

(𝑘 + 1)𝑔(𝑥(𝑘), 𝑤(𝑘))

(3.6)

+ 𝜇𝑇

(𝑘)𝐶𝑎

(𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)),

¯𝐽𝑁 (𝑘) =

𝑁−1
∑︁

𝑘=0

(𝐻 (𝑘) − 𝜆𝑇

(𝑘 + 1)𝑥(𝑘 + 1) − ¯𝜆𝑇

(𝑘 + 1)𝑤(𝑘 + 1)) + 𝜓(𝑥(𝑁), 𝑤(𝑁)),

(3.7)

where 𝐶𝑎 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) represents the active constraints at the time step 𝑘. It is worth noting
that 𝐶𝑎 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) is an empty vector for inactive constraints, and 𝐶𝑎 (𝑥(𝑘), 𝑢(𝑘), 𝑤(𝑘)) ∈

16

Optimal ControlPlant𝑥𝑘Nominal Initial State 𝑥𝑜(0)Nominal Preview 𝒘𝑜(0)Nominal Optimal Trajectory 𝒙𝑜0:𝑁,𝒖𝑜0:𝑁,𝒘𝑜0:𝑁Nominal Model (1), Nominal Preview Model (2), Constraints (3), Cost Function (4)𝑤𝑘Extended Neighboring Extremal𝛿𝑥𝑘=𝑥𝑘−𝑥𝑜(𝑘)𝛿𝑤𝑘=𝑤𝑘−𝑤𝑜(𝑘)𝛿𝑢𝑘=𝐾1∗𝑘𝛿𝑥𝑘+𝐾2∗(𝑘)𝛿𝑤𝑘𝑢∗𝑘=𝑢𝑜𝑘+𝛿𝑢𝑘Remote, Powerful ControllerOnboardPlantR𝑙𝑎

if we have 𝑙𝑎 (out of 𝑙) active constraints. Furthermore, 𝜇(𝑘) ∈ R𝑙𝑎

is the Lagrange multiplier

for the active constraints, and 𝜆(𝑘 + 1) ∈ R𝑛 and ¯𝜆(𝑘 + 1) ∈ R𝑛 represent the Lagrange multipliers

for the system dynamics (3.1) and the nominal preview model (3.2), respectively. It is worth noting

that the Lagrange multipliers 𝜇(𝑘), 𝜆(𝑘 + 1), and ¯𝜆(𝑘 + 1) are also referred as the co-states.

Assumption 2 (Active Constraints) At each time step 𝑘, the number of active constraints is not
greater than 𝑚, i.e., 𝐶𝑎

𝑢 (𝑘) is full row rank.

Since 𝑥𝑜 (𝑘), 𝑢𝑜 (𝑘), and 𝑤𝑜 (𝑘) (𝑘 ∈ [0, 𝑁]) represent the nominal optimal solution for

the nonlinear optimal control problem (3.5), they satisfy the following KKT conditions for the

augmented cost function (3.7):

𝐻𝑢 (𝑘) = 0, 𝑘 = 0, 1, ..., 𝑁 − 1,

𝜆(𝑘) = 𝐻𝑥 (𝑘), 𝑘 = 0, 1, ..., 𝑁 − 1,

𝜆(𝑁) = 𝜓𝑥 (𝑥(𝑁), 𝑤(𝑁)),

¯𝜆(𝑘) = 𝐻𝑤 (𝑘), 𝑘 = 0, 1, ..., 𝑁 − 1,

¯𝜆(𝑁) = 𝜓𝑤 (𝑥(𝑁), 𝑤(𝑁)),

𝜇(𝑘) ≥ 0, 𝑘 = 0, 1, ..., 𝑁 − 1,

(3.8)

where the subscripts 𝑢, 𝑥, and 𝑤 represent the partial derivatives of a function.

Now, using the KKT conditions (3.8) and the nominal solution 𝑥𝑜 (𝑘), 𝑢𝑜 (𝑘), and 𝑤𝑜 (𝑘), one

can calculate the Lagrange multipliers 𝜇(𝑘), 𝜆(𝑘 + 1), and ¯𝜆(𝑘 + 1) online as:

0 = 𝜙𝑢 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜) + 𝜆𝑇 (𝑘 + 1) 𝑓𝑢 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜) + 𝜇𝑇 (𝑘)𝐶𝑎

𝑢 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜),

𝜆(𝑘) = 𝜙𝑥 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜) + 𝜆𝑇 (𝑘 + 1) 𝑓𝑥 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜) + ¯𝜆𝑇 (𝑘 + 1)𝑔𝑥 (𝑥𝑜, 𝑤𝑜)

+ 𝜇𝑇 (𝑘)𝐶𝑎

𝑥 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜),

𝜆(𝑁) = 𝜓𝑥 (𝑥𝑜 (𝑁), 𝑤𝑜 (𝑁)),

(3.9)

¯𝜆(𝑘) = 𝜙𝑤 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜) + 𝜆𝑇 (𝑘 + 1) 𝑓𝑤 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜) + ¯𝜆𝑇 (𝑘 + 1)𝑔𝑤 (𝑥𝑜, 𝑤𝑜)

+ 𝜇𝑇 (𝑘)𝐶𝑎

𝑤 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜),

¯𝜆(𝑁) = 𝜓𝑤 (𝑥𝑜 (𝑁), 𝑤𝑜 (𝑁)).

17

Using the above equations, the Lagrange multipliers can be obtained as:

𝜇(𝑘) = −(𝐶𝑎

𝑢 (𝑘)𝐶𝑎
𝑢

𝑇

(𝑘))

−1

𝑢 (𝑘)𝜙𝑇
𝐶𝑎

𝑢 (𝑘) − (𝐶𝑎

𝑢 (𝑘)𝐶𝑎
𝑢

−1

𝑇

(𝑘))

𝑢 (𝑘) 𝑓 𝑇
𝐶𝑎

𝑢 (𝑘)𝜆(𝑘 + 1),

𝜆(𝑘) = 𝜙𝑥 (𝑘) + 𝜆𝑇 (𝑘 + 1) 𝑓𝑥 (𝑘) + ¯𝜆𝑇 (𝑘 + 1)𝑔𝑥 (𝑘) + 𝜇𝑇 (𝑘)𝐶𝑎

𝑥 (𝑘),

(3.10)

¯𝜆(𝑘) = 𝜙𝑤 (𝑘) + 𝜆𝑇 (𝑘 + 1) 𝑓𝑤 (𝑘) + ¯𝜆𝑇 (𝑘 + 1)𝑔𝑤 (𝑘) + 𝜇𝑇 (𝑘)𝐶𝑎

𝑤 (𝑘).

Note that 𝛿 ¯𝐽𝑁 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜, 𝜇𝑜, 𝜆𝑜, ¯𝜆𝑜) = 0, and Assumption 2 guarantees that 𝐶𝑎

𝑢 (𝑘)𝐶𝑎
𝑢

𝑇 (𝑘) is

invertible.

3.3.2 Extended Neighboring Extremal

For this part, we assume that the state and preview perturbations are small enough such that

they do not change the activity status of the constraint. To adapt to state and preview perturbations

from the nominal values, the ENE seeks to minimize the second-order variation of (2.14)4 subject

to linearized models and constraints. More specifically, the ENE algorithm solves the following

optimization problem with the initial conditions 𝛿𝑥(0) and 𝛿𝑤(0) as:

(𝛿xo, 𝛿uo, 𝛿wo) = arg min
𝛿x,𝛿u,𝛿w

𝐽𝑛𝑒
𝑁 (𝑘)

s.t.

𝛿𝑥(𝑘 + 1) = 𝑓𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑓𝑢 (𝑘)𝛿𝑢(𝑘) + 𝑓𝑤 (𝑘)𝛿𝑤(𝑘),

𝛿𝑤(𝑘 + 1) = 𝑔𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑔𝑤 (𝑘)𝛿𝑤(𝑘),

(3.11)

𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐶𝑎
𝐶𝑎

𝑢 (𝑘)𝛿𝑢(𝑘) + 𝐶𝑎

𝑤 (𝑘)𝛿𝑤(𝑘) = 0,

𝛿𝑥(0) = 𝛿𝑥0, 𝛿𝑤(0) = 𝛿𝑤0,

where

𝑁 (𝑘) = 𝛿2 ¯𝐽𝑁 (𝑘) =
𝐽𝑛𝑒

1
2

𝑇

𝑁−1
∑︁

𝑘=0












𝛿𝑥(𝑘)

𝛿𝑢(𝑘)











𝛿𝑥𝑇 (𝑁)𝜓𝑥𝑥 (𝑁)𝛿𝑥(𝑁) +

𝛿𝑤(𝑘)












𝐻𝑥𝑥 (𝑘) 𝐻𝑥𝑢 (𝑘) 𝐻𝑥𝑤 (𝑘)

𝐻𝑢𝑥 (𝑘) 𝐻𝑢𝑢 (𝑘) 𝐻𝑢𝑤 (𝑘)

𝐻𝑤𝑥 (𝑘) 𝐻𝑤𝑢 (𝑘) 𝐻𝑤𝑤 (𝑘)

1
2

𝛿𝑤𝑇 (𝑁)𝜓𝑤𝑤 (𝑁)𝛿𝑤(𝑁)

𝛿𝑥(𝑘)

𝛿𝑢(𝑘)

𝛿𝑤(𝑘)


































(3.12)

+

1
2

18

For (3.11) and (3.12), the Hamiltonian function and the augmented cost function are obtained

as:

𝐻𝑛𝑒 (𝑘) =

1
2

𝑇

𝛿𝑥(𝑘)

𝛿𝑢(𝑘)

𝛿𝑤(𝑘)


































𝐻𝑥𝑥 (𝑘) 𝐻𝑥𝑢 (𝑘) 𝐻𝑥𝑤 (𝑘)

𝐻𝑢𝑥 (𝑘) 𝐻𝑢𝑢 (𝑘) 𝐻𝑢𝑤 (𝑘)

𝐻𝑤𝑥 (𝑘) 𝐻𝑤𝑢 (𝑘) 𝐻𝑤𝑤 (𝑘)

𝛿𝑥(𝑘)

𝛿𝑢(𝑘)

𝛿𝑤(𝑘)


































+ 𝛿𝜆𝑇 (𝑘 + 1) ( 𝑓𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑓𝑢 (𝑘)𝛿𝑢(𝑘) + 𝑓𝑤 (𝑘)𝛿𝑤(𝑘))

+ 𝛿 ¯𝜆𝑇 (𝑘 + 1) (𝑔𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑔𝑤 (𝑘)𝛿𝑤(𝑘))

+ 𝛿𝜇𝑇 (𝑘)(𝐶𝑎

𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐶𝑎

𝑢 (𝑘)𝛿𝑢(𝑘) + 𝐶𝑎

𝑤 (𝑘)𝛿𝑤(𝑘)),

¯𝐽𝑛𝑒
𝑁 (𝑘) =

𝑁−1
∑︁

(𝐻𝑛𝑒

(𝑘) − 𝛿𝜆𝑇

(𝑘 + 1)𝛿𝑥(𝑘 + 1) − 𝛿 ¯𝜆𝑇

(𝑘 + 1)𝛿𝑤(𝑘 + 1))

𝑘=0
1
2

+

𝛿𝑥𝑇

(𝑁)𝜓𝑥𝑥 (𝑁)𝛿𝑥(𝑁) +

𝛿𝑤𝑇

1
2

(𝑁)𝜓𝑤𝑤 (𝑁)𝛿𝑤(𝑁),

(3.13)

(3.14)

where 𝛿𝜇(𝑘), 𝛿𝜆(𝑘), and 𝛿 ¯𝜆(𝑘) are the Lagrange multipliers. By applying the KKT conditions to

(3.14), one has

𝐻𝑛𝑒

𝛿𝑢 (𝑘) = 0,
𝛿𝜆(𝑘) = 𝐻𝑛𝑒

𝛿𝑥 (𝑘),

𝛿𝜆(𝑁) = 𝜓𝑥𝑥 (𝑁)𝛿𝑥(𝑁),

𝛿 ¯𝜆(𝑘) = 𝐻𝑛𝑒

𝛿𝑤 (𝑘),
𝛿 ¯𝜆(𝑁) = 𝜓𝑤𝑤 (𝑁)𝛿𝑤(𝑁),

𝑘 = 0, 1, ..., 𝑁 − 1,

𝑘 = 0, 1, ..., 𝑁 − 1,

𝑘 = 0, 1, ..., 𝑁 − 1,

(3.15)

𝛿𝜇(𝑘) ≥ 0,

𝑘 = 0, 1, ..., 𝑁 − 1.

To facilitate the development of the ENE algorithm, several auxiliary variables 𝑆(𝑘), 𝑊 (𝑘),

¯𝑆(𝑘), and ¯𝑊 (𝑘), 𝑘 = 1, 2, · · · , 𝑁 − 1, are introduced as

𝑆(𝑘) = 𝑍𝑥𝑥 (𝑘) −

(cid:104)

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇 (𝑘)

(cid:105)

𝐾 𝑜 (𝑘)









𝑊 (𝑘) = 𝑍𝑥𝑤 (𝑘) −

(cid:104)

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇 (𝑘)

(cid:105)

𝐾 𝑜 (𝑘)

19

,

𝐶𝑎

𝑍𝑢𝑥 (𝑘)

𝑥 (𝑘)








𝑍𝑢𝑤 (𝑘)

𝐶𝑎

𝑤 (𝑘)









(3.16)

,









¯𝑆(𝑘) = 𝑍𝑤𝑥 (𝑘) −

(cid:104)

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇 (𝑘)

(cid:105)

𝐾 𝑜 (𝑘)





𝑤 (𝑘)



where the terminal conditions for 𝑆(𝑘), 𝑊 (𝑘), ¯𝑆(𝑘), and ¯𝑊 (𝑘) are given by

¯𝑊 (𝑘) = 𝑍𝑤𝑤 (𝑘) −

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝐾 𝑜 (𝑘)

𝑇 (𝑘)









𝐶𝑎

(cid:104)

(cid:105)

,









𝐶𝑎

𝑍𝑢𝑥 (𝑘)

𝑥 (𝑘)








𝑍𝑢𝑤 (𝑘)

(3.17)

,

𝑆(𝑁) = 𝜓𝑥𝑥 (𝑁),

𝑊 (𝑁) = 0,

¯𝑆(𝑁) = 0,

¯𝑊 (𝑁) = 𝜓𝑤𝑤 (𝑁),

(3.18)

and

𝑍𝑢𝑥 (𝑘) = 𝐻𝑢𝑥 (𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑆(𝑘 + 1) 𝑓𝑥 (𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑊 (𝑘 + 1)𝑔𝑥 (𝑘),

𝑍𝑢𝑢 (𝑘) = 𝐻𝑢𝑢 (𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑆(𝑘 + 1) 𝑓𝑢 (𝑘),

𝑍𝑢𝑤 (𝑘) = 𝐻𝑢𝑤 (𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑆(𝑘 + 1) 𝑓𝑤 (𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑊 (𝑘 + 1)𝑔𝑤 (𝑘),

𝑍𝑥𝑥 (𝑘) = 𝐻𝑥𝑥 (𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝑆(𝑘 + 1) 𝑓𝑥 (𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝑊 (𝑘 + 1)𝑔𝑥 (𝑘) + 𝑔𝑇

𝑥 (𝑘) ¯𝑆(𝑘 + 1) 𝑓𝑥 (𝑘)

+ 𝑔𝑇

𝑥 (𝑘) ¯𝑊 (𝑘 + 1)𝑔𝑥 (𝑘),

𝑍𝑥𝑢 (𝑘) = 𝐻𝑥𝑢 (𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝑆(𝑘 + 1) 𝑓𝑢 (𝑘) + 𝑔𝑇

𝑥 (𝑘) ¯𝑆(𝑘 + 1) 𝑓𝑢 (𝑘),

𝑍𝑥𝑤 (𝑘) = 𝐻𝑥𝑤 (𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝑆(𝑘 + 1) 𝑓𝑤 (𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝑊 (𝑘 + 1)𝑔𝑤 (𝑘) + 𝑔𝑇

𝑥 (𝑘) ¯𝑆(𝑘 + 1) 𝑓𝑤 (𝑘)

+ 𝑔𝑇

𝑥 (𝑘) ¯𝑊 (𝑘 + 1)𝑔𝑤 (𝑘),

𝑍𝑤𝑥 (𝑘) = 𝐻𝑤𝑥 (𝑘) + 𝑓 𝑇

𝑤 (𝑘)𝑆(𝑘 + 1) 𝑓𝑥 (𝑘) + 𝑓 𝑇

𝑤 (𝑘)𝑊 (𝑘 + 1)𝑔𝑥 (𝑘) + 𝑔𝑇

𝑤 (𝑘) ¯𝑆(𝑘 + 1) 𝑓𝑥 (𝑘)

+ 𝑔𝑇

𝑤 (𝑘) ¯𝑊 (𝑘 + 1)𝑔𝑥 (𝑘),

𝑍𝑤𝑢 (𝑘) = 𝐻𝑤𝑢 (𝑘) + 𝑓 𝑇

𝑤 (𝑘)𝑆(𝑘 + 1) 𝑓𝑢 (𝑘) + 𝑔𝑇

𝑤 (𝑘) ¯𝑆(𝑘 + 1) 𝑓𝑢 (𝑘),

𝑍𝑤𝑤 (𝑘) = 𝐻𝑤𝑤 (𝑘) + 𝑓 𝑇

𝑤 (𝑘)𝑆(𝑘 + 1) 𝑓𝑤 (𝑘) + 𝑓 𝑇

𝑤 (𝑘)𝑊 (𝑘 + 1)𝑔𝑤 (𝑘) + 𝑔𝑇

𝑤 (𝑘) ¯𝑆(𝑘 + 1) 𝑓𝑤 (𝑘)

+ 𝑔𝑇

𝑤 (𝑘) ¯𝑊 (𝑘 + 1)𝑔𝑤 (𝑘),

(3.19)

20





















and

𝐾 𝑜 (𝑘) =

𝑍𝑢𝑢 (𝑘) 𝐶𝑎
𝑢

𝑇 (𝑘)

𝐶𝑎

𝑢 (𝑘)

0

−1










if 𝑘 ∈ K𝑎,

(3.20)









Theorem 1 (Extended Neighboring Extremal) Consider the optimization problem (3.11) and

𝑍−1
𝑢𝑢 (𝑘) 0

if 𝑘 ∈ K𝑖.



0

0

the Hamiltonian function (3.13). If 𝑍𝑢𝑢 (𝑘) > 0 for 𝑘 ∈

(cid:104)

0, 𝑁 − 1

(cid:105)

, then the ENE policy

𝛿𝑢(𝑘) = 𝐾∗

1 (𝑘)𝛿𝑥(𝑘) + 𝐾∗

2 (𝑘)𝛿𝑤(𝑘),

𝐾∗
1 (𝑘) = −

𝐾∗
2 (𝑘) = −

(cid:104)

(cid:104)

(cid:105)

(cid:105)

𝐼 0

𝐼 0

𝐾 𝑜 (𝑘)

𝐾 𝑜 (𝑘)
















𝐶𝑎

𝑥 (𝑘)

𝑍𝑢𝑥 (𝑘)









𝑍𝑢𝑤 (𝑘)







𝑤 (𝑘)

𝐶𝑎

,

,

(3.21)

approximates the perturbed solution for the nonlinear optimal control problem (3.5) in the presence

of state perturbation 𝛿𝑥(𝑘) and preview perturbation 𝛿𝑤(𝑘).

Proof 1 Using (3.13), (3.14), and the KKT conditions (3.15), one has

0 = 𝐻𝑢𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐻𝑢𝑢 (𝑘)𝛿𝑢(𝑘) + 𝐻𝑢𝑤 (𝑘)𝛿𝑤(𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝛿𝜆(𝑘 + 1) + 𝐶𝑎
𝑢

𝑇

(𝑘)𝛿𝜇(𝑘),

(3.22)

𝛿𝜆(𝑘) = 𝐻𝑥𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐻𝑥𝑢 (𝑘)𝛿𝑢(𝑘) + 𝐻𝑥𝑤 (𝑘)𝛿𝑤(𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝛿𝜆(𝑘 + 1) + 𝑔𝑇

𝑥 (𝑘)𝛿 ¯𝜆(𝑘 + 1)

𝑇

+ 𝐶𝑎
𝑥

(𝑘)𝛿𝜇(𝑘),

(3.23)

𝛿 ¯𝜆(𝑘) = 𝐻𝑤𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐻𝑤𝑢 (𝑘)𝛿𝑢(𝑘) + 𝐻𝑤𝑤 (𝑘)𝛿𝑤(𝑘) + 𝑓 𝑇

𝑤 (𝑘)𝛿𝜆(𝑘 + 1) + 𝑔𝑇

𝑤 (𝑘)𝛿 ¯𝜆(𝑘 + 1)

𝑇

+ 𝐶𝑎
𝑤

(𝑘)𝛿𝜇(𝑘),

(3.24)

21

where 𝛿𝜆(𝑁) = 𝜓𝑥𝑥 (𝑁)𝛿𝑥(𝑁) and 𝛿 ¯𝜆(𝑁) = 𝜓𝑤𝑤 (𝑁)𝛿𝑤(𝑁). Now, define the following general

relation:

𝛿𝜆(𝑘) = 𝑆(𝑘)𝛿𝑥(𝑘) + 𝑊 (𝑘)𝛿𝑤(𝑘) + 𝑇 (𝑘),

𝛿 ¯𝜆(𝑘) = ¯𝑆(𝑘)𝛿𝑥(𝑘) + ¯𝑊 (𝑘)𝛿𝑤(𝑘) + ¯𝑇 (𝑘).

(3.25)

(3.26)

Using (3.15), (3.25), and (3.26), one has 𝑇 (𝑁) = 0 and ¯𝑇 (𝑁) = 0. Substituting the linearized

model (3.11) and (3.25) into (3.22) yields

𝑍𝑢𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑍𝑢𝑢 (𝑘)𝛿𝑢(𝑘) + 𝑍𝑢𝑤 (𝑘)𝛿𝑤(𝑘) + 𝐶𝑎
𝑢

𝑇

(𝑘)𝛿𝜇(𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑇 (𝑘 + 1) = 0.

(3.27)

Using the linearized safety constraints (3.11) and (3.27), one has

= −𝐾 𝑜 (𝑘)

𝛿𝑢(𝑘)

𝛿𝜇(𝑘)

















− 𝐾 𝑜 (𝑘)

− 𝐾 𝑜 (𝑘)























𝛿𝑥(𝑘)

𝐶𝑎

𝑍𝑢𝑥 (𝑘)








𝑍𝑢𝑤 (𝑘)

𝑥 (𝑘)

𝛿𝑤(𝑘)

𝐶𝑎

𝑤 (𝑘)








𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1)

0

.









Now, substituting the model (3.11), (3.25) and (3.26) into (3.23) yields

𝛿𝜆(𝑘) = 𝑍𝑥𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑍𝑥𝑢 (𝑘)𝛿𝑢(𝑘) + 𝑍𝑥𝑤 (𝑘)𝛿𝑤(𝑘) + 𝐶𝑎
𝑥

𝑇

(𝑘)𝛿𝜇(𝑘) + 𝑓 𝑇

𝑥 (𝑘)𝑇 (𝑘 + 1)

+ 𝑔𝑇

𝑥 (𝑘) ¯𝑇 (𝑘 + 1).

Furthermore, substituting (3.28) into (3.29) yields

(cid:20)

𝑍𝑥𝑥 (𝑘) −

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

𝛿𝜆(𝑘) = (cid:169)
(cid:173)
(cid:173)
(cid:171)









(cid:20)

𝑍𝑥𝑤 (𝑘) −

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

+ (cid:169)
(cid:173)
(cid:173)
(cid:171)

+ 𝑓 𝑇

𝑥 (𝑘)𝑇 (𝑘 + 1) −

(cid:20)

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

22

















𝑍𝑢𝑥 (𝑘)
𝐶𝑎

𝑥 (𝑘)

(cid:170)
(cid:174)
(cid:174)
(cid:172)
𝑍𝑢𝑤 (𝑘)
𝐶𝑎

𝑤 (𝑘)









𝛿𝑥(𝑘)

𝛿𝑤(𝑘)



(cid:170)

(cid:174)

(cid:174)


(cid:172)

𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1)

0

+ 𝑔𝑇

𝑥 (𝑘) ¯𝑇 (𝑘 + 1).









(3.28)

(3.29)

(3.30)

From (3.16), (3.25) and (3.30), it can be concluded that

𝑇 (𝑘) = 𝑔𝑇

𝑥 (𝑘) ¯𝑇 (𝑘 + 1) + 𝑓 𝑇








Now, substituting the model (3.11), (3.25) and (3.26) into (3.24) yields

𝑥 (𝑘)𝑇 (𝑘 + 1) −

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝐾 𝑜 (𝑘)

𝑇 (𝑘)

(cid:104)

(cid:105)

𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1)

0









.

(3.31)

𝛿 ¯𝜆(𝑘) = 𝑍𝑤𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑍𝑤𝑢 (𝑘)𝛿𝑢(𝑘) + 𝑍𝑤𝑤 (𝑘)𝛿𝑤(𝑘) + 𝐶𝑎
𝑤

𝑇

(𝑘)𝛿𝜇(𝑘)

+ 𝑓 𝑇

𝑤 (𝑘)𝑇 (𝑘 + 1) + 𝑔𝑇

𝑤 (𝑘) ¯𝑇 (𝑘 + 1).

Furthermore, plugging (3.28) into (3.32) yields

(cid:20)

𝑍𝑤𝑥 (𝑘) −

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

𝛿 ¯𝜆(𝑘) = (cid:169)
(cid:173)
(cid:173)
(cid:171)









(cid:20)

𝑍𝑤𝑤 (𝑘) −

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

+ (cid:169)
(cid:173)
(cid:173)
(cid:171)

+ 𝑓 𝑇

𝑤 (𝑘)𝑇 (𝑘 + 1) −

(cid:20)

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

Using (3.17), (3.26) and (3.33), one has









𝑍𝑢𝑥 (𝑘)
𝐶𝑎

𝑥 (𝑘)



(cid:170)

(cid:174)

(cid:174)


(cid:172)

𝑍𝑢𝑤 (𝑘)
𝐶𝑎

𝑤 (𝑘)









𝛿𝑥(𝑘)

(cid:170)
(cid:174)
(cid:174)
(cid:172)

𝛿𝑤(𝑘)








𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1)

0

+ 𝑔𝑇

𝑤 (𝑘) ¯𝑇 (𝑘 + 1).









(3.32)

(3.33)

¯𝑇 (𝑘) = 𝑔𝑇

𝑤 (𝑘) ¯𝑇 (𝑘 + 1) + 𝑓 𝑇

𝑤 (𝑘)𝑇 (𝑘 + 1) −

(cid:104)

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇 (𝑘)

(cid:105)

𝐾 𝑜 (𝑘)

𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1)

0









.









(3.34)

Based on (3.31), (3.34), and the fact that 𝑇 (𝑁) = 0, ¯𝑇 (𝑁) = 0, one can conclude that for 𝑘 ∈
(cid:104)
, 𝑇 (𝑘) = 0, ¯𝑇 (𝑘) = 0. Thus, by using (3.28), the ENE policy (2.16) can be obtained. This

(cid:105)

1, 𝑁 − 1

completes the proof.

□

Remark 1 (Singularity) It is worth noting that the assumption of 𝑍𝑢𝑢 being positive definite (i.e.,

𝑍𝑢𝑢 (𝑘) > 0, 𝑘 ∈

(cid:104)

0, 𝑁 − 1

(cid:105)

) is essential for the ENE. 𝑍𝑢𝑢 (𝑘) > 0 is performed to calculate the

ENE such that it guarantees the convexity of (3.11). Considering 𝑍𝑢𝑢 (𝑘) > 0 and Assumption 2,
it is clear that 𝐾 𝑜 (𝑘) (3.20) is well defined. However, when the constraints involve only state and

23

preview (i.e., 𝐶𝑎
𝑢 (𝑘) is not full row rank), the matrix
𝐾 𝑜 is singular, leading to the failure of the proposed algorithm. This issue can be solved using the

𝑢 (𝑘) = 0), or when 𝑙𝑎 is greater than 𝑚 (i.e., 𝐶𝑎

constraint back-propagation algorithm presented in [15].

Remark 2 (Nominal Preview Model) If we do not have any idea about the nominal preview model

(3.2) for the existing preview information in the real system, we can simply use 𝑤(𝑘 + 1) = 𝑤(𝑘) as

the nominal preview model for the nonlinear optimal control problem (3.5) and the ENE algorithm.

However, it is clear that we achieve the best performance using the ENE when the nominal preview

model describes the preview information perfectly.

Algorithm 1 summarizes the ENE procedure for adaptation the pre-computed nominal control

solution 𝑢𝑜 (𝑘) to the small state perturbation 𝛿𝑥(𝑘) and the small preview perturbation 𝛿𝑤(𝑘) such
that it achieves the optimal control as 𝑢∗(𝑘) = 𝑢𝑜 (𝑘) + 𝛿𝑢(𝑘) using Theorem 1.

Algorithm 1 Extended Neighboring Extremal.
Input: The functions 𝑓 , 𝑔, 𝐶, 𝜙, and 𝜓, and the nominal trajectory x𝑜 (0 : 𝑁), u𝑜 (0 : 𝑁), and
w𝑜 (0 : 𝑁).
1: Initialize the matrices 𝜆𝑜 (𝑁), ¯𝜆𝑜 (𝑁), 𝑆(𝑁), 𝑊 (𝑁), ¯𝑆(𝑁), and ¯𝑊 (𝑁) using (3.9) and (3.18).
2: Calculate, in a backward run, the Lagrange multipliers 𝜇𝑜 (𝑘), 𝜆𝑜 (𝑘), and ¯𝜆𝑜 (𝑘) using (3.10).
3: Calculate, in a backward run, the matrices 𝑍 (𝑘), the gains 𝐾∗
2 (𝑘), and the matrices
𝑆(𝑘), 𝑊 (𝑘), ¯𝑆(𝑘), and ¯𝑊 (𝑘) using (3.19), (3.21), (3.16), and (3.17), respectively.
4: Given 𝑥𝑜 (0), 𝑤𝑜 (0), 𝛿𝑥(0), and 𝛿𝑤(0), in a forward run, calculate 𝛿𝑢(𝑘), 𝑢∗(𝑘), 𝑥(𝑘 + 1), and
𝑤(𝑘 + 1) using (3.21) and (3.1).

1 (𝑘) and 𝐾∗

3.3.3 Nominal Non-Optimal Solution and Large Perturbations

The ENE is derived under the assumption that a nominal optimal solution is available, and

the state and preview perturbations are small such that they do not change the activity status of

the constraints. In this subsection, we modify the ENE policy for a nominal non-optimal solution

and accordingly improve the algorithm to handle large state and preview perturbations which may

change the sets of inactive and active constraints.

For the nominal non-optimal sequences 𝑥𝑜 (𝑘), 𝑢𝑜 (𝑘), 𝑤𝑜 (𝑘), 𝜇𝑜 (𝑘), 𝜆𝑜 (𝑘), and ¯𝜆𝑜 (𝑘),

we assume that they satisfy the constraints described in (3.5) and (3.8) but may not satisfy the

24

optimality condition 𝐻𝑢 (𝑥𝑜, 𝑢𝑜, 𝑤𝑜, 𝜇𝑜, 𝜆𝑜, ¯𝜆𝑜) = 0. Under this circumstance, the cost function

(3.12) is modified as

𝑁 (𝑘) = 𝛿2 ¯𝐽𝑁 (𝑘) +
𝐽𝑛𝑒

𝑁−1
∑︁

𝑘=0

𝐻𝑇

𝑢 (𝑘)𝛿𝑢(𝑘) =

1
2

+

+

𝐻𝑥𝑥 (𝑘) 𝐻𝑥𝑢 (𝑘) 𝐻𝑥𝑤 (𝑘)

𝐻𝑢𝑥 (𝑘) 𝐻𝑢𝑢 (𝑘) 𝐻𝑢𝑤 (𝑘)

𝐻𝑤𝑥 (𝑘) 𝐻𝑤𝑢 (𝑘) 𝐻𝑤𝑤 (𝑘)

𝛿𝑥(𝑘)

𝛿𝑢(𝑘)

𝛿𝑤(𝑘)


































𝛿𝑤𝑇 (𝑁)𝜓𝑤𝑤 (𝑁)𝛿𝑤(𝑁)

𝑇

𝑘=0























𝑁−1
∑︁

𝛿𝑥(𝑘)

𝛿𝑢(𝑘)











1
𝛿𝑥𝑇 (𝑁)𝜓𝑥𝑥 (𝑁)𝛿𝑥(𝑁) +
2
𝑁−1
∑︁

𝛿𝑤(𝑘)

𝐻𝑇

𝑢 (𝑘)𝛿𝑢(𝑘).

1
2

Considering the optimal control problem (3.11) and the cost function (3.35), the Hamiltonian

𝑘=0

(3.35)

function is modified as

𝐻𝑛𝑒 (𝑘) =

1
2

𝑇

𝛿𝑥(𝑘)

𝛿𝑢(𝑘)

𝛿𝑤(𝑘)


































𝐻𝑥𝑥 (𝑘) 𝐻𝑥𝑢 (𝑘) 𝐻𝑥𝑤 (𝑘)

𝐻𝑢𝑥 (𝑘) 𝐻𝑢𝑢 (𝑘) 𝐻𝑢𝑤 (𝑘)

𝐻𝑤𝑥 (𝑘) 𝐻𝑤𝑢 (𝑘) 𝐻𝑤𝑤 (𝑘)

𝛿𝑥(𝑘)

𝛿𝑢(𝑘)

𝛿𝑤(𝑘)


































+ 𝛿𝜆𝑇 (𝑘 + 1)( 𝑓𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑓𝑢 (𝑘)𝛿𝑢(𝑘) + 𝑓𝑤 (𝑘)𝛿𝑤(𝑘))

(3.36)

+ 𝛿 ¯𝜆𝑇 (𝑘 + 1)(𝑔𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑔𝑤 (𝑘)𝛿𝑤(𝑘))

+ 𝛿𝜇𝑇 (𝑘)(𝐶𝑎

𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐶𝑎

𝑢 (𝑘)𝛿𝑢(𝑘) + 𝐶𝑎

𝑤 (𝑘)𝛿𝑤(𝑘)) + 𝐻𝑇

𝑢 (𝑘)𝛿𝑢(𝑘).

Now, the following theorem is presented to modify the ENE policy for the nominal non-optimal

solutions to the nonlinear optimal control problem (3.5).

Theorem 2 (Modified Extended Neighboring Extremal) Consider the optimization problem (3.11),

the KKT conditions (3.15), and the Hamiltonian function (3.36). If 𝑍𝑢𝑢 (𝑘) > 0 for 𝑘 ∈

(cid:104)

0, 𝑁 − 1

(cid:105)

,

then the ENE policy for a nominal non-optimal solution is modified as

𝛿𝑢(𝑘) = 𝐾∗

1 (𝑘)𝛿𝑥(𝑘) + 𝐾∗

2 (𝑘)𝛿𝑤(𝑘) + 𝐾∗

3 (𝑘)

𝐾∗
3 (𝑘) = −

(cid:104)

(cid:105)

𝐼 0

𝐾 𝑜 (𝑘),

25









𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1) + 𝐻𝑢 (𝑘)

0

,









(3.37)

where the gain matrices 𝐾∗

1, 𝐾∗

2, and 𝐾 𝑜 are defined in (3.20) and (3.21), and 𝑇 (𝑘) is a non-zero

variable defined in (3.42).

Proof 2 Using (3.15) and (3.36), (3.22) is modified as

𝐻𝑢𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐻𝑢𝑢 (𝑘)𝛿𝑢(𝑘) + 𝐻𝑢𝑤 (𝑘)𝛿𝑤(𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝛿𝜆(𝑘 + 1) + 𝐶𝑎
𝑢

𝑇

(𝑘)𝛿𝜇(𝑘) + 𝐻𝑢 (𝑘) = 0.

(3.38)

Substituting the linearized model (3.11) and (3.25) into (3.38) yields

𝑍𝑢𝑥 (𝑘)𝛿𝑥(𝑘) + 𝑍𝑢𝑢 (𝑘)𝛿𝑢(𝑘) + 𝑍𝑢𝑤 (𝑘)𝛿𝑤(𝑘) + 𝐶𝑎
𝑢

𝑇

(𝑘)𝛿𝜇(𝑘) + 𝑓 𝑇

𝑢 (𝑘)𝑇 (𝑘 + 1) + 𝐻𝑢 (𝑘) = 0.

Using the linearized safety constraints (3.11) and (3.39), one can obtain

= −𝐾 𝑜 (𝑘)

𝛿𝑢(𝑘)

𝛿𝜇(𝑘)

















− 𝐾 𝑜 (𝑘)

𝐶𝑎

𝑍𝑢𝑥 (𝑘)

𝑥 (𝑘)








𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1)

𝛿𝑥(𝑘) − 𝐾 𝑜 (𝑘)

− 𝐾 𝑜 (𝑘)

0
























(3.39)

(3.40)

𝛿𝑤(𝑘)

.

















𝐶𝑎

𝑍𝑢𝑤 (𝑘)

𝑤 (𝑘)








𝐻𝑢 (𝑘)

0









Substituting (3.40) into (3.29) yields

(cid:20)

𝑍𝑥𝑥 (𝑘) −

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

𝛿𝜆(𝑘) = (cid:169)
(cid:173)
(cid:173)
(cid:171)









𝑍𝑢𝑥 (𝑘)
𝐶𝑎

𝑥 (𝑘)



(cid:170)

(cid:174)

(cid:174)


(cid:172)

𝑍𝑢𝑤 (𝑘)
𝐶𝑎

𝑤 (𝑘)









𝛿𝑥(𝑘)

(cid:170)
(cid:174)
(cid:174)
(cid:172)

𝛿𝑤(𝑘)








𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1)









𝐻𝑢 (𝑘)







0

0

.









(cid:20)

𝑍𝑥𝑤 (𝑘) −

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

+ (cid:169)
(cid:173)
(cid:173)
(cid:171)

+ 𝑓 𝑇

𝑥 (𝑘)𝑇 (𝑘 + 1) −

(cid:20)

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

+ 𝑔𝑇

𝑥 (𝑘) ¯𝑇 (𝑘 + 1) −

(cid:20)

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

26

(3.41)









From (3.16), (3.25), and (3.41), it follows that

𝑇 (𝑘) = 𝑔𝑇

𝑥 (𝑘) ¯𝑇 (𝑘 + 1) + 𝑓 𝑇

𝑥 (𝑘)𝑇 (𝑘 + 1) −

(cid:104)

𝑍𝑥𝑢 (𝑘) 𝐶𝑎
𝑥

𝑇 (𝑘)

(cid:105)

𝐾 𝑜 (𝑘)









𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1) + 𝐻𝑢 (𝑘)

0

.








(3.42)

Now, plugging (3.40) into (3.32) yields

(cid:20)

𝑍𝑤𝑥 (𝑘) −

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

𝛿 ¯𝜆(𝑘) = (cid:169)
(cid:173)
(cid:173)
(cid:171)









(cid:20)

𝑍𝑤𝑤 (𝑘) −

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇

(𝑘)

(cid:21)

𝐾𝑜

(𝑘)

+ (cid:169)
(cid:173)
(cid:173)
(cid:171)

𝑍𝑢𝑥 (𝑘)
𝐶𝑎

𝑥 (𝑘)



(cid:170)

(cid:174)

(cid:174)


(cid:172)

𝑍𝑢𝑤 (𝑘)
𝐶𝑎

𝑤 (𝑘)









𝛿𝑥(𝑘)

(cid:170)
(cid:174)
(cid:174)
(cid:172)

𝛿𝑤(𝑘)








𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1)

+ 𝑓 𝑇

𝑤 (𝑘)𝑇 (𝑘 + 1) −

+ 𝑔𝑇

𝑤 (𝑘) ¯𝑇 (𝑘 + 1) −

(cid:20)

(cid:20)

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇

(𝑘)

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇

(𝑘)

(cid:21)

(cid:21)

𝐾𝑜

(𝑘)

𝐾𝑜

(𝑘)

Using (3.17), (3.26), and (3.43), one has
















0

.









𝐻𝑢 (𝑘)

0

(3.43)









¯𝑇 (𝑘) = 𝑔𝑇

𝑤 (𝑘) ¯𝑇 (𝑘 + 1) + 𝑓 𝑇

𝑤 (𝑘)𝑇 (𝑘 + 1) −

(cid:104)

𝑍𝑤𝑢 (𝑘) 𝐶𝑎
𝑤

𝑇 (𝑘)

(cid:105)

𝐾 𝑜 (𝑘)









𝑓 𝑇
𝑢 (𝑘)𝑇 (𝑘 + 1) + 𝐻𝑢 (𝑘)

0

.








(3.44)

Based on (3.40), (3.42) and (3.44), the modified ENE policy (3.37) is obtained. This completes the

proof.

□

Now, using (3.40), the relation between the state and preview perturbations and the Lagrange

multiplier perturbation is expressed as:

𝛿𝜇(𝑘) = 𝐾∗

4 (𝑘)𝛿𝑥(𝑘) + 𝐾∗

5 (𝑘)𝛿𝑤(𝑘),

𝐾∗
4 (𝑘) = −

𝐾∗
5 (𝑘) = −

(cid:104)

(cid:104)

0 𝐼

0 𝐼

(cid:105)

(cid:105)

𝐾 𝑜 (𝑘)

𝐾 𝑜 (𝑘)
















27

𝐶𝑎

𝑥 (𝑘)

𝑍𝑢𝑥 (𝑘)









𝑍𝑢𝑤 (𝑘)







𝑤 (𝑘)

𝐶𝑎

,

.

(3.45)

Moreover, using (3.37), the constraint perturbation is represented as

𝛿𝐶 (𝑘) = 𝐶𝑥 (𝑘)𝛿𝑥(𝑘) + 𝐶𝑢 (𝑘)𝛿𝑢(𝑘) + 𝐶𝑤 (𝑘)𝛿𝑤(𝑘)

= (𝐶𝑥 (𝑘) + 𝐶𝑢 (𝑘)𝐾∗

1 (𝑘))𝛿𝑥(𝑘) + (𝐶𝑤 (𝑘) + 𝐶𝑢 (𝑘)𝐾∗

2 (𝑘))𝛿𝑤(𝑘)

+ 𝐶𝑢 (𝑘)𝐾∗

3 (𝑘)( 𝑓 𝑇

𝑢 (𝑘)𝑇 (𝑘 + 1) + 𝐻𝑢 (𝑘)).

The perturbed Lagrange multiplier and the perturbed constraint are given by

𝜇(𝑘) = 𝜇𝑜 (𝑘) + 𝛿𝜇(𝑘),

𝐶 (𝑘) = 𝐶𝑜 (𝑘) + 𝛿𝐶 (𝑘).

(3.46)

(3.47)

(3.48)

Different activity statuses of the constraints may occur due to large perturbations. To address

this issue, we consider a line that connects the nominal variables 𝑥𝑜 (0) and 𝑤𝑜 (0) to the perturbed

variables 𝑥(0) and 𝑤(0). For the connecting line, we identify several intermediate points such that

the status of the constraint remains the same between two consecutive points. Since we respectively

have 𝜇(𝑘) = 0 and 𝐶 (𝑘) = 0 for the inactive and active constraints, we use (3.47) for the active

constraints to find the intermediate points which make the constraints inactive. Specifically, for the

active constraints, an 𝛼(𝑘) (0 ≤ 𝛼(𝑘) ≤ 1) is computed to have 𝜇𝑜 (𝑘) + 𝛼(𝑘)𝛿𝜇(𝑘) = 0. Moreover,

we employ (3.48) for the inactive constraints to find the intermediate points which make the

constraints active. For the inactive constraints, the 𝛼(𝑘) is computed to have 𝐶𝑜 (𝑘) + 𝛼(𝑘)𝛿𝐶 (𝑘) =

0. Thus, for 𝑘 ∈ [0, 𝑁 − 1], the intermediate points are achieved using the following equation:

𝛼(𝑘) =

𝜇𝑜
(𝑘)
𝛿𝜇(𝑘)

−

𝐶𝑜
(𝑘)
𝛿𝐶 (𝑘)

−






if 𝑘 ∈ K𝑎,

if 𝑘 ∈ K𝑖.

(3.49)

The smallest 𝛼(𝑘) is found such that the obtained perturbation changes the activity statuses of

the constraints at least at one time step 𝑘.

Algorithm 2 summarizes the modified ENE procedure for adaptation the pre-computed nominal

non-optimal control solution to the large state and preview perturbations such that it achieves

28

the optimal control as 𝑢∗(𝑘) = 𝑢𝑜 (𝑘) + 𝛿𝑢(𝑘) using Theorem 2. The algorithm identifies the

intermediate points and determines the modified ENE adaptation policy.

Algorithm 2 Modified Extended Neighboring Extremal.
Input: The functions 𝑓 , 𝑔, 𝐶, 𝜙, and 𝜓, and the nominal trajectory x𝑜 (0 : 𝑁), u𝑜 (0 : 𝑁), and
w𝑜 (0 : 𝑁).
1: Set 𝑗 = 0.
2: Initialize the matrices 𝜆𝑜 (𝑁), ¯𝜆𝑜 (𝑁), 𝑆(𝑁), 𝑊 (𝑁), ¯𝑆(𝑁), and ¯𝑊 (𝑁) using (3.9) and (3.18).
3: Calculate, in a backward run, the Lagrange multipliers 𝜇𝑜 (𝑘), 𝜆𝑜 (𝑘), and ¯𝜆𝑜 (𝑘) using (3.10).
4: Calculate, in a backward run, the matrices 𝑍 (𝑘), the gains 𝐾∗
4 (𝑘), and
5 (𝑘), and the matrices 𝑆(𝑘), 𝑊 (𝑘), 𝑇 (𝑘), ¯𝑆(𝑘),
¯𝑊 (𝑘), and ¯𝑇 (𝑘) using (3.19), (3.37), (3.45),
𝐾∗
(3.16), (3.17), (3.42), and (3.44), respectively.
5: Given initial state variation 𝛿𝑥(0) and initial preview variation 𝛿𝑤(0), in a forward run, calculate
𝛿𝜇(𝑘), 𝛿𝐶 (𝑘), 𝛼(𝑘), 𝛿𝑥(𝑘 + 1), and 𝛿𝑤(𝑘 + 1) using (3.45), (3.46), (3.49), (3.37), and the variations
of the system (3.11), respectively.
6: Set, in a forward run, 𝛼(𝑘) = 1 if 𝛼(𝑘) < 0 or 𝛼(𝑘) > 1. Then, find 𝜆 = min(𝛼(𝑘)). If 𝜆 = 0,
change the activity status of the constraint for the corresponding time step 𝑘 and go to Step 2.
7: Given 𝑥𝑜 (0), 𝑤𝑜 (0), 𝜆𝛿𝑥(0), and 𝜆𝛿𝑤(0), in a forward run, calculate 𝛿𝑢(𝑘), 𝑢(𝑘), 𝛿𝑥(𝑘 + 1),
𝛿𝑤(𝑘 + 1), 𝑥(𝑘 + 1), and 𝑤(𝑘 + 1) using (3.37) and the variations of the system (3.11).
8: If 0 < 𝜆 < 1, set 𝑥𝑜 (0) = 𝑥𝑜 (0) + 𝛼𝛿𝑥(0), 𝑤𝑜 (0) = 𝑤𝑜 (0) + 𝛼𝛿𝑤(0), 𝛿𝑥(0) = (1 − 𝛼)𝛿𝑥(0),
𝛿𝑤(0) = (1 − 𝛼)𝛿𝑤(0), and 𝑗 = 𝑗 + 1. Then, go to Step 2.
9: If 𝜆 = 1, in a forward run, calculate 𝑢∗(𝑘) = 𝑢𝑜 (𝑘) + (cid:205)
(3.37) and (3.1).

𝑗 𝛿𝑢 𝑗 (𝑘), 𝑥(𝑘 + 1), and 𝑤(𝑘 + 1) using

2 (𝑘), 𝐾∗

3 (𝑘), 𝐾∗

1 (𝑘), 𝐾∗

Remark 3 (Designing Parameters) Considering suitable nominal models, the main design pa-

rameters of the proposed approach come from the original optimization problem (3.5), which are

the prediction number 𝑁 and the designing weights in the stage cost 𝜙(𝑥, 𝑢, 𝑤) and the terminal

cost 𝜓(𝑥, 𝑤). The prediction number 𝑁 must be high enough so that the obtained optimal controller

stabilizes the system; however, higher 𝑁 causes higher computational cost to solve the optimization

problem. Moreover, the designing weights in the costs must be selected such that both minimum

tracking error and minimum control input are achieved.

Remark 4 (Implementation) The proposed ENE framework is easy to implement and light in
computation. Specifically, given a nominal initial state 𝑥𝑜 (0), a nominal preview 𝑤𝑜 (0 : 𝑁), a

control objective function to minimize, system and control constraints, a nominal optimal state

29

and control trajectory (𝑥𝑜, 𝑢𝑜) will be computed using an optimal control strategy. Note that this

nominal solution can be computed offline and stored online, can be performed on a remote powerful

controller (e..g, cloud), or computed ahead of time by utilizing the idling time of the processor.
1 (𝑘), 𝐾∗

2 (𝑘), 𝑘 = 0, 1, · · · , 𝑁 − 1 in (3.21) can also

In the same time, the ENE adaptation gains 𝐾∗

be computed along with the nominal control law. During the online implementation, the actual

initial state 𝑥(0) and the actual preview 𝑤 are likely different from the nominal values used for the

optimal control computations. Instead of recomputing the optimal control sequence, the control
correction (3.21) is computed, where 𝛿𝑥(𝑘) = 𝑥(𝑘) − 𝑥𝑜 (𝑘) and 𝛿𝑤(𝑘) = 𝑤(𝑘) − 𝑤𝑜 (𝑘) denote

the state perturbation and the preview perturbation, respectively. Then, the final control is used as
𝑢∗(𝑘) = 𝑢𝑜 (𝑘) + 𝛿𝑢(𝑘), and this implementation is easily extended for the modified ENE. As seen

from the steps discussed above, the proposed approach is easy to implement and involves negligible

online computational cost.

Remark 5 (Nonlinear Model Predictive Control) One can employ the nonlinear optimal control

problem (3.5) as the open-loop nonlinear model predictive control (NMPC) or the closed-loop

NMPC. For the open-loop version, providing the 𝑁-length nominal trajectory from the NMPC, the

ENE algorithm approximates the NMPC policy such that it calculates two time-varying 𝑁-length

feedback gains on the state and preview perturbations. Although the feedback gains are pre-

computed, the ENE is able to take feedback from the real system for the 𝑁 predictions in contrast

to the open-loop NMPC. On the other hand, for the colsed-loop NMPC, we save the ENE solution

but we only apply the first control input to the plant at each time step. Taking the feedback from

the real system, the ENE solution from the previous step is considered as the nominal non-optimal

solution, and the ENE algorithm is applied again to adapt the recent solution for the current time

step.

Remark 6 (Comparison) In comparison with the existing NE frameworks [43, 42, 44, 45], we

extend the regular NE approaches that only consider state deviations to a general setting that both

state and preview deviations are considered. This is a significant extension as many modern control

30

applications are employing preview information due to the increased availability of connectivity

[39, 40, 41]. The necessity of adapting to preview perturbations is also demonstrated in our

simulation studies, where we show that the proposed ENE can significantly outperform the regular

NE when the preview information has certain variations.

Figure 3.2: Cart-Inverted Pendulum.

3.4 Simulation Results

In this part, we demonstrate the performance of the proposed ENE framework for both small

and large perturbations via a simulation example. The simulation example is adopted from the

cart-inverted pendulum (see Fig. 3.2) whose system dynamics is described by:

(cid:165)𝑧 =

(cid:165)𝜃 =

𝐹 − 𝐾𝑑 (cid:164)𝑧 − 𝑚(𝐿 (cid:164)𝜃2 sin(𝜃) − 𝑔 sin(𝜃) cos(𝜃)) − 2𝑤𝑧
𝑀 + 𝑚 sin2(𝜃)
𝑤𝜃
𝑚𝐿2

(cid:165)𝑧 cos(𝜃) + 𝑔 sin(𝜃)
𝐿

−

,

,

(3.50)

where 𝑧 and 𝜃 denote the position of the cart and the pendulum angle. 𝑚 = 1kg, 𝑀 = 5kg, and

𝐿 = 2m represent the mass of the pendulum, the mass of the cart, and the length of the pendulum,
respectively. 𝑔 = 9.81m/s2 and 𝐾𝑑 = 10Ns/m are respectively the gravity acceleration and the

damping parameter. The variable force 𝐹 controls the system under a friction force 𝑤𝑧 and a

friction torque 𝑤𝜃. 𝑇 = 0.1s is considered as the sampling time for discretization of the model

31

𝝎𝒛𝝎𝒛𝝎𝜽𝑭𝒛𝑳𝜽𝒎𝑴(3.50), and we assume that we have certain preview of 𝑤𝑧 and 𝑤𝜃. The states, the outputs, the

preview information, and the control input constraint are respectively expressed as

𝑥 = [𝑥1, 𝑥2, 𝑥3, 𝑥4]𝑇 = [𝑧, (cid:164)𝑧, 𝜃, (cid:164)𝜃]

𝑇 ,

𝑦 = [𝑥1, 𝑥3]𝑇 = [𝑧, 𝜃]𝑇 ,

𝑤 = [𝑤1, 𝑤2, 𝑤3, 𝑤4]𝑇 = [0, 𝑤𝑧, 0, 𝑤𝜃]𝑇 ,

−300 ≤ 𝐹 ≤ 300.

The following values are used for the simulation: 𝑁 = 35, 𝑥𝑜 (0) = [0, 0, −𝜋, 0]𝑇 , 𝑤𝑜 (0) =
[0, 0.1, 0, 0.1]𝑇 . Moreover, the nominal preview model is represented as 𝑤𝑜 (𝑘+1) = −0.008𝑥𝑜 (𝑘)−
0.1𝑤𝑜 (𝑘). For the small perturbation setting, the initial state perturbation and the actual friction
profile are set as 𝛿𝑥(0) = [0.01, 0.01, 0.01, 0.01]𝑇 and 𝑤(𝑘) = 0.004 sin(𝑘) +0.004rand(𝑘) +0.002,

respectively. For the large perturbation setting, the initial state perturbation and the actual friction

profile are chosen as 𝛿𝑥(0) = [0.2, 0.2, 0.2, 0.2]𝑇 and 𝑤(𝑘) = 0.015 sin(𝑘) + 0.015rand(𝑘) + 0.01,

respectively.

Figs. 3.3-3.5 show the control performance of the open-loop NMPC, the standard NE, the ENE,

and the closed-loop NMPC subject to the small perturbations. For the open-loop NMPC, under

the nominal initial state 𝑥𝑜 (0) and preview 𝑤𝑜 (0), we obtain the N-length open-loop trajectory
(𝑥𝑜, 𝑢𝑜, 𝑤𝑜) and apply the open-loop control 𝑢𝑜 to the system as shown in Fig. 3.3.

It is

worth noting that the state and preview information are updated during the optimization problem

based on the considered nominal model (3.50) and the nominal preview model 𝑤𝑜 (𝑘 + 1) =
−0.008𝑥𝑜 (𝑘) − 0.1𝑤𝑜 (𝑘), respectively. However, since it is the open-loop version of the NMPC,

the controller does not take the feedback from the real states and preview, makes the least control

force, and leads to degraded performance due to the state and preview deviations as shown in Fig.

3.4. The NE is capable of taking the state feedback from the real system and adjusting the nominal

optimal control, the open-loop control trajectory obtained by the NMPC, for the state perturbations.

From Fig. 3.4, one can see that the NE does show an improved performance as compared to the

32

open-loop NMPC but it falls short against the ENE since it only handles the state perturbations

without adapting to the preview perturbations. In comparison with the open-loop NMPC and the

NE, the proposed ENE takes the state and preview feedback from the real system and achieves better

performance, where it promptly stabilizes the system with the minimum cost in the presence of state

and preview perturbations as shown in Fig. 3.5. Although we employ the ENE for the open-loop

NMPC, due to the feedback from the real system, the ENE shows a similar control performance

as the closed-loop NMPC for this case as shown in Figs. 3.4 and 3.5. However, the closed-loop

NMPC has high computational cost since it solves the optimization problem (3.5) at each step.

Figure 3.3: Control Input for Small Perturbation.

Figs. 3.6 and 3.7 illustrate the control performance of the open-loop NMPC, the NE, the ENE,

the modified NE, the modified ENE, and the closed-loop NMPC subject to large perturbations.

As shown in Fig. 3.6, one can see that the considered large perturbations change the activity

status of the input constraint, and it causes that the NE and the ENE violates the constraint due

to the absence of the intermediate points between the nominal initial state and preview and the

perturbed ones. However, the modified NE and the modified ENE satisfies the constraint, and

the modified ENE indicates a similar performance as the closed-loop NMPC as shown in Fig.

33

Figure 3.4: System Outputs for Small Perturbation.

Figure 3.5: Cost for Small Perturbation.

3.7. Moreover, to see the role of the nominal preview model on the proposed control scheme,

Figs. 3.8 and 3.9 compare the results of the ENE and the modified ENE for two nominal preview

models 𝑤𝑜 (𝑘 + 1) = 𝑤𝑜 (𝑘) and 𝑤𝑜 (𝑘 + 1) = −0.008𝑥𝑜 (𝑘) − 0.1𝑤𝑜 (𝑘) with the actual friction

profile 𝑤(𝑘) = 0.008 sin(𝑘) + 0.008rand(𝑘) + 0.004. One can see that the activity status of the

constraint is changed under the considered perturbation; however, it is not high enough to cause

34

the constraint violation for the ENE. Furthermore, it can be seen that both the modified ENE and

the ENE accomplish better control performance when the preview model 𝑤𝑜 (𝑘 + 1) = 𝑤𝑜 (𝑘) is

applied. Providing a suitable nominal preview model leads to well control performance by the

proposed ENE and modified ENE.

Table I compares the performances (i.e. ∥𝑦 − 𝑟 ∥) and the computational times of the proposed

controllers for the small perturbation. Based on the formulations, it is obvious that the ENE and

the modified ENE (MENE) show the same performance and computational time for the small

perturbations. We also have same result for the NE and the modified NE (MNE) for the small

perturbations. Table II compares the performances and the computational times of the proposed

controllers for the large perturbations. In Tables I and II, the closed-loop NMPC (CLNMPC) and

the open-loop NMPC (OLNMPC) show the best and the worst performance, respectively; however,

considering both performance and computational time, the modified ENE presents the best results.

The simulation setup is widely applicable as in many modern applications, a nominal preview

model is available while the actual corresponding signal can also be measured or estimated online.

For example, a wind energy forecast model is obtained using a deep federated learning approach

[35], which can be served as a nominal preview model, and the wind disturbance can also be

measured using light detection and ranging systems in real time [41]. For the considered cart-

inverted pendulum simulations, the nominal preview information is obtained using a nominal

model, i.e. 𝑤𝑜 (𝑘 + 1) = −0.008𝑥𝑜 (𝑘) − 0.1𝑤𝑜 (𝑘); however, for each time step 𝑘, we generate

the real preview information as 𝑤(𝑘) = 0.004 sin(𝑘) + 0.004rand(𝑘) + 0.002, which leads to a

perturbation from the nominal one. Providing a nominal solution based on the nominal state and

preview, the proposed ENE framework adapts the nominal control to the perturbations generated

by the measured/estimated real state and preview information. Furthermore, to simulate the

large perturbation case, we follow the same process but change the real preview information as

𝑤(𝑘) = 0.015 sin(𝑘) + 0.015rand(𝑘) + 0.01 for Figs. 3.6 and 3.7 and 𝑤(𝑘) = 0.008 sin(𝑘) +

0.008rand(𝑘) + 0.004 for Figs. 3.8 and 3.9.

35

Figure 3.6: Control Input for Large Perturbation.

Figure 3.7: System Outputs for Large Perturbation.

3.5 Chapter Summary

The ENE algorithm was developed to adapt a nominal trajectory to the state and preview

perturbations, and a multi-segment strategy was employed to handle the large perturbations. Sim-

ulations demonstrated the ENE’s technological advances over the NE and the NMPC, and the

nominal preview model is crucial to the effectiveness of the ENE. The proposed ENE framework is

36

Figure 3.8: Control Input for Different Nominal Preview Models.

Figure 3.9: System Outputs for Different Nominal Preview Models.

applicable to general optimal control problem setting as there is no assumption on the under/over-

actuation of the system. If a regular optimal control implementation can yield good performance,

the ENE is expected to yield comparable performance with less computation complexity, where

the computational load of the ENE grows linearly for the optimization horizon.

37

Table 3.1: Comparison of Controllers for Small Perturbations.

Control
CLNMPC
MENE
ENE
MNE
NE
OLNMPC

Performance
5.5735
5.6429
5.6429
5.9846
5.9846
19.2123

Time (per loop)
5.7179 𝑚𝑠
0.0659 𝑚𝑠
0.0659 𝑚𝑠
0.0658 𝑚𝑠
0.0658 𝑚𝑠
0.1770 𝑚𝑠

Table 3.2: Comparison of Controllers for Large Perturbations.

Control
CLNMPC
MENE
ENE
MNE
NE
OLNMPC

Performance
6.1609
6.2704
6.6838
6.7045
7.4038
41.8378

Time (per loop)
5.7179 𝑚𝑠
0.1225 𝑚𝑠
0.0659 𝑚𝑠
0.1224 𝑚𝑠
0.0658 𝑚𝑠
0.1770 𝑚𝑠

38

CHAPTER 4

DATA-ENABLED NEIGHBORING EXTREMAL CONTROL

In this chapter, we study the problem of data-driven optimal trajectory tracking for the nonlinear

systems with a non-parametric model. Given an initial I/O trajectory and a desired reference

trajectory, the data-enabled predictive control (DeePC) provides an optimal control sequence using

a data matrix on raw I/O data; however, this approach has shown high computational cost due to

the dimension of the decision variable. We propose a data-enabled neighboring extremal (DeeNE)

to approximate the DeePC policy and reduce its computational cost for the constrained nonlinear

systems. The DeeNE adapts a pre-computed nominal DeePC solution to the perturbations of

the initial I/O trajectory and the reference trajectory from the nominal ones. Simulation-based

analysis is used to gain insights into the effects of the DeeNE, and experimental results validate

that these insights carry over to the real-world systems. The results are demonstrated with a video

of successful trajectory tracking of KINOVA Gen3 (7-DoF Arm Robot). 1

4.1 Background

Optimization-based control strategies typically rely on accurate parametric representations of

real systems, but this can be challenging for complex systems. Therefore, data-driven optimal

controllers have become increasingly attractive to both academics and industry practitioners [20].

There are two paradigms of the data-driven optimal control: i) indirect data-driven optimal control

first identifies a model using the I/O data and then conducts control design based on the identified

model [10], and ii) direct data-driven optimal control circumvents the step of system identification

and obtains control policy directly from the I/O data [23]. The direct data-driven optimal control

may have higher flexibility and better performance compared to the indirect one [24].

1The material of this chapter is from “Computationally Efficient Data-Enabled Predictive Con-
trol for Arm Robots,” 2024 [47] and “Data-Enabled Neighboring Extremal Optimal Control: A
Computationally Efficient DeePC,” IEEE 62nd Conference on Decision and Control (CDC), 2023
[48].

39

Recently, a result in the context of behavioral system theory [25], known as Fundamental

Lemma [26], has received renewed attention in the direct data-driven optimal control. In the spirit

of the Fundamental Lemma, a direct data-driven optimal control, called data-enabled predictive

control (DeePC) [27], makes a transition from model-based optimal control strategies (e.g. model

predictive control (MPC)) to a data-driven one. When perfect (noiseless and uncorrupted) I/O data

is accessible, the DeePC can accurately predict the future behaviors of the LTI systems thanks to the

Fundamental Lemma. In this case, the DeePC has equivalent closed-loop behavior to conventional

MPC with a model and perfect state estimation [27]. However, in practice, perfect data is in

general not accessible to the controller due to measurement and process noises, which leads to

inaccurate estimations and predictions and may degrade the quality of the obtained optimal control

sequence. Moreover, the Fundamental Lemma has been proposed for the LTI systems and is not

perfect to learn the behaviors of the nonlinear systems. Therefore, the DeePC is robustified through

suitable regularizations to ensure good performance under noisy data and nonlinearities [49, 50].

Furthermore, it has been shown that a quadratic regularization is essential for stability [23].

Although the DeePC plays an inevitable role in optimal control strategies, it is computationally

expensive because of the dimension of the decision variable and solving an online optimization

problem at each time step. Several approaches have been proposed to optimize a lower dimension

decision variable and reduce the computational cost of the DeePC for the LTI systems. Subspace

predictive control (SPC) [51, 30] identifies a reduced model for the linear DeePC using the singular

value decomposition of the raw data; however, it is not a pure data-driven controller due to the

identification part. Null-space predictive control (NPC) [28] introduces a lower dimension decision

variable to reduce the computational cost of the DeePC, but it only works for the unconstrained

linear DeePC. Minimum-dimension DeePC [5] uses the singular value decomposition to make

more efficient numerical computation for the constrained linear DeePC. However, for the nonlinear

systems, the computational cost of the DeePC is still a challenging problem and needs to be solved.

It is worth noting that for the SPC and the minimum-dimension DeePC, the choice of the number

40

of the singular values to retain/cut is very critical and often not automatized.

In this chapter, we develop a data-enabled NE (DeeNE) framework for the nonlinear DeePC

problem with initial I/O and reference perturbations. Moreover, we treat the DeeNE problem

when nominal non-optimal solutions are present, and a modified control policy is developed to

guarantee the control performance. Promising results are demonstrated by applying the developed

controller to the cart inverted pendulum and the arm robot. The outline of this chapter is as

follows. The problem formulation and the preliminaries of the DeePC are provided in Section II.

Section III presents the proposed data-enabled neighboring extremal for the unknown nonlinear

systems. Section IV presents the simulation results and experimental verifications. Finally, the

conclusions are provided in Section V.

4.2 Problem Formulation

Consider a discrete-time nonlinear system in the following form:

𝑥(𝑘 + 1) = 𝑓 (𝑥(𝑘), 𝑢(𝑘)),

𝑦(𝑘) = ℎ(𝑥(𝑘), 𝑢(𝑘)),

(4.1)

where 𝑘 ∈ N+ denotes the time step, 𝑥 ∈ R𝑛 represents the state vector of the system, 𝑢 ∈ R𝑚 is
the control input, and 𝑦 ∈ R𝑝 denotes the outputs of the system. Moreover, 𝑓 : R𝑛 × R𝑚 → R𝑛 is
the system dynamics with 𝑓 (0, 0) = 0, and ℎ : R𝑛 × R𝑚 → R𝑝 represents the output dynamics.

Consider a safety constraint as

𝐶 (𝑦(𝑘), 𝑢(𝑘)) ≤ 0,

(4.2)

where 𝐶 : R𝑝 × R𝑚 → R𝑙.

Definition 3 (Closed-Loop Performance) Consider the nonlinear system (4.1) and a tracking

control problem with the desired trajectory 𝑟 (𝑘). Starting from an initial state 𝑥0, the closed-loop

system performance over 𝑁 steps is characterized by the following cost term:

𝐽𝑁 (y, u) =

𝑁−1
∑︁

𝑘=0

𝜙(𝑦(𝑘), 𝑢(𝑘)),

41

(4.3)

where u = [𝑢(0), 𝑢(1), · · · , 𝑢(𝑁 − 1)], y = [𝑦(0), 𝑦(1), · · · , 𝑦(𝑁 − 1)], and 𝜙(𝑦, 𝑢) is the stage

cost.

The model-based optimal control aims at optimizing the system performance over 𝑁 future

steps for the real system (4.1), which is reduced to the following constrained optimization problem:

(y∗, u∗) = arg min

y,u

𝐽𝑁 (y, u)

𝑠.𝑡.

𝑥(𝑘 + 1) = 𝑓 (𝑥(𝑘), 𝑢(𝑘))

𝑦(𝑘) = ℎ(𝑥(𝑘), 𝑢(𝑘))

𝐶 (𝑦(𝑘), 𝑢(𝑘)) ≤ 0.

(4.4)

The key ingredient for the optimal control (4.4) is an accurate parametric model of the system,

but obtaining such a model, using plant modeling or identification procedures, is often the most

time consuming and expensive part of control design.

4.2.1 Non-Parametric Representation of Unknown Systems

Inspired by Fundamental Lemma [26], the system model (4.1) is replaced by an algebraic

constraint that enables us to predict the length-𝑁 future input-output (I/O) trajectory for a given

length-𝑇𝑖𝑛𝑖 past (I/O) trajectory.

The Hankel matrices H(𝑢𝑑) and H(𝑦𝑑) are built from the offline collected I/O samples 𝑢𝑑 and

𝑦𝑑 as:

H(𝑢𝑑) =

𝑢2

𝑢1

𝑢2
...












𝑢𝑇𝑖𝑛𝑖+𝑁 𝑢𝑇𝑖𝑛𝑖+𝑁+1



𝑢3
...

· · · 𝑢𝑇−𝑇𝑖𝑛𝑖−𝑁+1
· · · 𝑢𝑇−𝑇𝑖𝑛𝑖−𝑁+2
. . .

...

· · ·

𝑢𝑇















,

(4.5)

where H(𝑢𝑑) ∈ R𝑚(𝑇𝑖𝑛𝑖+𝑁)×𝐿 needs to have full row rank to satisfy the persistency of excitation

requirement, and the number of its columns is denoted as 𝐿 = 𝑇 − 𝑇𝑖𝑛𝑖 − 𝑁 + 1. The Hankel matrix
of outputs H(𝑦𝑑) ∈ R𝑝(𝑇𝑖𝑛𝑖+𝑁)×𝐿 is built in an analogous way from the collected samples 𝑦𝑑.

42

Then, the Hankel matrices are partitioned in Past and Future subblocks as








where 𝑈𝑃 ∈ R𝑚𝑇𝑖𝑛𝑖×𝐿, 𝑈𝐹 ∈ R𝑚𝑁×𝐿, 𝑌𝑃 ∈ R𝑝𝑇𝑖𝑛𝑖×𝐿, and 𝑌𝐹 ∈ R𝑝𝑁×𝐿.


𝑈𝑃



𝑈𝐹





𝑌𝑃



𝑌𝐹




=: H(𝑢𝑑),

=: H(𝑦𝑑),









(4.6)

Lemma 2 (Fundamental Lemma [26]) Consider a controllable linear time-invariant (LTI) sys-
tem, there is a unique 𝑔 ∈ R𝐿 such that any length-𝑇𝑖𝑛𝑖 + 𝑁 trajectory of the system satisfies the
following linear equation under a full row rank H(𝑢𝑑) as


𝑢𝑖𝑛𝑖












where 𝑈𝑃, 𝑌𝑃, 𝑈𝐹, and 𝑌𝐹 are fixed data matrices obtained from the offline collected I/O data,


𝑈𝑃






𝑈𝐹



































(4.7)

𝑔 =

𝑦𝑖𝑛𝑖

𝑌𝑃

𝑌𝐹

𝑢

𝑦

,

(𝑢𝑖𝑛𝑖, 𝑦𝑖𝑛𝑖) is a given length-𝑇𝑖𝑛𝑖 initial trajectory, and (𝑢, 𝑦) is a length-𝑁 future trajectory which is

predicted online.

4.2.2 Data-Enabled Predictive Control

□

For a given initial trajectory (𝑢𝑖𝑛𝑖, 𝑦𝑖𝑛𝑖) collected from the real system (4.1), one can replace

the optimization problem (4.4) with data-enabled predictive control (DeePC) as [27, 36]

(y∗, u∗, 𝜎y

∗, 𝜎u

∗, g∗) = arg min
y,u,𝜎y,𝜎u,g

𝐽𝑁 (y, u, 𝜎y, 𝜎u, g)

𝑠.𝑡.

𝑌𝑃

𝑦𝑖𝑛𝑖

𝑔 =




𝑈𝑃
𝑢𝑖𝑛𝑖


















𝑈𝐹


















𝐶 (𝑦, 𝑢) ≤ 0,

𝑌𝐹

𝑢

𝑦

(4.8)















+

𝜎𝑢

𝜎𝑦

0

0





























43

where 𝜎𝑢 ∈ R𝑚𝑇𝑖𝑛𝑖 is an auxiliary slack variable to cover process noises, 𝜎𝑦 ∈ R𝑝𝑇𝑖𝑛𝑖 is an

auxiliary slack variable to cover measurement noises and nonlinearities, and 𝐽𝑁 (y, u, 𝜎y, 𝜎u, g) is

the modified cost function for data-driven controllers with noisy data and nonlinearities.

Now, using 𝑦 = 𝑌𝐹 𝑔, 𝑢 = 𝑈𝐹 𝑔, 𝜎𝑦 = 𝑌𝑃𝑔 − 𝑦𝑖𝑛𝑖, and 𝜎𝑢 = 𝑈𝑃𝑔 − 𝑢𝑖𝑛𝑖 as free optimization

variables, one can rewrite (4.8) as

g∗ = arg min

g

𝐽𝑁 (𝑌𝐹g, 𝑈𝐹g, 𝑌𝑃𝑔 − 𝑦𝑖𝑛𝑖, 𝑈𝑃𝑔 − 𝑢𝑖𝑛𝑖, 𝑔)

(4.9)

𝑠.𝑡. 𝐶 (𝑌𝐹 𝑔, 𝑈𝐹 𝑔) ≤ 0.

If the constraint 𝐶 (𝑦, 𝑢) was absent in (4.8), the problem is referred to the unconstrained

DeePC, and the solution is available in closed form with reduced computational burden. For this

case, one has 𝑢 = 𝑈𝐹 𝑔 = 𝐾𝑟
𝑑
R𝑚𝑁×(𝑚+𝑝)𝑇𝑖𝑛𝑖 are control gains, 𝑟 is the desired reference trajectory, and 𝑤𝑖𝑛𝑖 =

𝑤𝑖𝑛𝑖 as the DeePC policy, where 𝐾𝑟

𝑑 ∈ R𝑚𝑁×𝑝𝑁 and 𝐾𝑖𝑛𝑖
𝑖𝑛𝑖, 𝑦𝑇
𝑢𝑇
𝑖𝑛𝑖

𝑟 + 𝐾𝑖𝑛𝑖
𝑑

𝑑 ∈
(cid:105)𝑇

(cid:104)

is the given initial trajectory. However, the constrained DeePC (4.8) requires an iterative solver;

therefore, the DeePC may suffer from high computational cost since the dimension of the decision

variable 𝑔 depends on the length of the collected data 𝑇 in the Hankel matrix.

4.3 Main Result

In this section, given a nominal solution (𝑔𝑜, 𝑢𝑜, 𝑦𝑜), we propose a data-enabled neighboring

extremal (DeeNE) to approximate the DeePC policy in the presence of initial (I/O) and reference

trajectories perturbations. We assume that the initial I/O and reference trajectories perturbations

are small enough such that they do not change the activity status of the constraint. The resulting

equation helps us to reduce the time and effort in computing the data-driven optimal control for the

system (4.1).

4.3.1 Nominal Lagrange Multipliers

Considering (4.9), the augmented cost function are defined as:

¯𝐽𝑁 (𝑤𝑖𝑛𝑖, 𝑔, 𝑟, 𝜇) = 𝐽𝑁 (𝑤𝑖𝑛𝑖, 𝑔, 𝑟) + 𝜇𝑇 𝐶𝑎

(𝑤𝑖𝑛𝑖, 𝑔).

(4.10)

44

where 𝐶𝑎 (𝑤𝑖𝑛𝑖, 𝑔) represents the active constraints, and 𝜇 is the Lagrange multiplier associated

with the active constraints.

Let (𝑤𝑜

𝑖𝑛𝑖, 𝑔𝑜, 𝑟 𝑜) represents the nominal solution for the DeePC (4.8). The nominal solution

satisfies the following KKT conditions for the augmented cost function (4.10) as:

¯𝐽𝑔 (𝑤𝑖𝑛𝑖, 𝑔, 𝑟, 𝜇) = 0,

𝜇 ≥ 0,

(4.11)

where ¯𝐽𝑔 indicates ∇ ¯𝐽𝑁 /∇𝑔.
Assumption 3 (Active Constraints) 𝐶𝑎

𝑔 (𝑤𝑖𝑛𝑖, 𝑔) is full row rank.

Now, substituting the nominal solution (𝑤𝑜

𝑖𝑛𝑖, 𝑔𝑜, 𝑟 𝑜) into the above KKT condition, one can

calculate the Lagrange multiplier 𝜇 online. From (4.11), it follows that

𝐽𝑔 (𝑤𝑜

𝑖𝑛𝑖, 𝑔𝑜, 𝑟 𝑜) + 𝜇𝑇 𝐶𝑎

𝑔 (𝑤𝑜

𝑖𝑛𝑖, 𝑔𝑜) = 0.

Using the above equation, the Lagrange multiplier can be obtained online as:

𝜇 = −(𝐶𝑎

𝑔 𝐶𝑎
𝑔

𝑇

)

−1

𝐶𝑎
𝑔 𝐽𝑇
𝑔 .

(4.12)

(4.13)

Note that Assumption 3 guarantees that 𝐶𝑎

𝑔 𝐶𝑎
𝑔

𝑇 is invertible. Moreover, it is worth noting that

𝜇 = 0 if the constraint 𝐶 (𝑤𝑜
the nominal optimal Lagrange multiplier 𝜇𝑜.

𝑖𝑛𝑖, 𝑔𝑜) is not active. The Lagrange multiplier (4.13) is considered as

4.3.2 Data-Enabled Neighboring Extremal

For this part, we consider the nominal solution (𝑔𝑜, 𝑢𝑜, 𝑦𝑜) as an optimal solution obtained

by the DeePC. To adapt to initial I/O and reference perturbations from the nominal values, the

DeeNE seeks to minimize the second-order variation of (4.10) subject to linearized constraints.

More specifically, the DeeNE algorithm solves the following optimization problem with the given

information 𝛿𝑤𝑖𝑛𝑖 and 𝛿𝑟 as:

𝛿g∗ = arg min

𝛿g

𝐽𝑛𝑒
𝑁

𝑠.𝑡. 𝐶𝑎

𝑔 𝛿𝑔 = 0,

45

(4.14)

where

𝑁 = 𝛿2 ¯𝐽𝑁 =
𝐽𝑛𝑒

1
2

𝑇

𝛿𝑤𝑖𝑛𝑖

𝛿𝑔

𝛿𝑟


































¯𝐽𝑤𝑖𝑛𝑖 𝑤𝑖𝑛𝑖
¯𝐽𝑔𝑤𝑖𝑛𝑖
¯𝐽𝑟𝑤𝑖𝑛𝑖

¯𝐽𝑤𝑖𝑛𝑖𝑔
¯𝐽𝑔𝑔
¯𝐽𝑟𝑔

¯𝐽𝑤𝑖𝑛𝑖𝑟
¯𝐽𝑔𝑟
¯𝐽𝑟𝑟























𝛿𝑤𝑖𝑛𝑖

𝛿𝑔

𝛿𝑟

.












(4.15)

For (4.14), the augmented cost function are obtained as:

¯𝐽𝑛𝑒
𝑁 =

1
2

𝛿𝑤𝑖𝑛𝑖

𝛿𝑔

𝛿𝑟












𝑇












¯𝐽𝑤𝑖𝑛𝑖 𝑤𝑖𝑛𝑖
¯𝐽𝑔𝑤𝑖𝑛𝑖
¯𝐽𝑟𝑤𝑖𝑛𝑖












¯𝐽𝑤𝑖𝑛𝑖𝑔
¯𝐽𝑔𝑔
¯𝐽𝑟𝑔

¯𝐽𝑤𝑖𝑛𝑖𝑟
¯𝐽𝑔𝑟
¯𝐽𝑟𝑟























𝛿𝑤𝑖𝑛𝑖

𝛿𝑔

𝛿𝑟












where 𝛿𝜇 is the Lagrange multiplier.

By applying the KKT conditions to (4.16), one has

+ 𝛿𝜇𝑇 𝐶𝑎

𝑔 𝛿𝑔,

(4.16)

¯𝐽𝑛𝑒
𝛿𝑔 = 0,

𝛿𝜇 ≥ 0.

(4.17)

where ¯𝐽𝑛𝑒
𝛿𝑔

indicates ∇ ¯𝐽𝑛𝑒

𝑁 /∇𝛿𝑔.

Considering the DeePC (4.8), we have a nominal solution (𝑔𝑜, 𝑢𝑜, 𝑦𝑜) for an initial I/O trajectory
(𝑢𝑜
𝑖𝑛𝑖) and reference trajectory 𝑟 𝑜. For a new initial I/O trajectory (𝑢𝑖𝑛𝑖, 𝑦𝑖𝑛𝑖) and reference
𝑖𝑛𝑖, 𝑦𝑜
trajectory 𝑟, the optimal solution is approximated by 𝑢∗ = 𝑢𝑜 + 𝛿𝑢 using the DeeNE adaptation.

The objective is now to develop a DeeNE framework for the data-driven optimal trajectory tracking

problem. The following theorem presents the proposed DeeNE to approximate the DeepC policy

in the presence of initial I/O and reference perturbations.

Theorem 3 (Data-Enabled Neighboring Extremal) Consider the optimization problem (4.14),
the augmented cost function (4.16), and the KKT conditions (4.17). If ¯𝐽𝑔𝑔 > 0, then the DeeNE

46

policy

𝛿𝑔 = 𝐾∗

1𝛿𝑤𝑖𝑛𝑖 + 𝐾∗

2𝛿𝑟,

,









(4.18)

¯𝐽𝑔𝑤𝑖𝑛𝑖
0

¯𝐽𝑔𝑟

0

,























−1

(cid:104)

(cid:104)

𝐾∗

1 = −

𝐾∗

2 = −

(cid:105)

𝐾 𝑜

𝐼 0

(cid:105)

𝐾 𝑜

𝐼 0

𝐾 𝑜 =

𝑇

¯𝐽𝑔𝑔 𝐶𝑎
𝑔
𝐶𝑎
𝑔

0

















approximates the perturbed solution for the DeePC (4.8) in the presence of initial I/O perturbation

𝛿𝑤𝑖𝑛𝑖 and reference perturbation 𝛿𝑟.

Proof 3 Using (4.16) and the KKT conditions (4.17), one has

¯𝐽𝑔𝑤𝑖𝑛𝑖 𝛿𝑤𝑖𝑛𝑖 + ¯𝐽𝑔𝑔𝛿𝑔 + ¯𝐽𝑔𝑟 𝛿𝑟 + 𝐶𝑎
𝑔

𝑇 𝛿𝜇 = 0.

Now, using (4.19) and the linearized safety constraints (4.14), one has

𝑇

¯𝐽𝑔𝑔 𝐶𝑎
𝑔
𝐶𝑎
𝑔

0









𝛿𝑔

𝛿𝜇

























= −

¯𝐽𝑔𝑤𝑖𝑛𝑖
0

















𝛿𝑤𝑖𝑛𝑖 −

𝛿𝑟,

¯𝐽𝑔𝑟

0

















which yields

(4.19)

(4.20)

= −𝐾 𝑜

𝛿𝑤𝑖𝑛𝑖 − 𝐾 𝑜

𝛿𝑟.

(4.21)

𝛿𝑔

𝛿𝜇

















¯𝐽𝑔𝑤𝑖𝑛𝑖
0

















¯𝐽𝑔𝑟

0

















Thus, the DeeNE policy (4.18) is obtained, and the proof is completed.
Remark 7 (Singularity) It is worth noting that the assumption of ¯𝐽𝑔𝑔 being positive definite (i.e.,
¯𝐽𝑔𝑔 > 0) is essential for the DeeNE. ¯𝐽𝑔𝑔 > 0 is performed to calculate the DeeNE such that it
guarantees the convexity of (4.14). Considering ¯𝐽𝑔𝑔 > 0 and Assumption 3, it is clear that 𝐾 𝑜 in
𝑔 is not full row rank, the matrix 𝐾 𝑜 is singular, leading to the failure of
(4.18) is well defined. If 𝐶𝑎

□

the proposed algorithm. This issue can be solved using the constraint back-propagation algorithm

presented in [15].

47

Remark 8 (Control Input) Using the control policy (4.18), one can obtain 𝑔∗ = 𝑔𝑜 + 𝛿𝑔, then
𝑢∗ = 𝑢𝑜 + 𝛿𝑢 is obtained using 𝑢 = 𝑈𝐹 𝑔. Therefore, one can conclude that 𝛿𝑢 = 𝐾𝑟

𝑛𝑒𝛿𝑟 + 𝐾𝑖𝑛𝑖

𝑛𝑒 𝛿𝑤𝑖𝑛𝑖.

4.3.3 Nominal Non-Optimal Solution

The DeeNE is derived under the assumption that a nominal DeePC solution is available. In this

subsection, we modify the DeeNE policy for a nominal non-optimal solution so that we can use the

DeeNE solution as the nominal solution during the control process. For the nominal non-optimal

sequences (𝑤𝑜
satisfy the optimality condition ¯𝐽𝑔 (𝑤𝑜

𝑖𝑛𝑖, 𝑔𝑜, 𝑟 𝑜), we assume that they satisfy the constraints described in (4.8) but may not
𝑖𝑛𝑖, 𝑔𝑜, 𝑟 𝑜, 𝜇𝑜) = 0. Under this circumstance, the cost function

(4.15) is modified as:

𝑁 = 𝛿2 ¯𝐽𝑁 + ¯𝐽𝑇
𝐽𝑛𝑒

𝑔 𝛿𝑔 =

1
2

𝑇

𝛿𝑤𝑖𝑛𝑖

𝛿𝑔

𝛿𝑟


































¯𝐽𝑤𝑖𝑛𝑖 𝑤𝑖𝑛𝑖
¯𝐽𝑔𝑤𝑖𝑛𝑖
¯𝐽𝑟𝑤𝑖𝑛𝑖

¯𝐽𝑤𝑖𝑛𝑖𝑔
¯𝐽𝑔𝑔
¯𝐽𝑟𝑔

¯𝐽𝑤𝑖𝑛𝑖𝑟
¯𝐽𝑔𝑟
¯𝐽𝑟𝑟

𝛿𝑤𝑖𝑛𝑖

𝛿𝑔

𝛿𝑟


































+ ¯𝐽𝑇

𝑔 𝛿𝑔.

(4.22)

Considering the optimal control problem (4.14) and the cost function (4.22), the augmented

cost function is modified as:

¯𝐽𝑛𝑒
𝑁 =

1
2

𝑇

𝛿𝑤𝑖𝑛𝑖

𝛿𝑔

𝛿𝑟


































¯𝐽𝑤𝑖𝑛𝑖 𝑤𝑖𝑛𝑖
¯𝐽𝑔𝑤𝑖𝑛𝑖
¯𝐽𝑟𝑤𝑖𝑛𝑖

¯𝐽𝑤𝑖𝑛𝑖𝑔
¯𝐽𝑔𝑔
¯𝐽𝑟𝑔

¯𝐽𝑤𝑖𝑛𝑖𝑟
¯𝐽𝑔𝑟
¯𝐽𝑟𝑟

𝛿𝑤𝑖𝑛𝑖

𝛿𝑔

𝛿𝑟


































+ ¯𝐽𝑇

𝑔 𝛿𝑔 + 𝛿𝜇𝑇 𝐶𝑎

𝑔 𝛿𝑔.

(4.23)

Now, the following theorem is presented to modify the DeeNE policy for the nominal non-

optimal solutions to the data-driven nonlinear optimal control problem.

Theorem 4 (Modified Data-Enabled Neighboring Extremal) Consider the optimization prob-
lem (4.14), the KKT conditions (4.17), and the augmented cost function (4.23). If ¯𝐽𝑔𝑔 > 0, then the

DeeNE policy is modified for a nominal non-optimal solution as:

,

¯𝐽𝑔

0

















(4.24)

𝛿𝑔 = 𝐾∗

1𝛿𝑤𝑖𝑛𝑖 + 𝐾∗

2𝛿𝑟 + 𝐾∗
3

𝐾∗

3 = −

(cid:104)

𝐼 0

(cid:105)

𝐾 𝑜,

48

where the gain matrices 𝐾∗

1, 𝐾∗

2, and 𝐾 𝑜 are defined in (4.18).

Proof 4 Using the KKT conditions (4.17) and the modified augmented cost function (4.23), one

has

¯𝐽𝑔𝑤𝑖𝑛𝑖 𝛿𝑤𝑖𝑛𝑖 + ¯𝐽𝑔𝑔𝛿𝑔 + ¯𝐽𝑔𝑟 𝛿𝑟 + 𝐶𝑎
𝑔

𝑇 𝛿𝜇 + ¯𝐽𝑔 = 0.

Now, using (4.25) and the linearized safety constraints (4.14), one has

𝑇

¯𝐽𝑔𝑔 𝐶𝑎
𝑔
𝐶𝑎
𝑔

0









𝛿𝑔

𝛿𝜇

























= −

¯𝐽𝑔𝑤𝑖𝑛𝑖
0

















𝛿𝑤𝑖𝑛𝑖 −

𝛿𝑟 −

¯𝐽𝑔𝑟

0

















¯𝐽𝑔

0









which yields

= −𝐾 𝑜

𝛿𝑤𝑖𝑛𝑖 − 𝐾 𝑜

𝛿𝑟 − 𝐾 𝑜

𝛿𝑔

𝛿𝜇

















¯𝐽𝑔𝑤𝑖𝑛𝑖
0

















¯𝐽𝑔𝑟

0

















Thus, the modified DeeNE policy (4.24) is obtained, and the proof is completed.

,









.

¯𝐽𝑔

0

















(4.25)

(4.26)

(4.27)

□

Remark 9 (Quadratic Cost) One can consider a quadratic cost function 𝐽𝑁 (y, u, 𝜎y, 𝜎u, g) as:

𝐽𝑁 (y, u, 𝜎y, 𝜎u, g) = ∥𝑦 − 𝑟 ∥2

𝑄 + ∥𝑢∥2

𝑅 + 𝜆𝑦 ∥𝜎𝑦 ∥2

2 + 𝜆𝑢 ∥𝜎𝑢 ∥2

2 + 𝜆𝑔 ∥𝑔∥2
2,

(4.28)

where the positive semi-definite matrix 𝑄 ∈ R𝑝𝑁×𝑝𝑁 and the positive definite matrix 𝑅 ∈ R𝑚𝑁×𝑚𝑁

are weighting matrices, and the positive parameters 𝜆𝑦, 𝜆𝑢, 𝜆𝑔 ∈ R are regularization weights. For

the quadratic cost function (4.28), the DeePC is a quadratic program (QP) problem on the decision

variable 𝑔, which requires an iterative solver, i.e. an online QP solver such as qpOASES [52]. To

use the DeeNE, we have

¯𝐽𝑔 = 2((𝑌𝐹 𝑔 − 𝑟)𝑇 𝑄𝑌𝐹 + (𝑈𝐹 𝑔)𝑇 𝑅𝑈𝐹 + 𝜆𝑦 (𝑌𝑃𝑔 − 𝑦𝑖𝑛𝑖)𝑇𝑌𝑃 + 𝜆𝑢 (𝑈𝑃𝑔 − 𝑢𝑖𝑛𝑖)𝑇𝑈𝑃 + 𝜆𝑔𝑔𝑇 ),

¯𝐽𝑔𝑔 = 2(𝑌𝑇

𝐹 𝑄𝑌𝐹 + 𝑈𝑇

𝐹 𝑅𝑈𝐹 + 𝜆𝑦𝑌𝑇

𝑃 𝑌𝑃 + 𝜆𝑢𝑈𝑇

𝑃𝑈𝑃 + 𝜆𝑔),

𝑃 + 𝜆𝑢𝑈𝑇
𝑃),

¯𝐽𝑔𝑤𝑖𝑛𝑖 = −2(𝜆𝑦𝑌𝑇
¯𝐽𝑔𝑟 = −2𝑌𝑇

𝐹 𝑄,

where one can see that ¯𝐽𝑔𝑔 > 0.

49

(4.29)

Remark 10 (Robustness) The DeePC (4.8), the DeeNE (4.18), and the modified DeeNE (4.24)

show that even though an accurate prediction of the future behavior is unattainable in practice

due to the noisy I/O data and the nonlinearities, the obtained optimal control sequence provides

performance guarantees for the actually realized I/O cost.

Algorithm 1 summarizes the modified DeeNE procedure for adaptation the pre-computed

nominal control solution 𝑢𝑜 to the small initial I/O trajectory perturbation 𝛿𝑤𝑖𝑛𝑖 and reference
trajectory perturbation 𝛿𝑟 such that it achieves the optimal control as 𝑢∗ = 𝑢𝑜 + 𝛿𝑢 using Theorem

4. One knows that at the first step, we have the nominal optimal solution from the DeePC as

the input of the algorithm; thus, Theorem 1 and Theorem 2 represent same control policy since

¯𝐽𝑔 (𝑤𝑜

𝑖𝑛𝑖, 𝑔𝑜, 𝑟 𝑜, 𝜇𝑜) = 0 for nominal optimal solution. However, we use Theorem 4 for all time
steps since we use the DeeNE solution as the nominal solution for the future time steps. Moreover,

it is worth noting that choosing 𝑠 > 0 reduces the computational cost, and in some cases may

improve the control performance [27, 53].

and 𝐾∗
2

Algorithm 3 Data-Enabled Neighboring Extremal.
Input: The data matrices 𝑈𝑃, 𝑌𝑃, 𝑈𝐹, and 𝑌𝐹; the function 𝐶; the weighting matrices 𝑄 and 𝑅;
𝑖𝑛𝑖 and reference 𝑟 𝑜 trajectories;
the regularization weights 𝜆𝑦, 𝜆𝑢, and 𝜆𝑔; the nominal initial I/O 𝑤𝑜
and the nominal optimal solution (𝑔𝑜, 𝑢𝑜, 𝑦𝑜) from the DeePC.
1: Calculate the Lagrange multiplier using the nominal optimal solution 𝑔𝑜 and the nominal initial
I/O 𝑤𝑜
𝑖𝑛𝑖 and reference 𝑟 𝑜 trajectories in (4.13).
2: Calculate the gains 𝐾∗
using (4.18).
1
3: Given the real initial I/O 𝑤𝑖𝑛𝑖 and reference 𝑟 trajectories, calculate 𝛿𝑤𝑖𝑛𝑖 and 𝛿𝑟, respectively,
and then 𝛿𝑔 and 𝑔∗ using (4.18).
4: Compute the optimal I/O sequences 𝑢∗ = 𝑈𝐹 𝑔∗ and 𝑦∗ = 𝑌𝐹 𝑔∗.
5: Apply optimal control input (𝑢(𝑘), 𝑢(𝑘 + 1), · · · , 𝑢(𝑘 + 𝑠)) = (𝑢∗
some 𝑠 ≤ 𝑁 − 1.
6: Update the nominal initial I/O trajectory, reference trajectory, and optimal solution as 𝑤𝑜
𝑟 𝑜 = 𝑟, 𝑔𝑜 = 𝑔∗, 𝑢𝑜 = 𝑢∗ and 𝑦𝑜 = 𝑦∗.
7: Set k to k + s and update the initial I/O trajectory 𝑤𝑖𝑛𝑖 and the reference trajectory 𝑟 to the 𝑇𝑖𝑛𝑖
most recent I/O measurements.
8: Return to (1).

𝑠) to the plant for

𝑖𝑛𝑖 = 𝑤𝑖𝑛𝑖,

1, · · · , 𝑢∗

0, 𝑢∗

50

Figure 4.1: Cart-Inverted Pendulum.

4.4 Simulation Results

In this section, we demonstrate the performance of the proposed DeeNE framework via two

simulation examples, i.e., a cart-inverted pendulum and an arm robot.

4.4.1 Cart-Inverted Pendulum

For a cart-inverted pendulum (see Fig. 4.1), one has

(cid:165)𝑧 =

(cid:165)𝜃 =

𝐹 − 𝐾𝑑 (cid:164)𝑧 − 𝑚(𝐿 (cid:164)𝜃2 sin(𝜃) − 𝑔 sin(𝜃) cos(𝜃)) − 2𝑑𝑧
𝑀 + 𝑚 sin2(𝜃)
𝑑𝜃
𝑚𝐿2

(cid:165)𝑧 cos(𝜃) + 𝑔 sin(𝜃)
𝐿

−

,

,

(4.30)

where 𝑧 and 𝜃 denote the position of the cart and the pendulum angle. 𝑚 = 1𝑘𝑔, 𝑀 = 5𝑘𝑔, and

𝐿 = 2𝑚 represent the mass of the pendulum, the mass of the cart, and the length of the pendulum.
𝑔 = 9.81𝑚/𝑠2 and 𝐾𝑑 = 10𝑁 𝑠/𝑚 are the gravity acceleration and the damping parameter. The

variable force 𝐹 controls the system under a friction force 𝑑𝑧 and a friction torque 𝑑𝜃. 𝑇𝑠 = 0.02𝑠

is considered as the sampling time for discretization of the model (4.30), and we assume 𝑑𝑧 and 𝑑𝜃

as the process noises. The states, the outputs, the process noise, the measurement noise, and the

control input constraint are expressed as

𝑥 = [𝑥1, 𝑥2, 𝑥3, 𝑥4]𝑇 = [𝑧, (cid:164)𝑧, 𝜃, (cid:164)𝜃]

𝑇 ,

51

𝝎𝒛𝝎𝒛𝝎𝜽𝑭𝒛𝑳𝜽𝒎𝑴𝑦 = [𝑥1, 𝑥3]𝑇 + 𝑣 = [𝑧, 𝜃]𝑇 + 𝑣,

𝑑 = [𝑑1, 𝑑2, 𝑑3, 𝑑4]𝑇 = [0, 𝑑𝑧, 0, 𝑑𝜃]𝑇 ,

−50 ≤ 𝐹 ≤ 50.

where 𝑑 and 𝑣 represent the process noise and measurement noise, respectively.

The following values are used for the simulation: 𝑇𝑖𝑛𝑖 = 30, 𝑁 = 45, the simulation time
𝑇 = 200, 𝑥(0) = [0, 0, 𝜋/270, 0]𝑇 , and 𝑑𝑧, 𝑑𝜃 = 0.002(2𝑟𝑎𝑛𝑑 (1, 𝑇) − 1). We generate the first

initial trajectory (𝑢𝑖𝑛𝑖, 𝑦𝑖𝑛𝑖) using zero control input, i.e. 𝑢𝑖𝑛𝑖 = 𝑢(0 : 29) = 0, which leads to the
state 𝑥(30) = [0.0151, 0.0783, 0.1225, 0.6200]𝑇 . Figs. 4.2 and 4.3 show the control performances

of the DeeNE and the DeePC. For the DeePC, we use the DeePC policy (4.8), apply the length-s

optimal control sequence to the plant, and update the initial trajectory 𝑤𝑖𝑛𝑖 for the next step (see

Algorithm 2 in [27]). As we discussed in Algorithm 1, we use DeeNE policy (4.24) to avoid solving

the DeePC problem at each step and reduce the computational cost. As is obvious from the Fig.

4.2, one can see that the DeeNE policy approximates the DeePC policy very well and is capable

of adjusting the nominal DeePC by fully considering the initial I/O trajectory perturbations. From

Fig. 4.2, one can see that the DeeNE provides similar performance for trajectory tracking problem

as compared to the DeePC; however, the DeeNE reduces the computational cost of the DeePC very

well as shown in Table II. Table I compares the cost-based performance and the computational time

for the DeePC under various values of 𝑠, where demonstrates that we have the best performance

for the considered system with 𝑠 = 5. Table II illustrates the cost-based performance and the

computational time for both DeePC and DeeNE under two cases 𝑠 = 0 and 𝑠 = 5, and one can

see that the DeeNE with 𝑠 = 5 shows the best performance for the regulation of the cart-inverted

pendulum.

52

Figure 4.2: Control Input for Cart-Inverted Pendulum.

Figure 4.3: System Outputs for Cart-Inverted Pendulum.

Table 4.1: Comparison of Performance and Computational Cost for the DeePC with Various 𝑠.

DeePC
s = 0
s = 5
s = 10

Performance
22.3650
22.1485
22.7375

Time (per loop)
0.1419 𝑚𝑠
0.0284 𝑚𝑠
0.0159 𝑚𝑠

4.4.2 KINOVA Gen3

In order to evaluate the performance of the developed DeeNE, we prove its efficacy on a 7-DoF

robotic manipulator. We learn and control KINOVA Gen3, which is a light weight 7-DoF arm

robot. According to its specifications and for the sake of safety, we consider the minimum and

53

20406080100120140160180200-50-40-30-20-1001020PredictionInitial020406080100120140160180200-0.2-0.100.10.2020406080100120140160180200-0.2-0.100.10.2PredictionInitialTable 4.2: Comparison of Performance and Computational Cost for both DeePC and DeeNE.

Controller
DeePC (s = 0)
DeePC (s = 5)
DeeNE (s = 0)
DeeNE (s = 5)

Performance
22.3650
22.1485
24.0375
23.5335

Time (per loop)
0.1419 𝑚𝑠
0.0284 𝑚𝑠
0.0551 𝑚𝑠
0.0102 𝑚𝑠

maximum values for the joint angular velocities (control inputs) as [−𝜋/6, 𝜋/6]𝑟𝑎𝑑/𝑠. Limitations
on the Cartesian position coverage are all [−0.9, 0.9]𝑚. We give the joint angular velocities 𝑢 ∈ R7
to the arm robot and measure the pose of the robot 𝑦 ∈ R6 which indicates the 3D position and the

3D orientation of the end-effector. However, to avoid a discontinuous behavior in the orientation

part, we transfer the 3D orientation to 4D orientation using Quaternions. The protocol of data

collection is as follows. We have collected data from the 7-DoF arm robot for 50 trajectories with

𝑇𝑖 = 100 data points on each trajectory and the sampling time 𝑇𝑠 = 0.1𝑠. It is worth noting that

since we are generating the Hankel matrix using multiple signal trajectories, called mosaic-Hankel

matrix (a Hankel matrix with discontinuous signal trajectories), the number of data points on each

trajectory must be greater than the depth of the Hankel matrix, i.e. 𝑇𝑖 > 𝑇𝑖𝑛𝑖 + 𝑁 [54]. For

each trajectory, the initial joint angles and the inputs are chosen randomly according to a uniform

probability distribution. Due to the set up condition in the lab (desk structure, wall position, etc.),

we had to stop the robot if it was close to hit an object, ignore that trajectory, and continue the data

collection with another initial position and/or input values.

Details of DeePC are as follows. The reference trajectory 𝑟 (𝑘) ∈ R7 represents the desired

values for the pose of the robot. According to the quadratic cost function, the matrices 𝑄 =
5 × 104 × 𝐼𝑝𝑁 and 𝑅 = 1 × 102 × 𝐼𝑚𝑁 are considered to penalize the tracking error and control input
amplitude, respectively. The slack variables 𝜆𝑦, 𝜆𝑢 = 5 × 105 are used to make sure the feasibility
of the optimal control problem. The regularization parameter 𝜆𝑔 = 5 × 102 avoids the overfitting

issue due to the collected noisy data. Finally, the initial trajectory and the prediction lengths are

𝑇𝑖𝑛𝑖 = 35 and 𝑁 = 20, respectively. Since we have 𝑢 ∈ R7 and 𝑦 ∈ R7, the dimension of the

54

mosaic-Hankel matrix is H(𝑢𝑑, 𝑦𝑑) ∈ R770×2300 causing high computational cost for applying a

real-time DeePC on the 7-DoF arm robot. For DeePC, we use the DeePC policy, apply the first

𝑠 optimal control input 𝑢(𝑘 : 𝑘 + 𝑠) to the 7-DoF arm robot, measure the pose of the robot, and

update the initial trajectory 𝑤𝑖𝑛𝑖 and the reference trajectory 𝑟 for the next step (See Algorithm 2 in

[27]). For an initial pose of the arm robot, we generate the first initial trajectory (𝑢𝑖𝑛𝑖, 𝑦𝑖𝑛𝑖) using

random control inputs, i.e. (𝑢(0 : 34), 𝑦(0 : 34)). For a tracking performance index, we use Root

Mean Square Error (RMSE) between the desired reference trajectory and the pose of the arm robot

over the entire trajectory.

In this part, we compare the performance of the proposed DeeNE framework with DeePC

such that we evaluate the tracking performance and computational time for different open-loop

control scenarios 𝑠. For this part, we use the forward kinematics model of the 7-DoF arm robot

to evaluate the performance of the control schemes. The reference trajectory 𝑟 (𝑘) is consider

as a sinusoidal trajectory with 300 data points for the pose of the end-effector. For the desired

reference trajectory 𝑟 (𝑘), DeePC and DeeNE must accomplish tracking control task. For this case,

we use DeeNE policy to correct the open-loop DeePC solution at each step while we reduce the

computational time. The tracking performance and the computational time are studied for DeePC

and DeeNE frameworks under different open-loop control 𝑠. Fig. 4.4 compares the control input

for the open-loop control scenario 𝑠 = 20, which illustrates taht how DeeNE corrects the DeePC

policy. Figs. 4.5 and 4.6 indicate the position and orientation tracking performance, respectively,

where one can see that DeePC does not track the reference trajectory for 𝑠 = 20; however, DeeNE

tracks the reference very well. From Table 4.3, we can compare the tracking performance and the

computational time indices for both control algorithms under 𝑠 = 0, 𝑠 = 10, and 𝑠 = 20. One

can see that both controllers perform similar on the tracking performance for 𝑠 = 0, but DeeNE

provides lower computational time. However, as we increase the open-loop part of the controller,

i.e., 𝑠, the performance of DeePC goes down since it predicts the behavior of the system using the

last available initial and reference trajectories. On the other hand, DeeNE takes a feedback from

55

the system and update the initial and reference trajectories at each time step, which corrects the

DeePC predictions. Consequently, one can see that DeeNE enables us to have both high-precision

tracking performance and faster motion speed for the 7-DoF arm robot.

Figure 4.4: Control Input for 7-DoF Arm Robot (Simulation).

Figure 4.5: Position Tracking for 7-DoF Arm Robot (Simulation).

56

Figure 4.6: Orientation Tracking for 7-DoF Arm Robot (Simulation).

Table 4.3: Comparison of Performance and Computational Time for DeePC and DeeNE with
Different Open-Loop Control Scenarios.

Controller
DeePC (s = 0)
DeePC (s = 10)
DeePC (s = 20)
DeeNE (s = 0)
DeeNE (s = 10)
DeeNE (s = 20)

RMSE
0.0023
0.0048
0.0051
0.0024
0.0027
0.0031

Time (per loop)
0.2002 𝑠
0.0204 𝑠
0.0114 𝑠
0.0303 𝑠
0.0039 𝑠
0.0025 𝑠

4.5 Experimental Verifications

In this part, we apply both controllers on the real 7-DoF arm robot for a closed-loop control

scenario (i.e., 𝑠 = 0) to make sure the safety and stability of the robot under DeePC. For the

position of the end-effector, we consider the first part of the reference trajectory as the abbreviation

of Michigan State University as MSU including 1000 data points. For the orientation of the end-

effector, we consider the second part of the reference trajectory as 0.5 degree with 1000 data points.

For the desired reference trajectory 𝑟 (𝑘), DeePC must simultaneously accomplish tracking control

and setpoint control tasks for the position and orientation of the end-effector, respectively. For

this case, we use DeeNE policy to avoid solving the DeePC problem at each time step and reduce

57

the computational time, which provides a fast motion speed for the robot. Fig. 4.7 compares

the control inputs generated by both control algorithms, which illustrates the well-performance of

DeeNE on the approximation of the DeePC policy. Figs. 4.8 and 4.9 indicate the position and

orientation tracking performance, respectively, where one can see that both controllers tracks the

reference trajectory very well. From Table 4.4, we can compare the tracking performance and

the computational time indices for both control algorithms, which perform similar on the tracking

performance, but DeeNE provides lower computational time.

It is worth noting that since the

computational time of DeePC, i.e. 0.2𝑠, is higher than the sampling time, i.e. 0.1𝑠, we apply

DeePC on the arm robot for the first 0.1𝑠, stop the robot for the second 0.1𝑠 to receive the updated

response of DeePC, and then repeat the process using the updated control input. Consequently,

it is obvious that DeeNE enables us to have both high-precision tracking performance and faster

motion speed for the 7-DoF arm robot. Moreover, we cannot stop the system until receiving the

updated response of DeePC for safety-critical scenarios since it may cause an accident for the

robots/autonomous vehicles.

Figure 4.7: Control Input for 7-DoF Arm Robot (Experiment).

We next verify the performance of the control algorithms under the safety constraints, which

the arm robot must avoid unsafe regions and dynamic obstacles. In the second experiment, the

58

Figure 4.8: Position Tracking for 7-DoF Arm Robot (Experiment).

Figure 4.9: Orientation Tracking for 7-DoF Arm Robot (Experiment).

Table 4.4: Comparison of Performance and Computational Time for DeePC and DeeNE.

Controller
DeePC
DeeNE

RMSE
0.0142
0.0143

Time (per loop)
0.2005 𝑠
0.0307 𝑠

59

robot must track the same reference trajectory MSU; however, we consider an unsafe region on

the S part. This case addresses the control tasks that the reference trajectory is obtained offline

using path planning or by the operator, but the controller must avoid unsafe regions through the

given reference trajectory due to the dynamic obstacles. For simplicity, we have considered a

fixed unsafe region on the S part of MSU; thus, both DeePC and DeeNE can satisfy the safety

constraints and track the reference trajectory well as shown in Figs. 4.10-4.12. However, like the

previous task, Table 4.5 shows that the computational time of DeePC is higher than the sampling

time, which may cause an accident for the dynamic obstacles since the robot updated the control

input after receiving the response of DeePC. On the other hand, DeeNE quickly computes the

optimal control input for the updated initial and reference trajectories; therefore, it can avoid the

dynamic obstacles well. A video of the experiments performed can be found at the following link

https://www.youtube.com/watch?v=BlKTUgkAMVo.

Figure 4.10: Safe Control Input for 7-DoF Arm Robot (Experiment).

4.6 Chapter Summary

In this chapter, we proposed a computationally efficient method to implement the data-driven

optimal controllers (e.g. DeePC) that include nonlinearities in real time. The DeeNE algorithm

60

Figure 4.11: Safe Position Tracking for 7-DoF Arm Robot (Experiment).

Figure 4.12: Safe Orientation Tracking for 7-DoF Arm Robot (Experiment).

Table 4.5: Comparison of Performance and Computational Cost for DeePC and DeeNE with
Safety Guarantees.

Controller
DeePC
DeeNE

RMSE
0.0148
0.0149

Time (per loop)
0.2008 𝑠
0.0309 𝑠

61

was developed to approximate the DeePC policy in the presence of input/output and reference

trajectories perturbations. The developed DeeNE was based on the second-order variation of the

original DeePC problem such that the computational load of the DeeNE grows linearly for the

optimization horizon. This control approach alleviates the online computational burden and extend

the applicability of data-driven optimal controllers. Simulations of the cart inverted pendulum

system demonstrated the DeeNE’s technological advances over the DeePC. Moreover, simulation

and experimental verifications on the 7-DoF arm robot demonstrated the performance of the DeeNE

compared to the DeePC for a more complex system.

62

CHAPTER 5

ADAPTIVE DATA-ENABLED PREDICTIVE CONTROL

In this chapter, we focus on developing an adaptive data-driven optimal control for time-varying

systems. DeePC uses pre-collected input/output (I/O) data to construct Hankel matrices for online

predictive control. However, in systems with evolving dynamics, incorporating real-time data into

the DeePC framework becomes crucial to enhance control performance. We propose an adaptive

DeePC framework for time-varying systems, enabling the algorithm to update the Hankel matrix

online by adding real-time informative signals. By exploiting the minimum non-zero singular

value of the Hankel matrix, the developed online DeePC selectively integrates informative data

and effectively captures evolving system dynamics. Additionally, a numerical singular value

decomposition technique is introduced to reduce the computational complexity for updating a

reduced-order Hankel matrix. Simulation results on two cases, a linear time-varying system and

the vehicle anti-rollover control, demonstrate the effectiveness of the online reduced-order DeePC.1

5.1 Background

The Fundamental Lemma only holds for deterministic linear time-invariant (LTI) systems [55,

56]. For other systems such as nonlinear systems, stochastic systems, and time-varying systems, the

rank condition on the Hankel matrix is not sufficient to accurately determine the trajectory subspace,

which may lead to poor performance in the data-driven control. Several techniques employing slack

variables and regularization methods have been proposed to address these limitations and enhance

performance for the nonlinear systems and the stochastic systems [27, 29, 57]. Moreover, a robust

Fundamental Lemma has been proposed to ensure the persistently exciting (PE) input with sufficient

order for the stochastic LTI systems [55]. However, these techniques are only effective in local

regions captured by the pre-collected I/O data and cannot handle the new system dynamics that

emerge online. Therefore, it is necessary to update the Hankel matrix online using real-time I/O

1The material of this chapter is from “Online Reduced-Order Data-Enabled Predictive Control,”

arXiv preprint arXiv:2407.16066, 2024 [54].

63

data to predict system behavior accurately.

For time-varying systems, i.e., systems with evolving dynamics, online DeePC [58, 59] is

developed to continuously update the Hankel matrix using real-time data.

In [58], old data is

replaced with new data:

the first column (the oldest data point) is discarded from the Hankel

matrix, all columns are shifted back by one step, and the most recent real-time data is added as

the last column. However, this method requires a PE control input in real-time, which is achieved

by adding a suitable excitation, such as injecting noise into the control input during closed-loop

operation.

Injecting noise can deteriorate control performance and put unnecessary stress on

actuators. To address this, [56] presents a discontinuous online DeePC method that replaces the PE

requirement with a rank condition on mosaic-Hankel matrix (a Hankel matrix with discontinuous

I/O trajectories) proposed by [60]. This approach requires only an offline PE input trajectory, which

is produced based on the robust Fundamental Lemma [55], and updates the mosaic-Hankel matrix

if the rank condition is satisfied. However, as mentioned before, a rank condition (or PE condition)

is not sufficient to indicate informative data for non-deterministic LTI systems.

[61] proposes a continuous online DeePC by adding real-time I/O trajectories to the Hankel

matrix, which removes the PE requirement for the real-time control input as the rank condition is

always satisfied. However, continuously increasing the columns of the Hankel matrix leads to high

memory and computational costs. To mitigate this, two numerical singular value decomposition

(SVD) algorithms are used alternatively based on the rank of the Hankel matrix: when the rank of

the Hankel matrix is less than the number of rows, the numerical SVD algorithm [62], which is

designed based on the eigen-decomposition algorithm [63], is applied; and when the rank of the

Hankel matrix is equal to the number of rows, the numerical SVD algorithm [64] is employed.

The online DeePC scheme [61] faces three main limitations: (i) it requires adding all real-time

trajectories to the Hankel matrix, leading to high computational cost and the inclusion of data

without considering its informativeness; (ii) the numerical SVD algorithm [64], as mentioned in

[63], is not fast and can be unstable; and (iii) the dimension of the online SVD-based DeePC can be

64

reduced to a minimum possible dimension. These limitations motivate us to propose a new online

reduced-order DeePC. Our approach measures the informativeness of real-time data by observing

the minimum non-zero singular value of the Hankel matrix and uses the most informative data

discontinuously, which addresses challenge (i). By setting a well-tuned threshold on the minimum

non-zero singular value, we are able to capture the new system dynamics that emerge online and

update the Hankel matrix only with the most informative data. Moreover, to overcome challenges

(ii) and (iii), we modify the numerical SVD algorithm [62], as this algorithm only works for

low-rank modifications, and develop an online reduced-order DeePC with adaptive order. These

modifications result in lower computational complexity for the online DeePC and may improve the

control performance for some cases.

In this chapter, we develop an adaptive DeePC framework for the time-varying systems such that

the Hankel matrix is updated using real-time informative data. Promising results are demonstrated

by applying the developed adaptive controller to the linear time-varying system and the vehicle

anti-rollover control. The outline of this chapter is as follows. The problem formulation and the

preliminaries of the DeePC are provided in Section II. Section III presents the proposed adaptive

DeePC for the time-varying systems. Section IV presents the simulation results. Finally, the

conclusions are provided in Section V.

5.2 Problem Formulation

Consider the following unknown discrete-time system:

𝑥(𝑘 + 1) = 𝑓 (𝑥(𝑘), 𝑢(𝑘)),

𝑦(𝑘) = ℎ(𝑥(𝑘), 𝑢(𝑘)),

(5.1)

where 𝑘 ∈ N+ represents the time step, 𝑥 ∈ R𝑛 denotes the system state, 𝑢 ∈ R𝑚 indicates the
control input, and 𝑦 ∈ R𝑝 stands for the system output. Moreover, 𝑓 : R𝑛 × R𝑚 → R𝑛 indicates
the system dynamics with 𝑓 (0, 0) = 0, and ℎ : R𝑛 × R𝑚 → R𝑝 denotes the output dynamics.

Assuming the system (5.1) is a linear time-invariant (LTI) system, i.e.,

𝑓 (𝑥(𝑘), 𝑢(𝑘)) =

65

𝐴𝑥(𝑘) + 𝐵𝑢(𝑘) and ℎ(𝑥(𝑘), 𝑢(𝑘)) = 𝐶𝑥(𝑘) + 𝐷𝑢(𝑘), the Fundamental Lemma [26] provides

a non-parametric representation to describe the system behavior.

Definition 4 (Hankel Matrix) Given a signal 𝑤(𝑘) ∈ R𝑞, we denote by 𝑤1:𝑇 the restriction of

𝑤(𝑘) to the interval [1, 𝑇], namely 𝑤1:𝑇 = [𝑤⊤(1), 𝑤⊤(2), · · · , 𝑤⊤(𝑇)]⊤. The Hankel Matrix of

depth 𝐾 ≤ 𝑇 is defined as:

H𝐾 (𝑤1:𝑇 ) :=

𝑤(1)

𝑤(2)
...

𝑤(2)

𝑤(3)
...

· · · 𝑤(𝑇 − 𝐾 + 1)

· · · 𝑤(𝑇 − 𝐾 + 2)
. . .

...

𝑤(𝐾) 𝑤(𝐾 + 1)

· · ·

𝑤(𝑇)















.















(5.2)

Let 𝐿 := 𝑇 − 𝐾 + 1, then we have H𝐾 (𝑤1:𝑇 ) ∈ R𝑞𝐾×𝐿.

Definition 5 (Persistently Exciting) The sequence 𝑤1:𝑇 is persistently exciting (PE) of order 𝐾 if

H𝐾 (𝑤1:𝑇 ) has full row rank, i.e., 𝑟𝑎𝑛𝑘 (H𝐾 (𝑤1:𝑇 )) = 𝑞𝐾.

Lemma 3 (Fundamental Lemma [26]) Consider the system (5.1) as a controllable LTI system
with a pre-collected input/output (I/O) sequence (𝑢d
1:𝑇 ) of length 𝑇. Providing a PE input
1:𝑇
sequence 𝑢d
1:𝑇 of order 𝐾 + 𝑛, any length-𝐾 sequence (𝑢1:𝐾 , 𝑦1:𝐾 ) is an I/O trajectory of the LTI

, 𝑦d

system if and only if we have

for some real vector 𝑔 ∈ R𝐿.


𝑢1:𝐾







𝑦1:𝐾









=









H𝐾 (𝑢d
H𝐾 (𝑦d

1:𝑇 )

1:𝑇 )

𝑔,









Remark 11 (Rank of Hankel Matrix) Considering 𝑟 := 𝑟𝑎𝑛𝑘 (cid:169)
(cid:173)
(cid:173)
(cid:171)

citing control input sequence of order 𝐾 + 𝑛 ensures that 𝑚𝐾 + 1 ≤ 𝑟 ≤ 𝑚𝐾 + 𝑛. Furthermore,

H𝐾 (𝑢d
H𝐾 (𝑦d

1:𝑇 )

1:𝑇 )

















(cid:170)
(cid:174)
(cid:174)
(cid:172)

𝑟 = 𝑚𝐾 + 𝑛 if 𝐾 ≥ 𝑙, where 𝑙 ≤ 𝑛 is the observability index. See [65] for more details.

The Fundamental Lemma shows that the Hankel matrix in (5.3) spans the vector space of

all length-𝐾 signal trajectories that an LTI system can produce, provided that the collected input

66

(5.3)

, a persistently ex-

sequence is PE of order 𝐾 + 𝑛 and the underlying system is controllable. However, the Fundamental

Lemma requires a long continuous signal trajectory to construct the Hankel matrix. [60] extends

the Fundamental Lemma to accommodate multiple short signal trajectories, which we refer to as

discontinuous Fundamental Lemma in this paper. This extended Lemma is developed by using

a general data structure called mosaic-Hankel matrix, which incorporates a dataset consisting of

multiple short discontinuous signal trajectories.

Definition 6 (Mosaic-Hankel Matrix) Let 𝑊 = {𝑤1

1:𝑇𝑠 } be the set of 𝑠 discontinuous
sequences with length of 𝑇1, · · · , 𝑇𝑠. The mosaic-Hankel matrix of depth 𝐾 ≤ min(𝑇1, · · · , 𝑇𝑠) is

, · · · , 𝑤𝑠

1:𝑇1

defined as:

M𝐾 (𝑊) = [H𝐾 (𝑤1

), H𝐾 (𝑤2

1:𝑇1
𝑖=1 𝑇𝑖 and 𝐿 = 𝑇 − 𝑠(𝐾 − 1), then we have M𝐾 (𝑊) ∈ R𝑞𝐾×𝐿.

1:𝑇2

), · · · , H𝐾 (𝑤𝑠

1:𝑇𝑠 )].

Let 𝑇 = (cid:205)𝑠

(5.4)

Lemma 4 (Discontinuous Fundamental Lemma [60]) Consider the system (5.1) as a control-
lable LTI system with a pre-collected I/O sequence 𝑈d = {𝑢d,1
, · · · , 𝑦d,𝑠
1:𝑇𝑠 }
1:𝑇1
of length 𝑇 which consists of 𝑠 I/0 sequences of length 𝑇1, · · · , 𝑇𝑠. Providing an input sequence 𝑈d

1:𝑇𝑠 }, 𝑌 d = {𝑦d,1
1:𝑇1

, · · · , 𝑢d,𝑠

with

𝑟 := rank (cid:169)
(cid:173)
(cid:173)
(cid:171)


M𝐾 (𝑈d)



M𝐾 (𝑌 d)












(cid:170)
(cid:174)
(cid:174)
(cid:172)

= 𝑚𝐾 + 𝑛,

(5.5)

any length-𝐾 sequence (𝑢1:𝐾 , 𝑦1:𝐾 ) is an I/O trajectory of the LTI system if and only if we have


𝑢1:𝐾







𝑦1:𝐾









=









M𝐾 (𝑈d)
M𝐾 (𝑌 d)

𝑔,









(5.6)

for some real vector 𝑔 ∈ R𝐿.

It is worth noting that the Hankel matrix (5.2) represents a special case of the mosaic-Hankel

matrix (5.4) with 𝑠 = 1. The general form (5.4) also describes other special forms, such as Page

matrix or Trajectory matrix [65]. The main advantages of Lemma 4 compared to Lemma 3 are: i)

it uses multiple short discontinuous trajectories instead of one long continuous trajectory, and ii) it

67

replaces the PE condition on the input sequence with the rank condition (5.5) on the I/O matrix.

(5.5) is referred as generalized PE condition [60].

Both (5.3) and (5.6) can be regarded as the non-parametric representation for system (5.1). Let

𝑇ini, 𝑁 ∈ Z, and 𝐾 = 𝑇ini + 𝑁. The Hankel matrices H𝐾 (𝑢d
1:𝑇 ) (or mosaic-Hankel
matrices M𝐾 (𝑈d), M𝐾 (𝑌 d)) are divided into two parts (i.e., “past data” of length 𝑇ini and “future

1:𝑇 ), H𝐾 (𝑦d

(5.7)

(5.8)

data” of length 𝑁):

or

= H𝐾 (𝑢d

1:𝑇 ),


𝑈𝑃



𝑈𝐹












= H𝐾 (𝑦d

1:𝑇 ),


𝑌𝑃



𝑌𝐹












= M𝐾 (𝑈d),

= M𝐾 (𝑌 d),


𝑈𝑃



𝑈𝐹













𝑌𝑃



𝑌𝐹












where 𝑈p and 𝑈f denote the first 𝑇ini block rows and the last 𝑁 block rows of H𝐾 (𝑢d
1:𝑇 ) (or
M𝐾 (𝑈d)), respectively (similarly for 𝑌p and 𝑌f). The data-enabled predictive control (DeePC) is

formulated as [27, 36]

𝐽 (𝑦, 𝑢, 𝜎𝑦, 𝜎𝑢, 𝑔)

(5.9)

(𝑦∗, 𝑢∗, 𝜎∗

𝑦 , 𝜎∗

𝑢 , 𝑔∗) = arg min
𝑦,𝑢,𝜎𝑦,𝜎𝑢,𝑔

s.t.

𝑢

𝑌𝑃

𝑔 =
















𝑈𝑃



𝑈𝐹









𝑢 ∈ U, 𝑦 ∈ Y.


𝑢ini













𝑦ini

𝑌𝐹

𝑦

+















,

𝜎𝑢

0

𝜎𝑦

0





























In (5.9), 𝐽 (𝑦, 𝑢, 𝜎𝑦, 𝜎𝑢, 𝑔) represents the cost function. 𝑢ini = 𝑢𝑘−𝑇ini:𝑘−1 is the control input
sequence within a past time horizon of length 𝑇ini, and 𝑢 = 𝑢𝑘:𝑘+𝑁−1 is the control input sequence
within a prediction horizon of length 𝑁 (similarly for 𝑦ini and 𝑦). U, Y represent the input and
output constraints, respectively. 𝜎𝑢 ∈ R𝑚𝑇ini, 𝜎𝑦 ∈ R𝑝𝑇ini stand for auxiliary variables.

Both Lemma 3 and Lemma 4 are only valid for deterministic LTI systems.

[59] and [58]

respectively show that Lemma 3 can be extended to the nonlinear systems and the time-varying

68

systems by continuously updating the Hankel matrix, which require a real-time PE input sequence.

Moreover, [56] removes the real-time PE requirement for the time-varying systems by using Lemma

4. However, the proposed rank condition (or PE condition) is not sufficient to ensure the infor-

mativeness of the data for non-deterministic LTI systems. While the rank condition may hold

for the non-deterministic LTI systems, it can still lead to large prediction errors and poor control

performance. In this paper, we propose an online DeePC framework for the time-varying systems

to improve the rank condition and present a valid indicator for evaluating data informativeness.

5.3 Main Result

In this section, based on the data informativeness of the mosaic-Hankel matrix, we propose an

online DeePC framework for the time-varying systems. The data informativeness is evaluated with

the minimum non-zero singular value of the mosaic-Hankel matrix and is enhanced by adding the

most informative signals to the matrix. It should be mentioned that the rank condition is always

satisfied in real time since we add signal trajectories as additional columns. Moreover, we develop

an online reduced-order DeePC using a numerical singular value decomposition (SVD) to reduce

the computational cost of the control scheme.

5.3.1 Adaptive Data-Enabled Predictive Control

For the deterministic LTI systems, Lemma 3 and Lemma 4 are valid only if the data is sufficiently

informative. However, for the non-deterministic LTI systems, the collected data may result in a full-

rank mosaic-Hankel matrix (or Hankel matrix) but cannot ensure accurate prediction of the system

behavior. To address this issue, [55] proposes a quantitative measure of PE for the input trajectory

based on the minimum non-zero singular value of the Hankel matrix, enhancing the robustness of

the Fundamental Lemma against uncertainties. However, this approach is only effective locally

around collected I/O data and leads to poor performance for the time-varying systems. Therefore,

we focus on improving the performance of the discontinuous Fundamental Lemma for the time-

varying systems. Singular values of a matrix are defined as 𝜎1 ≥ 𝜎2 ≥ · · · ≥ 𝜎𝑟 ≥ · · · ≥ 𝜎end ≥ 0,

69

where 𝜎1 and 𝜎end are also referred as 𝜎max and 𝜎min, respectively. The singular value 𝜎𝑟

corresponds to the rank of the matrix and is the minimum non-zero singular value, i.e., all singular

values from 𝜎𝑟+1 to 𝜎min are zero.

[55] shows that the prediction error of the Fundamental Lemma can be arbitrarily large, even if

the rank condition on the Hankel matrix is met. The important factor is the data informativeness,

which is represented by the minimum non-zero singular value 𝜎𝑟 . Therefore, we can update the

mosaic-Hankel matrix with real-time signal trajectories if 𝜎𝑟 increases.

If the real-time signal

trajectory contains new information relevant to describing the system behavior (5.1), it increases

𝜎𝑟 ; otherwise, 𝜎𝑟 decreases if the real-time signal trajectory lacks new information. Thus, we

can set a threshold 𝜎thr for 𝜎𝑟 to ensure that only the most informative data is added provided

𝜎𝑟 ≥ 𝜎thr. To achieve better prediction for the time-varying systems using the Fundamental

Lemma, we formulate an online DeePC as follows:

𝐽 (𝑦, 𝑢, 𝜎𝑦, 𝜎𝑢, 𝑔)

(5.10)

(𝑦∗, 𝑢∗, 𝜎∗

𝑦 , 𝜎∗

𝑢 , 𝑔∗) = arg min
𝑦,𝑢,𝜎𝑦,𝜎𝑢,𝑔

s.t.

𝑢

𝑔 =















𝑈𝑜


𝑃


𝑈𝑜

𝐹


𝑌 𝑜

𝑃


𝑌 𝑜


𝐹

𝑢 ∈ U, 𝑦 ∈ Y,


𝑢ini













𝑦ini

𝑦

+















,

𝜎𝑢

0

𝜎𝑦

0





























where the matrices 𝑈𝑜
𝑃

, 𝑈𝑜
𝐹

, 𝑌 𝑜
𝑃

, and 𝑌 𝑜
𝐹

are updated online. Specifically, denote 𝑊 d as the combined

data set of 𝑈d and 𝑌 d, i.e., 𝑊 d = {𝑈d, 𝑌 d}. The corresponding mosaic-Hankel matrix M𝐾 (𝑊 d) is

defined as:

.

(5.11)

When 𝑘 < 𝐾, the matrices 𝑈𝑜
𝑃

are initialized with M𝐾 (𝑊 d) (i.e., 𝑈d and
𝑌 d as shown in (5.8)). When 𝑘 ≥ 𝐾, the most recent real-time I/O sequence 𝑤𝑘−𝐾+1:𝑘 :=

, 𝑈𝑜
𝐹

, 𝑌 𝑜
𝑃

M𝐾 (𝑊 d) :=

M𝐾 (𝑈d)
M𝐾 (𝑌 d)
















, and 𝑌 𝑜
𝐹

70

(cid:104)

𝑢⊤

𝑘−𝐾+1:𝑘

, 𝑦⊤

𝑘−𝐾+1:𝑘

(cid:105) ⊤

is first used to update the matrix

M𝐾 (𝑊 d) ←

(cid:104)

M𝐾 (𝑊 d), 𝑤𝑘−𝐾+1:𝑘

(cid:105)

.

Then, the rank and the minimum non-zero singular value of M𝐾 (𝑊 d) are calculated.
minimum non-zero singular value of M𝐾 (𝑊 d) is larger than the threshold 𝜎thr, i.e., 𝜎𝑟 ≥ 𝜎thr,
are updated with M𝐾 (𝑊 d); otherwise, the sequence
then the matrices 𝑈𝑜
𝑃

, and 𝑌 𝑜
𝐹

, 𝑈𝑜
𝐹

, 𝑌 𝑜
𝑃

If the

𝑤𝑘−𝐾+1:𝑘 is removed from M𝐾 (𝑊 d).

5.3.2 Adaptive Reduced-Order Data-Enabled Predictive Control

SVD techniques are effective in reducing computational complexity for data-driven control

methods [66, 61]. In this section, we incorporate a new numerical SVD algorithm into the online

DeePC framework such that the reduced-order mosaic-Hankel matrix and corresponding singular

values can be updated efficiently.

Considering 𝑟 = rank(M𝐾 (𝑊 d)), one can formulate the SVD of the mosaic-Hankel matrix

M𝐾 (𝑊 d) as follows:

M𝐾 (𝑊 d) = 𝑈Σ𝑉 ⊤ = [𝑈𝑟 𝑈𝑞𝐾−𝑟 ]

[𝑉𝑟 𝑉𝐿−𝑟 ]⊤,

(5.12)

Σ𝑟 0









0





0




where 𝑞 = 𝑚 + 𝑝, Σ ∈ R𝑞𝐾×𝐿 is the singular matrix, and 𝑈 ∈ R𝑞𝐾×𝑞𝐾 and 𝑉 ∈ R𝐿×𝐿 are left

and right singular vectors, respectively, such that 𝑈𝑈⊤ = 𝑈⊤𝑈 = 𝐼𝑞𝐾 and 𝑉𝑉 ⊤ = 𝑉 ⊤𝑉 = 𝐼𝐿.
Moreover, Σ𝑟 contains the top 𝑟 non-zero singular values, 𝑈𝑟 ∈ R𝑞𝐾×𝑟 , 𝑈𝑞𝐾−𝑟 ∈ R𝑞𝐾×(𝑞𝐾−𝑟), and
𝑉𝑟 ∈ R𝐿×𝑟 , 𝑉𝐿−𝑟 ∈ R𝐿×(𝐿−𝑟). Therefore, one can write

M𝐾 (𝑊 d)𝑔 = 𝑈𝑟 Σ𝑟𝑉 ⊤

𝑟 𝑔 = M′

𝐾 (𝑊 d)𝑔′,

(5.13)

where M′

𝐾 (𝑊 d) = 𝑈𝑟 Σ𝑟 ∈ R𝑞𝐾×𝑟 and 𝑔′ = 𝑉 ⊤

𝑟 𝑔 ∈ R𝑟 . If the pre-collected data is sufficient rich,

then we have 𝑚𝐾 + 𝑛 ≤ 𝑟 ≤ min(𝑞𝐾, 𝐿). Thus, one can approximate (5.13) using a rank order

𝑚𝐾 + 𝑛 ≤ 𝑟𝑎 ≤ 𝑟, as follows:

M𝐾 (𝑊 d)𝑔 ≈ 𝑈𝑟𝑎 Σ𝑟𝑎𝑉 ⊤

𝑟𝑎 𝑔 = M′′

𝐾 (𝑊 d)𝑔′′,

(5.14)

71

where M′′

𝐾 (𝑊 d) = 𝑈𝑟𝑎 Σ𝑟𝑎 ∈ R𝑞𝐾×𝑟𝑎 and 𝑔′′ = 𝑉 ⊤

𝑟𝑎 𝑔 ∈ R𝑟𝑎 .

Now, one can formulate an online reduced-order DeePC as follows:

𝐽 (𝑦, 𝑢, 𝜎𝑦, 𝜎𝑢, 𝑔′′)

(5.15)

(𝑦∗, 𝑢∗, 𝜎∗

𝑦 , 𝜎∗

𝑢 , 𝑔′′∗) = arg min

𝑦,𝑢,𝜎𝑦,𝜎𝑢,𝑔′′

s.t.

𝑢

𝑔′′ =

𝑈𝑜′′



𝑢ini



𝑃



𝑈𝑜′′






𝐹






𝑌 𝑜′′



𝑃






𝑌 𝑜′′






𝐹



𝑢 ∈ U, 𝑦 ∈ Y,

𝑦ini

𝑦

+















,

𝜎𝑢

0

𝜎𝑦

0





























where the matrices 𝑈𝑜′′
𝑃

, 𝑈𝑜′′
𝐹

, 𝑌 𝑜′′
𝑃

, and 𝑌 𝑜′′
𝐹

adaptive order 𝑟𝑎 such that 𝜎𝑟𝑎 ≥ 𝜎thr.

M′′

𝐾 (𝑊 d) for M𝐾 (𝑊 d) ←

(cid:104)

M𝐾 (𝑊 d), 𝑤𝑘−𝐾+1:𝑘

are updated online based on M′′

𝐾 (𝑊 d) under an
It should be mentioned that we use SVD to update
(cid:105). Moreover, the adaptive order 𝑚𝐾 +𝑛 ≤ 𝑟𝑎 ≤ 𝑟,

which is based on the threshold singular value 𝜎thr, allows an adaptive dimension for the reduced-
𝐾 (𝑊 d) regarding the data informativity. Indeed, the dimension of

order mosaic-Hankel matrix M′′

M′′

𝐾 (𝑊 d), i.e., 𝑟𝑎, is changed adaptively based on 𝜎𝑟𝑎 ≥ 𝜎thr.
For both proposed online DeePC frameworks, calculating the SVD at each time step is not

computationally efficient for real-time control. Therefore, inspired by [62], we propose a numerical
(cid:105) by taking advantage of our knowledge of
algorithm to compute the SVD of (cid:104)

M𝐾 (𝑊 d), 𝑤𝑘−𝐾+1:𝑘
the SVD of M𝐾 (𝑊 d), which reduces the computational time of the proposed online DeePC.

When rank(M𝐾 (𝑊 d)) < rows(M𝐾 (𝑊 d)), one can express (cid:104)

M𝐾 (𝑊 d), 𝑤𝑘−𝐾+1:𝑘

(cid:105) as 𝑋 +

𝐴𝐵⊤, where 𝑋 = [M𝐾 (𝑊 d), 0], 𝐴 = 𝑤𝑘−𝐾+1:𝑘 , and 𝐵 = [0, · · · , 0, 1]⊤. Therefore, one has

𝑋 + 𝐴𝐵⊤ = [𝑈𝑟 𝐴]








𝑟 0]⊤. Let 𝑃 be an orthogonal basis of the column space of (𝐼 − 𝑈𝑟𝑈⊤

[𝑉𝑥 𝐵]⊤.

Σ𝑟 0









0

𝐼

(5.16)

𝑟 ) 𝐴, i.e., the

where 𝑉𝑥 = [𝑉 ⊤

72

component of 𝐴 that is orthogonal to 𝑈𝑟 , and set 𝑅𝐴 = 𝑃⊤(𝐼 − 𝑈𝑟𝑈⊤

𝑟 ) 𝐴. Now, one can write

[𝑈𝑟 𝐴] = [𝑈𝑟 𝑃]

𝐼 𝑈⊤

𝑟 𝐴

0

𝑅𝐴









,









(5.17)

where similar to a QR decomposition, 𝑅𝐴 needs not be upper-triangular or square. Similarly, let

𝑅𝐵 = 𝑄⊤(𝐼 − 𝑉𝑥𝑉 ⊤

𝑥 )𝐵, where 𝑄 is the component of 𝐵 that is orthogonal to 𝑉𝑥. Thus, one has

[𝑉𝑥 𝐵] = [𝑉𝑥 𝑄]

𝐼 𝑉 ⊤

𝑥 𝐵

0

𝑅𝐵









.









Substituting (5.17) and (5.18) into (5.16), we have

𝑋 + 𝐴𝐵⊤ = [𝑈𝑟 𝑃]𝑆[𝑉𝑥 𝑄]⊤,

𝑆 =

𝐼 𝑈⊤

𝑟 𝐴

0

𝑅𝐴

























Σ𝑟 0

0

𝐼

















𝐼 𝑉 ⊤

𝑥 𝐵

0

𝑅𝐵

⊤

,









(5.18)

(5.19)

where one can write 𝑆 as:

𝑆 =

Using [63], diagonalizing 𝑆 = 𝑈𝑆Σ𝑆𝑉 ⊤
𝑆

⊤

Σ𝑟 0









0

+

𝑟 𝐴


𝑈⊤











0



gives rotations 𝑈𝑆 and 𝑉𝑆 of the extended subspaces


𝑉 ⊤
𝑥 𝐵







(5.20)

















𝑅𝐴

𝑅𝐵

.

[𝑈𝑟 𝑃] and [𝑉𝑥 𝑄] such that

𝑋 + 𝐴𝐵⊤ = ( [𝑈𝑟 𝑃]𝑈𝑆)Σ𝑆 ( [𝑉𝑥 𝑄]𝑉𝑆)⊤

(5.21)

is the desired SVD.

Remark 12 (Rank-1 Modifications) For the proposed numerical SVD, we are not limited to only

add one signal to the mosaic-Hankel matrix. The above formulations work for adding more signals

at the same time by defining the correct matrices 𝐴 and 𝐵. However, in our algorithm, we focus on

adding one signal to the matrix at each time step and calculate the SVD of the new matrix based on

73

the SVD of the original matrix, which is called Rank-1 modification in the numerical SVD. For Rank-
𝑟 ) 𝐴∥−1(𝐼−𝑈𝑟𝑈𝑇
1 modification, we define 𝑃 = ∥(𝐼−𝑈𝑟𝑈𝑇
Σ𝑟 𝑈𝑇

𝑟 ) 𝐴 and 𝑄 = ∥(𝐼−𝑉𝑥𝑉𝑇

𝑥 )𝐵∥−1(𝐼−𝑉𝑥𝑉𝑇

𝑥 )𝐵,

𝑟 𝐴

which yield 𝑆 =

since 𝑉𝑇

𝑥 𝐵 = [𝑉𝑇

𝑟 0] [0, ..., 0, 1]𝑇 = 0.









0

𝑅









When rank(M𝐾 (𝑊 d)) = rows(M𝐾 (𝑊 d)), one needs rewrite the SVD of the mosaic-Hankel

matrix (cid:104)

M𝐾 (𝑊 d), 𝑤𝑘−𝐾+1:𝑘

(cid:105) as

𝑋 + 𝐴𝐵⊤ = 𝑈𝑟 [Σ𝑟 𝑈⊤

𝑟 𝐴] [𝑉𝑥 𝐵]⊤

Therefore, we only need to provide an orthogonal matrix for [𝑉𝑥 𝐵], which yields 𝑆 =

(5.22)

(cid:104)

(cid:105)

+

Σ𝑟 0

𝑈⊤

𝑟 𝐴

⊤


𝑉 ⊤
𝑥 𝐵







𝑅𝐵









, and 𝑋 + 𝐴𝐵⊤ = (𝑈𝑟𝑈𝑆)Σ𝑆 ( [𝑉𝑥 𝑄]𝑉𝑆)⊤ is the desired SVD.

Remark 13 (Comparison) Compared to [61], the proposed online reduced-order DeePC mea-

sures data informativity of real-time signals by observing the minimum non-zero singular value

of the mosaic-Hankel matrix, selectively using the most informative signals instead of adding all

real-time trajectories. Moreover, the dimension of the online reduced-order DeePC starts from

𝑟𝑎 = 𝑚𝐾 + 𝑛 instead of the rank, which leads to the minimum possible dimension for the mosaic-

Hankel matrix. We also refine the numerical SVD algorithm, addressing its speed and stability

issues as noted in [63]. These three contributions improve the control performance and reduce

computational complexity.

5.4 Simulation Results

5.4.1 Linear Time-Varying System

In this subsection, we demonstrate the performance of the proposed online reduced-order

DeePC framework via a simulation example on a linear time-varying (LTV) system, described by:

𝑥(𝑘 + 1) = 𝐴(𝑘)𝑥(𝑘) + 𝐵(𝑘)𝑢(𝑘) + 𝑑 𝑝 (𝑘),

(5.23)

𝑦(𝑘) = 𝐶𝑥(𝑘) + 𝑑𝑚 (𝑘),

74

where 𝑑 𝑝 and 𝑑𝑚 represent process and measurement noises, respectively. The matrices 𝐴(𝑘),

𝐵(𝑘), and 𝐶 are constructed as

𝐴(𝑘) = 𝐴𝑜 + 𝜆(𝑘)

𝐵(𝑘) = 𝐵𝑜 + 𝜆(𝑘)

,















0

0

0

0.01


















0.0001










0.001

0.001

0

0

0.001

0

0.01

0

0.001

0

0

0.01

0

0

0.01

,

0.001


0.0001













0.001

0

𝐶 =


1 0 0 0



0 1 0 0












,

where 𝜆(𝑘) is a time-varying parameter, and 𝐴𝑜 and 𝐵𝑜 are set as follows:

0.041

0

0

0.033

0.924

0

0

0.937

,















.

𝐴𝑜 =

𝐵𝑜 =

0

0

0

0

0

0

0.918

0.921
















0.017 0.001






0.001 0.023






0.061








0.072





0

0

For the simulation, we consider the following values: 𝑇ini = 35, 𝑁 = 45, the simulation time

𝑇𝑐 = 2100, 𝑥(0) = [0.5, 0.5, 0.5, 0.5]⊤, and 𝑑 𝑝 and 𝑑𝑚 are considered random values such that

∥𝑑 𝑝 ∥, ∥𝑑𝑚 ∥ ≤ 0.002. Moreover, the initial trajectory (𝑢ini, 𝑦ini) is generated by applying zero

control input to the system and measuring the corresponding output.

75

The evolution of 𝜆(𝑘) is shown in Fig. 5.1, where one can see that the behavior of the LTV

system is repeating in real time. Fig. 5.1 also depicts the order of the reduced-order mosaic-Hankel

matrix for the proposed online DeePC (5.15), showing that the adaptive order 𝑟𝑎 changes when the

LTV system switches to different dynamics. For the online DeePC [61], the order is constant and

equal to the rank of the mosaic-Hankel matrix, which is 320 for the collected data set.

Figs. 5.2 and 5.3 show the control performance of the proposed online reduced-order DeePC

(5.15) in comparison with the traditional DeePC [27], the online DeePC [56] (replacing old data

with new data in the mosaic-Hankel matrix), and the online DeePC [61] (adding new data to the

mosaic-Hankel matrix). For the traditional DeePC [27], the DeePC policy (5.9) is employed such

that the first optimal control input is applied to the system, and the initial trajectory {𝑢ini, 𝑦ini}

is updated for the next step (see Algorithm 2 in [27]). From Figs. 5.2 and 5.3, one can see that

the proposed online DeePC shows better tracking performance in comparison with other control

schemes since it uses informative data to update the mosaic-Hankel matrix. The online DeePC

[56] does not show a reasonable performance when all PE data is removed from the mosaic-Hankel

matrix since the rank condition is not enough to evaluate suitable data. Table 5.1 demonstrates

that the developed online DeePC significantly reduces the computational time compared to other

controllers while keep the tracking performance well. Moreover, the average time for computing

the numerical SVD is 0.0261𝑚𝑠 for the online DeePC [61]; however, our proposed numerical SVD

(5.22) takes 0.0085𝑚𝑠, which leads to lower computational complexity for the the online DeePC

(5.15). The dimension of the online reduced-order DeePC (5.15) is another reason for better

computational time since it is the minimum possible dimension for the mosaic-Hankel matrix. It is

worth noting that Track. Perf. represents tracking performance, which is root mean squared error

as ∥𝑟𝑒 𝑓 𝑒𝑟𝑒𝑛𝑐𝑒 − 𝑦∥, for different control frameworks.

76

Figure 5.1: Order of Reduced-Order Mosaic-Hankel Matrix for LTV System.

Figure 5.2: Comparison of Control Inputs for LTV System.

Table 5.1: Comparison of Tracking Performance and Computational Cost for LTV System.

Controller

DeePC [8]
Online DeePC [13]
Online DeePC [19]
Online R.O. DeePC (Ours)

Track.
Perf.
4.83
3.39
3.05
3.05

Time (per loop)

0.21 𝑠
0.18 𝑠
0.09 𝑠
0.05 𝑠

77

200400600800100012001400160018002000-1-0.500.51200400600800100012001400160018002000165170175200400600800100012001400160018002000051002004006008001000120014001600180020000510300350400450500550600-0.500.511.5Figure 5.3: Comparison of System Outputs for LTV System.

5.4.2 Vehicle Rollover Avoidance

Rollover is a type of vehicle accident in which a vehicle tips over onto its side or roof. The

rollover propensity of a vehicle is changed for different road surfaces or carried loads. Therefore, it

is a big challenge to derive an accurate model for the vehicle dynamics which includes all operating

conditions. We apply our online reduced-order DeePC to safeguard a vehicle against rollover.

Considering a constant longitudinal speed for the vehicle, the steering wheel angle (SW) acts as

the command, which is generated either by a human operator or a higher-level planning algorithm.

For an arbitrary reference command SW, denoted as 𝑢𝑟 , the rollover constraint may not be satisfied.

Thus, DeePC is used as a safety filter for the reference command SW 𝑢𝑟 to obtain an admissible

input 𝑢. Indeed, DeePC ensures compliance with the load transfer ratio (LTR) constraint to prevent

potential rollovers.

Following [67], a linear model is considered to represent the vehicle dynamics, which vehicle

roll angle 𝑞, roll rate 𝑝, lateral velocity 𝑣, and yaw rate 𝛾 represent the states of the system.

Considering the system in the format of (5.23), we have 𝐴(𝑘) = 𝐴𝑜 + 0.01𝜆(𝑘) 𝐴𝑜, 𝐵(𝑘) =

78

20040060080010001200140016001800200000.51020040060080010001200140016001800200000.513003504004505005506000.60.70.8𝐵𝑜 + 0.01𝜆(𝑘)𝐵𝑜 as the time-varying matrices, and 𝐴𝑜, 𝐵𝑜, and 𝐶 are considered as:

0.0154 −6.81 × 10−5

0.997

−78.3

0.00499

−12.2 −65.3

−0.932 −0.799 −6.20














(cid:104)
−5.76 × 10−5 2.80 0.278 0.655

3.32

8.27

1.52

−3.89

−1.57

−1.49
(cid:105)𝑇

,

+ 𝐼4,















𝐴𝑜 = 𝑇𝑠

𝐵𝑜 = 𝑇𝑠

(cid:104)

𝐶 =

0.1200 0.0124 −0.0108 0.0109

(cid:105)

,

where 𝑥 =

(cid:104)

𝑞 𝑝 𝑣 𝛾

(cid:105)𝑇

, 𝑢 = 𝑆𝑊, 𝑦 = 𝐿𝑇 𝑅, and 𝑇𝑠 is sampling time. It should be mentioned that

the matrices 𝐴𝑜, 𝐵𝑜, and 𝐶 are obtained based on a CarSim model for a standard utility truck under

a constant longitudinal speed 80𝑘𝑚/ℎ. More specifically, the vehicle tracks a constant reference

longitudinal speed 80𝑘𝑚/ℎ using a feedback control on the gas pedal, which is not discussed here.

Through the LTR, the rollover constraint is defined as:

𝐿𝑇 𝑅 =

𝐹𝑧,𝑅 − 𝐹𝑧,𝐿
𝑚𝑔

,

where 𝑚𝑔 is the vehicle weight, and 𝐹𝑧,𝑅 and 𝐹𝑧,𝐿 stand for the total vertical force on the right-side

tires and the left-side tires, respectively. Note that |𝐿𝑇 𝑅| > 1 means wheels lifting off; thus, the

rollover constraint is imposed as:

− 1 ≤ 𝐿𝑇 𝑅 ≤ 1.

For the simulation, we consider the following values: 𝑇𝑖𝑛𝑖 = 10, 𝑁 = 15, the simulation time
𝑇𝑐 = 1200, the smapling time 𝑇𝑠 = 0.1, 𝑥(0) = [0, 0, 0, 0]𝑇 , and random values ∥𝑑 𝑝 ∥, ∥𝑑𝑚 ∥ ≤

0.002. Moreover, the first initial trajectory (𝑢𝑖𝑛𝑖, 𝑦𝑖𝑛𝑖) is generated by applying 𝑢(𝑘) = 55, 𝑘 =

1 : 𝑇𝑖𝑛𝑖 to the system and measuring the corresponding output. For this case, 𝜆(𝑘) and 𝑟𝑎 are

shown in Fig. 5.4, where one can see that the adaptive order 𝑟𝑎 changes when the system switches

to another dynamics. For the online DeePC [61], the rank of the mosaic-Hankel matrix is 50,

which is the order of the reduced-order mosaic-Hankel matrix. Like the previous case, Fig. 5.4

79

illustrates that the online DeePC (5.15) has lower order than the online DeePC [61], which leads to

better computational cost. Figs. 5.5 and 5.6 show the control performance of the proposed online

reduced-order DeePC (5.15) in comparison with the traditional DeePC [27], the online DeePC

[56] (replacing old data by new data in the mosaic-Hankel matrix), and the online DeePC [61]

(adding new data to the mosaic-Hankel matrix). Fig. 5.5 shows the tracking performance and

modification to the reference command SW for the control strategies. As shown in Fig. 5.6, one

can see that the proposed online reduced-order DeePC only satisfies the LTR constraint, and other

control schemes cannot satisfy the constraint. Table II demonstrates that the developed online

DeePC significantly reduces the computational time compared to other controllers while keep the

tracking performance and system safety well. It should be mentioned that Const. Viol. represents

constraint violation number for different control frameworks.

In this case, not only our online

DeePC has lower computational cost compared to the online DeePC [61], but also it only satisfies

the safety constraint due to employing useful information in the mosaic-Hankel matrix.

Figure 5.4: Order of Reduced-Order Mosaic-Hankel Matrix for Vehicle Rollover Avoidance.

80

1002003004005006007008009001000110001231002003004005006007008009001000110029303132Figure 5.5: Control Inputs for Vehicle Rollover Avoidance.

Figure 5.6: System Outputs for Vehicle Rollover Avoidance.

Table 5.2: Comparison of Safety Performance and Computational Cost for Vehicle Rollover
Avoidance.

Time (per loop)

0.024 𝑠
0.025 𝑠
0.010 𝑠
0.007 𝑠

Controller

DeePC [8]
Online DeePC [13]
Online DeePC [19]
Online R.O. DeePC (Ours)

Const.
Viol.
1
1
1
0

81

020040060080010001200-100-50050100150ReferenceDeePC [8]Online DeePC - Replacement [13]Online DeePC - Adding [19]Online Reduced-Order DeePC (Ours)400405410415-90-80-70-60-50020040060080010001200-1.5-1-0.500.511.52ConstraintDeePC [8]Online DeePC - Replacement [13]Online DeePC - Adding [19]Online Reduced-Order DeePC (Ours)405410415420-1.2-1-0.85.5 Chapter Summary

In this chapter, we proposed an online DeePC framework that incorporates real-time data

updates into the Hankel matrix, leveraging the minimum non-zero singular value to selectively

integrate informative signals. This approach effectively captures the dynamic nature of the system,

ensuring improved control performance. Furthermore, we introduced a numerical SVD technique

to mitigate the computational complexity associated with data integration. Simulation results

validated the efficacy of the proposed online reduced-order DeePC framework, demonstrating its

potential for achieving optimal control in evolving system environments.

82

CHAPTER 6

CONCLUSION

In this thesis, we addressed several existing challenges about the nonlinear optimal control,

including parametric model requirement, model uncertainties, and computational cost. We ap-

proached this topic with three goals in mind: (i) Improving the neighboring extremal (NE) optimal

control to handle the model uncertainties; (ii) Designing a data-driven neighboring extremal optimal

control; and (iii) Developing an adaptive data-enabled predictive control (DeePC).

6.1 Contributions

In Chapter 3, we introduced an extended NE (ENE) to handle the model uncertainties for the

NE, where effectively reduces the computational cost of the model-based nonlinear optimal control.

The developed ENE was based on the second-order variation of the original optimization problem,

which led to a set of Riccati-like backward recursive equations. The ENE adapted a nominal

trajectory to the state and preview perturbations, and a multi-segment strategy was employed

to guarantee closed-loop performance and constraint satisfaction for the large perturbations. In

Chapter 4, we introduced a data-enabled neighboring extremal (DeeNE) to remove the parametric

model requirement for the NE, where is very useful for high computational cost of DeePC. We also

developed a scheme to handle nominal non-optimal solutions so that we can use the DeeNE solution

as the nominal solution during the control process. The developed DeeNE was based on the second-

order variation of the original DeePC problem such that the computational load of the DeeNE grows

linearly for the optimization horizon. In Chapter 5, we introduced an adaptive DeePC framework

for time-varying systems, enabling the algorithm to update the Hankel matrix online by adding

real-time informative signals. By exploiting the minimum non-zero singular value of the Hankel

matrix, the developed online DeePC selectively integrated informative data and effectively captured

evolving system dynamics. Additionally, a numerical singular value decomposition technique was

introduced to reduce the computational complexity for updating a reduced-order Hankel matrix.

83

6.2 Future Works

As the future works, two interesting research directions are Nonlinear Fundamental Lemma

and Data-Enabled Control Barrier Function. Below are the detailed objectives of each topic.

6.2.1 Nonlinear Fundamental Lemma

The Fundamental Lemma accurately learns the system’s behaviour such that the subspace

of the I/O trajectories of a linear time invariant (LTI) system can be obtained from the column

span of a data Hankel matrix. However, this lemma is not perfect to learn the behaviors of the

nonlinear systems. Therefore, the DeePC is robustifies the Fundamental Lemma through a suitable

regularization to ensure good performance for the nonlinear systems. However, deriving a rigorous

mathematical theory to develop a Nonlinear Fundamental Lemma would highly show a better

performance for the nonlinear optimal control compared to regularization.

6.2.2 Data-Enabled Control Barrier Function

To guarantee system safety, control barrier function (CBF) has recently emerged as a promising

framework to efficiently handle system constraints [68]. The CBF implies forward invariance of a

safety set based on system dynamics, and robust CBF and adaptive CBF maintains system safety

in the presence of system uncertainties such as unknown disturbance, model mismatch, and state

estimation error [69, 70, 71, 72, 73]. A data-enabled CBF can be proposed for the nonlinear optimal

control to guarantee system safety using raw input/output data.

84

CHAPTER 7

PUBLICATIONS

The research conducted during the author’s time as a PhD student has involved close collab-

oration with a number of colleagues, and this section lists all the articles submitted or published

during this time.

Journal Publications:

• A. Vahidi-Moghaddam, K. Zhang, X. Yin, V. Srivastava, Z. Li, “Online Reduced-Order

Data-Enabled Predictive Control,” arXiv preprint arXiv:2407.16066, 2024.

• A. Vahidi-Moghaddam, K. Zhu, K. Zhang, and Z. Li, “Computationally Efficient Data-

Enabled Predictive Control for Arm Robots,” 2024.

• A. Vahidi-Moghaddam, K. Chen, K. Zhang, Z. Li, Y. Wang, and K. Wu.

"A Unified

Framework for Online Data-Driven Predictive Control with Robust Safety Guarantees,"

IEEE Transactions on Automation Science and Engineering, 2024.

• A. Vahidi-Moghaddam, K. Zhang, Z. Li, X. Yin, Z. Song, and Y. Wang, "Extended Neigh-

boring Extremal Optimal Control with State and Preview Perturbations," IEEE Transactions

on Automation Science and Engineering, 2023.

• Z. Li, A. Vahidi-Moghaddam, H. Modares, X. Wang, and J. Sun, "Finite-Time Distur-

bance Rejection for Nonlinear Systems using an Adaptive Disturbance Observer based on

Experience-Replay," International Journal of Adaptive Control and Signal Processing, vol.

36, no. 8, pp. 2065-2082, 2022.

• A. Vahidi-Moghaddam, M. Mazouchi, and H. Modares, "Memory-Augmented System Iden-

tification with Finite-Time Convergence," IEEE Control Systems Letters, vol. 5, no. 2, pp.

571-576, 2020.

85

Conference Publications:

• A. Vahidi-Moghaddam, K. Zhang, Z. Li, and Y. Wang, "Data-Enabled Neighboring Extremal

Optimal Control: A Computationally Efficient DeePC," in 2023 IEEE 62nd Conference on

Decision and Control (CDC), IEEE, 2023.

• A. Vahidi-Moghaddam, Z. Li, N. Li, K. Zhang, and Y. Wang, "Event-Triggered Cloud-based

Nonlinear Model Predictive Control with Neighboring Extremal Adaptations," in 2022 IEEE

61st Conference on Decision and Control (CDC), pp. 3724-3731. IEEE, 2022.

• A. Vahidi-Moghaddam, K. Chen, Z. Li, Y. Wang, and K. Wu, “Data-Driven Safe Predictive

Control using Spatial Temporal Filter-based Function Approximators,” in 2022 American

Control Conference (ACC), pp. 2803–2809, IEEE, 2022.

• A. Vahidi-Moghaddam, M. Mazouchi, and H. Modares, "Learning Dynamics System Mod-

els with Prescribed-Performance Guarantees using Experience-Replay," In 2021 American

Control Conference (ACC), pp. 1941-1946, IEEE, 2021.

86

BIBLIOGRAPHY

[1] David A Copp, Kyriakos G Vamvoudakis, and João P Hespanha. “Distributed output-
feedback model predictive control for multi-agent consensus”. In: Systems & Control Letters
127 (2019), pp. 52–59.

[2] Robert AE Zidek, Ilya V Kolmanovsky, and Alberto Bemporad. “Model predictive control
for drift counteraction of stochastic constrained linear systems”. In: Automatica 123 (2021),
p. 109304.

[3] Angelo Bratta et al. “Optimization-Based Reference Generator for Nonlinear Model Predic-

tive Control of Legged Robots”. In: Robotics 12.1 (2023), p. 6.

[4]

Jeremy Lore et al. “Model predictive control of boundary plasmas using reduced models de-
rived from SOLPS-ITER”. In: APS Division of Plasma Physics Meeting Abstracts. Vol. 2021.
2021, NM09–002.

[5] Kaixiang Zhang, Yang Zheng, and Zhaojian Li. “Dimension Reduction for Efficient Data-

Enabled Predictive Control”. In: arXiv preprint arXiv:2211.03697 (2022).

[6] Dinesh Krishnamoorthy, Ali Mesbah, and Joel A Paulson. “An adaptive correction scheme
for offset-free asymptotic performance in deep learning-based economic MPC”. In: IFAC-
PapersOnLine 54.3 (2021), pp. 584–589.

[7] Yajie Bao et al. “Learning-based Adaptive-Scenario-Tree Model Predictive Control with
Probabilistic Safety Guarantees Using Bayesian Neural Networks”. In: 2022 American Con-
trol Conference (ACC). IEEE. 2022, pp. 3260–3265.

[8] Elena Arcari, Andrea Carron, and Melanie N Zeilinger. “Meta learning MPC using finite-
dimensional Gaussian process approximations”. In: arXiv preprint arXiv:2008.05984 (2020).

[9] Elena Arcari et al. “Bayesian multi-task learning mpc for robotic mobile manipulation”. In:

IEEE Robotics and Automation Letters (2023).

[10] Amin Vahidi-Moghaddam et al. “Data-Driven Safe Predictive Control Using Spatial Tem-
poral Filter-based Function Approximators”. In: 2022 American Control Conference (ACC)
(2022).

[11] Amin Vahidi-Moghaddam et al. “A Unified Framework for Online Data-Driven Predictive
Control with Robust Safety Guarantees”. In: arXiv preprint arXiv:2306.17270 (2023).

[12] Robert L Grossman. “The case for cloud computing”. In: IT professional 11.2 (2009), pp. 23–

27.

[13] Saad Mubeen et al. “Delay mitigation in offloaded cloud controllers in industrial IoT”. In:

IEEE Access 5 (2017), pp. 4418–4430.

87

[14] Reza Ghaemi, Jing Sun, and Ilya Kolmanovsky. “Computationally efficient model predictive
control with explicit disturbance mitigation and constraint enforcement”. In: Proceedings of
the 45th IEEE Conference on Decision and Control. IEEE. 2006, pp. 4842–4847.

[15] Reza Ghaemi, Jing Sun, and Ilya Kolmanovsky. “Neighboring extremal solution for discrete-
time optimal control problems with state inequality constraints”. In: 2008 American Control
Conference. IEEE. 2008, pp. 3823–3828.

[16] Matt Wytock, Nicholas Moehle, and Stephen Boyd. “Dynamic energy management with
scenario-based robust MPC”. In: 2017 American Control Conference (ACC). IEEE. 2017,
pp. 2042–2047.

[17] Brett T Lopez, Jean-Jacques E Slotine, and Jonathan P How. “Dynamic tube MPC for
nonlinear systems”. In: 2019 American Control Conference (ACC). IEEE. 2019, pp. 1655–
1662.

[18] Carmen Amo Alonso et al. “Robust Distributed and Localized Model Predictive Control”.

In: arXiv preprint arXiv:2103.14171 (2021).

[19] Mohammad R Hajidavalloo et al. “MPC-Based Vibration Control and Energy Harvesting
using Stochastic Linearization for a New Energy Harvesting Shock Absorber”. In: 2021
IEEE Conference on Control Technology and Applications (CCTA). IEEE. 2021, pp. 38–43.

[20] Francoise Lamnabhi-Lagarrigue et al. “Systems & control for the future of humanity, research
agenda: Current and future roles, impact and grand challenges”. In: Annual Reviews in
Control 43 (2017), pp. 1–64.

[21] Zhong-Sheng Hou and Zhuo Wang. “From model-based control to data-driven control:
Survey, classification and perspective”. In: Information Sciences 235 (2013), pp. 3–35.

[22] Valentina Breschi, Andrea Sassella, and Simone Formentin. “On the design of regularized
explicit predictive controllers from input-output data”. In: IEEE Transactions on Automatic
Control (2022).

[23]

Julian Berberich et al. “Data-driven model predictive control with stability and robustness
guarantees”. In: IEEE Transactions on Automatic Control 66.4 (2020), pp. 1702–1717.

[24] Claudio De Persis and Pietro Tesi. “Formulas for data-driven control: Stabilization, opti-
mality, and robustness”. In: IEEE Transactions on Automatic Control 65.3 (2019), pp. 909–
924.

[25]

[26]

Jan C Willems and Jan W Polderman. Introduction to mathematical systems theory: a
behavioral approach. Vol. 26. Springer Science & Business Media, 1997.

Jan C Willems et al. “A note on persistency of excitation”. In: Systems & Control Letters
54.4 (2005), pp. 325–329.

88

[27]

Jeremy Coulson, John Lygeros, and Florian Dörfler. “Data-enabled predictive control: In the
shallows of the DeePC”. In: 2019 18th European Control Conference (ECC). IEEE. 2019,
pp. 307–312.

[28] Paolo Gherardo Carlet et al. “Data-driven continuous-set predictive current control for syn-
chronous motor drives”. In: IEEE Transactions on Power Electronics 37.6 (2022), pp. 6637–
6646.

[29] Ezzat Elokda et al. “Data-enabled predictive control for quadcopters”. In: International

Journal of Robust and Nonlinear Control 31.18 (2021), pp. 8916–8936.

[30] Linbin Huang et al. “Data-enabled predictive control for grid-connected power converters”.
In: 2019 IEEE 58th Conference on Decision and Control (CDC). IEEE. 2019, pp. 8130–
8135.

[31] Reza Ghaemi, Soryeok Oh, and Jing Sun. “Path following of a model ship using model
predictive control with experimental verification”. In: Proceedings of the 2010 American
control conference. IEEE. 2010, pp. 5236–5241.

[32] Hyeongjun Park et al. “Real-time model predictive control for shipboard power management
using the IPA-SQP approach”. In: IEEE Transactions on Control Systems Technology 23.6
(2015), pp. 2129–2143.

[33] Yanhui Xie et al. “Model predictive control for a full bridge DC/DC converter”. In: IEEE

Transactions on Control Systems Technology 20.1 (2011), pp. 164–172.

[34] Hyeongjun Park, Ilya Kolmanovsky, and Jing Sun. “Model predictive control of spacecraft
relative motion maneuvers using the IPA-SQP approach”. In: Dynamic Systems and Control
Conference. Vol. 56123. American Society of Mechanical Engineers. 2013, V001T02A001.

[35] Amirhossein Ahmadi et al. “Deep Federated Learning-Based Privacy-Preserving Wind

Power Forecasting”. In: IEEE Access 11 (2022), pp. 39521–39530.

[36] Linbin Huang et al. “Robust data-enabled predictive control: Tractable formulations and per-
formance guarantees”. In: IEEE Transactions on Automatic Control 68.5 (2023), pp. 3163–
3170.

[37] Amin Vahidi-Moghaddam et al. “Extended Neighboring Extremal Optimal Control with
State and Preview Perturbations”. In: IEEE Transactions on Automation Science and Engi-
neering (2023).

[38] Amin Vahidi-Moghaddam et al. “Event-Triggered Cloud-based Nonlinear Model Predic-
tive Control with Neighboring Extremal Adaptations”. In: 2022 IEEE 61st Conference on
Decision and Control (CDC). IEEE. 2022, pp. 3724–3731.

89

[39] Mohammad R. Hajidavalloo et al. “Simultaneous Suspension Control and Energy Harvesting
Through Novel Design and Control of a New Nonlinear Energy Harvesting Shock Absorber”.
In: IEEE Transactions on Vehicular Technology 71.6 (2022), pp. 6073–6087.

[40] Mohammad Reza Amini et al. “Cabin and battery thermal management of connected and
automated HEVs for improved energy efficiency using hierarchical model predictive control”.
In: IEEE Transactions on Control Systems Technology 28.5 (2019), pp. 1711–1726.

[41]

Jason Laks et al. “Model predictive control using preview measurements from lidar”. In:
49th AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace
Exposition. 2011, p. 813.

[42] Reza Ghaemi, Masoud Abbaszadeh, and Pierino G Bonanni. “Optimal flexibility control of
large-scale distributed heterogeneous loads in the power grid”. In: IEEE Transactions on
Control of Network Systems 6.3 (2019), pp. 1256–1268.

[43] Shiva Bagherzadeh, Hossein Karimpour, and Mehdi Keshmiri. “Neighboring extremal non-
linear model predictive control of a rigid body on SO (3)”. In: Robotica (2023), pp. 1–
22.

[44] Rohit Gupta, Anthony M Bloch, and Ilya V Kolmanovsky. “Combined homotopy and neigh-
boring extremal optimal control”. In: Optimal Control Applications and Methods 38.3 (2017),
pp. 459–469.

[45] Anthony M Bloch, Rohit Gupta, and Ilya V Kolmanovsky. “Neighboring extremal optimal
control for mechanical systems on Riemannian manifolds”. In: J. Geom. Mech 8.3 (2016),
pp. 257–272.

[46] Hyeongjun Park, Jing Sun, and Ilya Kolmanovsky. “A tutorial overview of IPA-SQP ap-
proach for optimization of constrained nonlinear systems”. In: Proceeding of the 11th World
Congress on Intelligent Control and Automation. IEEE. 2014, pp. 1735–1740.

[47] Amin Vahidi-Moghaddam et al. “Computationally Efficient Data-Enabled Predictive Control

for Arm Robots”. In: arXiv preprint (2024).

[48] Amin Vahidi-Moghaddam et al. “Data-Enabled Neighboring Extremal Optimal Control: A
Computationally Efficient DeePC”. In: 2023 IEEE 62st Conference on Decision and Control
(CDC). IEEE. 2023.

[49] Florian Dörfler, Jeremy Coulson, and Ivan Markovsky. “Bridging direct and indirect data-
driven control formulations via regularizations and relaxations”. In: IEEE Transactions on
Automatic Control 68.2 (2022), pp. 883–897.

[50] Valentina Breschi, Alessandro Chiuso, and Simone Formentin. “Data-driven predictive con-

trol in a stochastic setting: A unified framework”. In: Automatica 152 (2023), p. 110961.

90

[51] Wouter Favoreel, Bart De Moor, and Michel Gevers. “SPC: Subspace predictive control”.

In: IFAC Proceedings Volumes 32.2 (1999), pp. 4004–4009.

[52] Francesco Toso et al. “On-line continuous control set MPC for PMSM drives current loops
at high sampling rate using qpOASES”. In: 2019 IEEE Energy Conversion Congress and
Exposition (ECCE). IEEE. 2019, pp. 6615–6620.

[53] Lars Grüne et al. “Analysis of unconstrained nonlinear MPC schemes with time varying
control horizon”. In: SIAM Journal on Control and Optimization 48.8 (2010), pp. 4938–
4962.

[54] Amin Vahidi-Moghaddam et al. “Online Reduced-Order Data-Enabled Predictive Control”.

In: arXiv preprint arXiv:2407.16066 (2024).

[55]

[56]

[57]

Jeremy Coulson et al. “A quantitative notion of persistency of excitation and the robust
fundamental lemma”. In: IEEE Control Systems Letters 7 (2022), pp. 1243–1248.

Johannes Teutsch et al. “An Online Adaptation Strategy for Direct Data-driven Control”. In:
arXiv preprint arXiv:2304.03386 (2023).

Julian Berberich et al. “Data-driven model predictive control: closed-loop guarantees and
experimental results”. In: at-Automatisierungstechnik 69.7 (2021), pp. 608–618.

[58] Stefanos Baros et al. “Online data-enabled predictive control”. In: Automatica 138 (2022),

p. 109926.

[59]

[60]

[61]

Julian Berberich et al. “Linear tracking MPC for nonlinear systems—Part II: The data-driven
case”. In: IEEE Transactions on Automatic Control 67.9 (2022), pp. 4406–4421.

Ivan Markovsky and Florian Dörfler. “Identifiability in the behavioral setting”. In: IEEE
Transactions on Automatic Control 68.3 (2022), pp. 1667–1677.

Jicheng Shi and Colin N Jones. “Efficient Recursive Data-enabled Predictive Control: an
Application to Consistent Predictors”. In: arXiv preprint arXiv:2309.13755 (2023).

[62] Matthew Brand. “Fast low-rank modifications of the thin singular value decomposition”. In:

Linear algebra and its applications 415.1 (2006), pp. 20–30.

[63] Ming Gu and Stanley C Eisenstat. A stable and fast algorithm for updating the singular value

decomposition. 1993.

[64]

[65]

James R Bunch and Christopher P Nielsen. “Updating the singular value decomposition”.
In: Numerische Mathematik 31.2 (1978), pp. 111–129.

Ivan Markovsky and Florian Dörfler. “Behavioral systems theory in data-driven analysis,
signal processing, and control”. In: Annual Reviews in Control 52 (2021), pp. 42–64.

91

[66] Kaixiang Zhang et al. “Dimension Reduction for Efficient Data-Enabled Predictive Control”.
In: IEEE Control Systems Letters 7 (2023), pp. 3277–3282. doi: 10.1109/LCSYS.2023.
3322965.

[67] Ricardo Bencatel et al. “Reference governor strategies for vehicle rollover avoidance”. In:

IEEE Transactions on Control Systems Technology 26.6 (2017), pp. 1954–1969.

[68] Aaron D Ames et al. “Control barrier functions: Theory and applications”. In: 2019 18th

European control conference (ECC). IEEE. 2019, pp. 3420–3431.

[69] Kunal Garg and Dimitra Panagou. “Robust control barrier and control Lyapunov functions
with fixed-time convergence guarantees”. In: 2021 American Control Conference (ACC).
IEEE. 2021, pp. 2292–2297.

[70] Mitchell Black, Ehsan Arabi, and Dimitra Panagou. “A fixed-time stable adaptation law for
safety-critical control under parametric uncertainty”. In: 2021 European Control Conference
(ECC). IEEE. 2021, pp. 1328–1333.

[71] Axton Isaly et al. “Adaptive safety with multiple barrier functions using integral concurrent
learning”. In: 2021 American Control Conference (ACC). IEEE. 2021, pp. 3719–3724.

[72] Brett T Lopez, Jean-Jacques E Slotine, and Jonathan P How. “Robust Adaptive Control
Barrier Functions: An Adaptive & Data-Driven Approach to Safety (Extended Version)”. In:
arXiv preprint arXiv:2003.10028 (2020).

[73] Andrew J Taylor and Aaron D Ames. “Adaptive safety with control barrier functions”. In:

2020 American Control Conference (ACC). IEEE. 2020, pp. 1399–1405.

92