VERIFICATION OF PROBABILISTIC HYPERPROPERTIES
ON MARKOV MODELS

By

Oyendrila Dobe

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

Computer Science - Doctor of Philosophy

2024

ABSTRACT

Formal verification encapsulates the process of ensuring the correctness of systems with

respect to user-specified requirements, expressed as formal properties. This dissertation

explores the different aspects of the verification of systems involving uncertainty, specified

at an abstract level as Markov models, against system-level specifications, expressed as

hyperproperties.

In this context, we refer to models as frameworks used to capture the behaviour of systems

while abstracting away the exact details of its inner workings. Such system models are rarely

deterministic e.g., we may choose to simplify or remove elements that are irrelevant to the

verification, we might not be aware of exact details of certain components in the system,

or we might not be able to observe every aspect of the system. To this end, we chose to

represent systems as Markov models due to their flexibility in representing uncertainty (in

terms of nondeterminism, randomization, and partial observability), and its simplicity in

using the current state to determine the future evolution of the system.

The current ubiquitous nature of software demands verification of not only its isolated

behaviours, but also ones relating to security, privacy, robustness, and efficiency. These

additional requirements are more challenging to verify as they require simultaneous rea-

soning across multiple system behaviours. Such requirements can be formally specified as

hyperproperties. Prominent examples include noninterference of secret inputs on publicly

observable outputs, observational determinism of public outputs, optimal path planning in

robotics, individual fairness in models, side-channel timing attacks, conformance of different

system versions, etc.

Given this combination of model and properties, we explore the following:

• We explored the parameter synthesis problem and developed an SMT-based solution

for synthesizing values for unknown or missing parameters of the system, under the

assumption of satisfaction of a given specification.

• We generalize the probabilistic hyperlogic HyperPCTL by extending it to formally specify

requirements involving nondeterministic choices and rewards in models. We discuss the

expressive power of this extended logic, and how it makes the model checking problem

for this logic undecidable, in general.

• To provide tractable solutions to the model checking problem, we focused on a decidable

fragment of HyperPCTL with a single existential quantifier and proposed an SMT-

encoding-based model checking algorithm.

• We identify two different fragments of HyperPCTL based on their ability to express the

reachability requirement in most of our applications. For these two different fragments,

we provide a randomized convex addition of a scheduler and a distribution-based ap-

proach respectively.

• We have developed prototypical implementations for each of the proposed algorithms.

Specifically, to model check the nondeterministic and reward property extensions, we

have built a tool HyperProb, which implements our SMT-based algorithm to sym-

bolically model check restricted fragments of HyperPCTL.

• To tackle the challenges of scalability, we explored statistical model checking as a pos-

sible solution to model check a fragment of HyperPLTL and extended the tool PLASMA

to evaluate our theory.

I dedicate this dissertation to my parents, Juthika and Swapan Dobe,
for always letting me pursue my (often eccentric) ideas
&
to my younger self
for choosing to take on this life-changing journey.

iv

ACKNOWLEDGEMENTS

As I run the final lap of this marathon of obtaining this degree, I cannot help but feel

grateful for all the amazing people, memories, and lessons I have encountered on my way.

This journey has humbled me and helped me understand myself better beyond any limit I

had ever imagined. For everything I have accomplished, I have done so by standing on the

shoulders of giants who have made this path a little easier for me at every step. My only

hope is to be able to pay it forward in life. I want to use this space to give credit to a few

notable people who walked along with me on this journey and helped me pace my way to

the end line.

Adhering to traditions, I would first like to extend my gratitude towards my advisor and

mentor, Borzoo Bonakdarpour. I am thankful to him for taking me under his umbrella at a

confusing point in my graduate studies and introducing me to the world of hyperproperties.

His guidance all along the way has helped me grow both as a researcher and a person. Despite

our differences and several tough conversations, his unwavering support has, time and again,

kept me on track to finish this marathon. At its core, this advisor-advisee relationship

has been fuelled by cooked-to-perfection barbecues, bar-night discussions, late-night paper

submissions (delayed by my silly mistakes), and a shared trait of stubbornness. It is my

honour to have a life-long conflict of interest with him for future paper submissions.

I am thankful to my committee members Prof. Betty Cheng, Prof. Sandeep Kulkarni,

and Prof. Yiming Deng, for their time and feedback on my research, and for accommodating

my requests during my comprehensive exam and final defense. I am especially thankful to

Prof. Cheng for her detailed feedback on my dissertation. I would also like to extend my

thanks to all my collaborators, especially Prof. Erika Abraham and Prof. Ezio Bartocci.

It has been a great experience working alongside them and learning from them during our

collaborations. I have been fortunate to have crossed paths with Stefan Schuup and Lina

Gerlach, collaborators that I now call friends.

Taking a step back, I want to thank Prof. Amiya Halder, my beloved HoD ma’am,

v

B.N. Ray uncle, and Chatterjee uncle for encouraging me to pursue higher education.

I

am sincerely grateful to Sinha jethu for all his help and mentorship during the application

process.

Next in my gratitude queue would be my fellow lab-mates of TART: Ritam Ganguly, Anik

Momtaz, Tzu-Han Hsu, Eshita Zaman, and Arshia Rafieioskouei. Thank you for making the

lab a fun workplace. It was a pleasure discussing formal methods, politics, food, frustrations,

and prank ideas with you. I have learnt and soaked different attributes of your work ethics

and cherish our time spent together.

Among my fellows runners, running their own marathons, I thank- Debrudra Mitra and

Arna Ganguly for being there at one of my lowest points in this journey; I will forever

be indebted to you for lending me your shoulders, Soham Vanage and Ritam Ganguly for

being the initiators of crazy ideas that Debrudra da and I jumped in to pursue without

complaints, Ali Saffary for always lending me a listening pair of ears, David Ackley for

bringing weird humour to our conversations, Akash Dutta for being my partner-in-crime,

Umesh Chinalachi for his emotionless honest advice, Tiana Zefreen and Anik Momtaz for

accepting my weirdness, Saurabh Mishra for being the life of our parties, Arya Gupta for our

stimulating discussions, and Emma, Sabrina for making me feel at home in the final year. I

also thank Nishan Chatterjee and Vyomdhaval Bhatt for being my comfort places.

I am grateful for the existence of the MSU Badminton Club and its members, especially

Jayci Simon; the club helped me escape the daily grind, find my strength, and express myself

authentically. As part of the Spartan Girls Who Code community, I have learned so much

about mentoring, leadership, and preparing the next generation of steminists. I am thankful

to Teresa Vandersloot and Prof. Laura Dillon for accepting me as a part of this amazing

community run by strong fellow Spartans. I hope to continue to support this community

going forward.

A very important part of my graduate school experience was my internships which con-

vinced me of my future career choice. I am indebted to Nafi Diallo for believing in me and

vi

selecting me for my first internship opportunity. I enjoyed working under brilliant managers

like Temesghen Kahsai and Aaron Tomb who gave me interesting problems, the freedom to

pursue them independently, and their guidance. I thank my mentors Byron Cook and Rus-

tan Leino for inspiring me by sharing their passions and visions. And finally, I am extremely

grateful to have found a mentor in Nadia Labai who made my experiences enjoyable.

I am grateful to the CSE department at MSU for providing the space and financial support

for my research. I want to thank our non-teaching staff, especially, Vincent Mattison and

Brenda Hodge for making most of my department-related issues vanish in no time.

My research has been supported by generous funds from the National Science Foundation

and I am grateful for their continued support towards pushing the boundaries of science.

My final words are for the most important people in my life, my parents. Ma and Bapi

thank you for being the source of my strength, inspiration, and clarity. It has been a long

journey and you have sacrificed a lot for it. I hope to make it all worth it. Thank you for

being patient with me.

vii

TABLE OF CONTENTS

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1 Modelling of Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Model Checking for Hyperproperties
. . . . . . . . . . . . . . . . . . . . . .
1.4 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5 Thesis statement
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.7 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1 Discrete-Time Markov models . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Discrete-Time Markov Models with Rewards . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Probabilistic Hyperproperties

Chapter 3 Parameter Synthesis for Probabilistic Hyperproperties . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1
3.2 Parameter Synthesis Algorithm for ReachHyperPCTL . . . . . . . . . . . . .
3.3 Case Studies and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 4 Probabilistic Hyperproperties with Nondeterminism . . . . . .
4.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 HyperPCTL for MDPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 The Expressiveness Power of HyperPCTL . . . . . . . . . . . . . . . . . . .
4.4 Applications of HyperPCTL on MDPs
. . . . . . . . . . . . . . . . . . . . .
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 5 Decidable Fragment of Probabilistic Hyperproperties

with Nondeterminism . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.1
5.2 Proof of decidability of the restricted fragment . . . . . . . . . . . . . . . . .
5.3 Model Checking for Non-probabilistic Memoryless Schedulers . . . . . . . . .
5.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 6 Probabilistic Hyperproperties with Rewards . . . . . . . . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1
6.2 HyperPCTL with Rewards . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.3 Applications of HyperPCTL with Rewards . . . . . . . . . . . . . . . . . . .
6.4 Model Checking Algorithm for Reward Operators . . . . . . . . . . . . . . .

x

xi

1
2
4
7
9
13
13
15

18
18
21
25

29
29
35
41
50

51
51
54
57
62
64

66
66
66
70
78
80

82
82
85
91
94

viii

97
6.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Chapter 7 HyperProb: A Model Checker for Probabilistic

Hyperproperties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
7.1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Input to the Tool
7.2
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.3 Tool Structure and Usage
7.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Chapter 8 Efficient Probabilistic Model Checking for Relational

Reachability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
8.1 Preliminaries
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
8.2 Model checking algorithm for (2σ2s) fragment . . . . . . . . . . . . . . . . . 116
8.3 Distribution-Based Approach for the (1σ1s) Fragment . . . . . . . . . . . . . 120
8.4 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
8.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Chapter 9 Lightweight Verification of Hyperproperties . . . . . . . . . . . 134
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
9.1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
9.2 Preliminaries
9.3 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
9.4 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
9.5 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
9.6 Experimentation/Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Chapter 10 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
10.1 Discrete and Continuous Time Logics . . . . . . . . . . . . . . . . . . . . . . 160
10.2 Hyperproperties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
10.3 Parameter Synthesis
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
. . . . . . . . . . . . . . . 165
10.4 Model Checking of Probabilistic Hyperproperties
10.5 Statistical Model Checking . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Chapter 11 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . 168
11.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
11.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

ix

LIST OF TABLES

Table 1.1: Details of publications.

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

15

Table 3.1: Experimental results for thread scheduling. . . . . . . . . . . . . . . . . .

46

Table 5.1: Experimental results. TA: Timing attack. PW: Password leakage. TS:
. . . . . . . . . . . .

Thread scheduling. PC: Probabilistic conformance.

Table 6.1: Semantics of φ = P(φ1Uφ2), partly depending on p = (cid:80)

M,σ,s′ ∈ [0, 1] ∪ {⊥}. Here,
φ
(cid:75)
(cid:74)

.
(cid:75)

is short for

(cid:74)
Table 6.2: Experimental results. VR: Verification result. TA: Timing attack. PC:
Probabilistic conformance. RO: Robotics example. HS: Herman’s algo-
rithm. IJ: Israeli-Jaflon’s algorithm. ✓: the result is true. ×: the result
is false. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

s′∈Sσ P (s, s′) ·
M,σ,s. . . . . . . . . . . .
(cid:75)

.
(cid:74)

80

89

99

Table 7.1: Experimental results and comparison. TA: Timing attack. PW: Pass-
word leakage. TS: Thread scheduling. PC: Probabilistic conformance.
TO: Timeout. N: Prototype presented in [ ´ABBD20b]. O: HyperProb,.
SE: SMT encoding. SS: SMT solving. #op: Formula size (number of
operators). #st: Number of states. #tr: Number of transitions.

. . . . . 112

Table 8.1: Parameters used for scheduling three tasks τ0, τ1, τ2. . . . . . . . . . . . . 129

Table 8.2: Experimental results for scheduler generation using Alg. 17. #st: number
of states in the model, #tran: number of transitions in the model, #act:
number of actions in the model.

. . . . . . . . . . . . . . . . . . . . . . . 131

Table 8.3: Experimental results for scheduler generation using Alg. 18. #st: number
of states in the model, #tran: number of transitions in the model, #act:
number of actions in the model.

. . . . . . . . . . . . . . . . . . . . . . . 132

Table 9.1: Model details of grey-box case studies.

. . . . . . . . . . . . . . . . . . . 155

Table 9.2: Data from experimentation. #sch: number of scheduler-tuples sampled,

#tr: number of trace tuples sampled per scheduler tuple,
k: length of traces sampled. α = β = 0.01.

. . . . . . . . . . . . . . . . . 156

x

LIST OF FIGURES

Figure 1.1: Models to represent a roll of a dice.

. . . . . . . . . . . . . . . . . . . . .

Figure 1.2: Traces considered in different types of logic.

. . . . . . . . . . . . . . . .

Figure 1.3: Implementation of dice by coin tosses using Knuth-Yao protocol. . . . . .

Figure 1.4: Overall idea of model checking.

. . . . . . . . . . . . . . . . . . . . . . .

3

5

7

8

Figure 2.1: Example DTMC.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19

Figure 2.2: MDP showing the actions and probabilistic distributions.

. . . . . . . . .

20

Figure 2.3: Example Markov models with rewards.

. . . . . . . . . . . . . . . . . . .

22

Figure 3.1: The randomized response protocol.

. . . . . . . . . . . . . . . . . . . . .

30

Figure 3.2: Semantics example.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

Figure 3.3: A PDTMC (left) and the result of eliminating s1 in [DJJ+15] (mid) and
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

in Alg. 4 (right).

39

Figure 3.4: Synthesized probability distribution for the randomized response protocol.

43

Figure 3.5: Parametric die-coin using Knuth-Yao protocol. . . . . . . . . . . . . . . .

45

Figure 3.6: Parametric DTMC for the probabilistic noninterference example program
. . . . . . . . . . . . . . . . . . . . . . . .

with two threads th 1 and th 2.

46

Figure 3.7: Synthesized probability distribution for a randomized scheduler.

. . . . .

47

Figure 3.8: A parametric DTMC model for the dining cryptographers protocol with
. . . . . . . . . . . . . . . .

three cryptographers and three biased coins.

49

Figure 4.1: Example DTMC.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

52

Figure 4.2: MDPs that require different types of schedulers.

. . . . . . . . . . . . . .

52

Figure 4.3: Modular exponentiation.

. . . . . . . . . . . . . . . . . . . . . . . . . . .

62

Figure 4.4: String comparison.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

63

Figure 5.1: Example of mapping SAT to HyperPCTL model checking.

. . . . . . . . .

68

xi

Figure 6.1: Example Markov models. . . . . . . . . . . . . . . . . . . . . . . . . . . .

83

Figure 6.2: HyperPCTL syntax.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

Figure 6.3: Existing Semantics rules for HyperPCTL. . . . . . . . . . . . . . . . . . . .

87

Figure 6.4: Semantics for reward operators in HyperPCTL.

. . . . . . . . . . . . . . .

88

Figure 6.5: Modular exponentiation in RSA. . . . . . . . . . . . . . . . . . . . . . . .

91

Figure 6.6: The maze on the left satisfies φtarget, while on the right it violates φtarget.

93

Figure 6.7: Herman’s algorithm [Her90] for process i and example for three processes.

94

Figure 7.1: PRISM model generating the MDP in Fig.7.2 . . . . . . . . . . . . . . . . 103

Figure 7.2: MDP of the PRISM program in Fig 7.1.

. . . . . . . . . . . . . . . . . . . 103

Figure 7.3: Grammar defining HyperPCTL inputs to HyperProb, where the NUM token

is a decimal number and NAME is a non-empty string.

. . . . . . . . . . 104

Figure 7.4: Overview of the docker container with the tool and its dependencies. . . . 106

Figure 7.5: Dataflow inside the tool.

. . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Figure 7.6: Modular exponentiation.

. . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Figure 7.7: String comparison.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Figure 8.1: (Partial) MDP modelling information leakage via side-channel.

. . . . . . 117

Figure 8.2: DTMCs induced by different schedulers on MDP in Fig. 8.1.

. . . . . . . 119

Figure 8.3: Modular exponentiation.

. . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Figure 8.4: String comparison.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Figure 9.1: Two robots attempting to reach the same goal. . . . . . . . . . . . . . . . 149

Figure 9.2: Grid divided into regions to ensure opacity. . . . . . . . . . . . . . . . . . 150

Figure 9.3: DC with n = 4 (Pr ≥ 0.1 ± 0.01).

. . . . . . . . . . . . . . . . . . . . . . 157

Figure 9.4: Plots showing the change in ratio based on sampling across schedulers.

. 158

xii

Chapter 1

Introduction

A system is described as a group of interrelated components that work together to

accomplish a single goal. Everything around us can be considered a system - mobile phones,

traffic management, robotic arms, spacecraft, stock prices, patient flow in hospitals, etc.

When building these systems our aim should be to ensure their robustness, security, and

conformance to specific expectations. This problem is comparatively simpler if the systems

are deterministic. In reality, most systems are rarely deterministic. There is always a degree

of uncertainty involved in these systems contributed by noise disturbances, random time

delays, stochastic inputs, human errors, unknown inputs from environments, randomness in

algorithms to break deadlocks, assigning probability in output distributions in ML models,

etc. Hence, we are interested in reasoning about systems under uncertainty. In the rest of

this material, we specifically use the term randomness for cases where we can quantify the

uncertainty of the system and we use the term uncertainty to only describe cases where the

uncertainty is not quantifiable and has to be represented nondeterministically.

Formal verification is described as the process of using mathematical reasoning to prove

or disprove the correctness of a system against a set of specifications. For our purposes,

we use Markov models to represent the systems we study. When verifying systems, we

need to consider multiple observations of the system simultaneously to ensure its correctness

and security. Such specifications are referred to as hyperproperties. The lack of organized

theory and verification tools for handling such properties has motivated the range of research

1

described here.

1.1 Modelling of Systems

Models — A system exhibits observable behaviours due to their interactions between its

sub-components as well as its environment. In verification, we study the extent to which

these behaviours are analogous to the expectations for the same. However, when verifying

a particular aspect of the system, we might not need all the intricate details of its working

and can abstract the components irrelevant to the current context. However, these models

should be mathematically precise and unambiguous, and obtaining these models is no easy

feat. We can find incompleteness, ambiguities, and inconsistencies even while just accurately

modeling a complex system [BK08].

Historically, such system models have been generated in the form of guarded com-

mands [Dij75], Petri nets [Rei85], process algebra [DN11], Kripke structures ( Sec.1.1), timed

automata [RD94], etc.

Transition Systems — These are directed graphs where states of the system are abstracted

as nodes, and the changes in the states are modeled as edges of the graph, known as transi-

tions [BK08]. Kripke structures [Kri63] is a variation of transition systems where each state

is labeled by atomic propositions to express simple facts about it. Each transition may be

labeled by the action that causes the corresponding evolution of the system.

Example 1.
Let us consider the simple example of modeling a dice. We describe
this using a Kripke structure in Fig. 1.1a. We use s0 to denote the initial state with
the incoming transition from nowhere. Using six actions, labeled roll, we show the
possible transitions to the six faces of the dice, written as diei, i ∈ {1, . . . , 6}.

Although Kripke structures are enough to represent the states of a system along with

how the system evolves, they cannot give any details on how likely a transition is compared

2

roll

roll

l

l

o

r

roll

r

o

ll

r

o

l

l

s0

1/6

1/6

6

/

1

1/6

1

/

6

1

/

6

s0

1

1

1

1

1

1

1

1

1

1

1

1

1

/

8

1

/

6

1/8

6

/

1

1/4

1/6

biased

s0

unbiased

1/6

1/6

6

/

1

1/6

1

/

6

1

/

6

1

1

1

1

1

1

(a) Kripke structure

(b) DTMC

(c) Markov Decision Process

Figure 1.1: Models to represent a roll of a dice.

to the others. Hence, we need a more efficient modeling structure to capture the probabilistic

distributions of the transitions from one state to another.

Markov Processes — They capture the stochastic nature of systems. They abide by the

Markov property which requires the transition from a state to its next to depend only on the

current state and none of the previously visited states. In our work, we only consider discrete-

time systems. Discrete-Time Markov Chains (DTMC) [Chu67] ensure each transition occurs

at discrete moments in time as opposed to continuous time systems where the evolution of

the system is described by the rate of transition at each state.

Example 2.
In Fig. 1.1b, we describe the roll of the dice with the additional infor-
mation of how probable each of the transitions is. In this case, each of the transitions
is equally likely, hence they have a value of 1/6. Although not every system has equally
likely transitions, the sum of the probabilities of all its transitions should always add
up to one.

In order to represent our systems as DTMCs, we need to be aware of the probabilistic

distributions it entails. However, in reality, when modeling systems that interact with their

environment, it might not be possible to have the exact details of the inputs the environment

3

provides. In such cases, Markov Decision Processes (MDPs) [Put90] allow us to represent

the actions of the environment using non-determinism.

Example 3.
In Fig. 1.1c, we describe the rolling of two possible die. If we choose the
unbiased dice, we will get transitions similar to Fig. 1.1b. But if we choose a biased
dice, we can get skewed transitions. In this biased case, the outcome of getting a face
with five is more likely than the others. Notice how we have not specified how likely
it is to get a biased die compared to an unbiased one. This choice of which die to use
can be non-deterministic.

In the sections and chapters that follow, we specifically consider these two types of models

and describe them formally.

We use DTMCs to model purely probabilistic systems and MDPs to model probabilistic

systems with non-deterministic actions and evolving in discrete time.

1.2 Specifications

Each state in our model is a snapshot of the system at that point in time. It reflects the

values of variables of the system in that time instance. In verification, our aim is to check the

conformance of these values in states against expected values. We express our requirements

as specifications. We can use propositional logic to simply verify if certain values are true in

a specific state. However, the states transition into each other with time. In discrete-time

systems, we assume states evolve at each unit time step. In such situations, propositional

logic is not enough to reason about the timely evolution of the system. Temporal logics

introduces the concept of time into logic. This term was coined by Arthur Prior [Uck12]. In

program verification, Amir Pnueli [Pnu77] presented the basis for temporal reasoning.

4

Example 1.
Let us assume Drink(coffee) is a Boolean function that returns true
if I am drinking coffee and false otherwise. We can express ‘I am drinking coffee’
in propositional logic as Drink (coffee). To express ‘I will drink coffee’, we will need
Drink (coffee) means that either at the current moment or at some
temporal logic.
time in the future, I will be drinking coffee.

1.2.1 Trace-based Specifications

Traces — This forms the basis of our reasoning over systems. Each trace represents an

execution of the system we have modeled. In verification, we essentially reason about the

truth of our specifications on these traces. We can imagine them as a sequence of states as

shown in Fig. 1.2a. The model in Fig. 1.1a can produce these traces.

s0

s0

s0

s0

1/6

1/6

6

/

1

1/6

1

/

6

1

/

6

1

1

1

1

1

1

(a) Linear traces for
LTL

(b) Computation trees for CTL

(c) Probabilistic computation
trees for PCTL

Figure 1.2: Traces considered in different types of logic.

We can use Linear Temporal Logic(LTL) [Pnu77] to reason about these individual traces.

Using this logic, we can reason if something happens across all traces or not. However, we

often also need to reason simultaneously about all paths that are possible from a given state

in a possibilistic manner as shown in Fig. 1.2b for the model in Fig.1.1a. This concept was

introduced in [BAMP81] as Computational Tree Logic (CTL). Using this logic, we argue if

something happens at least once or always across the tree. An obvious extension of this

5

form of logic would be to argue about the probability of something happening instead of

just its possibility. This can be done using Probabilistic Computation Tree Logic (PCTL)

first introduced in [HJ94]. There have been several extensions of these logics to increase its

expressive power. The specifications we define using these logics are called trace properties as

each property corresponds to a set of traces that satisfy the property. So, a system satisfies

the given specification if the set of traces generated by the system is a subset of the traces

accepted by this property.

In LTL, we can specify

Example 2.
diei to check if we can reach a state labeled
diei now or in the future. In PCTL, we can specify P=0.16(
die1) to check that the
sum of probability of reaching states labeled die1 is equal to 0.16. Notice that we
might not be able to reach such a state in every execution. Hence we cannot use LTL
to reason about it, instead, we have to consider a branching time logic.

1.2.2 System Specifications

Several policies, however, cannot be verified by examining just the set of traces of a

system. An easy example can be conformance. During the development of a system, we

can design it according to our needs. In this phase, we usually abstract away the details of

implementation. Once built, we would then need to check if the implementation conforms

with the design modeled, based on some criteria. In Fig. 1.3 on the left we show the DTMC

for a die roll and on the left we show how the same idea can be implemented using multiple

coin tosses according to the Knuth Yao protocol [KY76]. Conformance, in this case, can aim

to verify if the probabilistic distribution of getting the different faces of the dice is the same

in both cases. To verify such a property we will need to compare the computation trees of

the two models simultaneously. Such specifications are called hyperproperties.

Hyperproperties — These are used to express relations between executions of a system.

Each hyperproperty is a set of sets of traces, H, that satisfy it. Hence, these are also

known as system properties. A system is said to satisfy a hyperproperty if it generates

6

s0

1
6
1
6
1
6
1
6
1
6
1
6

1

1

1

1

1

1

s′
0

1
2

1
2

s′
1

s′
2

1
2

1
2

1
2
1
2

1
2
1
2

s′
3

s′
4

s′
5

s′
6

1
2

1
2
1
2

1
2
1
2

1
2

1

1

1

1

1

1

Figure 1.3: Implementation of dice by coin tosses using Knuth-Yao protocol.

a set of traces that belongs to the set H. Several security policies like non-interference

and observational determinism, system properties like average response time, and average

up-time, are defined as hyperproperties. It allows us to reason and compare states across

multiple traces, which may or may not belong to the same model. Recent formalization of the

general concept of hyperproperties [CS08] has inspired extensions of previously mentioned

logics to its hyper counterparts. PCTL, specifically, has been extended into HyperPCTL

allowing quantification over initial states of a model [ ´AB18].

Example 3. Compared to e.g. 1.2, to express ‘I drink coffee everyday at the same
time’, we will need hyperproperties. The property shown below says that if I am
drinking coffee at some time during one day (denoted by s), I’ll always drink coffee at
the same time on any other day (denoted by s′).

∀s.∀s′.

(cid:0)Drink (coffee)s ↔ Drink (coffee)s ′

(cid:1)

1.3 Model Checking for Hyperproperties

Model checking, shown in Fig. 1.4, is a technique of formal verification that involves

reasoning along all possible behaviours of the model to prove or disprove a specification.

This is a fully automated (‘push-button’) technique. A strong assumption in this process is

that the model we use, to represent the system, and the specification used, to represent the

7

System Model

Satisfied/Witness

Model Checker

e

u

T r

False

Specification

Unsatisfied/Counterexample

Figure 1.4: Overall idea of model checking.

requirement, is accurate. Note that in model checking we can pose our specification either

as a forall case (commonly referred to as verification) or an exists case (commonly referred

to as synthesis). For verification, we intend to check the property across all combinations of

traces (depending on the number of quantifiers in the specification). Hence, the specification

can either be satisfied or we can provide counterexample. For synthesis, we intend to search

for a combination of traces. Hence, a satisfaction would yield a witness and a failure would

result in an unsatisfied result.

Since model checking aims to check all possible cases in a system, it is rigorous.

It

has been extensively applied for safety-critical systems instead of testing. One of the main

features is that it returns a counterexample when a specification fails in the system. This

helps to find the location of the failure. In case of hyperproperties, the model is usually a

composition of several copies of the same system or multiple systems. The number of models

in the composition is determined by the number of quantifiers in the hyperproperty, which

essentially is the number of traces we are comparing simultaneously.

The challenge lies in finding optimized model checking algorithms that scale well for large

state spaces. This is handled by finding a balance between the accuracy of the model checker

and the extent of model checking involved. On one hand, we can exhaustively model check

the whole system which gives the most accurate result but is time and memory-consuming.

On the other hand, we can use bounded model checking (where we limit the size of traces)

or statistical model checking (where we model check a sample of the traces and infer the

8

result with some confidence) which are less accurate but faster and scales better. We have

to make our choice depending on the system we are model checking - exhaustive for safety-

critical systems and less accurate model checking for cases where have tight time and memory

budget.

In all our subsequent work we explore different aspects and challenges of model check-

ing probabilistic hyperproperties on DTMCs and MDPs.

1.4 Motivation

Markov models have been extensively used to represent systems due to their transitions

being independent of their history. Although keeping a history does have its advantages,

Markov models provide a simpler insight into the workings of the system, thus, assisting in

its verification. They have been widely used in modeling randomized distributed systems

to represent self-stabilizing algorithms, consensus protocols, mutual exclusion, conformance,

etc. In communication protocols, they have been used to represent broadcasting protocols,

device discovery, IP configuration protocols, etc. In security, they have been used to model

contract signing, message exchange, cryptography, non-repudiation protocols, etc. In terms

of path planning synthesis, Markov models have been widely used to design grids and move-

ment of UAVs, robots, and vehicles. Note that the original works introducing the above ideas

might not have suggested the use of Markov chains for their representation, but different

model checking software has modeled the above problems as Markov Chains to verify the

systems.

A recent study [CS08] has shown that several important policies related to security,

privacy, and robustness are hyperproperties. Below are a few of the concepts that cannot be

represented or verified as trace properties.

• Secure information flow — When verifying systems, one of the main aspects is

9

the interaction between systems and its users. Each user has an authorization level

assigned to them (access to high/secret or low/public information). When interacting

with the system, they provide inputs to the system and observe its outputs. Infor-

mation leak refers to the unwanted exposure of sensitive information to unauthorized

users, endangering the privacy of the information owners. Information leaks cannot

be detected by observing individual outputs.

In such cases, we look for similarity

of output among traces possessing different secret inputs. Security policies regarding

information flow enforce restrictions on how the system generates its outputs and en-

sure there is no direct flow of information from secret channels to publicly observable

channels. Probabilistic information flow argues that the probabilistic distribution of

the public channel outputs does not differ drastically based on different secret inputs.

Non-interference [GM82] enforces that for every trace t in the system, there should be

another trace that has none of the high inputs in t but produces the same low observ-

able output as t. The observable outputs can be execution time, power consumption,

value of variables, etc. The essence of this concept is to prevent the user from knowing

that a specific trace is caused by specific secret values only. Observational determin-

ism [ZM03] enforces the equivalence between corresponding states of two traces if they

both begin in states that are observably equivalent. This will prevent an intruder from

understanding how secret values change the course of the execution of a system. Both

these policies require simultaneous observation of multiple traces, and hence cannot be

expressed by existing trace properties.

Service level agreements — When designing systems, we aim to ensure their perfor-

mance satisfies certain statistics specified in service level agreements. One such statistic

is the average response time of a system, i.e., the average time that elapses between

the reception of a request and the execution of its response. We can verify if each

trace has an individual response time within the bound provided, but that will be a

stronger property than necessary. We can certainly have a trace that has a much higher

10

response time, or one with a significantly lower response time, yet the average across

all traces can still be within the specified bound. Thus we need to consider multiple

traces simultaneously, to find the average value, requiring the use of hyperproperties.

Differential privacy — When working with anonymous data, one of the major poli-

cies is to ensure that we do not leak any information or indication about the owner

of the information, either directly or via some statistic. Differential privacy [DR14]

ensures that the output of a query on a dataset is similar irrespective of whether our

data is contributing to the dataset. This is of immense importance in the current

world. For example, we all receive targeted advertisements but we don’t want them

to reveal private details of our lives (say, the skin problems we suffer from) through

the ads that appear in our internet browser. Since we need to argue about both cases

where the specific owner is and is not a part of the database simultaneously, we will

need hyperproperties to express this policy.

Probabilistic causation — This focuses on arguing about how likely an event is

to cause another. The relation is bidirectional in the sense that the cause leads to

the event and without the cause the event is significantly less probable [Hit21]. The

concept of finding general causes of events can be visualized as trace properties where

we find all possible causes (say, smoking, genetics, pollution) for an event (say, lung

cancer). However, when we want to argue about the cause of a specific effect we will

need to consider all the probable causes simultaneously and find the most likely cause.

This is referred to as token causation. This will require hyperproperties to express the

search for such a cause.

Probabilistic conformance — System designs are used to guide their implementa-

tions. Conformance answers questions like “Was this a correct optimization?” or “Was

this a safe refactoring?” [Way20]. Extending this idea to probabilistic systems, we

want to ensure that no matter how the system has been implemented or changed, the

11

probabilistic distribution of the outcomes does not change. This is one way of deter-

mining if the design ideas have been successfully translated into the implementation

or if the changes in our implementation still satisfy the requirements of the original

system. Fig. 1.3 shows one such example.

Fairness — This is a growing concern among artificially intelligent systems. Since a

lot of the societal decisions about us are being assisted by pre-trained AI systems, we

want to ensure the system itself is fair to the whole population. Fairness can be divided

into group fairness and individual fairness. Group fairness checks if a group of minority

population is less likely to have access to a service or opportunity specifically due to

their minority status. Individual fairness refers to the fact that two similar individuals

should be treated similarly and has been recognized as a hyperproperty [ADDN17].

Robotics path planning — Non-hyperlogics are helpful in synthesizing paths in

grids. However, if we want to argue about optimality (a robot using a specific strategy

is always able to reach a goal faster), performance (which strategy has less energy con-

sumption), or robustness (the robot can always reach the goal under any disturbance),

we need to use hyperproperties [WZBP19].

Distributed algorithms — We often use randomization to break symmetry in order

to tackle impossibility results. Although one can reason about the expected perfor-

mance of a randomized distributed algorithm by the traditional reward models, from

a design perspective, it is desirable to determine and mitigate states from where con-

vergence to the objective of the algorithm takes much longer than others. This would

require the involvement of hyperproperties to argue about reachability from multiple

states.

This list is by no means a complete view but provides an insight into the range of

applications of probabilistic hyperproperties. This has motivated us to pursue research

on logic to represent them more efficiently and techniques to verify them on discrete-time

12

systems.

1.5 Thesis statement

For any verification problem, the two most important choices are the structure of the

model and the type of property considered. This dissertation focuses on one such specific

combination of these choices:

• When considering real-world systems, the presence of noise and errors cannot be erad-

icated completely. This gives rise to additional complexities. Discrete-time Markovian

models help in capturing these concepts while keeping the model simple enough for

feasible computation. Hence, our choice of models was that of DTMCs and MDPs.

• We focus on quantitative hyperproperties for probabilistic systems. The high expres-

sivity of the logic for this purpose (HyperPCTL and a fragment of HyperPLTL) makes

it challenging to create efficient model checking algorithms. With the majority of its

applications being in expressing security and privacy policies, these properties cannot

be merely ‘tested’ to ensure correctness. Hence, we focus on an in-depth exploration

of the complexity of these properties and attempt to devise different verification algo-

rithms for fragments of these logics.

Given this context, this dissertation proposes and aims to defend the following statement:

HyperPCTL can be used to concisely express quantitative system requirements as

probabilistic hyperproperties. Despite their high complexity, we can utilize exhaustive

and approximate approaches to model check specific fragments of this logic that can

express a range of prominent applications.

1.6 Contributions

To validate my thesis statement, I have conducted research on the following three fronts:

13

• Logic: We have focused on proposing a generalized logic HyperPCTL (extended the

idea of HyperPCTL proposed in [ ´AB18]) to express probabilistic hyperproperties. This

involved non-trivial extensions of the logic to allow the expression of properties on

nondeterministic systems and incorporated connections to reward models. We also

allow arguments about the truth of the property over schedulers which is used to

resolve non-determinism in such systems.

• Algorithm: We explore the complexity involved in model checking the general logic.

We have proved that an exhaustive model checking solution to this problem is accurate

but undecidable, in general, and becomes decidable if we limit the type of schedulers

to a memoryless, non-probabilistic type. We prove that this restriction makes the

problem NP-complete for an existential scheduler quantifier (co-NP complete for a

universal quantifier). Based on these results, we have explored different approaches to

handle the problem:

– Exhaustive: We devised an algorithm to encode the model checking problem

into an SMT-solving problem. It involves the automatic generation of Z3-based

constraints combining the information of both the model and the hyperproperty.

– Statistical: We focused on an interesting fragment of HyperPLTL and devised an

approximate sound but incomplete method that scales efficiently on models with

large state space.

– Fragment specific: We focused on specific fragments of the logic in terms of a

number of schedulers and initial states to device algorithms that synthesize ran-

domized and/or memoryful schedulers based on scheduler combination or distri-

bution transformers.

Additionally, we propose an SMT-based algorithm to solve the parameter synthesis

problem for a fragment of HyperPCTL for synthesizing missing transition probabilities

given that the model should satisfy a given hyperproperty.

14

• Implementation: We have built prototypical implementations of all our algorithms

and evaluated them on relevant case studies to focus on their strengths and weaknesses.

– For our exhaustive algorithm, we have built a tool HyperProb. The tool takes

a PRISM program and a probabilistic hyperproperty in HyperPCTL as a string,

parses them to undergo automated generation of Z3 constraints, uses Z3 to solve

the constraint set and returns the satisfaction result along with a witness (for

satisfaction of existential quantifier) and a counterexample (for unsatisfaction of

universal quantifier).

– For our statistical approach we focused on an existing tool PLASMA to utilize its

strength in handling scheduler synthesis and extending it to handle universally

quantified probabilistic hyperproperties in HyperPLTL.

– We have separately implemented prototypes of our fragment-specific algorithms

and intend to incorporate them into HyperProb as additional features.

1.7 Organization

Publication

Focus Area Model

Logic

Associated with

Parameter Synthesis for
Probabilistic Hyperproperties

Algorithm,
Implementation

DTMC

Non-nested
HyperPCTL

LPAR’20

Probabilistic Hyperproperties
with Nondeterminism

Logic

MDP

HyperPCTL

ATVA’20

Model Checking Hyperproperties
for Markov Decision Processes

Logic
Algorithm

MDP

HyperPCTL
(1σ)

Information and
Computation’22

Probabilistic Hyperproperties
with Rewards

Logic, Algorithm,
Implementation

MDP

HyperPCTL

NFM’22

HyperProb: A Model Checker for
Probabilistic Hyperproperties

Implementation MDP

HyperPCTL

FM’21

Efficient Probabilistic Model Checking
for Relational Reachability

Algorithm,
Implementation

MDP

HyperPCTL
(1σ1s), (1σ2s)

CSF’24
(Submitted)

Lightweight Verification
of Hyperproperties

Algorithm,
Implementation

MDP

HyperPLTL

ATVA’23

Table 1.1: Details of publications.

15

Table 1.1 elaborates on the publications associated with this dissertation. The rest of

the chapters are organized as follows:

• In Chapter 2, we formally describe the preliminary concepts on which our current work

is built. It describes the basics of the structure of our models as well as the logic.

• In Chapter 3, we describe our work [ ´ABBD20a] on the synthesis of values for parameters

in our models based on specifications they must satisfy, expressed as probabilistic

hyperproperties.

• In Chapter 4, we describe our work [ ´ABBD20b] to extend its expressibility to include

non-deterministic aspect of models by allowing quantification over schedulers and pro-

vide undecidability results for the model checking problem of the general logic.

• In Chapter 5, we describe our work [D ´ABB22] that focuses on the decidable fragment

of HyperPCTL along with proofs, and our SMT-based model checking algorithm.

• In Chapter 6, we describe our work of extending the expression of HyperPCTL to reason

over reward models [DW ´A+22].

• In Chapter 7, we describe the working of the model checker HyperProb [DABB21] that

involves all of the above extensions of HyperPCTL. It elaborates on the inner structure

and execution details of the tool along with detailed evaluation.

• In Chapter 8, we describe our scalable model checking algorithms for two specific frag-

ments of HyperPCTL. We synthesize randomized schedulers using the convex addition of

schedulers and memoryful, deterministic schedulers using distribution transformation-

based search.

• In Chapter 9, we describe our scalable model checking algorithm [DSB+23] for a specific

fragment of HyperPLTL by extending PLASMA along with thorough evaluation.

16

• In Chapter 10, we describe related works that have either formed the basis of our work

or motivated our research.

• In Chapter 11 we provide concluding discussions on extensions and applications of this

line of work.

17

Chapter 2

Preliminaries

This chapter formally defines the frameworks we use to describe our models and specifica-

tions in the following chapters. We define Markov chains and their variations which are used

to model our systems. We further provide the syntax and semantics of the logic HyperPCTL

which we extend in the following chapters. Commonly used in the rest of the chapters, we

use R, Q, and N to denote real, rational, and natural (including zero) numbers, respectively.

We use n = {1, . . . , n} for n ∈ N.

2.1 Discrete-Time Markov models

Definition 2.1.1. A discrete-time Markov chain (DTMC) is a tuple M=(S, P, AP, L) with
the following components:

• S is a non-empty finite set of states;

• P : S × S → [0, 1] is a transition probability function with (cid:80)

s′∈S

P(s, s′) = 1, for all

s ∈ S;

• AP is a finite set of atomic propositions, and

• L : S → 2AP is a labelling function.

■

18

{init}
0.4

s0

s1

{init}
0.3

0.2

0.7

{a}

s2

0.4

s3

s4

0.8

1

s5

0.2

1

s6

{a}

1

1

Figure 2.1: Example DTMC.

Fig 2.1 shows a simple DTMC. An (infinite) path of M is an infinite sequence π =

s0s1s2 . . . ∈ S ω of states with P(si, si+1) > 0, for all i ≥ 0; we write π[i] for si. Let

Paths s(M) denote the set of all (infinite) paths of M starting in s, and Paths s

fin(M) denote

the set of all non-empty finite prefixes of paths from Paths s(M), which we call finite paths.

For a finite path π = s0 . . . sk ∈ Paths s0

fin(M), k ≥ 0, we define |π| = k. We will also use

the notations Paths (M) = ∪s∈SPaths s(M) and Paths fin(M) = ∪s∈SPaths s
t ∈ S is reachable from a state s ∈ S in M if there exists a finite path in Paths s

fin(M). A state

fin(M) with

last state t; we use Paths s,T

fin (M) to denote the set of all finite paths from Paths s

fin(M) with

last state in T ⊆ S. A state s ∈ S is absorbing if P(s, s) = 1.

The cylinder set CylM(π) of a finite path π ∈ Paths s

fin(M) is the set of all

in-

finite paths of M with prefix π. The probability space for M and state s ∈ S is

(Paths s(M), {∪π∈RCylM(π) | R ⊆ Paths s

fin(M)}, Pr M

s ), where the probability of the cylin-

der set of π ∈ Paths s

fin(M) is Pr M

s (CylM(π)) = Π|π|

i=1

P(π[i−1], π[i]).

Note that the cylinder sets of two finite paths starting in the same state are either disjoint

or one is contained in the other. According to the definition of the probability spaces, the

total probability for a set of cylinder sets defined by the finite paths R ⊆ Paths s
is Pr M(R) = (cid:80)

s (π) with R′ = {π ∈ R | no π′ ∈ R \ {π} is a prefix of π}. To

π∈R′ Pr M

fin(M)

improve readability, we sometimes omit the DTMC index M in the notations when it is

clear from the context.

Definition 2.1.2. The parallel composition of two DTMCs Mi = (Si, Pi, APi, Li), i = 1, 2,
is the DTMC M1 × M2 = (S, P, AP, L) with the following components:

19

• S = S1 × S2;

• P : S × S → [0, 1] with P((s1, s2), (s′

1, s′

2)) = P1(s1, s′

1) · P2(s2, s′

2), for all states

(s1, s2), (s′

1, s′

2) ∈ S;

• AP = AP1 ∪ AP2, and

• L : S → 2AP with L((s1, s2)) = L1(s1) ∪ L2(s2).

■

Definition 2.1.3. A Markov decision process (MDP ) is a tuple M = (S, Act, P, AP, L) with
the following components:

• S is a non-empty finite set of states;

• Act is a non-empty finite set of actions;

• P : S × Act × S → [0, 1] is a transition probability function such that for all s ∈ S the
P(s, α, s′) = 1} is not empty

set of enabled actions in s Act(s) = {α ∈ Act | (cid:80)
and (cid:80)

P(s, α, s′) = 0 for all α ∈ Act \ Act(s);

s′∈S

s′∈S

• AP is a finite set of atomic propositions, and

• L : S → 2AP is a labeling function.

■

{h>0}

s0

s1

{h≤0}

β

1
2

α2
3

α

1
2

1
4

3
4

β

1
3

1
2

1
2

{l=1}

s2

τ

1

s3 {l=2}

τ

1

Figure 2.2: MDP showing the actions and probabilistic distributions.

Fig. 2.2 shows a simple MDP. Schedulers can be used to eliminate the non-determinism in

MDPs, inducing DTMCs with well-defined probability spaces.

Definition 2.1.4. A scheduler for an MDP M = (S, Act, P, AP, L) is a tuple σ =
(Q, act, mode, init), where

• Q is a countable set of modes;

• act : Q × S × Act → [0, 1] is a function for which (cid:80)
α∈Act\Act(s) act(q, s, α) = 0 for all s ∈ S and q ∈ Q;

(cid:80)

α∈Act(s) act(q, s, α) = 1 and

20

• mode : Q × S → Q is a mode transition function, and

• init : S → Q is a function selecting a starting mode for each state of M.

■

Let ΣM denote the set of all schedulers for the MDP M. A scheduler is finite-memory if

Q is finite, memoryless if |Q| = 1, and non-probabilistic if act(q, s, α) ∈ {0, 1} for all q ∈ Q,

s ∈ S and α ∈ Act.

Definition 2.1.5. Assume an MDP M = (S, Act, P, AP, L) and a scheduler σ =
(Q, act, mode, init) ∈ ΣM for M. The DTMC induced by M and σ is defined as Mσ =
(S σ, Pσ, AP, Lσ) with S σ = Q × S,

Pσ((q, s), (q′, s′)) =

(cid:40) (cid:80)

α∈Act(s) act(q, s, α) · P(s, α, s′)

0

if q′ = mode(q, s)
otherwise

and Lσ(q, s) = L(s) for all s, s′ ∈ S and all q, q′ ∈ Q.

■

A state s′ is reachable from s ∈ S in MDP M is there exists a scheduler σ for M such

that s′ is reachable from s in Mσ. A state s ∈ S is absorbing in M if s is absorbing in Mσ

for all schedulers σ for M. We sometimes omit the MDP index M in the notations when it

is clear from the context.

2.2 Discrete-Time Markov Models with Rewards

For any domain D and any v = (v0, . . . , vn−1) ∈ Dn, we define v[i] = vi for i ∈

{0, . . . , n−1}. The concepts below have been adapted from [BK08] and extended to work

for hyperlogics.

When defining costs or rewards for Markov models, we can assign rewards to states or

transitions. In this work we limit to the assignment of non-negative rewards to states and

support multi-dimensional reward vectors.

Definition 2.2.1. A Discrete Time Markov Chain with (k-ary) rewards (DTMCR) is a
tuple M = (S, P, AP, L, rew) with

• a non-empty set of states S,

21

• a transition function P : S × S → [0, 1] ⊆ R with (cid:80)

s′∈S P (s, s′) = 1 for all s ∈ S,

• a finite set of atomic propositions AP,

• a labeling function L : S → 2AP and

• a reward function rew : S → Rk

≥0.

{init}
0.4

{a}

s2
2

s0
5

0.4

1
∅

s5
1

0.7

0.2

s3
2
0.8

s1
6

∅
0.2

s6
1

{init}
0.3

s4
2

∅
1

{a}

s0
3

1
2

{h>0}

α

1
4

3
4

β

1
2

α2
3

s1
3

1
3

{h≤0}

β

1
2

1
2

(cid:40)

(cid:41)

s2
1

τ

1

l=1
end

(cid:40)

(cid:41)

s3
1

τ

1

l=2
end

(b) An MDPR.

1

1

(a) A DTMCR.

{h>0}

s2
1

1

s0
3

(cid:40)

3
4

2
1
3
4

s1
2

1
3

{h≤0}

(cid:41)

l=1
end

(cid:40)

(cid:41)

l=2
end

s3
1

1

(c) An induced DTMCR.

Figure 2.3: Example Markov models with rewards.

Fig. 2.3a shows an example DTMCR with unary rewards. Assume a DTMCR M =

(S, P, AP, L, rew). An infinite path is a sequence of states π = s0s1 . . . ∈ S ω with P (si, si+1) >

0 for all i ∈ N. A non-empty prefix of an infinite path is a finite path π = s0 . . . sn−1 ∈ S +

of length |π| = n ∈ N \ {0}. Let Paths s(M) (Paths s

fin(M)) be the set of all infinite (finite)

paths starting in s ∈ S. A state t ∈ S is reachable from s ∈ S if there exists a path in

Paths s

fin(M) ending in t. A state s ∈ S is absorbing iff P (s, s) = 1.

For a finite path π ∈ Paths s

fin(M), we define its cylinder set CylM(π) as the set of all

infinite paths with π as a prefix. The probability of the cylinder set of π ∈ Paths s

fin(M)

is defined as Pr M

s (CylM(π))=Π|π|−1

i=0 P (si, si+1). For sets R⊆Paths s

fin(M) we have Pr M

s (R)=

(cid:80)

π∈R′ Pr M

s (π), where R′ contains all finite paths from R that have no extensions in R.

22

These notions induce for each s ∈ S the probability space,

(cid:16)

Paths s(M), (cid:8) (cid:83)

π∈R CylM(π) | R ⊆ Paths s

fin(M)(cid:9), Pr M

s

(cid:17)

.

Note that the cylinder sets of two finite paths starting in the same state are either disjoint

or one is contained in the other.

For a reward function rew : S → Rk

≥0 and i ∈ {0, . . . , k−1} we define rewi : S → R≥0 to

assign the ith state reward rewi(s) = rew(s)[i] to all s ∈ S. The ith cumulative reward for a
finite path, π = s0s1 . . . sn−1 is defined as rewi(π) = (cid:80)n−1

j=0 rewi(sj). Note that non-negative

rewards assure monotonic increase of cumulative rewards with path extensions.

To argue about simultaneous runs across two DTMCRs, we define their parallel compo-

sition.

Definition 2.2.2. Assume two DTMCRs Mi = (Si, Pi, APi, Li, rewi) with ki-ary rewards,
i ∈ {1, 2}. We define the parallel composition M1×M2=(S1×S2, P, AP1 ∪ AP2, L, rew) with
(k1+k2)-ary rewards, such that for all (s1, s2), (s′

2) ∈ S × S:

1, s′

• P (cid:0)(s1, s2),(s′

1, s′

2)(cid:1)=P1(s1, s′

1)·P2(s2, s′

2),

• L((s1, s2))=L1(s1)∪L2(s2) and

• rew((s1, s2)) = (rew1(s1), rew2(s2)).

Next, we extend the probabilistic nature of DTMCRs with non-determinism.

Definition 2.2.3. A Markov Decision Process with k-ary rewards (MDPR) is a tuple M =
(S, Act, P, AP, L, rew) with

• a non-empty set of states S,

• a non-empty finite set of actions Act,

• a transition function P : S × Act × S → [0, 1] ⊆ R such that for each s ∈ S we have
s′∈S P (s, α, s′) ∈ {0, 1}. For all α ∈ Act, there is at least one action that can be
s′∈S P (s, α, s′) = 1} and for

(cid:80)
chosen in each state, such that α ∈ Act(s) = {α ∈ Act| (cid:80)
α ∈ Act \ Act(s), (cid:80)
s′∈S P (s, α, s′) = 0,

• a finite set of atomic propositions AP,

• a labelling function L : S → 2AP, and

23

• a reward function rew : S → Rk

≥0.

Fig.2.3b shows an example MDPR. In each state, for the next execution step, any of the

enabled actions can be chosen non-deterministically. Schedulers are used to eliminate this

non-determinism.

Definition 2.2.4. A scheduler for an MDPR M = (S, Act, P, AP, L, rew) is a tuple σ =
(Q, act, mode, init) with

• a countable set of modes Q,

• a function act : Q × S × Act → [0, 1] ⊆ R such that for every s ∈ S and q ∈ Q,

(cid:80)

α∈Act(s) act(q, s, α) = 1 and (cid:80)

α∈Act\Act(s) act(q, s, α) = 0 ,

• a mode transition function mode : Q × S → Q, and

• init : S → Q assigning to each state of M a starting mode.

Let ΣM be the set of all schedulers for M. A scheduler is finite-memory if Q is finite,

memoryless if |Q| = 1, and non-probabilistic if act(q, s, α) ∈ {0, 1} for all q ∈ Q, s ∈ S and

α ∈ Act.

Definition 2.2.5. Assume an MDPR M = (S, Act, P, AP, L, rew) with k-ary rewards and
a scheduler σ = (Q, act, mode, init) for M. Then M and σ induce the DTMCR with k-ary
rewards Mσ = (Sσ, P σ, AP, Lσ, rewσ), where Sσ = Q × S,

P σ((q, s), (q′, s′)) =




(cid:80)

0


α∈Act(s) act(q, s, α) · P (s, α, s′)

if q′ = mode(q, s)

if q′ ̸= mode(q, s) ,

with Lσ(q, s) = L(s) and rew σ(q, s) = rew(s), for all q ∈ Q and s ∈ S.

If σ is memoryless, we sometimes omit its mode and write (s) instead of (q, s). For the

MDPR in Fig. 2.3b and a scheduler that chooses action α in states s0, s1 and action τ in

states s2, s3, the induced DTMCR is shown in Fig. 2.3c.

Different executions in several models can be seen as executions in the composition of

the models. To simplify notation, in this dissertation we restrict ourselves to comparing

executions in the same model, leading to the notion of self-composition.

24

Definition 2.2.6. Assume an MDPR M = (S, Act, P, AP, L, rew) and a sequence σ =
(σ0, . . . , σn−1) ∈ (ΣM)n of schedulers for M.
let Mi =
: S → 2APi with Li(s) =
(S, Act, P, APi, Li, rew) with APi = {ai | a ∈ AP}, and Li
{ai | a ∈ L(s)}. We define the n-ary self composition of M under σ as the DTMCR
0 × . . . × Mσn−1
Mσ = (Sσ, P σ, APσ, Lσ, rewσ) = Mσ0
n−1 .

For i ∈ {0, . . . , n−1},

In the above definition, Mσi
i

is the DTMCR induced by Mi and σi. Note that the

reward of a state s = ((q0, s0), . . . , (qn−1, sn−1)) ∈ S σ in the n-ary self-composition Mσ

is the sequence rewσ(s) = (rew(s0), . . . , rew(sn−1)), i.e.

the ith state reward in the jth

execution is rewσ
reward in the jth execution as rewj,i(π) = (cid:80)|π|−1

k=0 rewj,i(π[k]).

j,i(s) = rewi(sj). For a finite path π in Mσ, we denote its cumulative ith

2.3 Probabilistic Hyperproperties

HyperPCTL [ ´AB18] was the first logic proposed to express probabilistic hyperproperties. It

generalized PCTL by allowing explicit quantification over initial states, and hence, multiple

computation trees. This laid out the syntax, semantics, and main applications which we

have extended in this dissertation.

2.3.1 HyperPCTL Syntax

HyperPCTL (quantified) state formulas φq are inductively defined as follows:

quantified formula

φq

::= ∀ˆs.φq

| ∃ˆs.φq

| φnq

non-quantified formula φnq

::= true | aˆs | φnq ∧ φnq | ¬φnq | φpr φpr

probability expression

φpr

::= P(φpath) | f (φpr

1 , . . . , φpr
k )

path formula

φpath

::=

φnq | φnq U φnq | φnq U [k1,k2] φnq

where ˆs is a state variable from an infinite set ˆS, φnq is a quantifier-free state formula, a ∈ AP

is an atomic proposition, φpr is a probability expression, ∼ ∈ {<, ≤, =, >, ≥}, f : [0, 1]k → R

are k-ary elementary functions to express arithmetic operations (binary addition, binary

subtraction, binary multiplication) over probabilities where constants are viewed as 0-ary

25

functions, and φpath is a path formula, such that k1 ≤ k2 ∈ N≥0. The probability operator P

allows the usage of probabilities in arithmetic constraints and relations.

A HyperPCTL construct φ (probability expression φpr, state formula φq, φnq or path

formula φpath) is well-formed if each occurrence of any aˆs with a ∈ AP and ˆs ∈ ˆS is in the

scope of a state quantifier for ˆs.

HyperPCTL formulas are well-formed HyperPCTL state formulas, where we additionally

allow standard syntactic sugar like false = ¬true, φ1∨φ2 = ¬(¬φ1∧¬φ2),

φ = true U φ,

and P( φ) = 1 − P( ¬φ).

2.3.2 HyperPCTL Semantics

HyperPCTL state formulas are evaluated in the context of self-composition of a DTMC as

described above. We use () to denote the empty sequence (of any type) and ◦ for concate-

nation. Intuitively, these sequences store instantiations for state variables. The satisfaction

of a HyperPCTL quantified formula by M is defined by

M |= φ

iff M, () |= φ .

The formula evaluates the logical value of the quantified state and path subformulas in

the context of a DTMC and an n-tuple state combination from the composed DTMC. The

semantics evaluates HyperPCTL formulas by structural recursion. Let Q denote a quantifier

from {∀, ∃}. For instantiating a state quantifier Qˆs by a state s, we concatenate sn+1 at the

end of the existing state tuple, where n is the number of quantifiers already processed. We

also replace each aˆs in the scope of the given quantifier by asn+1, resulting in a formula that

we denote by φ[ˆs ⇝ (n + 1)].

Formally, the semantics judgment rules are as follows:

M, s |= true,

M, s |= ai

iff ai ∈ L(si),

M, s |= φ1 ∧ φ2

iff M, s |= φ1 and M, s |= φ2,

26

M, s |= ¬φ

iff M, s ̸|= φ,

M, s |= ∀ˆs.φ

M, s |= ∃ˆs.φ

iff ∀sn+1 ∈ S. M, s ◦ sn+1 |= φ[ˆs ⇝ sn+1]

iff ∃sn+1 ∈ S. M, s ◦ sn+1 |= φ[ˆs ⇝ sn+1]

M, s |= φpr

1 ∼ φpr

2

P(φpath)
(cid:74)
f (φpr
(cid:74)

M,s
(cid:75)
1 , . . . φpr
k )

φpr
1

φpr
2
(cid:74)

(cid:74)

M,s
(cid:75)

M,s ∼
(cid:75)

iff
= Pr M(cid:0){π ∈ Paths s(M) | M, π |= φpath}(cid:1)
φpr
k
(cid:74)

φpr
1

(cid:1)

M,s . . . ,
(cid:75)
where M is an DTMC; n ∈ N≥0 is non-negative integer; s is a state of M; a ∈ AP is an

M,s
(cid:75)

M,s = f (cid:0)
(cid:74)
(cid:75)

atomic proposition and i ∈ {1, . . . , n}; φ, φ1, φ2 are HyperPCTL state formulas; φpr

1 · · · φpr

k are

probability expressions, and φpath is a HyperPCTL path formula whose satisfaction relation is

as follows:

M, π |= φ

iff M, π[1] |= φ

M, π |= φ1Uφ2

iff ∃j ≥ 0.

(cid:16)

M, π[j] |= φ2 ∧ ∀i ∈ [0, j). M, π[i] |= φ1

(cid:17)

M, π |= φ1U [k1,k2] φ2 iff ∃j ∈ [k1, k2].

(cid:16)

M, π[j] |= φ2 ∧ ∀i ∈ [0, j).M, π[i] |= φ1

(cid:17)

where π = s0s1 · · · is a path of M; and k1 ≤ k2 ∈ N≥0.

Example — Consider the example from Fig. 2.1 and the HyperPCTL property below.

φ = ∀s. ∀s′.(inits ∧ inits′) → (cid:0)Pr (

as) = Pr (

as′)(cid:1)

(2.1)

The formula in 2.1 is satisfied by M if for all pairs of initial states (labeled by the atomic

proposition init) the probability to satisfy a is the same, i.e., for each (si, sj) ∈ S 2 with init

∈ L(si) and init ∈ L(sj) it holds that M, (si, sj) |= Pr (a1) = Pr (a2). The probability of

reaching a from s0 is 0.4 + (0.2 × 0.2) = 0.44. Moreover, the probability of reaching a from

s1 is 0.3 + (0.7 × 0.2) = 0.44. Hence, we have M |= φ.

The model checking problem of probabilistic hyperproperties over DTMCs was decidable

and exponential in the number of quantifiers [ ´AB18]. They also proposed a symbolic model

checking algorithm that recursively evaluates the truth of sub-formulas φsub and labels the

root node of the computation tree with φsub if the sub-formula is true. On termination

27

of this labeling algorithm, if the starting states (per the quantifiers) contain the required

satisfaction labels corresponding to the hyperproperty, the algorithm is true else false.

This dissertation focuses on extending HyperPCTL [ ´AB18], both its theory base in terms

of logic and complexity results, and the model checking algorithm for model checking prob-

abilistic hyperproperties to allow for non-determinism, rewards, and fragment-specific solu-

tions.

28

Chapter 3

Parameter Synthesis for Probabilistic
Hyperproperties

In this chapter, we discuss the problem of parameter synthesis for our probabilistic hy-

perproperties. Given a model with unknown parameters and a HyperPCTL specification that

we want to be satisfied in the model, we want to study the complexity of the problem to

synthesize values for the unknown parameters and propose a symbolic constraint-based al-

gorithm for the same.

3.1

Introduction

We first motivate the problem through a simple example. Consider the concept of differ-

ential privacy [DR14, DMNS16], that is, a commitment by a data holder to a data subject

that he/she will not be affected by allowing his/her data to be used in any study or analysis.

More formally, let ϵ be a positive real number and A be a randomized algorithm that makes

a query to an input database and produces an output. Algorithm A is called ϵ-differentially

private, if for all databases D1 and D2 that differ on a single element, and all subsets S of

possible outputs of A, we have:

Pr [A(D1) ∈ S] ≤ eϵ · Pr [A(D2) ∈ S]

(3.1)

29

{t=y}

s1

0.5

0 . 5

s2

{t=n}

{t=y}

s1

p

p

s2

{t=n}

5
.
0

0 . 5

{r=y}

s4

s3

∅

5
.
0

0.5

p
−
1

q

s5

{r=n}

{r=y}

s4

s3

∅

p
−
1

1

−

q

s5

{r=n}

1

1

(a) DTMC

1

1

(b) Parametric DTMC

Figure 3.1: The randomized response protocol.

One way to guarantee differential privacy is by introducing randomized response to create

noise and provide plausible deniability. For example, let A be an embarrassing or illegal

activity. In a social study, each participant is faced with the query, “Have you engaged in

activity A in the past week?” and is instructed to respond by the following protocol: (1) flip

a fair coin, (2) if tail, then answer truthfully, and (3) if head, then flip the coin again and

respond “Yes” if head and “No” if tail. Thus, there are no good or bad responses and an

answer cannot be incriminating. The discrete-time Markov chain (DTMC) of this protocol

conducted by a fair coin is shown in Fig. 3.1a, where t = y (respectively, t = n) denotes that

the truth is ‘Yes’ (respectively, ‘No’) and r = y (respectively, r = n) denotes the fact that

the response is ‘Yes’ (respectively, ‘No’). It is straightforward to show that this protocol is

(ln 3)-differentially private.

Now, let us imagine that we intend to change this protocol in order to make it (ln 2)-

differentially private. To this end, one can first transform the DTMC shown in Fig. 3.1a into

a parametric DTMC (see Fig. 3.1b) that allows two different types of coins to be used during

the protocol, hence, parameters p and q. Then, we solve the parameter synthesis problem by

finding a value for parameters p and q that result in an (ln 2)-differentially private protocol.

Differential privacy is a probabilistic hyperproperty, as it prescribes a probability relation

between a set of independent executions. Although the parameter synthesis problem has

been extensively studied in the context of conventional properties, to our knowledge, it has

not yet been solved in the context of hyperproperties.

30

Here, our goal is to solve the parameter synthesis problem for a fragment of the temporal

logic HyperPCTL [ ´AB18]. HyperPCTL lifts the well-known probabilistic temporal logic PCTL

by allowing to express stochastic relations between computations starting in different initial

states. The fragment studied here, called ReachHyperPCTL, is restricted to non-nested proba-

bility operators. For the above randomized response protocol the following ReachHyperPCTL

formula states that whenever a computation starts in a state σ and another computation in

σ′ with a different truth value, the probabilities to get the same response satisfy the (ln 3)-

differential privacy condition:

φdp = ∀σ.∀σ′.

(cid:20)(cid:18)

(t=n)σ ∧ (t=y)σ′

(cid:20)(cid:18)

⇒

(cid:19)

(cid:19)

(t=y)σ ∧ (t=n)σ′

⇒

(cid:18)

(cid:16)

P

(cid:18)

(cid:16)

P

(r=n)σ

(cid:17)

≤ eln 3 · P

(cid:16)

(r=n)σ′

(cid:17)(cid:19)(cid:21)

∧

(r=y)σ

(cid:17)

≤ eln 3 · P

(cid:16)

(r=y)σ′

(cid:17)(cid:19)(cid:21)

(3.2)

Given a parametric DTMC D and ReachHyperPCTL formula ψ, we solve the parameter

synthesis problem for ReachHyperPCTL in two steps. In the first step, we compute for each

possible instantiation of the quantified (initial) states an arithmetic formula over the model

parameters that is true for exactly those parameter configurations that instantiate D to

satisfy ψ.

In a second step, we use those formulas to compute not only single solutions,

but whole regions of satisfying parameter configurations: we decompose the configuration

domain and identify smaller regions in which either all or none of the configurations lead to

the satisfaction of ψ.

We illustrate the application of our technique by using four case studies. Our first example

is differential privacy, as described above. The second example is probabilistic noninterfer-

ence [III92], which establishes a connection between information theory and information flow

by employing probabilities to address covert channels. The third case study is probabilistic

conformance, where we want to find the parameter values of two systems (e.g., a model and

an implementation) such that they conform with each other with respect to a specification.

The last case study is the dining cryptographers problem [Cha88], where we show that the

anonymity of the cryptographers is assured even when using a biased coin in the protocol.

31

3.1.1 The Logic ReachHyperPCTL

Here we consider parametric DTMCs and the problem to synthesize parameter configu-

rations that satisfy a certain probabilistic hyperproperty. As this problem involves symbolic

encodings of reachability probabilities and the truth values of hyperproperties, in order to

provide an effective synthesis algorithm, we restrict ourselves to a fragment called ReachHy-

perPCTL, which excludes nested probability operators.

Syntax. ReachHyperPCTL is syntactically defined over a set AP of atomic propositions by

the following abstract grammar:

ψ ::= ∀σ.ψ

p

::= P( φ)

φ ::= aσ

(cid:12)
(cid:12)
(cid:12) ∃σ.ψ
(cid:12)
(cid:12)
(cid:12)
(cid:12)
(cid:12)
(cid:12) φ ∧ φ

(cid:12)
(cid:12)
(cid:12) aσ
P(φUφ)
(cid:12)
(cid:12)
(cid:12) ¬φ

(cid:12)
(cid:12)
(cid:12) ¬ψ

(cid:12)
(cid:12)
(cid:12) ψ ∧ ψ
(cid:12)
(cid:12)
(cid:12) f (p, . . . , p)

(cid:12)
(cid:12)
(cid:12) p ∼ p

where a ∈ AP is an atomic proposition, ∼∈ {<, ≤, =, ≥, >}, σ are state variables from a

countably infinite set V, and f : [0, 1]k → R are k-ary elementary functions to express arith-

metic operations over probabilities, where constants are viewed as 0-ary functions. We call

φ and ψ state formulas, aσ an indexed atomic proposition and p a probability expression. We

denote by F the set of all ReachHyperPCTL state formulas and probability expressions (over

AP). The difference to HyperPCTL is that temporal operator may be applied to Boolean

combinations of atomic propositions only (operands φ instead of ψ). We use standard syn-

tactic sugar false = aσ∧¬aσ, true = ¬false, φ1 ∨ φ2 = ¬(¬φ1∧¬φ2),

φ = true Uφ,

φ = ¬ ¬φ, etc. An occurrence of an indexed atomic proposition aσ in a ReachHyperPCTL

state formula ψ is free if it is not in the scope of a quantifier bounding σ and otherwise

bound. ReachHyperPCTL sentences are ReachHyperPCTL state formulas in which all occur-

rences of all indexed atomic propositions are bound. ReachHyperPCTL (quantified ) formulas

are ReachHyperPCTL sentences. Each ReachHyperPCTL quantified formula can be transformed

into an equivalent formula in prenex normal form Q1σ1. . . . Qnσn.ψ, where each Qi ∈ {∀, ∃}

32

is a quantifier, σi is a state variable, and ψ is a quantifier-free ReachHyperPCTL formula. In

the following, we assume all ReachHyperPCTL quantified formulas to be in prenex normal

form.

Example. The formula

∀σ1.∃σ2.

(cid:16)

P(

aσ1) = P(

(cid:17)

bσ2)

(3.3)

holds if for each state s1, there exists another state s2, such that the probability to finally

reach a state labeled with a from s1 equals the probability of reaching b from s2.

Semantics. We present the semantics of ReachHyperPCTL based on the n-ary self-

composition of a DTMC. We emphasize that it is possible to define the semantics in terms

of the non-self-composed DTMC, but it will essentially result in a very similar setting, but

more difficult to understand.

Definition 3.1.1. The n-ary self-composition of a PDTMC M = (S, V, P, AP, L) is the
PDTMC Mn = (Sn, V, Pn, APn, Ln) with

• Sn = S × . . . × S is the n-ary Cartesian product of S,

• Pn(cid:0)s, s′) = Πi∈nP(si, s′

i) for all s = (s1, . . . , sn) ∈ Sn and s′ = (s′

1, . . . , s′

n) ∈ Sn,

• APn = ∪i∈nAPi, where APi = {ai | a ∈ AP} for i ∈ n, and

• Ln(s)= ∪i∈n Li(si) for all s=(s1, . . ., sn) ∈ Sn with Li(si)={ai | a ∈ L(si)} for i ∈ n. ■

The satisfaction relation for ReachHyperPCTL sentences by a DTMC M=(S, P, AP, L) is

defined by:

M |= ψ

iff M, () |= ψ

where () is the empty sequence of states. Thus, the satisfaction relation |= defines the

values of ReachHyperPCTL quantified, state, and path formulas in the context of a DTMC

M = (S, P, AP, L) and an n-tuple s = (s1, . . . , sn) ∈ Sn of states (which is () for n = 0).

Intuitively, the state sequence s stores instantiations for quantified state variables. The

33

semantics evaluates ReachHyperPCTL formulas by structural recursion. Quantifiers are in-

stantiated and the instantiated values for state variables are stored in the state sequence s.

To maintain the connection between a state in this sequence and the state variable which

it instantiates, we introduce the auxiliary syntax ai with a ∈ AP and i ∈ N>0, and if we

instantiate σ in ∃σ.ψ or ∀σ.ψ by state s, then we append s at the end of the state sequence

and replace all aσ that is bound by the given quantifier by ai with i being the index of s

in the state sequence. We will express the meaning of path formulas based on the n-ary

self-composition of M; the index i for the instantiation of σ also fixes the component index

in which we keep track of the paths starting in σ. The semantics judgment rules to evaluate

formulas in the context of a DTMC M = (S, P, AP, L) and an n-tuple s = (s1, . . . , sn) ∈ Sn

of states are the following:

M, s |= ∀σ.ψ

iff M, (s1, . . . , sn, sn+1) |= ψ[APn+1/APσ] for all sn+1 ∈ S

M, s |= ∃σ.ψ

iff M, (s1, . . . , sn, sn+1) |= ψ[APn+1/APσ] for some sn+1 ∈ S

M, s |= ai

iff a ∈ L(si)

M, s |= ψ1 ∧ ψ2

iff M, s |= ψ1 and M, s |= ψ2

M, s |= ¬ψ

iff M, s ̸|= ψ

M, s |= p1 ∼ p2

iff

M,s ∼
p1
(cid:75)
(cid:16)

(cid:74)

p2

(cid:74)

M,s
(cid:75)

= Pr

{π ∈ Paths s(Mn) | M, π[1] |= φ}

(cid:17)

M,s

P( φ)
(cid:75)
(cid:74)
P(φ1Uφ2)
(cid:75)
(cid:74)

M,s

(cid:16)

= Pr

{π ∈ Paths s(Mn) | exists j ≥ 0 such that M, π[j] |= φ2

and π[i] |= φ1 for all 0 ≤ i < j}

(cid:17)

f (p1, . . . , pk)
p1
M,s = f (
(cid:74)
(cid:75)
(cid:74)
M, s |= φ1 ∧ φ2

M,s, . . . ,
(cid:75)

pk
(cid:74)

M,s)
(cid:75)

iff M, s |= φ1 and M, s |= φ2

M, s |= ¬φ

iff M, s ̸|= φ

where ψ, ψ1, and ψ2 are ReachHyperPCTL state formulas; the substitution ψ[APn+1/APσ]

replaces for each atomic proposition a ∈ AP each free occurrence of aσ in ψ by an+1; a ∈ AP

is an atomic proposition and 1 ≤ i ≤ n; p1 and p2 are probability expressions and ∼∈ {<, ≤

34

{init}
0.4

s0

s1

{init}
0.3

0.2

0.7

{a}

s2

0.4

s3

s4

0.8

1

s5

0.2

1

s6

{a}

1

1

Figure 3.2: Semantics example.

, =, ≥, >}; φ is a ReachHyperPCTL path formula.

Example. The ReachHyperPCTL formula

ψ = ∀σ.∀σ′.(initσ ∧ initσ′) ⇒

(cid:16)

P(

aσ) = P(

(cid:17)

aσ′)

(3.4)

is satisfied by the DTMC M in Figure 3.2 if for all pairs of init-labelled states, the probability

to reach a is the same, i.e., for each (si, sj) ∈ S2 with init ∈ L(si) and init ∈ L(sj), it holds

that M, (si, sj) |= P(

a1) = P(

a2). The probability of reaching a from s0 is 0.4 + (0.2 ×

0.2) = 0.44. Moreover, the probability of reaching a from s1 is (0.7 × 0.2) + (0.3 × 1) = 0.44.

Hence, M |= ψ.

3.2 Parameter Synthesis Algorithm for ReachHyper-

PCTL

Assume in the following a ReachHyperPCTL quantified formula (i.e. sentence) ψ in prenex

normal form ψ = Q1σ1. . . . Qnσn.ψ′ with quantifiers Qi ∈ {∀, ∃} for i = 1, . . . , n. Assume

furthermore, a parametric DTMC D = (S, V, P, AP, L) with valid parameter configuration

domain I and let Dn = (Sn, V, Pn, APn, Ln) be the n-ary self-composition of D, defined over

the same set of atomic propositions as ψ, where n is the number of quantifiers in ψ. Our aim

in this section is to provide an algorithm for the synthesis of valid parameter configurations

for D such that ψ is satisfied.

The problem is to decide whether a given fixed valid parameter configuration leads to

35

the satisfaction of ψ is decidable. Moreover, for a box R of valid parameter configurations

of D, also the problem to decide whether all, none, or some of the parameter configurations

in R lead to the satisfaction of ψ is solvable.

In the following, we propose a parameter

synthesis algorithm that will use these computations to decompose a set of valid parameter

configurations into a finite set of subsets, and for each of those subsets provide information

whether all, none or some configuration in it leads to the satisfaction of the formula. This

way, we provide not only a single configurations that satisfy the requirements but even sets

of them, and point also to regions that contain no satisfying configurations.

Algorithm 1: Main parameter synthesis algorithm

Input : D: PDTMC; ψ: ReachHyperPCTL formula;
I: a box of valid parameter configurations;
maxit: iteration limit.

Output: (Rgreen, Rwhite, Rred): a decomposition of I into boxes from which all

(Rgreen), none (Rred) resp. some (Rwhite) configurations make D satisfy ψ.

1 Function Main(D, ψ, I, maxit)
SymbolicEncoding(D, ψ, 0);
2
return checkParameterSpace(D, ψ, I, maxit);

3

The main method is shown in Algorithm 1. In line 2, we first compute for each state

s = (s1, . . . , sn) ∈ Sn of the n-ary self-composition Dn a real-arithmetic formula Symb(ψ′, s)

over the model parameters that are true for exactly those parameter configurations under

which ψ′ holds in state s of Dn. Given a set description R of parameter configurations,

unsatisfiability of the formula R ∧ Symb(ψ′, s) will thus mean that there is no satisfying

configuration in R, whereas unsatisfiability of R ∧¬Symb(ψ′, s) means that all configurations

in R are satisfying for ψ′. If both are satisfiable then some configurations in R satisfy ψ′ and

some violate it.

Once the symbolic truth values of the input formula are constructed, in line 3 of Algorithm

1 we try to determine regions in the parameter space such that the input formula either

evaluates to true under all parameter values in the region or it evaluates to false for all

of them. We will use the constants green = 1 respectively red = −1 to encode these

36

properties, and we will use white = 0 to express that none of these properties hold.

Next, we explain the two above-mentioned computations. The symbolic values for ψ′

and all of its sub-formulas are computed by Algorithm 2. Besides the PDTMC model and

a probability expression of a ReachHyperPCTL formula F whose value needs to be computed,

the algorithm receives as a third input how many quantifiers we have already processed;

this is needed to be able to determine the position of quantifiers during recursive calls on

sub-formulas.

If F is atomic then we can compute its value in a given state by looking

at the state labeling. If F is non-atomic then it is the application of an operator to some

operands; the interesting case is when F is the application of a probability operator, in all

other cases we call the same method recursively to compute symbolic values for the operands

Algorithm 2: Symbolic value encoding: Main algorithm

Input : D=(S, V, P, AP, L): PDTMC; φ: ReachHyperPCTL formula or expression;

i: number of already processed quantifiers.

1 Function SymbolicEncoding(D, φ, i)
2

if φ is (∀σ. ψ) or (∃σ. ψ) then

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

i := i + 1;
SymbolicEncoding(D, ψ[APi/APσ], i);

else if φ is aj then

foreach s = (s1, . . . , sn) ∈ Sn do

if a ∈ L(sj) then Symb(φ, s) := true else Symb(φ, s) := false;

else if φ is ψ1 ∧ ψ2 then

SymbolicEncoding(D, ψ1, i); SymbolicEncoding(D, ψ2, i);
foreach s = (s1, . . . , sn) ∈ Sn do Symb(φ, s) := Symb(ψ1, s) ∧ Symb(ψ2, s);

else if φ is ¬ψ then

SymbolicEncoding(D, ψ, i);
foreach s = (s1, . . . , sn) ∈ Sn do Symb(φ, s) := ¬Symb(ψ, s);

else if φ is P( φ) then SymbolicEncodingNext(D, φ);
else if φ is P(φ1Uφ2) then SymbolicEncodingUntil(D, φ);
else if φ is p1 ∼ p2 then

SymbolicEncoding(D, p1, i); SymbolicEncoding(D, p2, i);
foreach s = (s1, . . . , sn) ∈ Sn do Symb(φ, s) := Symb(p1, s) ∼ Symb(p2, s);

else if φ is c then foreach s = (s1, . . . , sn) ∈ Sn do Symb(φ, s) := c;
else if φ is p1 op p2 with op ∈ {+, −, ∗} then

SymbolicEncoding(D, p1, i); SymbolicEncoding(D, p2, i);
foreach s = (s1, . . . , sn) ∈ Sn do Symb(φ, s) := Symb(p1, s) op Symb(p2, s);

37

Algorithm 3: Symbolic value encoding: Computation for next

Input : D = (S, V, P, AP, L): PDTMC; ReachHyperPCTL expression P( φ)

1 Function SymbolicEncodingNext(D, P( φ))
2

K = { s ∈ Sn | Dn, s |= φ };
foreach s ∈ Sn do Symb(P( φ), s) := (cid:80)

3

s′∈K P(s, s′);

Algorithm 4: Symbolic value encoding: Computation for until

Input : D = (S, V, P, AP, L): PDTMC; ReachHyperPCTL expression P(φ1Uφ2)

1 Function SymbolicEncodingUntil(D, P(φ1Uφ2))
2

S1 := {s ∈ Sn | Dn, s |= φ2};
S0 := {s ∈ Sn | Dn, s |= ¬φ1 ∧ ¬φ2};
S? := Sn \ (S1 ∪ S0);
foreach s ∈ S1 do

Symb(P(φ1Uφ2), s) := 1; Pn(s, s) := 1;
foreach successor s2 ∈ Sn \ {s} of s do Pn(s, s2) := 0;

foreach s ∈ S0 do

Symb(P(φ1Uφ2), s) := 0; Pn(s, s) := 1;
foreach successor s2 ∈ Sn \ {s} of s do Pn(s, s2) := 0;

foreach s ∈ S? do

if Pn(s, s) ̸∈ {0, 1} then

foreach successor s2 ∈ Sn \ {s} of s do Pn(s, s2) *=
Pn(s, s) := 0;

1
1−Pn(s,s) ;

foreach predecessor s1 ∈ Sn \ {s} of s do

foreach successor s2 ∈ Sn \ {s} of s do
Pn(s1, s2) += Pn(s1, s) · Pn(s, s2);

Pn(s1, s) := 0;

foreach s ∈ S? do Symb(P(φ1Uφ2), s) := (cid:80)

P(s, s2);

s2∈S1

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

and subsequently syntactically connect those by the respective operator.

There are two cases for the probability operator, one for the probability of a next-

expression and one for the until. The symbolic encodings for them are computed by the

Algorithms 3 and 4, respectively. The former is quite straightforward: to encode the value

of F = P( φ) we first determine the set K of those states of Dn that satisfy φ and then

for each s ∈ Sn the value of F can be encoded by summing up for each direct successor of s

that is included in K the probability to move there (in one step). Note that φ is a Boolean

combination of atomic propositions, therefore its truth can be easily determined for each

state.

38

p

s1

s0

q
1−q

s2

s3

p · q

p · q

s0

s1

s2

s3

s0

s1

q
1−q

s2

s3

1−p

(1−p) + p · (1 − q)

(1−p) + p · (1 − q)

Figure 3.3: A PDTMC (left) and the result of eliminating s1 in [DJJ+15] (mid) and in Alg.
4 (right).

The case for F being a probability expression P(φ1Uφ2) is a bit more involved. We use

state elimination, similar to the method used in [DJJ+15] to symbolically express reachability

probabilities by arithmetic expressions (rational functions). However, whereas in [DJJ+15]

such a reachability probability needs to be computed for a single initial state, for Reach-

HyperPCTL properties we need it for all states of a parametric DTMC. An algorithm that

computes these probability expressions independently, repeatedly applying the method from

[DJJ+15] to each state, would work but it would re-do a lot of computations.

Instead, we apply a slight modification to the standard state elimination approach to

make some additional bookkeeping. We first identify states for which the probability is

known to be one (state set S1) resp. zero (S0), and make them absorbing (lines 2–10).

Then for each remaining state s, we remove self-loops, and connect pairs of predecessors and

successors by direct transitions without visiting s in-between, and then remove the incoming

edges of s. As illustrated in Fig. 3.3, the difference to the approach in [DJJ+15] is that we do

not remove the outgoing edges of s, such that after having iterated over all states (lines 11–

18), direct transitions from all states to the absorbing ones will remain that allow to express

the reachability properties for all states similarly as it was done for the next operator (line

19).

Once we have for all states s ∈ Sn a symbolic description of the satisfaction of ψ′ in s, we

can start to search for satisfying and violating parameter configurations using Algorithm 5.

It maintains three sets, each of which contains zero or more boxes. Boxes from Rgreen contain

only satisfying parameter configurations, boxes from Rred only unsatisfying ones, whereas

39

boxes from Rwhite are mixed and contain both configuration types. We call the boxes from

the respective sets accordingly green, red, or white.

Algorithm 5: checkParameterSpace

Input : D: PDTMC; ψ: ReachHyperPCTL formula;
I: a box of valid parameter configurations;
maxit: upper iteration limit.

Output: (Rgreen, Rwhite, Rred): three sets of boxes decomposing I, such that each

box from Rgreen, Rred resp. Rwhite contains configurations from which all,
none resp. some make D satisfy ψ.

1 Function checkParameterSpace(D, ψ, I, maxit)
2

Rgreen := ∅; Rred := ∅; Rwhite := ∅; R := {I}; l := 1;
while R ̸= ∅ do

let R ∈ R; R := R\{R};
color := checkRegion(D, ψ, R, ());
if color= green then Rgreen := Rgreen ∪ {R}
else if color= red then Rred := Rred ∪ {R}
else

if l < maxit then
S := split(R);

l += |S|; R := R ∪ S;

else Rwhite := Rwhite ∪ {R};

return (Rgreen, Rwhite, Rred);

3

4

5

6

7

8

9

10

11

12

A queue R contains at the start of the initial box. Iteratively, we take a box R from R

and determine with Algorithm 6 its colour. If the colour is green or red then we put the box

into the corresponding set Rgreen resp. Rred. Otherwise, if the colour is white then we split

R into smaller boxes which are then added to R for further processing. For the split, any

heuristics can be used, in the hope that the smaller boxes will become conclusive in their

colour. To ensure termination, after an upper limit of maxit boxes have been scheduled for

processing in R, we finish by checking the remaining boxes in the queue without splitting

and collect the inconclusive ones in Rwhite.

Finally, the last module to discuss is Algorithm 6 which determines the colour of a box R,

i.e. the truth value of ψ = Q1x1. . . . Qnxn.ψ′ under configurations from R. Let us first have a

look at the lines 11-13, where the truth value of ψ′ is checked for a fixed state (s1, . . . , sn) of

the n-ary self-composition. Here, for a box R = [l1, u1]×. . .×[ln, un] we overload notation and

40

use R also to denote its logical description (cid:86)n

i=1 li ≤ xi ∧ xi ≤ ui. Since Symb(ψ, (s1, . . . , sn))

encodes the value of ψ in (s1, . . . , sn), the formula R ∧ Symb(ψ, (s1, . . . , sn)) is true for all

configurations in R that satisfy ψ. If this formula is unsatisfiable then we know that no config-

uration in R satisfies ψ and return the colour red. In contrary, if R ∧ ¬Symb(ψ, (s1, . . . , sn))

is unsatisfiable then we know that none of the configurations in R violate ψ and the colour

of the box is green. Otherwise, if both formulas are satisfiable then some configurations in

R satisfy ψ and some others do not, therefore the colour of the box is white.

It depends on the quantifiers for which states we need to execute this check, as imple-

mented in lines 2-9. For each existential quantifier Qi = ∃ we need to find just a single

state that makes the box green in order to make the formula true, whereas for universal

quantifiers Qi = ∀ it needs to hold for each state. For the latter case it means also that if

the box is red for one state then we know that According to this, quantifiers are instantiated

from left to right, and the previously described code in lines 11-13 is applied to check the

colour of the box for the chosen n-ary state vector.

As a result of the satisfiability checks in line 11 of Algorithm 6, for purely existentially

quantified formulas we can also provide a satisfying configuration for each white box.

Given the soundness of [DJJ+15], which we use basically unchanged (with the additional

bookkeeping shown in Fig. 3.3) to compute symbolic probabilities for the states, our compu-

tations in lines 11–13 are sound for each state. Furthermore, for universal state quantifiers,

we take the weakest satisfaction result under all states, and for existential state quantifiers

the strongest one, such that the soundness of our algorithm is easy to see assuming soundness

of [DJJ+15]).

3.3 Case Studies and Evaluation

We developed a prototypical implementation of our algorithm in Python, with the help

of several libraries that facilitate the handling of complex mathematical equations involved.

There is extensive use of STORMPY [stob], which is a set of Python bindings for the prob-

41

Algorithm 6: checkRegion

Input : D = (S, V, P, AP, L): PDTMC; ψ: ReachHyperPCTL formula;

R: a box of valid parameter configurations;

(s1, . . . , si−1) ∈ Si−1.

Output: color : one of the colors green=1, white=0 or red=-1 encoding whether
ψ is satisfied by D under all, some respectively none of the configurations
in R.

1 Function checkRegion(D, ψ, R, (s1, . . . , si−1))
2

if ψ is Qixi. . . . Qnxn.ψ′ then

if Qi = ∃ then color := red else color := green;
foreach si ∈ S do

color′ := checkRegion(D, Qi+1xi+1. . . . Qnxn.ψ′, R, (s1, . . . , si−1, si));
if Qi = ∃ then color := max{color, color′};
else if Qi = ∀ then color := min{color, color′};
if (Qi = ∃ ∧ color = green) or (Qi = ∀ and color = red) then break

return color

else

if R ∧ Symb(ψ, (s1, . . . , sn)) is unsatisfiable then return red
else if R ∧ ¬Symb(ψ, (s1, . . . , sn)) is unsatisfiable then return green
else return white

3

4

5

6

7

8

9

10

11

12

13

abilistic model checker STORM [DJKV17]. It has provided an efficient solution for parsing,

building, and storage of parametric DTMC models. Internally, STORMPY uses pycarl [pyc],

the python binding of CARL, an Open Source C++ Library for Computer Arithmetic and

Logic. Several data structures and data types have been used from pycarl and STORMPY to

handle complex polynomials, equations, and rational numbers. Finally, we have used the

SMT solver Z3 [dMB08] to implement lines 11 and 12 of Algorithm 6. All of our experiments

are run on a MacBook Pro laptop with a 2.7 GHz i7 processor with 8GB of RAM. We set

maxit = 1500 in Algorithm 5, the process always the oldest inconclusive box in R (FIFO)

and split inconclusive d-dimensional white boxes into 2d new box by splitting in the middle

in each dimension. We start with two smaller examples (randomized response and prob-

abilistic conformance) and then switch to larger case studies (probabilistic noninterference

and information leakage).

42

(a) (ln 3)-differential privacy.

(b) (ln 2)-differential privacy.

Figure 3.4: Synthesized probability distribution for the randomized response protocol.

3.3.1 Randomized Response

We first synthesize configurations for the randomized response protocol described in Sec-

tion 3.1. We experimented with the following scenarios. In the first scenario, we synthesized

parameters to achieve ln 3-differential privacy with parameters p and q as shown in Fig 3.1b.

The green area in Fig. 3.4a includes the valid values (p = 0.75 and q = 0.25, or, p = q = 0.5).

The white area consists of the values that remain unknown due to the termination of the

algorithm after 1500 rounds. In the second scenario, our goal is to achieve ln 2-differential

privacy. Again, the green area in Fig. 3.4b includes the valid values (e.g., p = 2

3 and q = 0.25).

The time spent to synthesize parameters in all the above scenarios was 0.1s.

3.3.2 Probabilistic Conformance

The notion of conformance describes how well a system implements correctly a given

specification in terms of observable behaviours, or, whether two systems (e.g., a model and

an implementation) conform with each other with respect to a specification. In our setting,

we model both specifications and implementation as PDTMCs.

As an example, let us consider the specification of a protocol, where a A 6-sided die is

rolled as long as we get the number six (state d6). Figure 3.5 (left) illustrates graphically our

example. Our specification states also that the die, after behaving fairly the first time, can

43

be biased only once according to a parameter p. Now, our goal is to implement this protocol

using an adaptation of the Knuth-Yao algorithm [KY76] that was designed originally for

simulating a 6-sided die by repeatedly tossing a fair coin, illustrated in Figure 3.5 (right).

In the considered adaptation, we allow bias in the tossing of the coin according to the same

parameter p, only when the state d′

6 (this state represents the number six in the simulated

die) is encountered.

In this experiment, we are interested to find the value of p, such that the implementation

of the protocol conforms with its specifications according to the probability of terminating

in each one of the five possible states that represent the numbers of the die from one to five.

This property can be formally expressed using the following ReachHyperPCTL formula:

φpc = ∀σ.∃σ′.

(cid:16)

s0σ ∧ s′

0σ′

(cid:17)

⇒

5
(cid:94)

(cid:16)

i=1

P(

diσ ) = P(

(cid:17)
d′
iσ′ )

(3.5)

We synthesized parameter value p = 0.5, meaning that applying a fair coin ensures

conformance of the implementation (the right model in Figure 3.5) with the specification

(the left PDTMC in Figure 3.5). The synthesis for this experiment was 28s, where 27s was

spent in Algorithms 2-4 and 1s in Algorithms 5-6. The imbalance is mainly due to the fact

that the PDTMCs have multiple nested cycles. We also note that p = 0.5 is the only valid

solution.

3.3.3 Probabilistic Noninterference in Randomized Schedulers

Noninterference is an information-flow security policy that enforces that a low-privileged

user (e.g., an attacker) should not be able to distinguish two computations from their publicly

observable outputs if they only vary in their inputs by a high-privileged user (e.g., a secret).

Probabilistic noninterference [III92] establishes connection between information theory

and information flow by employing probabilities to address covert channels. Intuitively, it

requires that the probability of every low-observable trace pattern is the same for every

low-equivalent initial state. Consider the following example [Smi03] of a program with two

44

s0

1
6
1
6
1
6
1
6
1
6
1
6

1

1

1

1

1

p
3

p
3

d1

d2

d3

d4

d5

d6

p
3

1−p
3

s′
0

p

1−p
3

1−p
3

1
2

1
2

1
2

s′
1

s′
2

1 − p

1
2

1
2
1
2

1
2
1
2

s′
3

s′
4

s′
5

s′
6

1

1

1

1

1

1
2

1
2
1
2

1
2
1
2

1
2

d′
1
d′
2
d′
3

d′
4
d′
5
d′
6

Figure 3.5: Parametric die-coin using Knuth-Yao protocol.

threads th 1 and th 2:

th 1 : while h > 0 do {h := h − 1}; l := 2

||

th 2 :

l := 1

(3.6)

where h is a secret input by a high-privileged user and l is a public output observable by low-

privileged users. Figure 3.6 depicts PDTMC models of this program for secret input h = 0

(left) and h = 1 (right). The solid (resp., dotted) transitions correspond to thread th 1 (resp.,

th 2), i labels initial states, proposition, w denotes execution of the while-loop condition

checking, and f denotes the terminating state. The parameter configuration p = 0.5 resp.

q = 0.5 models a fair scheduler that chooses each of the threads with equal probability for

the execution of the next atomic statement. Probabilistic noninterference requires that l

obtains the value of 1 (and 2) with equal probability, regardless of the initial value of h:

φpni = ∀σ.∀σ′.

(cid:18)

iσ ∧ (h=0)σ ∧ iσ′ ∧ (h=1)σ′

⇒

(cid:19)

(cid:18)(cid:16)

P(cid:0)

(cid:16)

P(cid:0)

(fσ ∧ (l=1)σ)(cid:1) = P(cid:0)

(fσ ∧ (l=2)σ)(cid:1) = P(cid:0)

(fσ′ ∧ (l=1)σ′)(cid:1)(cid:17)
(fσ′ ∧ (l=2)σ′)(cid:1)(cid:17)(cid:19)

∧

(3.7)

However, when using a fair scheduler, the likely outcome of the race between the two

assignments l := 1 and l := 2 depends on the initial value of h: the larger the initial value

45

{h = 0, i}

s0

p

1 − p
s1

s2 {l = 1}

{w}
p
−

p

1
s3 {l = 2} s4 {l = 1}

{w}
1

1

1

s5

1

{w}
s′
6

q

−

1

{h = 1, i}

q

−

1

q

{w}
s′
q
1

−

1

q

{l = 1}

s′
0

q

{l = 1}
s′
4

1

{h = 0}
s′
3

q

−

1

q

{l = 1}

s′
7

1

s6 {l = 1, f }

{l = 2, f }

s7

1

1

{l = 2}

s′
10

{l = 1, f }

1

s′
9
1
s′
12

1

{l = 1}

{h = 0}

{w}

s′
2
1
s′
5
1
s′
8
1
s′
11 {w}
1
s′
13

1

{l = 2, f }

Figure 3.6: Parametric DTMC for the probabilistic noninterference example program with
two threads th 1 and th 2.

Running time (s)

Alg.2-4 Alg.5-6

Total

#white #red
boxes boxes

red #samples
area

Input
h

(0, 1)

(0, 5)

(0, 10)

2.90

100.03

102.93

15.61

55.73

143.58

159.2

259.3

315.06

(0, 15)

113.58

459.60

573.18

(1, 2)

(3, 5)

(4, 8)

8.33

114.55

122.88

31.95

72.23

204.42

236.38

397.91

470.14

(8, 14)

213.96

2924.61

3138.07

378

374

374

377

368

411

371

378

748

752

0.79

0.815

752

0.8164

749

758

715

0.711

0.706

0.831

755

0.6622

748

0.825

477

421

480

413

425

496

481

496

Table 3.1: Experimental results for thread scheduling.

of h, the greater the probability that the final value of l is 2. For example, it is easy to

recognize in Fig. 3.6 that for the secret input h = 0 the final value is l = 1 with probability

1/4 and l = 2 with probability 3/4, but for the input h = 1 the final value is l = 1 with

probability 1/16 and l = 2 with probability 15/16. Thus, it holds that for two independent

executions with initial h values 0 resp. 1 the larger h value leads to a lower probability for

l = 1 upon termination i.e., this program does not satisfy φpni.

Now, let us repair this system by allowing the scheduler to use biased coins for different

secret input values h. Table 3.1 shows experimental results for different input value combina-

46

(a) (h = 3)σ, (h = 5)σ′.

(b) (h = 0)σ, (h = 10)σ′.

(c) (h = 14)σ, (h = 8)σ′.

Figure 3.7: Synthesized probability distribution for a randomized scheduler.

tions, for which we want to determine parameters that assure probabilistic noninterference.

The table shows for each considered pair of h-values the time spent in Algorithms 1–4 and

Algorithms 5–6, the total running time, the number of returned white and red boxes (no

green boxes have been detected), the percentage of the configuration domain covered by

red boxes (i.e. provably non-satisfying area), and the number of satisfying configurations

(samples) detected. Figure 3.7 shows the 2-dimensional plot of synthesized valid values of

parameters p and q for different pair values of h. As can be seen, as the values of h converge

(e.g., in Fig. 3.7a), the probabilities of p and q also converge, as the resulting DTMC is

balanced. On the contrary, as the values of h diverge (e.g., in Fig. 3.7b), the probabilities of

p and q also diverge, as the resulting DTMC is more imbalanced.

3.3.4 Information Leakage in Dining Cryptographers

Three cryptographers gather around a table for dinner. The waiter informs them that

the meal has been paid for by someone, who could be either one of the three cryptographers

or the master. The cryptographers respect each other’s privacy but want to find out whether

the master paid. To decide this, they execute the following two-stage protocol [Cha88]:

• Each cryptographer flips a coin and informs only the cryptographer on the right about

the outcome.

• Each cryptographer who did not pay for the dinner announces whether the two coins

that it can see (the own flipped one and the one on the left-hand neighbor flipped) are

47

the same (“agree”) or different (“disagree”).

• If a cryptographer paid for dinner, then it instead states the opposite (“disagree” if

the coins are the same and “agree” if the coins are different).

A parametric DTMC model of the protocol with three cryptographers and three biased

coins (with parameters p1, p2, and p3, respectively) is illustrated in Figure 3.8. It consists of

four independent sub-PDTMCS, staying for the four cases who paid: the master (M ), the

first (C1), the second (C2) resp. third (C3) cryptographer. We depict all four PDTMCS in

one illustration as they differ only in their state labeling. The labels payM and payi, i ∈ 3

encode that the master resp. cryptographer i paid; ti resp. hi encodes tail resp. head flipped

by cryptographer i; the labels ai resp. di encode that cryptographer i announced “agree”

resp. “disagree”; finally, done stays for a terminated protocol. In the text, the above state

identifiers si are lower indexed with the cases to distinguish between si

C3.We
are interested in deciding which parts of the valid parameter domain [0, 1]3 ⊆ R3 satisfy the

C1, si

C2, si

M , si

following ReachHyperPCTL formula (⊕ denotes the exclusive-or operator):

φdc = ∀σ. ∀σ′.

(cid:95)

(cid:16)
(

i∈3
(cid:16)

P

(cid:124)

payi

σ) ∧ (

(cid:95)

i∈3

(cid:17)

payi

σ′)

⇒

(3.8)

σ ⊕ a2

σ ⊕ a3

σ))

(doneσ ∧ (a1
(cid:123)(cid:122)
F1

(cid:17)

(cid:125)

(cid:16)

= P
(cid:124)

σ′ ⊕ a2
(doneσ′ ∧ (a1
(cid:123)(cid:122)
F2

σ′ ⊕ a3

σ′))

(cid:17)

(cid:125)

In other words, if the master does not pay, then the different outcomes are observed with

the same probabilities independently of the fact which cryptographer has paid. A careful

reader recognizes that the above property holds for all parameters. Intuitively, independently

of the flip outcomes, when ordered in a circle, the number of changes in the outcomes will

be always even. Thus, the number of “agree”s will be even if and only if an even number

of cryptographers lie. Therefore, when the master paid (zero lies) we have an even number

of “agree”s, and when one of the cryptographers paid then one lies and we have an odd

number.

To check this property, we follow Algorithm 1 and call first the SymbolicEncoding method

48

p 1

all: {h1}
s1

p

2

all: {h1, h2}

s3

1

−

p
2

s0
M :{pay0}
C1:{pay1}
C2:{pay2}
C3:{pay3}

1 − p1

p

2

all: {h1, t2}
s4

all: {t1, h2}

s5

all: {t1}
s2

1

−

p
2

all: {t1, t2}
s6

1

1

1

1

p3

−

p

p3

−

p

p3

−

p

p3

−

p

3

3

3

3

s7

s8

s9

s10

s11

s12

s13

s14

all: {h1, h2, h3}

all: {h1, h2, t3}

all: {h1, t2, h3}

all: {h1, t2, t3}

all: {t1, h2, h3}

all: {t1, h2, t3}

all: {t1, t2, h3}

all: {t1, t2, t3}

1

1

1

1

1

1

1

1

s15

1

s16

1

s17

1

s18

1

s19

1

s20

1

s21

1

s22

1

M :{a1,a2,a3,done}
C1:{d1,a2,a3,done}
C2:{a1,d2,a3,done}
C3:{a1,a2,d3,done}

M :{a1,d2,d3,done}
C1:{d1,d2,d3,done}
C2:{a1,a2,d3,done}
C3:{a1,d2,a3,done}

M :{d1,d2,a3,done}
C1:{a1,d2,a3,done}
C2:{d1,a2,a3,done}
C3:{d1,d2,d3,done}

M :{d1,a2,d3,done}
C1:{a1,a2,d3,done}
C2:{d1,d2,d3,done}
C3:{d1,a2,a3,done}

M :{d1,a2,d3,done}
C1:{a1,a2,d3,done}
C2:{d1,d2,d3,done}
C3:{d1,a2,a3,done}

M :{d1,d2,a3,done}
C1:{a1,d2,a3,done}
C2:{d1,a2,a3,done}
C3:{d1,d2,d3,done}

M :{a1,d2,d3,done}
C1:{d1,d2,d3,done}
C2:{a1,a2,d3,done}
C3:{a1,d2,a3,done}

M :{a1,a2,a3,done}
C1:{d1,a2,a3,done}
C2:{a1,d2,a3,done}
C3:{a1,a2,d3,done}

Figure 3.8: A parametric DTMC model for the dining cryptographers protocol with three
cryptographers and three biased coins.

from Algorithm 2 on the 2-ary self-composition of the PDTMC in Figure 3.8. The states

of this self-composition are pairs (si

T1, sj
T2

) with i, j ∈ 22 and T1, T2 ∈ {M, C1, C2, C3}.

Note that the self-composition is synchronous, i.e., each the non-absorbing state has four

successors; for example, the state (s0

C1, s0

C2) has the successors (1) (s1

C1, s1

C2) with probability

p1 · p1, (2) (s1

C1, s2

C2) with probability p1 · (1 − p1), (3) (s2

C1, s1

C2) with probability (1 − p1) · p1,

and (4) (s2

C1, s2

C2) with probability (1 − p1) · (1 − p1).

The formula 3.8 is trivially satisfied by all states where the left-hand-side of the implica-

tion is false, i.e., the only relevant initial states (instantiating σ and σ′) are (s0

T1, s0

T2) with

T1, T2 ∈ {C1, C2, C3}. Due to the synchronous nature of the self-composition, both runs

start in σ resp. σ′ will execute the same number of steps, i.e. stay at the same “depth” in

Figure 3.8. For all such state pairs, the probability expressions F1 and F2 in Formula 3.8

both simplify to 1, therefore the equality F1 = F2 holds independently from the parameter

configuration.

Starting with the parameter domain [0, 1]3, our implementation reports that the whole

box [0, 1]3 is green, without any splits. However, the (symbolic) transition matrix is quite

large (we have 7744 states in the 2-ary self-composition and our implementation does not

detect non-reachable states), so it takes about 40 minutes running time to get this answer.

49

3.4 Summary

When designing a system, often designers have to abide by certain pre-specified require-

ments. Working backward, the randomness or bias in the system is then designed according

to the requirement it has to satisfy. This is the parameter synthesis problem. In this work,

we focused on defining and solving the parameter synthesis problem for probabilistic hyper-

properties on DTMCs in particular. We have defined the problem, the fragment of the logic

we considered for simplicity, and proposed an algorithm to solve the problem. We further

demonstrated our approach in a few interesting case studies. Although computationally

challenging, the problem is interesting and useful.

50

Chapter 4

Probabilistic Hyperproperties with
Nondeterminism

4.1

Introduction

Hyperproperties [CS10] can express probabilistic relations between multiple executions

of a system and can describe the requirements of probabilistic systems. For example, in

information-flow security, adding probabilities is motivated by establishing a connection be-

tween information theory and information flow across multiple traces. A prominent example

is probabilistic schedulers that open up an opportunity for an attacker to set up a proba-

bilistic covert channel. Or, probabilistic causation compares the probability of occurrence of

an effect between scenarios where the cause is or is not present.

The state of the art on probabilistic hyperproperties has exclusively been studied in the

context of discrete-time Markov chains (DTMCs). The temporal logic HyperPCTL [ ´AB18],

extends PCTL by allowing explicit and simultaneous quantification over computation trees.

For example, the DTMC in Fig. 4.1 satisfies the following HyperPCTL formula:

ψ = ∀ˆs.∀ˆs′.

(cid:16)

initˆs ∧ initˆs′

(cid:17)

(cid:16)

P(

⇒

aˆs) = P(

(cid:17)

aˆs′)

(4.1)

which means that the probability of reaching proposition a from any pair of states ˆs and

ˆs′ labelled by init should be equal. Other works on probabilistic hyperproperties for DTMCs

51

{init}
0.4

s0

s1

{init}
0.3

0.2

0.7

{a}

s2

0.4

s3

s4

0.8

1

s5

0.2

1

s6

{a}

1

1

Figure 4.1: Example DTMC.

include parameter synthesis [ ´ABBD20a] and statistical model checking [WZBP19, WZBP20].

An important gap in the spectrum is the verification of probabilistic hyperproperties con-

cerning models that allow nondeterminism, in particular, Markov decision processes (MDP).

Nondeterminism plays a crucial role in many probabilistic systems. For instance, nonde-

terministic queries can be exploited to make targeted attacks on databases with private

information [GMB17].

{h>0}

s0

s1

{h≤0}

β

1
2

α2
3

α

1
2

1
4

3
4

β

1
3

1
2

1
2

{l=1}

s2

τ

1

s3 {l=2}

τ

1

(a) MDP showing the actions and probabilis-
tic distributions.

β 1

α 1

s0

α 1

s1

(b) MDP requiring probabilistic schedulers.

Figure 4.2: MDPs that require different types of schedulers.

To motivate the idea, consider the MDP in Fig. 4.2a, where h is a high secret and l is

a low publicly observable variable. To protect the secret, there should be no probabilistic

dependencies between observations of l and the value of h. On one hand, an attacker that

chooses a scheduler that always takes action α from states s0 and s1 can learn whether or

not h ≤ 0 by observing the probability of obtaining l = 1 (or l = 2). On the other hand, a

scheduler that always chooses action β does not leak any information about the value of h.

Thus, a natural question to ask is whether a certain property holds for all or some schedulers.

52

With the above motivation, we focus on probabilistic hyperproperties in the context of

MDPs. Such hyperproperties inherently need to consider different nondeterministic choices

in different executions, and naturally call for quantification over schedulers. There are several

challenges to achieving this. In general, there are schedulers whose reachability probabilities

cannot be achieved by any memoryless non-probabilistic scheduler, and, hence finding a

scheduler is not reducible to checking non-probabilistic memoryless schedulers, as it is done

in PCTL model checking for MDPs. Consider for example the MDP in Fig. 4.2b, for which

we want to know whether there is a scheduler such that the probability of reaching s1

from s0 equals 0.5. There are two non-probabilistic memoryless schedulers, one choosing

action α and the other, action β in s0. The first one is the maximal scheduler for which

s1 is reached with probability 1, and the second one is the minimal scheduler leading to

probability 0. However, the probability 0.5 cannot be achieved by any non-probabilistic

scheduler. Memoryless probabilistic schedulers can neither achieve probability 0.5:

if a

memoryless scheduler would take action α with any positive probability, then the probability

to reach s1 is always 1. The only way to achieve the reachability probability 0.5 (or any

value strictly between 0 and 1) is by a probabilistic scheduler with memory, e.g., taking α

and β in s0 with probabilities 0.5 each when this is the first step on a path, and β with

probability 1 otherwise.

To this effect, we first extend the temporal logic HyperPCTL [ ´AB18] to the context of

MDPs. To this end, we augment the syntax and semantics of HyperPCTL to quantify over

schedulers and relate probabilistic computation trees for different schedulers. For example,

the following formula generalizes (4.1) by requiring that the respective property should hold

for all computation trees starting in any states ˆs and ˆs′ of the DTMC induced by any

scheduler ˆσ:

∀ˆσ.∀ˆs(ˆσ).∀ˆs′(ˆσ).

(cid:16)

init ˆs ∧ init ˆs′

(cid:17)

(cid:16)

P(

⇒

aˆs) = P(

(cid:17)

aˆs′)

(4.2)

On the negative side, we show that the problem to check HyperPCTL properties for

MDPs is in general undecidable. On the positive side, we show that the problem be-

53

comes decidable when we restrict the scheduler quantification domain to memoryless non-

probabilistic schedulers. We also show that this restricted problem is already NP-complete

(respectively, coNP-complete) in the size of the given MDP for HyperPCTL formulas with a

single existential (respectively, universal) scheduler quantifier. Subsequently, we propose

an SMT-based encoding to solve the restricted model checking problem. We have im-

plemented our method and analyse it experimentally on three case studies: probabilistic

scheduling attacks, side-channel timing attacks, and probabilistic conformance (available at

https://github.com/TART-MSU/HyperProb).

4.2 HyperPCTL for MDPs

We now describe the syntax and semantics of our extension of the temporal logic Hyper-

PCTL to the context of MDPs.

4.2.1 HyperPCTL Syntax

HyperPCTL (quantified) state formulas φq are inductively defined as follows:

quantified formula

φq

::= ∀ˆσ.φq

| ∃ˆσ.φq

| ∀ˆs(ˆσ).φq

| ∃ˆs(ˆσ).φq

| φnq

non-quantified formula φnq

::= true | aˆs | φnq ∧ φnq | ¬φnq | φpr < φpr

probability expression

φpr

::= P(φpath) | f (φpr

1 , . . . , φpr
k )

path formula

φpath

::=

φnq | φnq U φnq | φnq U [k1,k2] φnq

where ˆσ is a scheduler variable 1 from an infinite set ˆΣ, ˆs is a state variable from an infinite

set ˆS, φnq is a quantifier-free state formula, a ∈ AP is an atomic proposition, φpr is a

probability expression, f : [0, 1]k → R are k-ary elementary functions to express operations

over probabilities, arithmetic operators (binary addition, unary/binary subtraction, binary

multiplication) over probabilities, where constants are viewed as 0-ary functions, and φpath

is a path formula, such that k1 ≤ k2 ∈ N≥0. The probability operator P allows the usage of

1We use the notation ˆσ for scheduler variables and σ for schedulers, and analogously ˆs for state variables

and s for states.

54

probabilities in arithmetic constraints and relations.

A HyperPCTL construct φ (probability expression φpr, state formula φq, φnq or path

formula φpath) is well-formed if each occurrence of any aˆs with a ∈ AP and ˆs ∈ ˆS is in the
scope of a state quantifier for ˆs(ˆσ) for some ˆσ ∈ ˆΣ, and any quantifier for ˆs(ˆσ) is in the

scope of a scheduler quantifier for ˆσ. We restrict ourselves to quantifying first the schedulers

then the states, i.e., different state variables can share the same scheduler. One can consider

also local schedulers when different players cannot explicitly share the same scheduler, or in

other words, each scheduler quantifier belongs to exactly one of the quantified states.

HyperPCTL formulas are well-formed HyperPCTL state formulas, where we additionally

allow standard syntactic sugar like false = ¬true, φ1∨φ2 = ¬(¬φ1∧¬φ2),

φ = true U φ,

and P( φ) = 1 − P( ¬φ). For example, the HyperPCTL state formula ∀ˆσ.∃ˆs(ˆσ).P( aˆs) <

0.5 is a HyperPCTL formula. The HyperPCTL state formula P( aˆs)<0.5 is not a HyperPCTL

formula, but can be extended to such. The HyperPCTL state formula ∀ˆs(ˆσ).∃ˆσ.P( aˆs)<0.5

is not a HyperPCTL formula, and it even cannot can be extended to such.

4.2.2 HyperPCTL Semantics

Definition 4.2.1. The n-ary self-composition of an MDP M = (S, Act, P, AP, L) for a
sequence σ = (σ1, . . . , σn) ∈ (ΣM)n of schedulers for M is the DTMC parallel composition
Mσ = Mσ1
is the DTMC induced by Mi and σi, and where
Mi = (S, Act, P, APi, Li) with APi = {ai | a ∈ AP} and Li(s) = {ai | a ∈ L(s)}, for all
■
s ∈ S.

n , where Mσi

1 × . . . × Mσn

i

HyperPCTL state formulas are evaluated in the context of an MDP M = (S, Act, P, AP, L),

a sequence σ = (σ1, . . . , σn) ∈ (ΣM)n of schedulers, and a sequence ⃗r = ((q1, s1), . . . , (qn, sn))

of Mσ states; we use () to denote the empty sequence (of any type) and ◦ for concatena-

tion. Intuitively, these sequences store instantiations for scheduler and state variables. The

satisfaction of a HyperPCTL quantified formula by M is defined by

M |= φ

iff M, (), () |= φ .

55

The semantics evaluates HyperPCTL formulas by structural recursion. Let in the following

Q, Q′, . . . denote quantifiers from {∀, ∃}. When instantiating Qˆσ.φ by a scheduler σ ∈ ΣM,

we replace in φ each sub-formula Q′ˆs(ˆσ).φ′, that is not in the scope of a quantifier for ˆσ by

Q′ˆs(σ).φ′, and denote the result by φ[ˆσ ⇝ σ]. For instantiating a state quantifier Qˆs(σ).φ

by a state s, we append σ = (Q, act, mode, init) and (init(s), s) at the end of the respective

sequences, and replace each aˆs in the scope of the given quantifier by as, resulting in a

formula that we denote by φ[ˆs ⇝ s]. To evaluate probability expressions, we use the n-ary

self-composition of the MDP.

Formally, the semantics judgment rules are as follows:

M, σ, ⃗r |= true

M, σ, ⃗r |= ai

iff ai ∈ Lσ(⃗r)

M, σ, ⃗r |= φ1 ∧ φ2

iff M, σ, ⃗r |= φ1 and M, σ, ⃗r |= φ2

M, σ, ⃗r |= ¬φ

iff M, σ, ⃗r ̸|= φ

M, σ, ⃗r |= ∀ˆσ.φ

iff ∀σ ∈ ΣM. M, σ, ⃗r |= φ[ˆσ ⇝ σ]

M, σ, ⃗r |= ∃ˆσ.φ

iff ∃σ ∈ ΣM. M, σ, ⃗r |= φ[ˆσ ⇝ σ]

M, σ, ⃗r |= ∀ˆs(σ).φ

iff ∀sn+1 ∈ S. M, σ ◦ σ, ⃗r ◦ (init(sn+1), sn+1) |= φ[ˆs ⇝ sn+1]

M, σ, ⃗r |= ∃ˆs(σ).φ

iff ∃sn+1 ∈ S. M, σ ◦ σ, ⃗r ◦ (init(sn+1), sn+1) |= φ[ˆs ⇝ sn+1]

M, σ, ⃗r |= φpr

1 < φpr

2

P(φpath)
M,σ,⃗r
(cid:75)
M,σ,⃗r = f (cid:0)
1 , . . . φpr
k )
(cid:75)

f (φpr

(cid:74)

(cid:74)

φpr
2
(cid:74)

φpr
1
(cid:74)

(cid:75)

M,σ,⃗r

M,σ,⃗r <

iff
= Pr Mσ (cid:0){π ∈ Paths⃗r(Mσ) | M, σ, π |= φpath}(cid:1)
φpr
k
(cid:74)

M,σ,⃗r . . . ,

φpr
1
(cid:74)

M,σ,⃗r

(cid:1)

(cid:75)

(cid:75)

(cid:75)

where M is an MDP; n ∈ N≥0 is non-negative integer; σ ∈ (ΣM)n; ⃗r is a state of Mσ;

a ∈ AP is an atomic proposition and i ∈ {1, . . . , n}; φ, φ1, φ2 are HyperPCTL state formulas;

σ = (Q, act, mode, init) ∈ ΣM is a scheduler for M; φpr

1 · · · φpr

k are probability expressions,

56

and φpath is a HyperPCTL path formula whose satisfaction relation is as follows:

M, σ, π |= φ

iff M, σ, ⃗r1 |= φ

M, σ, π |= φ1Uφ2

iff ∃j ≥ 0.

(cid:16)

M, σ, ⃗rj |= φ2 ∧ ∀i ∈ [0, j). M, σ, ⃗ri |= φ1

(cid:17)

M, σ, π |= φ1U [k1,k2] φ2 iff ∃j ∈ [k1, k2].

(cid:16)

M, σ, ⃗rj |= φ2 ∧ ∀i ∈ [0, j).M, σ, ⃗ri |= φ1

(cid:17)

where π = ⃗r0⃗r1 · · · with ⃗ri = ((qi,1, si,1), . . . , (qi,n, si,n)) is a path of Mσ; formulas φ, φ1, and

φ2 are HyperPCTL state formulas, and k1 ≤ k2 ∈ N≥0.

4.3 The Expressiveness Power of HyperPCTL

The standard PCTL semantics define that to satisfy a PCTL formula P∼c(φ) in a given

MDP state s, all schedulers should induce a DTMC that satisfies P∼c(φ) in s. Though it

should hold for all schedulers, it is known that there exist minimal and maximal schedulers

that are non-probabilistic and memoryless, therefore it is sufficient to restrict the reasoning

to such schedulers. Since for MDPs with finite state and action spaces, the number of such

schedulers is finite, PCTL model checking for MDPs is decidable. Given this analogy, one

would expect that HyperPCTL model checking should be decidable, but it is not.

Theorem 4.3.1. HyperPCTL model checking for MDPs is in general undecidable.

Before we prove the above theorem, let us explore shortly, the source of increased ex-

pressiveness with respect to PCTL that makes HyperPCTL undecidable. State quantification

cannot be the source, as the state space is finite and thus, there are finitely many possible

state quantifier instantiations.

Assume an MDP M = (S, Act, Pr , AP, L) with a state s ∈ S that is uniquely labeled by

the proposition init ∈ L(s), and let a, b ∈ AP. In PCTL, each probability bound needs to

be satisfied under all schedulers. For example:

M, s |=PCTL

P<0.5(aUb)

⇔ M |=HyperPCTL ∀ˆσ( ˆM).∀ˆs( ˆMˆσ).(initˆs → P(aˆs U bˆs) < 0.5)

57

Alternatively, we can state:

M, s

|=PCTL

P<0.5(a U b)

⇔ ∀σ∈ΣM. 1, Mσ[1], ((init σ(s), s))

|=HyperPCTL

P(a1 U b1)<0.5

where init σ(s) is the starting mode of scheduler σ in state s. Generally, the HyperPCTL frag-

ment which starts with a single universal scheduler quantifier and contains a single bound

on a single probability operator is still decidable. However, when a PCTL formula has sev-

eral probability bounds, its satisfaction requires each bound to be satisfied by all schedulers
(cid:16)

(cid:17)
P<0.5(a Ub) ∨ P>0.5(aUb)

is equivalent to

independently. For example, M, s |=PCTL

M |=HyperPCTL ∀ˆσ.∀ˆs(ˆσ).(initˆs → P(aˆs Ubˆs) < 0.5) or
M |=HyperPCTL ∀ˆσ.∀ˆs(ˆσ).(initˆs → P(aˆsUbˆs) > 0.5)

but not equivalent to the HyperPCTL formula

M |=HyperPCTL ∀ˆσ.∀ˆs(ˆσ).(initˆs → (P(aˆsUbˆs) < 0.5 ∨ P(aˆsUbˆs) > 0.5))

which states that the probability is either less than or larger than 0.5 under all sched-

ulers, which is true if there exists no scheduler under which the probability is 0.5 (see also

[BBGK12]). Thus, even for a fragment restricted to universal scheduler quantification, com-

binations of probability bounds allows HyperPCTL to express existential scheduler synthesis

problems.

Finally, consider a scheduler quantifier followed by state quantifiers, whose scope may

contain probability expressions. This means that we start several “experiments” in parallel,

each one represented by a state quantifier. However, we may use in all experiments the same

scheduler. Informally, this allows us to express the existence or absence of schedulers with

certain probabilistic hyperproperties for the induced DTMCs. It would however also make

sense to flip this quantifier order, such that state quantifiers are followed by scheduler quan-

tifiers. This would mean, that we can use different schedulers in the different concurrently

running experiments. This would be meaningful e.g., when users can provide input to the

system, i.e., when the scheduler choice lies with the “observers” of the individual experi-

ments, and they can adapt their schedulers to observations made in the other concurrently

58

running experiments.

Proof of Theorem 4.3.1 —

To prove Theorem 4.3.1, we reduce the emptiness problem in probabilistic B¨uchi automata

(PBA), which is known to be undecidable [BBG08], to our problem. PBA can be viewed as a

nondeterministic B¨uchi automaton where the nondeterminism is resolved by a probabilistic

choice. That is, for any state q and letter a in alphabet Σ, either q does not have any

a-successor or there is a probability distribution for the a-successors of q.

Definition 4.3.1. A probabilistic B¨uchi automaton (PBA) over a finite alphabet Σ is a tuple
P = (Q, δ, Σ, F ), where,

• Q is a finite state space,

• δ : Q × Σ × Q → [0, 1] is the transition probability function, such that for all q ∈ Q

and a ∈ Σ: (cid:80)

q′∈Q(q, a, q′) ∈ {0, 1} for all q ∈ Q and a ∈ Σ,

• F ⊆ Q is the set of accepting states.

A run for an infinite word w = a1a2 · · · ∈ Σω is an infinite sequence π = q0q1q2 · · · of

states in Q, such that qi+1 ∈ δ(qi, ai+1) = {q′ | δ(qi, ai+1, q′) > 0} for all i ∈ N≥0. Let Inf(π)

denote the set of states that are visited infinitely often in π. Run π is called accepting if

Inf(π) ∩ F ̸= ∅. Given an infinite input word w ∈ Σω, the behaviour of P is given by the

infinite Markov chain that is obtained by unfolding P into a tree using w. This is similar

to an induced Markov chain from an MDP by a scheduler. Hence, standard concepts for

Markov chains can be applied to define the acceptance probability of w in P, denoted by

Pr P(w) or briefly Pr (w), by the probability measure of the set of accepting runs for w in P.

We define the accepted language of P as: L(P) = {w ∈ Σω | Pr P(w) > 0}. The emptiness

problem is to decide whether or not L(P)=∅ for a given input P.

Mapping Our idea of mapping the emptiness problem in PBA to HyperPCTL model

checking for MDPs is as follows. We map a PBA to an MDP such that the words of the

PBA are mimicked by the runs of the MDP. In other words, the letters of the words in the

59

PBA appear as propositions on states of the MDP. This way, the existence of a word in the

language of the PBA corresponds to the existence of a scheduler that produces a satisfying

computation tree in the induced Markov chain of the MDP.

MDP model: Let P = (Q, δ, Σ, F ) be a PBA with alphabet Σ. We obtain an MDP

M = (S, Act, P, AP, L) as follows:

• The set of states is S = Q × Σ.

• The set of actions is Act = Σ.

• The transition probability function P is defined as follows:

(cid:16)

P

(q, a), b, (q′, a′)

(cid:17)

=





δ(q, b, q′)

if a′ = b

0

otherwise

• The set of atomic propositions is AP = Σ ∪ {f }, where f ̸∈ Σ (we use f to label the

accepting states).

• The labeling function L is defined for each a ∈ Σ and q ∈ Q as follows:

L(q, a) =






{a, f }

if q ∈ F

{a}

otherwise

HyperPCTL formula: The HyperPCTL formula in our mapping is

φmap = ∃ˆσ( ˆM).∃ˆs( ˆMˆσ).∀ˆs′( ˆMˆσ).

(cid:18)

P(cid:0) (cid:94)

(cid:19)
(aˆs ↔ aˆs′)(cid:1) = 1

∧

(cid:18)

P

(cid:16)

a∈AP\{f }

P(cid:0) P(

(cid:17)
fˆs) = 1(cid:1) = 1

(cid:19)

> 0

(4.3)

Intuitively, the above formula establishes the connection between the PBA emptiness prob-

lem and HyperPCTL model checking problem for MDPs. In particular:

60

• The existence of a scheduler ˆσ(M) in φmap corresponds to the existence of a word w

in L(P);

• the state quantifiers and the left conjunct ensure that the path in the induced Markov

chain and the PBA follow the sequence of actions (respectively, letters) in the witness

to ˆσ(M) (respectively, w), and

• the right conjunct mimics that a state in F is visited with non-zero probability if and

only if a state labeled by proposition f is visited infinitely often in the MDP with

non-zero probability.

Reduction We now show that L(P) ̸= ∅ if and only if M |= φmap. We distinguish two

cases:

(→) Suppose we have L(P) ̸= ∅. This means there exists a word w ∈ Σω, such that

Pr P(w) > 0. We use w to eliminate the existential scheduler quantifier and instantiate
ˆσ( ˆM) in formula φmap. This induces a DTMC and now, we show that the induced

DTMC satisfies the following HyperPCTL formula:

∃ˆs( ˆMˆσ).∀ˆs′( ˆMˆσ).

(cid:18)

P(cid:0) (cid:94)

(cid:19)
(aˆs ↔ aˆs′)(cid:1) = 1

∧

(cid:18)

(cid:16)

P

a∈AP\{f }

P(cid:0) P(

(cid:17)
fˆs) = 1(cid:1) = 1

(cid:19)

> 0

(4.4)

To this end, observe that the right conjunct is trivially satisfied because Pr P(w) > 0.

That is since a state in F is visited infinitely often with non-zero probability in P,

a state labeled by f in M is also visited infinitely often with non-zero probability.

The left conjunct is also satisfied by the construction of the mapped MDP since the

sequence of letters in w appear in all paths of the induced DTMC as propositions.

(←) The reverse direction is pretty similar. Since the answer to the model checking problem

is affirmative, a witness to scheduler quantifier ˆσ exists. This scheduler induces a

61

DTMC whose paths follow the same sequence of propositions. This sequence indeed

provides us with the word w for P. Finally, since the right conjunct in φmap is satisfied

by the MDP, we are guaranteed that w reaches an accepting state in F infinitely often

with non-zero probability.

This concludes the proof.

4.4 Applications of HyperPCTL on MDPs

Side-channel timing leaks open a channel to an attacker to infer the value of a secret by

observing the execution time of a function. For example, the heart of the RSA public-key

encryption algorithm is the modular exponentiation algorithm that computes (ab mod n),

where a is an integer representing the plain text and b is the integer encryption key.

1 v o i d mexp ( ) {

2

3

4

5

6

7

8

c = 0 ; d = 1 ;
w h i l e ( i >= 0 ) {

i = k ;

i = i −1; c = c ∗ 2 ;
d = ( d∗d ) % n ;
( b ( i ) = 1 )
i f

c = c +1;
d = ( d∗ a ) % n ;

}

9
10 }
11 /∗∗∗∗∗∗∗∗∗∗∗∗/
12 t = new Thread ( mexp ( ) ) ;
13 j = 0 ; m = 2 ∗ k ;
14 w h i l e ( j < m & ! t . s t o p )
15 /∗∗∗∗∗∗∗∗∗∗∗∗/

j ++;

Figure 4.3: Modular exponentiation.

A careless implementation can leak b through a probabilistic scheduling channel (see

Fig. 4.3 on the left). This program is not secure since the two branches of the if have

different timing behaviours. Under a fair execution scheduler for parallel threads, an attacker

thread can infer the value of b by running in parallel to a modular exponentiation thread and

iteratively incrementing a countervariable until the other thread terminates (lines 12-14).

62

1 i n t

s t r c m p ( c h a r ∗ r ) {
c h a r ∗ s = ’ Bg\ $4 \ 0 ’ ;
i = 0 ;
w h i l e ( s [ i ]

!= ’ \ 0 ’ ) {

2

3

4

5

6

7

8
9 }

( s [ i ] ! = r [ i ] )

i ++;
i f
}
r e t u r n 1 ;

r e t u r n 0 ;

Figure 4.4: String comparison.

To model this program by an MDP, we can use two nondeterministic actions for the two

branches of the if statement, such that the choice of different schedulers corresponds to the

choice of different bit configurations b(i) for the key b. This algorithm should satisfy the

following property: the probability of observing a concrete value in the counter j should be

independent of the bit configuration of the secret key b:

∀ˆσ1.∀ˆσ2.∀ˆs(ˆσ1).∀ˆs′(ˆσ2).

(cid:16)

initˆs ∧ initˆs′

(cid:17)

⇒

P(

(j = l)ˆs) = P(

(cid:17)

(j = l)ˆs′)

(4.5)

m
(cid:94)

(cid:16)

l=0

Another example of timing attacks that can be implemented through a probabilistic

scheduling side channel is password verification which is typically implemented by comparing

an input string with another confidential string (see Fig 4.4).

Also here, an attacker thread can measure the time necessary to break the loop, and use

this information to infer the prefix of the input string matching the secret string.

Scheduler-specific observational determinism policy (SSODP) [NSH13] is a confi-

dentiality policy in multi-threaded programs that defend against an attacker that chooses an

appropriate scheduler to control the set of possible traces. In particular, given any scheduler

and two initial states that are indistinguishable with respect to a secret input (i.e., low-

equivalent), any two executions from these two states should terminate in low-equivalent

63

states with equal probability. Formally, given a proposition h representing a secret:

∀ˆσ.∀ˆs(ˆσ).∀ˆs′(ˆσ).(cid:0)hˆs ⊕ hˆs′

(cid:1) ⇒

(cid:94)

(cid:0)P(

l∈L

lˆs) = P(

lˆs′)(cid:1)

(4.6)

where l ∈ L are atomic propositions that classify low-equivalent states and ⊕ is the exclusive-

or operator. A stronger variation of this policy is that the executions are stepwise low-

equivalent:

∀ˆσ.∀ˆs(ˆσ).∀ˆs′(ˆσ).(cid:0)hˆs ⊕ hˆs′

(cid:1) ⇒ P (cid:0) (cid:94)

(cid:0)(P lˆs) = (P lˆs′)(cid:1)(cid:1) = 1.

(4.7)

l∈L

Probabilistic conformance describes how well a model and an implementation conforms

with each other with respect to a specification. As an example, consider a 6-sided die. The

probability to obtain one possible side of the die is 1/6. We would like to synthesize a pro-

tocol that simulates the 6-sided die behaviour only by repeatedly tossing a fair coin. We

know that such an implementation exists [KY76], but we aim to find such a solution auto-

matically by modeling the die as a DTMC and by using an MDP to model all the possible

coin-implementations with a given maximum number of states, including 6 absorbing final

states to model the outcomes. In the MDP, we associate to each state a set of possible non-

deterministic actions, each of them choosing two states as successors with equal probability

1/2. Then, each scheduler corresponds to a particular implementation. Our goal is to check

whether there exists a scheduler that induces a DTMC over the MDP, such that repeatedly

tossing a coin simulates die-rolling with equal probabilities for the different outcomes:

∃ˆσ.∀ˆs(ˆσ).∃ˆs′(ˆσ).

(cid:16)

initˆs ∧ initˆs′

(cid:17)

⇒

6
(cid:94)

(cid:16)

l=1

P(

(die = l)ˆs) = P(

(die = l)ˆs′)

(cid:17)

(4.8)

4.5 Summary

In this chapter, we have elaborated on the extension of syntax and semantics needed

to accommodate non-determinism in HyperPCTL. We have proved that the model checking

problem for MDPs, in general, is undecidable, owing to the extensive set of possible schedulers

64

that can be used to resolve non-determinism in the system. Next, we will focus on a restricted

segment of this logic.

65

Chapter 5

Decidable Fragment of Probabilistic
Hyperproperties with
Nondeterminism

5.1

Introduction

Following from the previous chapter, due to the undecidability of HyperPCTL formulas for

MDPs, we focus in this chapter on the restricted semantics of HyperPCTL, where scheduler

quantification ranges over non-probabilistic memoryless schedulers only.

It is easy to see

that limiting ourselves to non-probabilistic memoryless schedulers makes the model checking

problem decidable, as there are only finitely many such schedulers. Regarding complexity,

we have the following result.

Theorem 5.1.1. The problem to decide for MDPs the truth of HyperPCTL formu-
las with a single existential (respectively, universal) scheduler quantifier over non-
probabilistic memoryless schedulers is NP-complete (respectively, coNP-complete) in the
state set size of the given MDP.

5.2 Proof of decidability of the restricted fragment

In order to show membership to NP, let M be an MDP and φ = ∃ˆσ( ˆM).φ′ be a Hy-

perPCTL formula, where φ′ is a state quantified formula. We show that given a solution

66

to the problem, we can verify the solution in polynomial time. Observe that given a non-

probabilistic memoryless scheduler as a witness to the existential quantifier ∃ˆσ( ˆM), one can

compute the induced DTMC and then verify the DTMC against the resulting HyperPCTL

formula in polynomial time in the size of the induced DTMC [ ´AB18].

Inspired by the proof technique introduced in [BF18], for the lower bound, we reduce the

SAT problem to our model checking problem. The SAT problem is as follows:

Let y = y1 ∧ y2 ∧ · · · ∧ ym be a Boolean formula where each yj, for j ∈ [1, m],

is a disjunction of at least three literals using propositions {x1, x2, . . . , xn}. Is y

satisfiable, i.e., is there an assignment of truth values to x1, x2, . . . , xn, such that

y evaluates to true?

Mapping We now present a mapping from an arbitrary SAT problem instance to

the model checking problem of an MDP and a HyperPCTL formula of

the form

∃ˆσ( ˆM). ∃ˆs( ˆMˆσ). ∀ˆs′( ˆMˆσ). φ. Then, we show that the MDP satisfies this formula if and

only if the answer to the SAT problem is affirmative. Fig. 5.1 shows an example.

MDP: For a given propositional logic formula in conjunctive normal form, we define the

MDP M = (S, Act, P, AP, L) as follows.

• (Atomic propositions AP) We include four atomic propositions: p and ¯p to mark the

positive and negative literals in each clause and c and ¯c to mark paths that correspond
to clauses of the SAT formula. Thus, AP = (cid:8)p, ¯p, c, ¯c}.

• (Set of states S) We now identify the members of S:

– For each clause yj, where j ∈ [1, m], we include a state rj, labeled by proposition

c. We also include a state r0 labeled by ¯c.

– For each clause yj, j ∈ [1, m], we introduce the following n states:

(cid:110)

vj
i | i ∈ [1, n]

(cid:111)
.

67

Figure 5.1: Example of mapping SAT to HyperPCTL model checking.

Each state vj

i is labelled with proposition p if xi is a literal in yj, or with ¯p if ¬xi

is a literal in yj.

– For each Boolean variable xi, where i ∈ [1, n], we include two states: a state si

labelled with p and a state ¯si labelled with ¯p.

• (Set of actions Act) The set of actions is Act = {α, β, γ}. Intuitively, the scheduler

chooses action α (respectively, β) at a state si or ¯si to assign true (respectively, false)

to variable xi+1. Action γ is the sole action available at all other states.

• (Transition probability function P ) We now identify the members of P . All transitions

have probability 1, so we only discuss the actions.

– We add transitions (rj, γ, vj

1) for each j ∈ [1, m], where from rj, the probability

of reaching vj

1 is 1.

68

⑥⑥⑥⑥②✹❂②✸❂✭✿①✶❴①✷❴✿①✹✮✭✿①✸❴①✹❴✿①✺✮✭①✶❴①✹❴①✺✮✭①✶❴✿①✷❴①✸✮❢❝❣❢❝❣❢❝❣❢❝❣❢♣❣❢✖♣❣❢✖♣❣❢♣❣❢♣❣❢✖♣❣❢♣❣❢♣❣❢♣❣❢❣❢❣✈✶✶✈✷✶✈✸✶✈✹✶✈✶✷✈✷✷✈✸✷✈✹✷✈✶✹✈✶✺✈✷✺✈✷✹✈✸✸✈✹✺✈✹✸✈✷✸✈✸✹✈✶✸✈✸✺✈✹✹s✷❢✖♣❣❢✖♣❣❢♣❣s✶s✸s✹s✺✖s✶✖s✷✖s✸✖s✹✖s✺☞❀(cid:0)✁tr❛✂✄☎✁✵tr❛✂✄☎✆✝✆✶✆✷✆✸✆✹☞❀(cid:0)☞❀(cid:0)☞❀(cid:0)☞❀(cid:0)☞❀(cid:0)☞❀(cid:0)☞❀(cid:0)☞❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)❢✖♣❣❢♣❣❢✖♣❣❢❣❢✖♣❣❢❣❢❣❢❣❢✖♣❣❢♣❣☛❀(cid:0)②✷❂②✶❂❢♣❣❢❣❢❣❢♣❣❢♣❣❢✖❝❣✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)✌❀(cid:0)☛❀(cid:0)☛❀(cid:0)☛❀(cid:0)☛❀(cid:0)☛❀(cid:0)☛❀(cid:0)☛❀(cid:0)☛❀(cid:0)❢✖♣❣– We also add transitions (vj

i , γ, vj

i+1) for each i ∈ [1, n), connecting the states

representing literals in each clause yj, j ∈ [1, m].

– For each i ∈ [1, n), we include four transitions (si, α, si+1), (si, β, ¯si+1), (¯si, α, si+1),

and (¯si, β, ¯si+1). The intuition here is that when the scheduler chooses action α

at state si or ¯si, variable xi+1 evaluates to true and when the scheduler chooses

action β at state si or ¯si, variable xi+1 evaluates to false in the SAT instance.

We also include two transitions (r0, α, s1) and (r0, β, ¯s1) with the same intended

meaning.

– Finally, we include self-loops (sn, γ, sn), (¯sn, γ, ¯sn), and (vj

n, γ, vj

n), for each j ∈

[1, m].

HyperPCTL formula: The HyperPCTL formula in our mapping is:

φmap = ∃ˆσ( ˆM).∃ˆs( ˆMˆσ).∀ˆs′( ˆMˆσ). ¯cˆs ∧

(cid:18)
cˆs′ → P

(cid:18) (cid:16)

(pˆs ∧ pˆs′) ∨ (¯pˆs ∧ ¯pˆs′)

=1

(5.1)

(cid:17)(cid:19)

(cid:19)

The intended meaning of the formula is that if there exists a scheduler that makes the

formula true by choosing the α and β actions, this scheduler gives us the assignment to the

Boolean variables in the SAT instance. This is achieved by making all clauses true, hence,

the ∀ˆs′( ˆMˆσ) sub-formula.

Reduction We now show that the given SAT formula is satisfiable if and only if the MDP

obtained by our mapping satisfies the HyperPCTL formula φmap.

(→) Suppose that y is satisfiable. Then, there is an assignment that makes each clause yj,

where j ∈ [1, m], true. We now use this assignment to instantiate a scheduler for the

formula φmap. If xi = true, then we instantiate scheduler ˆσ such that in state si−1

or ¯si−1, it chooses action α. Likewise, if xi = false, then we instantiate scheduler ˆσ,

such that in state si−1 or ¯si−1, it chooses action β. We now show that this scheduler
instantiation evaluates formula φmap to true. First observe that ˆs( ˆMˆσ) can only be
instantiated with state r0 and ˆs′( ˆMˆσ) can only be instantiated with states rj, where

69

j ∈ [1, m]. Otherwise, the left side of the implication in φmap becomes false, making

the formula vacuously true. Since each yj is true, there is at least one literal in yj that

is true. If this literal is of the form xi, then we have xi = true and the path that

starts from r0 will include si, which is labelled by p. Hence, the values of p, in both
paths that start from ˆs( ˆMˆσ) and ˆs′( ˆMˆσ) are eventually equal. If the literal in yj is of
the form ¬xi, then xi = false and the path that starts from ˆs( ˆMˆσ) will include ¯si.

Again, the values of ¯p are eventually equal. Finally, since all clauses are true, all paths

that start from ˆs′( ˆMˆσ) reach a state where the right side of the implication becomes

true.

(←) Suppose our mapped MDP satisfies formula φmap. This means that there exists a

scheduler and state ˆs( ˆMˆσ) that makes the sub-formula ∀ˆs′( ˆMˆσ) true, i.e., since ˆs can

uniquely be instantiated by r0 due to its labeling by c, the path that starts from r0

results in making the inner PCTL formula true for all paths that start from rj, where

1 ≤ j ≤ m, as the left of the implication is false for all other states. We obtain the truth

assignment to the SAT problem as follows. If the scheduler chooses action α to state

si, then we assign xi = true. Likewise, if the scheduler chooses action β to state ¯si,

then we assign xi = false. Observe that since in no state p and ¯p are simultaneously

true and no path includes both si and ¯si, variable xi will have only one truth value.

Similar to the forward direction, it is straightforward to see that this valuation makes

every clause yj of the SAT instance true.

5.3 Model Checking for Non-probabilistic Memoryless

Schedulers

Due to the undecidability of model checking HyperPCTL formulas for MDPs, we now re-

strict the semantics, where scheduler quantification ranges over non-probabilistic memoryless

schedulers only. It is easy to see that this restriction makes the model checking problem de-

70

Algorithm 7: Main SMT encoding algorithm
Input : M = (S, Act, P, AP, L): MDP;

Qˆσ.Q1ˆs1(ˆσ). . . . Qnˆsn(ˆσ).φnq: HyperPCTL formula.

Output: Whether M satisfies the input formula.
1 Function Main(M, Qˆσ. Q1ˆs1(ˆσ). . . . Qnˆsn(ˆσ). φnq)
2

α∈Act(s) σs = α) // scheduler choice

E := (cid:86)
s∈S((cid:87)
if Q is existential then

E := E∧ Semantics(M, φnq, n)
E := E∧ Truth(M, ∃ˆσ. Q1ˆs1(ˆσ). . . . Qnˆsn(ˆσ). φnq)
if check(E) = SAT then return TRUE
else return FALSE
else if Q is universal then

// Qi is ∀ if Qi = ∃ and ∃ else
E := E∧ Semantics(M, ¬φnq, n)
E := E∧ Truth(M, ∃ˆσ.Q1ˆs1(ˆσ). . . . Qnˆsn(ˆσ).¬φnq)
if check(E) = SAT then return FALSE
else return TRUE

3

4

5

6

7

8

9

10

11

12

cidable, as there are only finitely many such schedulers that can be enumerated. Regarding

complexity, we have the following property.

Theorem 5.3.1. The problem to decide for MDPs the truth of HyperPCTL formu-
las with a single existential (respectively, universal) scheduler quantifier over non-
probabilistic memoryless schedulers is NP-complete (respectively, coNP-complete) in the
state set size of the given MDP.

Next, we propose an SMT-based technique for solving the model checking problem for

non-probabilistic memoryless scheduler domains, and for the simplified case of having a

single scheduler quantifier; the general case for an arbitrary number of scheduler quantifiers

is similar, but a bit more involved, so the simplified setting might be more suitable for

understanding the basic ideas.

The main method listed in the Algorithm 7 constructs a formula E that is satisfiable if

and only if the input MDP M satisfies the input HyperPCTL formula with a single scheduler

quantifier over the non-probabilistic memoryless scheduler domain. Let us first deal with the

case that the scheduler quantifier is existential. In line 2 we encode possible instantiations

71

σ for the scheduler variable ˆσ, for which we use a variable σs for each MDP state s ∈ S

to encode which action is chosen in that state.

In line 4 we encode the meaning of the

quantifier-free inner part φnq of the input formula, whereas line 5 encodes the meaning of

the state quantifiers, i.e.

for which sets of composed states φnq needs to hold in order

to satisfy the input formula.

In lines 6–7 we check the satisfiability of the encoding and

return the corresponding answer. Formulas with a universal scheduler quantifier ∀ˆσ.φ are

semantically equivalent to ¬∃ˆσ.¬φ. We make use of this fact in lines 8–12 to check first the

satisfaction of encoding for ∃ˆσ.¬φ and return the inverted answer.

The Semantics method, shown in Algorithm 8, applies structural recursion to encode the

meaning of its quantifier-free input formula. As variables, the encoding uses (1) propositions

probs,φnq ∈ {0, 1} holdss,φnq ∈ {true, false} to encode the truth of each Boolean sub-formula

φnq of the input formula in each state s ∈ S n of the n-ary self-composition of M, (2) numeric

variables probs,φpr ∈ [0, 1] ⊆ R to encode the value of each probability expression φpr in the

input formula in the context of each composed state s ∈ S n, (3) variables holdsToInts,φpr ∈

{0, 1} to encode truth values in a pseudo-Boolean form, i.e. we set holdsToInts,φpr = 1 for

holdss,φnq = true and probs,φpr = 0 else and (4) variables ds,φ to encode the existence of a

loop-free path from state s to a state satisfying φ.

There are two base cases: the Boolean constant true holds in all states (line 2), whereas

atomic propositions hold in exactly those states that are labeled by them (line 3). For

conjunction (line 5) we recursively encode the truth values of the operands and state that

the conjunction is true if and only if both operands are true. For negation (line 8) we again

encode the meaning of the operand recursively and flip its truth value. For the comparison

of two probability expressions (line 10), we recursively encode the probability values of

the operands and state the respective relation between them for the satisfaction of the

comparison.

The remaining cases encode the semantics of probability expressions. The cases for con-

stants (line 22) and arithmetic operations (line 23) are straightforward. For the probability

72

Algorithm 8: SMT encoding for the meaning of the input formula

Input : M = (S, Act, P, AP, L): MDP;

φ: quantifier-free HyperPCTL formula or expression;
n: number of state variables in φ.

Output: SMT encoding of the meaning of φ in the n-ary self-composition of M.

1 Function Semantics(M, φ, n)
if φ is true then E := (cid:86)
2
else if φ is aˆsi then

3

s∈Sn holdss,φ

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

E := ((cid:86)

s∈Sn, a∈L(si)(holdss,φ)) ∧ ((cid:86)

s∈Sn, a /∈L(si)(¬holdss,φ));

else if φ is φ1 ∧ φ2 then

E := Semantics(M, φ1, n)∧ Semantics(M, φ2, n)∧
(cid:86)

s∈Sn((holdss,φ∧holdss,φ1∧holdss,φ2) ∨ (¬holdss,φ∧(¬holdss,φ1∨¬holdss,φ2)))

else if φ is ¬φ′ then

E := Semantics(M, φ′, n) ∧ (cid:86)

else if φ is φ1 < φ2 then

s∈Sn(holdss,φ ⊕ holdss,φ′)

E := Semantics(M, φ1, n)∧ Semantics(M, φ2, n)∧

(cid:86)

s∈Sn((holdss,φ ∧ probs,φ1 < probs,φ2) ∨ (¬holdss,φ ∧ probs,φ1 ≥ probs,φ2))

else if φ is P( φ′) then

E := Semantics(M, φ′, n)∧

(cid:86)

s∈Sn

(cid:0)(holdsToInts,φ′ = 1 ∧ holdss,φ′) ∨ (holdsToInts,φ′ = 0 ∧ ¬holdss,φ′)(cid:1)

foreach s = (s1, . . . , sn) ∈ S n do

foreach ⃗α = (α1, . . . , αn) ∈ Act(s1) × . . . × Act(sn) do

E := E ∧ (cid:0)(cid:2) (cid:86)n
(cid:80)

i=1 σsi = αi
s′∈supp(α1)×...×supp(αn)((Πn

(cid:3) → (cid:2)probs,φ =
P(si, αi, s′

i=1

i)) · holdsToInts′,φ′)(cid:3)(cid:1);

else if φ is P(φ1Uφ2) then E := SemanticsUnboundedUntil(M, φ, n)
else if φ is P(φ1U [k1,k2]φ2) then E := SemanticsBoundedUntil(M, φ, n)
else if φ is c then E := (cid:86)
s∈Sn(probs,φ = c)
/* op ∈ {+, −, ∗} */
else if φ is φ1 op φ2

then
E := Semantics(M, φ1, n)∧ Semantics(M, φ2, n)∧
(cid:86)

s∈Sn(probs,φ = (probs,φ1 op probs,φ2));

return E;

P( φ′) (line 13), we encode the Boolean value of φ′ in the variables holdss,φ′ (line 14), turn

them into arithmetic pseudo-Boolean values holdsToInts,φ′ (1 for true and 0 for false, line

15), and state that for each composed state, the probability value of P( φ′) is the sum of

the probabilities to get to a successor state where the operand φ′ holds; since the successors

and their probabilities are scheduler-dependent, we need to iterate over all scheduler choices

and use supp(αi) to denote the support {s ∈ S αi(s) > 0} of the distribution αi (line 17).

The encodings for the probabilities of unbounded until formulas (line 20) and bounded until

73

Algorithm 9: SMT encoding for the meaning of unbounded until formulas

Input : M = (S, Act, P, AP, L): MDP; φ: HyperPCTL unbounded until formula of

the form P(φ1Uφ2); n: number of state variables in φ.
Output: SMT encoding of φ’s meaning in the n-ary self-composition of M.

1 Function SemanticsUnboundedUntil(M, φ = P(φ1Uφ2), n)
E := Semantics(M, φ1, n)∧ Semantics(M, φ2, n)
2
foreach s = (s1, . . . , sn) ∈ S n do

3

4

5

6

7

8

9

E := E ∧ (holdss,φ2 → probs,φ=1) ∧ ((¬holdss,φ1 ∧ ¬holdss,φ2) → probs,φ=0)
foreach ⃗α = (α1, . . . , αn) ∈ Act(s1) × . . . × Act(sn) do
(cid:16)(cid:2)holdss,φ1 ∧ ¬holdss,φ2 ∧ (cid:86)n
(cid:3) →
s′∈supp(α1)×...×supp(αn)((Πn
i)) · probs′,φ)∧
s′∈supp(α1)×...×supp(αn)(holdss′,φ2∨ds,φ2>ds′,φ2)))(cid:3)(cid:17)

E := E ∧
(cid:2)probs,φ = (cid:80)
(probs,φ>0 → ((cid:87)

i=1 σsi = αi
P(si, αi, s′

i=1

return E;

formulas (line 21) are listed in Algorithm 9 and 10, respectively.

For the probabilities P(φ1Uφ2) to satisfy an unbounded until formula, the method Seman-

ticsUnboundedUntil shown in Algorithm 9 first encodes the meaning of the until operands

(line 2). For each composed state s ∈ S n, the probability of satisfying the until formula in s

is encoded in the variable probs,P(φ1U φ2). If the second until-operand φ2 holds in s then this

probability is 1 and if none of the operands are true in s then it is 0 (line 4). Otherwise,

depending on the scheduler σ of M (line 5), the value of probs,P(φ1U φ2) is a sum, adding

up for each successor state s′ of s the probability to get from s to s′ in one step times

the probability to satisfy the until-formula on paths starting in s′ (line 7). However, these

encodings work only when at least one state satisfying φ2 is reachable from s with a positive

probability: for any bottom SCC whose states all violate φ2, the probability P(φ1Uφ2) is

obviously 0, however, assigning any fixed value from [0, 1] to all states of this bottom SCC

would yield a fixed-point for the underlying equation system. To assure correctness, in line

8 we enforce smallest fixed-points by requiring that if probs,P(φ1U φ2) is positive then there

exists a loop-free path from s to any state satisfying φ2. In the encoding of this property

we use fresh variables ds,φ2 and require a path over states with strong monotonically de-

creasing ds,φ2-values to a φ2-state (where the decreasing property serves to exclude loops).

74

Algorithm 10: SMT encoding for the meaning of bounded until formulas

Input : M = (S, Act, P, AP, L): MDP; φ: HyperPCTL bounded until formula of
the form P(φ1U [k1,k2]φ2); n: number of state variables in φ.

Output: SMT encoding of φ’s meaning in the n-ary self-composition of M.

1 Function SemanticsBoundedUntil(M, φ = P(φ1U [k1,k2]φ2), n)
2

if k2 = 0 then

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

E := Semantics(M, φ1, n)∧ Semantics(M, φ2, n)
foreach s = (s1, . . . , sn) ∈ S n do

E := E ∧ (holdss,φ2→probs,φ=1) ∧ (¬holdss,φ2→probs,φ=0)

else if k1 = 0 then

E := SemanticsBoundedUntil(M, P(φ1U [0,k2−1]φ2), n)
foreach s = (s1, . . . , sn) ∈ S n do

E := E ∧ (holdss,φ2→probs,φ=1) ∧ ((¬holdss,φ1 ∧ ¬holdss,φ2)→probs,φ=0)
foreach ⃗α = (α1, . . . , αn) ∈ Act(s1) × . . . × Act(sn) do
(cid:16)(cid:2) holdss,φ1 ∧ ¬holdss,φ2 ∧ (cid:86)n
(cid:3) → (cid:2)probs,φ =
i=1 σsi = αi
i)) · probs′,P(φ1U [0,k2−1]φ2)) (cid:3)(cid:17)

s′∈supp(α1)×...×supp(αn)((Πn

P(si, αi, s′

E := E ∧

(cid:80)

i=1

else if k1 > 0 then

E := SemanticsBoundedUntil(M, P(φ1U [k1−1,k2−1]φ2), n)
foreach s = (s1, . . . , sn) ∈ S n do

E := E ∧ (¬holdss,φ1 → probs,φ = 0)
foreach ⃗α = (α1, . . . , αn) ∈ Act(s1) × . . . × Act(sn) do

E := E ∧

(cid:16)(cid:2) holdss,φ1 ∧ (cid:86)n
s′∈supp(α1)×...×supp(αn)((Πn

(cid:80)

i=1

i=1 σsi = αi
P(si, αi, s′

(cid:3) → (cid:2) probs,φ =
i)) · probs′,P(φ1U [k1−1,k2−1]φ2)) (cid:3)(cid:17)

return E;

The domain of the distance-variables ds,φ2 can be e.g. integers, rationals or reals; the only

restriction is that is should contain at least |S|n ordered values. Especially, it does not need

to be lower bounded (note that each solution assigns to each ds,φ2 a fixed value, leading a

finite number of distance values).

The SemanticsBoundedUntil method, listed in Algorithm 10, encodes the probability

P(φ1U [k1,k2]φ2) of a bounded until formula in the numeric variables probs,P(φ1U [k1,k2]φ2) for
all (composed) states s ∈ S n and recursively reduced time bounds. There are three main

cases: (i) the satisfaction of φ1U [0,k2−1]φ2 requires to satisfy φ2 immediately (lines 2–5); (ii)

φ1U [0,k2−1]φ2 can be satisfied by either satisfying φ2 immediately or satisfying it later, but

in the latter case φ1 needs to hold currently (lines 6–12); (iii) φ1 has to hold and φ2 needs to

75

Algorithm 11: SMT encoding of the truth of the input formula

Input : M = (S, Act, P, AP, L): MDP;

∃ˆσ.Q1ˆs1(ˆσ). . . . Qnˆsn(ˆσ).φnq: HyperPCTL formula.
Output: Encoding of the truth of the input formula in M.

1 Function Truth(M, ∃ˆσ. Q1ˆs1(ˆσ). . . . Qnˆsn(ˆσ). φnq)
2

foreach i = 1, . . ., n do

3

4

5

if Qi = ∀ then Bi :=”(cid:86)
else Bi :=”(cid:87)

si∈S”

si∈S”

return B1 . . . Bn holds(s1,...,sn),φnq;

be satisfied sometime later (lines 13–19). To avoid the repeated encoding of the semantics

of the operands, we do it only when we reach case (i) where recursion stops (line 3). For the

other cases, we recursively encode the probability to reach a φ2-state over φ1 states where

the deadlines are reduced with one step (lines 7 resp. 14) and use these to fix the values

of the variables probs,P(φ1U [k1,k2]φ2), similarly to the unbounded case but under additional

consideration of time bounds.

Finally, the Truth method listed in Algorithm 11 encodes the meaning of the state quan-

tification:

it states for each universal quantifier that instantiating it with any MDP state

should satisfy the formula (conjunction over all states in line 3), and for each existential

state quantification that at least one state should lead to satisfaction (disjunction in line 4).

Theorem 5.3.2. Algorithm 7 returns a formula that is true iff its input HyperPCTL
formula is satisfied by the input MDP.

We note that the satisfiability of the generated SMT encoding for a formula with an

existential scheduler quantifier does not only prove the truth of the formula but provides

also a scheduler as a witness, encoded in the solution of the SMT encoding. Conversely,

the unsatisfiability of the SMT encoding for a formula with a universal scheduler quantifier

provides a counterexample scheduler.

76

5.3.1 Example of the Encoding

Consider the problem where we are trying to verify the following property on the MDP

in Fig. 4.2a.

∃ˆσ( ˆM).∀ˆs( ˆMˆσ).∃ˆs′( ˆMˆσ).

(cid:16)
(h>0 )ˆs∧(h≤0 )ˆs′

(cid:17)

(cid:16)

P(

→

(l=1)ˆs)=P(

(l=2)ˆs′)

(cid:17)

(5.2)

We encode the actions in the MDP as described in line 2 of Algorithm 7. Here, σi refers to

the action in si.

Esch = (σ0 = α ∨ σ0 = β) ∧ (σ1 = α ∨ σ1 = β) ∧ (σ2 = τ ) ∧ (σ3 = τ )

(5.3)

We handle the encoding of the state quantifiers using Algorithm 11. We use φnq to

represent the quantifier-free part of the above property and holdssi,sj ,φnq refers to the encoding

to ensure φnq holds in the composed state of (si, sj).

Etruth = (holdss0,s0,φnq ∨ . . . ∨ holdss0,s3,φnq ) ∧

. . . ∧ (holdss3,s0,φnq ∨ . . . ∨ holdss3,s3,φnq )

(5.4)

We handle atomic propositions as described in line 3 in Algorithm 8. For example, we

have the encoding of (h>0 )ˆs below. Please note here that the first quantifier is relevant and

the second is not. Also, (h>0 )ˆs is true only in s0, hence we encode the atomic proposition

with a negation for all other states.

E(h>0 )ˆs

= (holdss0,s0,(h>0 )ˆs

) ∧ (¬holdss1,s0,(h>0 )ˆs

∧ . . . ∧ ¬holdss3,s0,(h>0 )ˆs

)

(5.5)

To encode operators, we include both the satisfaction and dissatisfaction clauses in the

encoding. Depending on the atomic propositions involved, one of the clauses would be

satisfied. For example, the encoding for conjunction of

(cid:16)

(h>0 )ˆs∧(h≤0 )ˆs′

(cid:17)

for (s0, s0) would

77

be as follows.

Econj = (holdss0,s0,(h>0 )ˆs

∧ holdss0,s0,(h≤0 )ˆs′ ∧ holdss0,s0,(h>0 )ˆs∧(h≤0 )ˆs′ ) ∨

((¬holdss0,s0,(h>0 )ˆs

∨ ¬holdss0,s0,(h≤0 )ˆs′ ) ∧ ¬holdss0,s0,(h>0 )ˆs∧(h≤0 )ˆs′ )

(5.6)

The encoding of

is similar to Algorithm 9 except that we consider φ1 to be true and

ignore its encoding. For example, the encoding of P(

(l=1)ˆs) for (s0, s0) and action (αˆs, αˆs′)

would be as below. Note that since the first quantifier is relevant, we will only encode (s2, s0)

and (s3, s0) as the successor states.

E =(holdss0,s0,(l=1)ˆs

→ probs0,s0,P(

(l=1)ˆs) = 1) ∧ (probs0,s0,P(

(l=1)ˆs) ≥ 0)∧

(¬holdss0,s0,(l=1)ˆs

∧ σ0 = α ∧ σ1 = α) →

(cid:16)

probs0,s0,P(

(l=1)ˆs) =

(3/4 × 1 × probs2,s0,P(
(cid:16)

(l=1)ˆs)) + (1/4 × 1 × probs3,s0,P(

(l=1)ˆs))

(cid:17)

∧

probs0,s0,P(

(l=1)ˆs) > 0 → (holdss2,s0,(l=1)ˆs

∨ (holdss3,s0,(l=1)ˆs

∨ ds0,s0,(l=1)ˆs

>ds3,s0,(l=1)ˆs

>ds2,s0,(l=1)ˆs

)

∨ ds0,s0,(l=1)ˆs
(cid:17)
)

(5.7)

For multiple scheduler quantifiers: To extend this algorithm to work for multiple sched-

ulers, we first have to encode the combination of actions for every state combination. Hence

in line 2 of Algorithm 7, we will need n nested loops to encode all the possible scheduler

choices of n scheduler quantifiers.

In the rest of the algorithm, we have to make similar

changes to account for the combinations of actions possible from each state s ∈ S n. So we

will need similar nested loops in cases like line 5 of Algorithm 9.

5.4 Evaluation

We developed a prototypical implementation of our algorithm in Python, with the help

of several libraries. There is extensive use of STORMPY [stob, DJKV17], which provides

an efficient solution to parsing, building, and storage of MDPs. We used the SMT-solver

Z3 [dMB08] to solve the logical encoding generated by Algorithm 7. All of our experiments

78

were run on a MacBook Pro laptop with a 2.3GHz i7 processor with 32GB of RAM. The

results are presented in Table 5.1.

As the first case study, we model and analyse information leakage in the modular expo-

nentiation algorithm (function modexp in Fig. 4.4); the corresponding results in Table 5.1

are marked by TA. We experimented with 1, 2, and 3 bits for the encryption key (hence,

m ∈ {2, 4, 6}). The specification checks whether there is a timing channel for all possible

schedulers, which is the case for the implementation in modexp.

Our second case study is the verification of password leakage through the string compar-

ison algorithm (function str cmp in Fig 4.4). Here, we also experimented with m ∈ {2, 4, 6};

results in Table 5.1 are denoted by PW.

In our third case study, we assume two concurrent processes. The first process decrements

the value of a secret h by 1 as long as the value is still positive, and after this it sets a low

variable l to 1. A second process just sets the value of the same low variable l to 2. The

two threads run in parallel; as long as none of them terminated, a fair scheduler chooses for

each CPU cycle the next executing thread. As discussed in Section 4.1, this MDP opens a

probabilistic thread scheduling channel and leaks the value of h. We denote this case study

by TS in Table 5.1, and compare observations for executions with different secret values h1

and h2 (denoted as h = (h1, h2) in the table). There is an interesting relation between the

execution times for TA and TS. For example, although the MDP for TA with m = 4 has 60

reachable states and the MDP for TS comparing executions for h = (0, 15) has 35 reachable

states, verification of TS takes 20 times more than TA. We believe this is because the MDP

of TS is twice deeper than the MDP of TA, making the SMT constraints more complex.

Our last case study is on probabilistic conformance, denoted PC. The input is a DTMC

that encodes the behaviour of a 6-sided die as well as a structure of actions having probability

distributions with two successor states each; these transitions can be pruned using a scheduler

to obtain a DTMC which simulates the die outcomes using a fair coin. Given a fixed state

space, we experiment with different numbers of transitions. In particular, we started from the

79

Case
study

m = 2
m = 4
m = 6

m = 2
m = 4
m = 6

h = (0, 1)
h = (0, 15)
h = (4, 8)
h = (8, 15)

s=(0)
s=(0,1)
s=(0,1,2)

TA

PW

TS

PC

Running time (s)

#SMT #subformulas #states #transitions

SMT encoding SMT solving

Total variables

5.43
114.00
1721.00

5.14
207.00
3980.00

0.83
60.00
11.86
60.00

277.00
822.00
1690.00

0.31
20.00
865.00

0.30
40.00
1099.00

0.07
1607.00
17.02
1606.00

5.74
134.00
2586.00

8.14
247.00
5079.00

0.90
1667.00
28.88
1666.00

1996.00
5808.00
58095.00

2273.00
6630.00
59785.00

8088
50460
175728

8088
68670
274540

1379
34335
12369
34335

21220
21220
21220

50654
368062
1381118

43432
397852
1641200

7913
251737
87097
251737

1859004
5349205
11006581

24
60
112

24
70
140

7
35
21
35

20
20
20

46
136
274

46
146
302

13
83
48
83

158
280
404

Table 5.1: Experimental results. TA: Timing attack. PW: Password leakage. TS: Thread
scheduling. PC: Probabilistic conformance.

implementation in [KY76] and then we added all the possible nondeterministic transitions

from the first state to all the other states (s=0), from the first and second states to all

the others (s=0,1), and from the first, second, and third states to all the others (s=0,1,2).

Each time we were able not only to satisfy the formula, but also to obtain the witness

corresponding to the scheduler satisfying the property.

Regarding the running times listed in Table 5.1, we note that our implementation is

only prototypical and there are possibilities for numerous optimizations. Most importantly,

for purely existentially or purely universally quantified formulas, we could define a more

efficient encoding with much less variables. However, it is clear that the running times for

even relatively small MDPs are large. This is simply because of the high complexity of the

verification of hyperproperties. In addition, the HyperPCTL formulas in our case studies have

multiple scheduler and/or state quantifiers, making the problem significantly more difficult.

5.5 Summary

One important gap in HyperPCTL was the inability to express properties involving non-

determinism in systems. In this chapter, we have discussed the reasoning of hyperproperties

80

over schedulers. This new dimension added an extra layer of complexity due to the in-

crease in expressiveness of the language.

In terms of practical results, we have proposed

an NP-complete algorithm for the logic which only allows reasoning over memoryless and

deterministic schedulers. We have also demonstrated our approach in a few interesting case

studies. However, the logic still had an important gap, in terms of its inability to argue over

reward models, which caused us to further explore it as described in the next chapter.

81

Chapter 6

Probabilistic Hyperproperties with
Rewards

6.1

Introduction

Stochastic phenomena appear in many systems such as those that interact with the

physical environment (e.g., due to environmental uncertainties, thermal fluctuations, ran-

dom message loss, and processor failure). Traditionally, system specifications that deal with

uncertainties are expressed in some form of probabilistic temporal logic such as PCTL and

PCTL∗ [BK08]. These logics can express the properties of single probabilistic computation

trees. The temporal logic HyperPCTL generalizes PCTL to express probabilistic hyperproperties

by allowing quantification over multiple computation trees and expressing the probability re-

lation among them. For instance, consider the Markov Decision Process (MDP) in Fig. 6.1a.

The HyperPCTL formula

∀ˆσ.∀ˆs(ˆσ).∀ˆs′(ˆσ).

(cid:16)

(h > 0 )ˆs ∧ (h ≤ 0 )ˆs′

(cid:17)

(cid:16)

⇒

P (l = 1)ˆs = P (l = 1)ˆs′

(cid:17)

(6.1)

requires that the probability of reaching a state with proposition l = 1 from any pair of states

ˆs and ˆs′ labelled by h > 0 and h ≤ 0 respectively, should be equal for the Discrete Time

Markov Chain (DTMC) induced by any scheduler ˆσ. In addition to the probability relation

between certain events and computations, it is natural to analyse the average behaviour of

82

s0
3

1
2

{h>0}

α

1
4

3
4

s1
3

1
3

β

1
2

α2
3

β

1
2

1
2

{h≤0}

{h>0}

(cid:40)

(cid:41)

s2
1

τ

1

l=1
end

(cid:40)

(cid:41)

s3
1

τ

1

l=2
end

s0
3

(cid:40)

3
4

1
2
3
4

s1
2

1
3

{h≤0}

(cid:41)

l=1
end

(cid:40)

(cid:41)

l=2
end

s3
1

1

s2
1

1

(a) An MDPR.

(b) An induced DTMCR.

Figure 6.1: Example Markov models.

Markov models as well as the interrelation of average behaviours in different executions. For

example:

• Service-level agreements (e.g., average system response time and uptime) are generally

concerned with the average performance metrics of a system among a set of executions.

This is, of course, a system-wide performance requirement rather than the property of

individual executions.

• Side-channel timing leaks can potentially reveal sensitive information through the exe-

cution time of a function call. The execution time can be captured as a reward model

where each instruction is associated with a cost and the probabilistic hyperproperty

expresses that every pair of executions should exhibit the same expected execution

cost.

• Distributed algorithms often use randomization to break symmetry to tackle impossibil-

ity results. Although one can reason about the expected performance of a randomized

distributed algorithm by the traditional reward models, from a design perspective, it

is desirable to determine and mitigate states from where convergence to the objective

of the algorithm takes much longer than others.

These examples motivate the need to somehow augment probabilistic hyperproperties with

reward constraints.

83

With this motivation, our first contribution is to make the connection between reward

models and probabilistic hyperproperties.

In the context of a hyperproperty, analogous

to the probability relation between multiple executions in a HyperPCTL formula, a reward

mechanism should be able to express the expected reward relation along different quanti-

fied computation trees. To this end, we extend the syntax and semantics of HyperPCTL by

allowing arithmetic functions over expected rewards and comparing them over multiple ex-

ecutions. For instance, for the MDP in Fig. 6.1a one may express whether there exist two

schedulers such that starting from any two states, labeled with h>0 and h≤0, resp., the

expected reward of reaching an end-labeled state is the same using the following property:

(cid:16)
∃ˆσ1.∃ˆσ2.∀ˆs(ˆσ1).∀ˆs′(ˆσ2).

(h>0)ˆs ∧ (h≤0)ˆs′

(cid:17)

→

(cid:16)

Rˆs(

endˆs) = Rˆs′(

(cid:17)

endˆs′)

(6.2)

In the MDP in Fig. 6.1a, if we instantiate ˆs with s0, and choose the action α, we collect a

reward of (3 + 3

4 × 1 + 1

4 × 1) = 4, on reaching s2 and s3 with label end. Similarly, if we

instantiate ˆs′ with s1, and choose the action α, we collect a reward of (3 + 2

3 × 1 + 1

3 × 1) = 4,

on reaching s2 and s3 with label end. Hence, we can prove the existence of schedulers that

satisfy the above property in the MDP in Fig. 6.1a. On a closer look, no matter which action

we choose at s0 and s1, the property is always satisfied. Also, if we instantiate ˆs and ˆs′ with

any other states different from s0 resp. s1, the property is vacuously true. On the contrary,

if we replace the equality of rewards with inequality then the property is false as there are no

such schedulers. Besides comparing reward values, our HyperPCTL extension offers further

expressive power to e.g. measure accumulated rewards in an execution until an observable

property, say termination, gets satisfied in another one.

Our second contribution is an algorithm for model checking HyperPCTL formulas with

rewards for MDPs. Since the general verification problem is shown to be undecidable, we

focus on memoryless non-probabilistic schedulers which yield a decidable problem, for which

we propose a model checking algorithm based on logical problem encoding and SMT solving.

We have implemented a prototype of our method and analysed it experimentally on three

84

case studies: (1) side-channel timing attacks, (2) probabilistic performance conformance,

and (3) randomized path planning for multi-agent robotics applications.

6.2 HyperPCTL with Rewards

In this section, we elaborate on the syntax and semantics of the newly added reward-based

operators in HyperPCTL. We additionally discuss the cases where we encounter undecidability

of rewards and how we can avoid them in specific cases.

6.2.1 Syntax of HyperPCTL with Rewards

Hyperproperties of executions in an MDPR can be specified using the logic HyperPCTL. As

shown in Fig. 6.2, a quantified formula ϕq starts with a sequence of quantifiers over scheduler

variables ˆσ ∈ ˆΣ, fixing the schedulers under which executions are considered. Inside, a state-

quantified formula ϕsq defines a sequence of quantifiers over state variables ˆs ∈ ˆS, where

each quantifier specifies a new execution from a given state under a given scheduler. Note

that different executions might use the same scheduler.

In the scope of these quantifiers is a non-quantified state formula ϕnq, which can be the

constant true, an atomic proposition indexed with a state variable, conjunction, negation,

or a relational constraint comparing two arithmetic expressions via ∼∈ {>, ≥, =, ̸=, <, ≤}.

ϕq
ϕsq
ϕnq
ϕar
ϕpath

| ∃ˆσ.ϕq

| ϕsq
::= ∀ˆσ.ϕq
::= ∀ˆs(ˆσ).ϕsq
| ∃ˆs(ˆσ).ϕsq
::= true | aˆs | ϕnq ∧ ϕnq
::= P(ϕpath) | Rˆs,i(ϕpath) | f (ϕar, . . . , ϕar)
::=

| ϕnq
| ¬ϕnq

| ϕnq U [k1,k2] ϕnq

| ϕnq U ϕnq

ϕnq

| ϕar ∼ ϕar

Figure 6.2: HyperPCTL syntax.

Arithmetic expressions are constructed from probability expressions, reward expressions,

or applying arithmetic function symbols (e.g., addition, subtraction, multiplication, etc.,

where constants are 0-ary functions) to arithmetic expressions. Note that the reward operator

R is indexed with a state variable ˆs specifying the execution for which we consider the reward,

85

and an integer i specifying the reward component; for models with unary rewards, like in our

examples, we skip the second index (as it is always 0). Finally, the parameters of probabilistic

and reward expressions are path formulas, which apply one of the temporal operators, next

( ), unbounded until (U), or bounded until (U [k1,k2], k1 ≤ k2 ∈ N≥0) to non-quantified state

formulas.

A HyperPCTL formula is a quantified formula in that every occurrence of an indexed

atomic proposition aˆs is in the scope of a state quantifier for ˆs(ˆσ), which in turn is in

the scope of a scheduler quantifier for ˆσ. W.l.o.g., in the following we assume that each

scheduler or state variable is quantified at most once.

In addition to standard syntactic

sugar ∨, →,

,

, . . ., we can express expected cumulative reward over the next t ∈ N steps

and expected reward in the state reached after t steps as follows:

Rˆs,i(Ct) = Rˆs,i(trueU [t,t]true) and Rˆs,i(It) =






Rˆs,i(Ct) − Rˆs,i(Ct−1)

if t>0

Rˆs,i(Ct)

else .

(6.3)

6.2.2 Semantics of HyperPCTL with Rewards

HyperPCTL formulas are evaluated recursively in the context of an MDPR M, a sequence

σ of actions, and a sequence s of states, both of the same length. Intuitively, the length

of these sequences says how many executions we run in parallel, and the ith elements in

these sequences specify the ith execution of the scheduler and the initial state in the induced

DTMCR, respectively. An MDPR M satisfies a HyperPCTL formula ϕ (written M |= ϕ) iff

M, (), () |= ϕ.

In the semantic rules shown in Fig. 6.3, the substitution φ[ˆσ⇝σ] remembers the instan-

tiation of a scheduler variable ˆσ by a concrete scheduler σ = (Q, act, mode, init) through

syntactically transforming in φ each ∀ˆs(ˆσ) and ∃ˆs(ˆσ) into ∀ˆs(σ) and ∃ˆs(σ), resp. When

instantiating the nth state quantifier ∀ˆs(σ) or ∃ˆs(σ) by a state s, we “start” an nth execu-

tion in state (init(s), s) of Mσ, which corresponds to extending the previously (n−1)-ary

self-composition of M to arity n. We remember this by adding σ and s at the end of the

86

corresponding sequences in the context (using concatenation ◦), and applying the substitu-

tion φ[ˆs⇝n] to replace each indexed atomic proposition aˆs and each reward operator Rˆs,i in

φ by an and Rn,i, respectively.1 We recall from Chapter 4 the semantics of constructs that

are not related to rewards:

M, σ, s |= ∀ˆσ.φ
M, σ, s |= ∃ˆσ.φ
M, σ, s |= ∀ˆs(σ).φ
M, σ, s |= ∃ˆs(σ).φ
M, σ, s |= true
M, σ, s |= ai
M, σ, s |= φ1 ∧ φ2
M, σ, s |= ¬φ
M, σ, s |= φar
P(φpath)
M,σ,s
(cid:74)
(cid:75)
1 , . . . , φar
f (φar
k )
(cid:74)

1 ∼ φar
2

iff M, σ, s |= φ[ˆσ⇝σ] for all σ ∈ ΣM
iff M, σ, s |= φ[ˆσ⇝σ] for some σ ∈ ΣM
iff M, σ ◦ σ, s ◦ (init(s), s) |= φ[ˆs⇝|σ|] for all s∈S
iff M, σ ◦ σ, s ◦ (init(s), s) |= φ[ˆs⇝|σ|] for some s∈S

iff ai ∈ Lσ(s)
iff M, σ, s |= φ1 and M, σ, s |= φ2
iff M, σ, s ̸|= φ
iff
M,σ,s ∼
= Pr Mσ (cid:0){π ∈ Paths s(Mσ) | M, σ, π |= φpath}(cid:1)
(cid:75)
φar
φar
1
k
(cid:74)
(cid:74)

M,σ,s, . . . ,
(cid:75)

φar
1
(cid:74)
M,σ,s = f (cid:0)
(cid:75)

M,σ,s
(cid:75)

φar
2

M,σ,s

(cid:1)

(cid:74)

(cid:75)

Figure 6.3: Existing Semantics rules for HyperPCTL.

We describe the semantics for Rj,i(φpath) in Fig. 6.4 (note that instantiating a state

quantifier for ˆs(σ) replaces each Rˆs,i occurrence by Rj,i, where j is the position of the

quantifier). The value of Rj,i( φnq) is the current ith reward plus the expected ith reward

of the successor state in the jth execution if the probability that the successor state satisfies

φnq is 1; otherwise, the value is undefined. The value of Rj,i(φnq

1 U φnq

2 ) is the expected

cumulative ith reward in the jth execution, accumulated until the first time a (global self-

composition) state is reached that satisfies φnq

2 , in case the probability of satisfying φnq

1 U φnq

2

is 1; otherwise, the value is undefined. The semantics of Rj,i(φnq

1 U [k1,k2] φnq

2 ) is similar, but

the rewards are accumulated until the first satisfaction of φnq

2 within time [k1, k2]. Formally,

the semantics for

not the case,

Rj,i(φpath)
M,σ,s is as follows, given that
(cid:75)
(cid:74)
Rj,i(φpath)
M,σ,s is undefined.
(cid:75)
(cid:74)

P(φpath)
M,σ,s = 1. If that is
(cid:75)

(cid:74)

Since adding rewards to HyperPCTL causes arithmetic values to be potentially undefined,

we need to extend the above semantics to handle the propagation of undefined values. For

1Instead of syntactical substitutions, we could also use binding functions to map scheduler variables to

schedulers and state variables to indices in the state sequence in the context.

87

Pathss

fin(Mσ)(φnq

1 Uφnq
2 )

Pathss

fin(Mσ)(φnq

1 U [k1,k2]φnq

Rj,i( φnq)
(cid:74)
Rj,i(φnq
(cid:74)
Rj,i(φnq
(cid:74)

M,σ,s
(cid:75)
1 Uφnq
2 )
M,σ,s
(cid:75)
1 U [k1,k2]φnq
2 )
M,σ,s
(cid:75)

= {s0 . . . sn ∈ Paths s
M,σ,si |= φnq

1 ∧ ¬φnq

fin(Mσ) | M, σ, sn |= φnq
for i = 0, . . . , n−1}
fin(Mσ) | k1 ≤ n ≤ k2 and

2

2 and

2 ) = {s0 . . . sn ∈ Paths s
M,σ,sn |= φnq
2 and
M,σ,si |= φnq
1
M,σ,si |= φnq
j,i(s) + (cid:80)
π∈Paths s

= rew σ
= (cid:80)
= (cid:80)

π∈Paths s

for i = 0, . . . , k1−1 and

1 ∧ ¬φnq

2

for i = k1, . . . , n−1}

s′∈Sσ P σ(s, s′) · rewσ

j,i(s′)
2 )(Pr σ(π) · rewσ

fin (Mσ)(φnq
fin (Mσ)(φnq

1 U φnq
1 U [k1,k2]φnq

j,i(π))
2 )(Pr σ(π) · rewσ

j,i(π))

Figure 6.4: Semantics for reward operators in HyperPCTL.

each syntactic case, the above semantics remain unchanged if all involved statements used

in the definition are defined. It would be an easy job to set the values in all other cases to

undefined. However, even if some of the arguments are undefined, we still might be able to

conclude a defined value. For example, if one of the operands in a conjunction is false then

the conjunction is inevitably false, even if the other operand is undefined. In extension to

the above semantics for the cases when all terms used in the definition are defined, below

we fix the semantics for the remaining cases to reduce the occurrence of undefined values.

We extend the Boolean domain of true (1) and false (0) with undefined (⊥). We use

the |= relation as before when all sub-expressions (and thus the formula) are known to be

defined, and use

·
(cid:75)

(cid:74)

· otherwise. Logical constants as well as atomic propositions are always

defined. The value of a conjunction is undefined if and only if none of the operands is false

and not both operands are true, whereas a negation is undefined if and only if the negated

formula is undefined.

The value of a universally state-quantified formula ∀ˆs(σ).φ is undefined if the value of φ

is undefined for at least one instantiation of the formula with a state and is not false for any

other instantiation. Likewise, the value of an existentially state-quantified formula ∃ˆs(σ).φ

is undefined if the value of φ is undefined for at least one instantiation of the formula with a

state and is not true for any other instantiation. The undefinedness of scheduler quantifiers

is analogous.

88

Row

1

2
3
4
5

6
7
8
9

φ1
(cid:74)

*

(cid:75)

φ2
(cid:74)

1

(cid:75)

p

*

φ
(cid:75)
(cid:74)

1

0
⊥
⊥
1

0
⊥
1
1

0
0
0
0

⊥
⊥
⊥
⊥

*
0

0
0
̸= 0 ⊥
p

*

*
*
1

⊥
⊥
1
̸= 1 ⊥

Table 6.1: Semantics of φ = P(φ1Uφ2), partly depending on p = (cid:80)
[0, 1] ∪ {⊥}. Here,

is short for

s′∈Sσ P (s, s′) ·

M,σ,s′ ∈
φ
(cid:75)

(cid:74)

.
(cid:75)

(cid:74)

.
(cid:74)

M,σ,s.
(cid:75)

Also, the domain of arithmetic values gets extended with the undefined value ⊥. Arith-

metic function applications f (φ1, . . . , φk) and arithmetic constraints φ1 ∼ φ2 are unde-

fined if and only if any of their parameters are undefined. However, for probabilistic

until φ = P(φ1Uφ2) we can exploit available information to increase the number of de-

fined cases, even if the satisfaction of one of the operands is undefined in the current

state, as shown in Table 6.1. The information we exploit for the semantics in a state

s are probabilistic until values in the successor state, or more precisely, the value of
p = (cid:80)

M,σ,s′ ∈ [0, 1] ∪ {⊥}, which we consider undefined iff one of the
φ
(cid:75)

s′∈Sσ P (s, s′) ·

(cid:74)

successor probabilities is undefined.

Table 6.1 extends the original probabilistic until semantics from above with the undefined

cases, using ∗ to denote an arbitrary (defined or undefined) arithmetic value. This table is

split into three parts. The first part states that if φ2 is true then the formula value is 1. The

second part covers the case where φ2 is false, where the violation of φ1 leads to the violation

of the formula, and if φ1 is true then the formula probability equals the value of p.

An interesting case in the second block is when φ1 is undefined: though in most cases

the formula is also undefined, if we know that the probability to satisfy the until formula

in the future is 0 then we can safely state that the probability to satisfy the same in the

89

current state is also 0. Similarly in the third block, if φ1 is true in the current state and the

probability to satisfy the until formula in the future is 1 then, irrelevant of the value of φ2,

the probability to satisfy the until formula from the current state is always 1.

Reward expressions are undefined if the respective path property is not satisfied with

probability 1. For the reward expression Rj,i( φ), this is the only case in which it is

undefined. To evaluate φ = Rj,i(φ1Uφ2), if φ2 is true in the current state then we need to

know only the current state’s reward; in this case, the reward is defined independently of the

successor states. If φ2 is false currently then the reward is computed from the current state

reward plus the expected successor φ-values, thus undefinedness of the reward expression in

a successor state causes undefinedness in the current state. However, if φ2 is undefined in the

current state then we do not know which of these two cases apply; the only case where this

does not matter is if the reward expression evaluates in all successor states to 0, namely then

the value of φ is the current state reward. Thus if φ2 is undefined in the current state then

the reward expression is undefined in all but this special case, even if the probability of the

until formula is 1. The definedness of bounded until formulas are similar to the unbounded

case for both probability and reward expressions, except that we now also need to account

for the bounds.

However, with these definitions, we only exploit some but not all information, to deter-

mine the definedness of a property. Assume, for example, the property that from a state

s, the probability to eventually satisfy φ is less than p. It might be the case that in some

states reachable from s the value of φ is undefined, triggering the above probability to be

undefined by our algorithm. However, φ might be reachable along another path with a prob-

ability larger than p, in which case we could have safely stated that it is at least p. Hence, it

can be a direction of future research to find a tighter bound on the definedness of a property.

90

1 v o i d mexp ( ) {

2

3

4

5

6

7

8

9

10

c = 0 ; d = 1 ;
w h i l e ( i >= 0 ) {

i = k ;

i = i −1; c = c ∗ 2 ;
d = ( d∗d ) % n ;
( b ( i ) = 1 ) {
i f

c = c +1;
d = ( d∗ a ) % n ;

}

}

Figure 6.5: Modular exponentiation in RSA.

6.3 Applications of HyperPCTL with Rewards

The introduction of rewards in the logic allows us to quantify certain metrics like time or

cost along the path where we are sure to reach our goal state. Additionally, it allows us to

filter our solutions based on these metrics. Below we discuss a few extensions to our existing

applications that are possible due to the reward-extension of HyperPCTL.

6.3.1 Timing Attacks

Side-channel timing leaks can potentially reveal sensitive information. For example, RSA

uses the modular exponentiation algorithm on the right to compute ab mod n, where a is

the message and b is the encryption key. This implementation is flawed because of the if in

line 6. Due to the lack of an else branch, its execution will take longer if b contains more

1-bit. An attacker could therefore run a thread in parallel to measure the execution time of

the algorithm to derive the number of 1-bits in the encryption key.

To prevent such vulnerabilities, we would like the execution time to be independent of

the bit values in the encryption key, which is captured by assigning a reward of 1 to each

state in the MDPR. Here, each state represents the current position in the code and loop

iteration. This results in the following HyperPCTL formula:

91

∀ˆσ1.∀ˆσ2.∀ˆs(ˆσ1).∀ˆs′(ˆσ2). (initˆs ∧ initˆs′) → (Rˆs(

endˆs) = Rˆs′(

endˆs′))

(6.4)

6.3.2 Probabilistic Conformance

The aim here is to ensure that an implementation conforms with the system it is simu-

lating. We consider the implementation of a 6-sided die with repeated tossing of a fair coin

using the Knuth-Yao algorithm [KY76]. For conformance, the probabilistic distribution of

reaching the 6 sides of a die should be equal in both cases. We model this problem with an

MDP consisting of two components: the first component describes the die and its states rep-

resent the faces of the die after being rolled. The second component describes the multiple

coin tosses and its states represent the unique combined results of the tosses. Extending this

model with rewards allows us to synthesize efficient implementations: if we assign to every

state, except the absorbing states, a state reward of 1, the expected reward on reaching one

of the absorbing states in the coin implementation will be equal to the expected number

of coin tosses in it. If we limit the rewards collected in such a path, we can filter the im-

plementations with minimum intermediate states. The following formula specifies that the

expected number of coin tosses in such an implementation must be less than 4:

∃ˆσ.∀ˆs(ˆσ).∃ˆs′(ˆσ).dieInitˆs →

ϕ ∧ Rˆs′(

(cid:32)

(cid:33)

(die = l)ˆs′)) < 4

6
(cid:95)

(
l=1

with ϕ = coinInitˆs′ ∧

6
(cid:94)

l=1

(P(

(die = l)ˆs) = P(

(die = l)ˆs′))

(6.5)

6.3.3 Cost Analysis in Multi-Agent Path Planning

We consider the examples in Fig. 6.6 where two robots R1, R2 aim to reach the target cell

end starting their journey from two different initial cells (start1, start2). The robots’ behavior

is modeled as an MDPR where each cell occupied represents a state. Nondeterministic actions

represent all possible moves of the robot from each cell, while the successful maneuvering

after having executed an action is captured by a probability distribution.

92

Figure 6.6: The maze on the left satisfies φtarget, while on the right it violates φtarget.

Fences prevent a robot from moving in a certain direction disabling possible actions in

a particular cell, while the presence of ramps or uneven terrain can increase/decrease the

probability of correct robot maneuvers. The occupancy of each state has a cost in terms of

energy consumption modeled as a positive reward. We want to check that for all possible

(memoryless) schedulers, when robots R1, R2 start their mission from their respective initial

conditions and they can both reach the target state with probability 1, then the expected

energy consumption for robot R1 is less than the expected energy consumption for robot R2.

This can be expressed as the following probabilistic hyperproperty:

φtarget = ∀ˆσ.∀ˆs(ˆσ).∀ˆs′(ˆσ).ψ →

(cid:16)

Rˆs(

endˆs) < Rˆs′(

(cid:17)

endˆs′)

where ψ =

(cid:16)

start1 ˆs ∧ start2 ˆs′ ∧ P(

endˆs) = 1 ∧ P(

endˆs′) = 1

(cid:17)

(6.6)

6.3.4 Probabilistic Self-stabilizing Systems

In distributed systems, randomization is often used to break symmetry between processes

to tackle impossibility results. For instance, self-stabilizing token circulation in a ring is

impossible in a non-probabilistic setting but Herman’s algorithm [Her90] (see Fig. 6.7) uses

randomization to ensure recovery to a stable state (i.e., there is only one token circulating)

with probability one.

In such an algorithm, from certain initial states, convergence to a

stable state may be faster than others and if faults hit those states with a higher probability,

it reduces the average convergence time significantly. Thus, designers of self-stabilizing

algorithms often use state encodings to tackle slow recovery [FBT13]. The following formula

93

R11R212211113333333R11R212211113111333𝑠𝑡𝑎𝑟𝑡!𝑠𝑡𝑎𝑟𝑡"𝑒𝑛𝑑𝑟𝑎𝑚𝑝𝑠𝑟𝑒𝑤𝑎𝑟𝑑𝑓𝑒𝑛𝑐𝑒𝑢𝑛𝑒𝑣𝑒𝑛𝑡𝑒𝑟𝑟𝑎𝑖𝑛intends to check whether there exists a state from which the convergence time is twice slower

than from some other state:

∀ˆσ.∃ˆs(ˆσ).∃ˆs′(ˆσ).

(cid:16)

Rˆs(

stableˆs) > 2 · Rˆs′(

stableˆs′)

(cid:17)

(6.7)

Note: Herman’s algorithm yields a DTMCR and, thus, the choice of scheduler quantifi-

cation is irrelevant.

1: Variable: xi : boolean ∈ {0, 1}
2: Guarded Commands:
3: xi = xi−1 −→ p : xi := 0 + (1 − p) : xi := 1;
4: xi ̸= xi−1 −→ 1 : xi := xi−1;

π0
x0=0

π1
x1=0

=⇒

π2
x2=0

π1
x1=1

π0
x0=0

π2
x2=1

Figure 6.7: Herman’s algorithm [Her90] for process i and example for three processes.

6.4 Model Checking Algorithm for Reward Operators

HyperPCTL provides an increased level of expressiveness over PCTL and PCTL∗, causing

the model checking problem for MDPRs to be undecidable even without rewards, as shown

in [ ´ABBD20b]. To achieve decidability for HyperPCTL without rewards, in [ ´ABBD20b] we

restricted the domain of scheduler quantification to memoryless non-probabilistic schedulers.

For this restricted domain, the model checking problem is NP-complete (or coNP-complete)

when the scheduler quantification is existential (or universal). We provided a model check-

ing algorithm by logically encoding HyperPCTL satisfaction problems as linear real-arithmetic

formulas and used an SMT solver to check the encodings for satisfiability. Elaborate expla-

nations of encoding non-reward operators can be found in [ ´ABBD20b].

After adding rewards, the model checking problem restricted to finite memoryless sched-

ulers is still decidable. Similar to the standard model checking problem for Markov Reward

Models, computing the expected reward earned until a certain set of states is reached, has a

polynomial time complexity in the size of the MDP: the problem can be solved by determin-

ing a linear real-arithmetic equation system via graph reachability analysis and solving it.

94

This means adding rewards does not change the class of complexity of the model checking

problem as identified in [ ´ABBD20b].

However, adding rewards to the problem requires a major adaption of the logical en-

coding. The reason is that expected reward values might be undefined, and undefinedness

might propagate from the inner sub-formulas to the formula value. The main contributions

of this section are (1) to extend the model checking algorithm from [ ´ABBD20b] to encode

the semantics of reward-related HyperPCTL expressions and (2) to modify the previous en-

codings to model undefinedness propagation for the remaining language components. To

ease understanding, in the following, we consider unary-reward models and a single exis-

tential scheduler quantifier in our properties; extension to multi-dimensional rewards and

several scheduler quantifiers without quantifier alternation is doable by little modifications

to the algorithms. Given their finite domain, support for scheduler quantifier alternation is

possible, too, but it would require more involved extensions. Assume as input an MDPR

model M and a HyperPCTL formula ϕ. In [ ´ABBD20b] we used Boolean variables holdss,ϕ to

encode the truth value of a Boolean-valued formula ϕ in state s. In this work, we replace

the two-valued domain for these variables with a three-valued domain over the values true

(1), false (0) and undefined (⊥). Furthermore, we use variables vals,ϕ to store the numerical

value of an arithmetic expression ϕ in state s. To also encode the definedness of arithmetic

values, we introduce additional Boolean variables defs,ϕ which should be true if and only if

the corresponding value is defined. Finally, to encode a scheduler, we use for each state of

M a variable σs to store the chosen action.

The starting point of the encoding is Algorithm 12, which begins by encoding the sched-

uler choice 2 in line 2. The semantics of the non-quantified inner formula ϕnq under a given

scheduler choice in each of the states are encoded in line 3. This basic encoding E is extended

in two directions: formula T encodes that ϕ can be made true by some suitable quantifier

instantiation, whereas U encodes that ϕ can be made true or undefined. Only if none of

2For n scheduler quantifiers, we would simply need to include such a scheduler encoding for each of the

schedulers σ1, . . . , σn, and in the rest of the encoding, refer to the respective schedulers σi instead of σ.

95

Algorithm 12: Main SMT encoding algorithm
Input: M = (S, Act, P, AP, L, rew): MDPR;

φ: HyperPCTL formula.

Output: Whether M satisfies φ.

1 Function Main(M, φ = ∃ˆσ.Q1ˆs1(ˆσ). . . . Qnˆsn(ˆσ).φnq):
2

s∈S((cid:87)

α∈Act(s) σs = α)

E := (cid:86)
E := E ∧ Semantics(M, φnq, n)
T := E ∧ Eval(M, φ, {1})
U := E ∧ Eval(M, φ, {⊥, 1})
if check(T ) = SAT then return TRUE
else if check(U ) = SAT then return UNDEF
else return FALSE

3

4

5

6

7

8

these two cases apply (i.e. if both formulas are unsatisfiable), we conclude that M does not

satisfy ϕ. Not listed in the algorithm is the case of a universal scheduler quantifier, where

we use negation to get an existential formula, apply the listed algorithm, and negate the

answer.

Algorithm 13: Encoding certain formula values
Input: M = (S, Act, P, AP, L, rew): MDPR;
ϕ: HyperPCTL formula; v⊆{0, 1, ⊥}.

Output: Encoding that M, (), () |= ∃ˆσ.Q1ˆs1. . . . Qnˆsn.(ϕnq ∈ v).

1 Function Eval(M, ϕ = ∃ˆσ.Q1ˆs1. . . . Qnˆsn.ϕnq, v):
2

foreach i = 1, . . . , n do

if Qi = ∀ then Bi := ” (cid:86)

si∈S ” else Bi := ” (cid:87)

si∈S ”

return B1 . . . Bn (holds(s1,...,sn),ϕnq ∈ v)

3

4

The semantics of formulas is encoded by Algorithm 14. We omit the pseudocode of sub-

algorithms that were needed also without rewards; these are similar to those in [ ´ABBD20b]

but get extended with the encoding of definedness as explained in Section 6.2.2. Relevant for

rewards is line 14, calling the method RewSemantics in Algorithm 15 to encode the semantics

of the reward operators. In the case of rewards over the next operator ϕ = Rˆsi( ϕ′), we

first encode the probability P( ϕ′); ϕ is undefined if this probability is not 1 (line 5). If the

probability is defined, then the reward is the expected reward of the successors in the ith

execution (line 7).

96

Algorithm 14: SMT encoding for the meaning of an input formula

Input: M = (S, Act, P, AP, L, rew): MDPR; φ: quantifier-free HyperPCTL formula

or expression; n: number of state variables in φ.

Output: SMT encoding of the meaning of φ in n-ary self-composition of M.

1 Function Semantics(M, φ, n):
if φ is true then E := (cid:86)
2
else if φ is aˆsi then

3

s∈Sn holdss,φ=1

4

5

6

7

8

9

10

11

12

13

14

15

E := ((cid:86)

s∈Sn, a∈L(si)(holdss,φ=1)) ∧ ((cid:86)

s∈Sn, a̸∈(si)(holdss,φ=0))

else if φ is ¬φ′ then

E := Semantics(M, φ′, n) ∧ (cid:86)
(cid:86)

s∈Sn(holdss,φ′=1 → holdss,φ=0) ∧ (cid:86)

s∈Sn(holdss,φ′=0 → holdss,φ=1)∧

s∈Sn(holdss,φ′=⊥ → holdss,φ=⊥)

1 ∼ φar

1 , . . . , φar

2 then E := SemanticsComp(M, φ, n)

k ) then E := SemanticsArithmetic(M, φ, n)

else if φ is φ1 ∧ φ2 then E := SemanticsConjunction(M, φ, n)
else if φ is φar
else if φ is f (φar
else if φ is P( φ′) then E := SemanticsNext(M, φ, n)
else if φ is P(φ1Uφ2) then E := SemanticsUnboundedUntil(M, φ, n)
else if φ is P(φ1U [k1,k2]φ2) then E := SemanticsBoundedUntil(M, φ, n)
else E := RewSemantics(M, φ, n)
return E

To encode the reward of unbounded until formulas, we first need to encode the probability

of the until formula, since this probability needs to be 1 for a defined reward value. Then we

call the RewardUnboundedUntil method from Algorithm 16, which implements the semantics

of the reward of unbounded until from Section 6.2.2. Undefinedness is covered in line 5 when

the probability of the unbounded until is either not defined or not 1, and in the lines 9-

10 when the probability of the unbounded until is 1, φ2 is not true and either a successor

reward is undefined, or ϕ2 is undefined and the successor rewards are not zero. The method

RewardBoundedUntil for reward expressions with bounded until not shown here, is similar

to the unbounded case but needs additional bookkeeping about the time interval within

which φ2 needs to be satisfied.

6.5 Evaluation

We have implemented a prototype of the presented algorithm by extending our tool

HyperProb [DABB21] to support rewards. The implementation has been coded in Python

97

Algorithm 15: SMT encoding for the meaning of reward operators

Input: M = (S, Act, P, AP, L, rew): MDPR; φ: quantifier-free HyperPCTL formula

or expression; n: number of state variables in φ.

Output: SMT encoding of the meaning of φ in n-ary self-composition of M.

1 Function RewSemantics(M, φ, n):
2

if φ is Rˆsi( φ′) then

E := Semantics(M, P( φ′), n)
foreach s = (s1, . . . , sn) ∈ Sn do

E := E ∧ ((vals,P( φ′) ̸= 1 ∨ ¬defs,P( φ′)) ↔ ¬defs,φ)
foreach α = (α1, . . . , αn) ∈ Act(s1) × . . . × Act(sn) do

E := E ∧ ([defs,φ ∧ (cid:86)n
rew(si) + (cid:80)
else if φ is Rˆsi(φ1U [k1,k2]φ2) then

j=1 σsj = αj] → [vals,φ =

s′∈supp(α1)×...×supp(αn)(((cid:81)n

j=1 P (sj, αj, s′

j)) · rew(s′

i))])

E := SemanticsBoundedUntil(M, P(φ1U [k1,k2]φ2), n)
E := E ∧ RewardBoundedUntil(M, φ, n)

else if φ is Rˆsi(φ1Uφ2) then

E := SemanticsUnboundedUntil(M, P(φ1Uφ2), n)
E := E ∧ RewardUnboundedUntil(M, φ, n)

return E

Algorithm 16: SMT encoding for the reward of unbounded until

Input: M = (S, Act, P, AP, L, rew): MDPR; φ: HyperPCTL unbounded until
formula of the form Rˆsi(φ1Uφ2); n: number of state variables in φ.
Output: SMT encoding of φ’s meaning in the n-ary self-composition of M.

1 Function RewardUnboundedUntil(M, φ = Rˆsi(φ1Uφ2), n):
2

φ′ := P(φ1Uφ2); E := true
foreach s = (s1, . . . , sn) ∈ Sn do

E := E ∧ (holdss,φ2 = 1 → (vals,φ = rew(si) ∧ defs,φ))
E := E ∧ ((vals,φ′ ̸= 1 ∨ ¬defs,φ′) → ¬defs,φ)
foreach α = (α1, . . . , αn) ∈ Act(s1) × . . . × Act(sn) do
E := E ∧ ((vals,φ′ = 1 ∧ defs,φ′ ∧ holdss,φ2 ̸= 1 ∧ (cid:86)n
[vals,φ = rew(si) + (cid:80)
(¬defs,φ ↔ [((cid:87)
(holdss,φ2 = ⊥ ∧ vals,φ ̸= rew(si))])])

s′∈supp(α1)×...×supp(αn)(((cid:81)n
s′∈supp(α1)×...×supp(αn) ¬defs′,φ)∨

j=1 σsj = αj) →

i=1 P (sj, αj, s′

j)) · vals′,φ)∧

return E

3

4

5

6

7

8

9

10

11

12

13

14

3

4

5

6

7

8

9

10

11

using the libraries Lark [lar] for parsing the input formula, and Stormpy [stob] for parsing the

input MDPR. The generated constraints are then solved by the SMT solver Z3 [dMB08].

Our implementation cannot handle all possible cases of undefinedness. We currently do

not calculate the extent of partial definedness of a property in a model. We check whether

98

Case
study

1-bit key
16-bit key
30-bit key
45-bit key

VR

×
×
×
×

✓
s=(0)
✓
s=(0,1,2)
s=(0,. . .,4) ✓
s=(0,. . .,6) ✓
✓
×
✓
×
✓
×
✓
×

3x3
3x3
4x4
4x4
5x5
5x5
6x6
6x6

n = 3
n = 5

n = 3
n = 4
n = 5
n = 6

✓
✓

✓
✓
✓
✓

TA

PC

RO

HS

IJ

Running time (s)

Encoding Solving

#SMT
Total variables

0.11
16.41
143.49
774.53

0.01
3.69
44.64
1304.98

0.12
20.10
188.13
2079.51

344
19244
62868
137448

5.03
6.66
8.82
11.64

0.87
0.93
3.55
3.43
13.07
13.19
44.52
44.65

0.1
0.95

0.08
0.24
0.89
3.93

2.03
8.91
35
53.05

0.05
0.05
0.28
0.25
0.5
0.98
1.04
7.48

0.01
0.13

0.01
0.04
0.33
19.39

7.06
15.57
43.82
64.69

0.92
0.98
3.83
3.68
13.57
14.17
45.56
52.13

0.11
1.08

0.09
0.28
1.22
23.32

7281
7281
7281
7281

2179
2179
6561
6561
15651
15651
32041
32041

489
2369

169
601
2233
8569

#sub #states #transitions

formulas

1008
49728
160160
348080

34681
61631
88581
115531

7622
7622
21572
21476
48302
48302
96096
96096

4655
7047

698
2194
7010
23362

8
68
124
184

20
20
20
20

18
18
32
32
50
50
72
72

8
32

7
15
31
63

10
100
184
274

186
494
802
1110

66
66
160
148
250
250
398
398

28
244

21
56
140
336

Table 6.2: Experimental results. VR: Verification result. TA: Timing attack. PC: Proba-
bilistic conformance. RO: Robotics example. HS: Herman’s algorithm. IJ: Israeli-Jaflon’s
algorithm. ✓: the result is true. ×: the result is false.

the states queried in the property are reachable with a probability of one and proceed in

the calculation of rewards in such cases. Hence, we have evaluated case studies, where the

reachability probabilities are always one.

The concept of rewards has eased the modeling of case studies with respect to count-

ing of expected steps needed to reach a state. Hence, for timing attack and probabilistic

conformance case studies, the number of transitions and states are less when compared to

the models used in [ ´ABBD20b]. The implementation also returns a witness/counterexample

whenever possible, allowing us to synthesize schedulers. Note that, though the ensemble

of schedulers in the executions (i.e. σ in the semantical context) define a scheduler in the

self-composition, not all schedulers of the self-composition can be defined this way, posing a

major difference between scheduler synthesis for PCTL and for HyperPCTL.

99

For the TA case study, we have modeled the problem with {1, 16, 30, 45}-bit encryption

keys. We have verified the HyperPCTL formula described in Section 6.3.1. The property does

not hold on the given model and our implementation finds this bug. Since our implementation

can handle only one scheduler quantifier, we have added a second copy of the model to the

input MDPR such that the single scheduler can assign different actions to the states in the

two copies of the model.

For the PC case study, we have verified the property described in Section 6.3.2. We have

started with a model with all possible transitions, represented non-deterministically, from

the initial state s0. For all other states, we allowed only the transitions that would give us

a correct solution. We challenged our implementation to synthesize a scheduler that will

satisfy the required probabilities within the given reward bound. We scaled the model by

incrementally allowing all possible combinations of transitions using nondeterministic actions

in each state and limited the expected coin tosses to be 4 for each experiment. For all the

cases, our implementation was successful in finding a solution, which we verified manually

as correct.

For the RO case study, we have verified the property described in Section 6.3.3. We

have scaled the model in terms of maze size and verified both positive and negative cases of

pathfinding. On self-stabilizing systems, we have verified several properties and described

one of them in Section 6.3.4. This property is satisfied and we have successfully found a

witness. We have reported the timing data for this property in Table 6.2. We have verified

the property in models representing both Herman’s (HS) and Israeli-Jaflon’s (IJ) [IJ90]

algorithms. Since Herman’s algorithm is only valid for odd processes, we tried verification

over {3, 5} processes. For Israeli-Jaflon’s, we tried it over {3, 4, 5, 6} process.

The experiments have been performed in a Docker container running on a system with

a 2.3 GHz i7 processor and 32 GB of RAM. Because of the incomplete implementation of

handling of undefined values, which would add a significant number of additional constraints,

the reported execution times are lower than they would normally be. From Table 6.2, it is

100

clear that the execution times for even relatively small MDPRs are large. This is because of

the inherent complexity of the problem, to which reward operators add a new dimension of

complexity.

6.6 Summary

Probabilistic logics allow us to argue about the reachability of a state or set of states.

However, they cannot differentiate between paths used to reach the states as ‘good’ or ‘bad’

based on criteria like robustness, less energy or power consumption, avoidance of obstacles,

etc.

In this work, we focused on extending our previously proposed general language to

connect it to reward models. This provided additional power to express the concept of ‘best’

or ‘efficient’ paths among all possible paths found. We have also extended our previously

proposed SMT-based algorithm and demonstrated our approach in a few interesting case

studies. This extension came with the challenge of undefined reward values that are caused

by the basic definition of rewards being infinite for unreachable states. As part of future

work for this extension, we have to modify our implementation to accommodate for undefined

results and consider other forms of rewards such as transition-based results and conversion

between state and transition rewards.

101

Chapter 7

HyperProb: A Model Checker for
Probabilistic Hyperproperties

7.1

Introduction

In this chapter, we introduce the tool HyperProb, a model checker for verifying proba-

bilistic hyperproperties expressed in the temporal logic HyperPCTL [ ´AB18, ´ABBD20b] on

Markov Decision Processes (MDP) given as a PRISM model [KNP11]. HyperProb reduces

the model checking problem to a satisfiability modulo theory (SMT) problem, implemented

by the SMT-solver Z3 [dMB08]. Along with the Boolean verdict of the model checking

problem, HyperProb may provide a witness or a counterexample to the hyperproperty repre-

sented as a deterministic memoryless scheduler. A witness is provided when the probabilistic

hyperproperty containing an existential quantifier, holds, and it can be used to synthesize

the induced discrete-time Markov chain (DTMC) satisfying the desired probabilistic hyper-

property. A counterexample is provided when the probabilistic hyperproperty containing a

universal quantifier over all possible schedulers, does not hold, and it can be exploited to

synthesize an adversarial attack that may violate the desired probabilistic hyperproperty.

Our implementation is available at: https://www.cse.msu.edu/tart/tools. This tool is the

culmination of the different extensions to the logic and model checking algorithm we dis-

cussed in the previous chapters.

102

7.2

Input to the Tool

1 mdp
2 module basic_mdp
3 h : [0..1];

// high input

4

ℓ: [0..2]; // low output

( h =0) → 1/2: (ℓ ’=1) + 1/2: (ℓ ’=2);
( h =1) → 1/2: (ℓ ’=1) + 1/2: (ℓ ’=2);
(!ℓ=0) → 1: true ;

5 [ alpha ] (ℓ=0 & h =0) → 3/4: (ℓ ’=1) + 1/4: (ℓ ’=2);
6 [ alpha ] (ℓ=0 & h =1) → 2/3: (ℓ ’=1) + 1/3: (ℓ ’=2);
7 [ beta ]
8 [ beta ]
9 [ tau ]
10 endmodule
(ℓ=0) endinit
11 init
12 label " h0 " = ( h =0);
13 label " h1 " = ( h =1);
14 label "ℓ1"
15 label "ℓ2"

= (ℓ=1); //ℓ1 label on s2 , s4
= (ℓ=2); //ℓ2 label on s3 , s5

// initial states
// h0 label on s0
// h1 label on s1

Figure 7.1: PRISM model generating the MDP in Fig.7.2

{h0}

α

1
4

3
4

s0

1
2

β
1
2

s1

1
3

α
2
3

{h1}

β

1
2

1
2

s2

}
1
ℓ
,
0
h
{

s3

}
2
ℓ
,
0
h
{

s4

}
1
ℓ
,
1
h
{

s5

}
2
ℓ
,
1
h
{

τ 1

τ 1

τ 1

τ 1

Figure 7.2: MDP of the PRISM program in Fig 7.1.

Input modeling language. The input model is provided as a high-level PRISM1 program

[KNP11]. We illustrate the language here on a simple example. This program takes a high-

security input h and computes a low-security output ℓ. The execution steps are represented

symbolically by probabilistic actions. For example, line 5 in Fig. 7.1 declares that action

alpha can be executed if ℓ = 0 and h = 0, and the action resets ℓ to the value 1 with

1https://www.prismmodelchecker.org/

103

φsched
φstate

ϕ

c

p

ψ

::= “AS” NAME “.” φstate
::= ϕ | “A” NAME “.” φstate

| “ES” NAME “.” φstate

| “E” NAME “.” φstate

::= “t” | “f” | NAME “(” NAME “)” | “∼” ϕ |

ϕ “&” ϕ | ϕ “|” ϕ | ϕ “=>” ϕ | ϕ “< − >” ϕ | c

::= p “<” p | p “<=” p | p “=” p | p “>=” p | p “>” p

::= “P” ψ | p “+” p | p “-” p | p “.” p | NUM

::= “(X” ϕ “)” | “(” ϕ “U” ϕ “)” |

“(” ϕ “U[” NUM “,” NUM “]” ϕ “)” | “(F” ϕ “)”

Figure 7.3: Grammar defining HyperPCTL inputs to HyperProb, where the NUM token is a
decimal number and NAME is a non-empty string.

probability 3/4, and to the value 2 with probability 1/4. Line 10 defines that ℓ = 0 initially.

Modelling each state explicitly yields the Markov decision process (MDP) depicted in Fig.

7.2, where the state labelling with atomic propositions is defined in the lines 11-14 in Fig.

7.1. For the given PRISM program, we use STORMpy to parse the model and to generate the

underlying MDP.

Specification language: To protect the secret value h, there should be no probabilistic

dependency between observations on the low variable ℓ and the value of h. For example,

an attacker that chooses a scheduler that always takes action α from states s0 and s1 can

learn whether or not h = 0 by observing the probability of obtaining ℓ = 1 (or ℓ = 2). Such

properties, which compare observations on system execution when using different schedulers

in different initial states are probabilistic hyperproperties. These properties, which should be

model checked by HyperProb, can be specified in the temporal logic HyperPCTL [ ´ABBD20b].

We refer to [ ´ABBD20b] for a detailed description of HyperPCTL but show the tool’s input

grammar in Fig. 7.3. For example, the property that we expect from the MDP in Fig 7.2 is

that for any scheduler (AS sched.) and any two initial states s1 and s2 (A s1. A s2.) with

different high input values (h0 (s1) ∧ h1 (s2)), the probability to reach the low value 1 from

104

s1 (P F ℓ1(s1)) is the same as from s2 (P F ℓ1(s2)), and analogously for the low value 2:

AS sched. A s1. A s2. (h0 (s1) ∧ h1 (s2)) =>

(P F ℓ1(s1) = P F ℓ1(s2)) & (P F ℓ2(s1) = P F ℓ2(s2))

(7.1)

Note that in the above formula, we use expressions of the form a(s) to state that the

atomic proposition a holds in the computation tree starting from the (quantified) state s.

For this formula, our tool will provide a violation as output with a deterministic memoryless

scheduler that would leak information to an attacker.

7.3 Tool Structure and Usage

In this section, we elaborate on the implementation details of the tool and its usage along

with examples.

7.3.1 Implementation

HyperProb, is an optimized implementation of the algorithm presented in [ ´ABBD20b].

Given an MDP and a HyperPCTL property, it verifies if the given hyperproperty holds in the

input MDP. Depending on the scheduler quantifier in the hyperproperty, if it holds, we get

a witness to the property (in the case of ∃) or if the hyperproperty does not hold, we get a

counterexample in the model (in the case of ∀). The witness or counterexample is defined

by the actions chosen at each state to obtain the required DTMC.

The tool has been implemented using Python3 and depends on several python packages.

Computer Arithmetic and Logic (CArL) [CARa], is an open source C++ library for handling

of complex mathematical computations and is needed by STORM. carl-parser [CARb] is

an ANTLR based parser which is meant as an extension to CArL. pyCArL [pyc] essentially

provides python bindings for CArL and is a dependency for STORM [stoa]. STORM is an

academically developed probabilistic model checker. STORMpy [stob] is the python binding

for STORM that we use to parse the input model. For the ease of usage, we have provided

105

Figure 7.4: Overview of the docker container with the tool and its dependencies.

a docker image for the user. The image comes pre-installed with Ubuntu, all the required

dependencies, and the tool. Docker makes it easier for the user to run the tool, as they do

not have to install any of the dependencies mentioned above. The main advantage of docker

is that running of the tool becomes independent of the operating system the user has. The

overall view of the docker container can be seen in Fig. 7.4.

Inside the tool, as shown in Fig. 7.5 , we first parse the model using STORMpy to store

them in an optimized way and for easy retrieval of the details of the model like labels,

transitions, states, and actions. We, then, parse the input hyperproperty into a syntax tree

that allows us to recursively encode the property in the next stages. Using the parsed model,

we first encode all possible actions in each state as SMT constraints. Using the parsed model

and the parsed property, we encode, as SMT constraints, both, the quantifier combinations

by generating constraints that should be satisfied at all states (for ∀) or by at least one state

(for ∃), and the semantic interpretation of the operators in the formula.

In the final step of the algorithm, we feed these constraints to the SMT solver Z3 [dMB08].

If our scheduler quantifier is a ∃, and the constraints are satisfiable, the tool outputs a

witness to the property, as a set of actions that should be chosen at each state of the model.

If our scheduler quantifier is a ∀, and the constraints are unsatisfiable, the tool outputs a

counterexample to the property as a set of actions that should be chosen at each state of

106

MDP model

Hyperproperty

Parser for
model

Parser for
property

Encode quantifier
combinations

Encode scheduler
choices

Z3 solver

Encode property
semantics

Satisfaction
and
Witness

Dissatisfaction
and
Counterexample

Figure 7.5: Dataflow inside the tool.

the model. For all the other combinations of the scheduler quantifier and satisfiability, the

tool returns a true/false but with no sequence of actions.

HyperProb incorporates additional optimizations whose impact is reflected in Table 7.1

(columns N referring to original implementation in [ ´ABBD20b], and O referring to the opti-

mized implementation). In particular, instead of considering all possible state combinations

when encoding a state formula for Z3, we consider only the relevant state combinations. For

example, let us consider the scenario in Fig. ?? with six states, and its HyperPCTL specifica-

tion as described in Section 7.2 with two state quantifiers. Our goal is to encode the atomic

proposition h0(s1). In the unoptimized version, this is encoded for 36 state combinations

((s01, s02), (s01, s12), . . . , (s11, s02), . . . , (s51, s52)), since we have two copies of the MDP. How-

ever, in HyperProb, we consider the information that while encoding h0(s1), only states of the

second copy of the MDP are relevant. Hence, we encode it for just six state combinations,

((s01, s02), . . . , (s01, s52)). We keep the first state for the irrelevant copy as the first member

of every state combination. This not only reduces the number of constraints generated but

also relaxes the constraints Z3 needed work on.

7.3.2 Usage

In order to use the tool, we will first need to ensure that our system has docker [doc]

installed in it. Then, download the docker image and create a container from it.

Inside

107

the container, all dependency packages are installed in the /opt folder under the root direc-

tory. The main tool package is located in /home/HyperProb folder under the root directory.

We can add our model file anywhere in this folder. The tool can be run by invoking the

source.py script with appropriate inputs in the format,

python source.py file path for model hyperproperty

Here file path for model refers to the file path with respect to /home/HyperProb directory as

base and the hyperproperty is written according to the grammar in Section 7.2. For example,

if your file is /home/HyperProb/models/mdp.nm, your command would be,

python source.py models/mdp.nm hyperproperty

The commands to replicate all our case studies have been placed under /home/Hyper-

Prob/benchmark files/Experiments.txt.

7.4 Evaluation

In this section, we remind the readers briefly of our benchmarks and discuss our experi-

mental results, comparing them to our previous implementations.

7.4.1 Case Studies

Side-channel timing leaks

open a channel for attackers to infer the value of a secret by

observing the execution time of a function. For example, the heart of the RSA public-key

encryption algorithm is the modular exponentiation algorithm that computes (ab mod n),

where a is an integer representing the plaintext and b is the integer encryption key. A careless

implementation can leak b through a probabilistic scheduling channel (see Fig. 7.6). This

program is not secure since the two branches of the if have different timing behaviours.

Under a fair execution scheduler for parallel threads, an attacker thread can infer the value

108

1 v o i d mexp ( ) {

2

3

4

5

6

7

8

c = 0 ; d = 1 ;
w h i l e ( i >= 0 ) {

i = k ;

i = i −1; c = c ∗ 2 ;
d = ( d∗d ) % n ;
( b ( i ) = 1 )
i f

c = c +1;
d = ( d∗ a ) % n ;

}

9
10 }
11 /∗∗∗∗∗∗∗∗∗∗∗∗/
12 t = new Thread ( mexp ( ) ) ;
13 j = 0 ; m = 2 ∗ k ;
14 w h i l e ( j < m & ! t . s t o p )
15 /∗∗∗∗∗∗∗∗∗∗∗∗/

j ++;

Figure 7.6: Modular exponentiation.

of b by running in parallel to a modular exponentiation thread and iteratively incrementing a

counter variable until the other thread terminates (lines 12-14). To model this program by an

MDP, we can use two nondeterministic actions for the two branches of the if statement, such

that the choice of different schedulers corresponds to the choice of different bit configurations

b(i) for the key b. This algorithm should satisfy the following property: the probability of

observing a concrete value in the counter j should be independent of the bit configuration

of the secret key b:

∀ˆσ1.∀ˆσ2.∀ˆs(ˆσ1).∀ˆs′(ˆσ2).

(cid:16)

initˆs ∧ initˆs′

(cid:17)

⇒

P(

(j = ℓ)ˆs) = P(

(cid:17)

(j = ℓ)ˆs′)

(7.2)

m
(cid:94)

(cid:16)

ℓ=0

Another example of a timing attack that can be implemented through a probabilistic

scheduling side channel is password verification. It is typically implemented by comparing

an input string with another confidential string (see Fig 7.7). Also here, an attacker thread

can measure the time necessary to break the loop and use this information to infer the prefix

of the input string matching the secret string.

109

1 i n t

s t r c m p ( c h a r ∗ r ) {
c h a r ∗ s = ’ Bg\ $4 \ 0 ’ ;
i = 0 ;
w h i l e ( s [ i ]

!= ’ \ 0 ’ ) {

( s [ i ] ! = r [ i ] )

i ++;
i f
}
r e t u r n 1 ;

r e t u r n 0 ;

2

3

4

5

6

7

8
9 }

Figure 7.7: String comparison.

Scheduler-specific observational determinism policy (SSODP) [NSH13] is a confi-

dentiality policy in multi-threaded programs that defends against an attacker choosing an

appropriate scheduler to control the set of possible traces. In particular, given any scheduler

and two initial states that are indistinguishable with respect to a secret input (i.e., low-

equivalent), any two executions from these two states should terminate in low-equivalent

states with equal probability. Formally, given a proposition h representing a secret:

∀ˆσ.∀ˆs(ˆσ).∀ˆs′(ˆσ).(cid:0)hˆs ⊕ hˆs′

(cid:1) ⇒

(cid:94)

(cid:0)P(

ℓ∈L

ℓˆs) = P(

ℓˆs′)(cid:1)

(7.3)

where ℓ ∈ L are atomic propositions that classify low-equivalent states and ⊕ is the exclusive-

or operator. A stronger variation of this policy is that the executions are stepwise low-

equivalent:

∀ˆσ.∀ˆs(ˆσ).∀ˆs′(ˆσ).(cid:0)hˆs ⊕ hˆs′

(cid:1) ⇒ P (cid:0) (cid:94)

(cid:0)(P ℓˆs) = (P ℓˆs′)(cid:1)(cid:1) = 1.

(7.4)

ℓ∈L

Probabilistic conformance describes how well a model and an implementation conform

with each other with respect to a specification. As an example, consider a six-sided die. The

probability to obtain one possible side of the die is 1/6. We want to synthesize a protocol

that simulates the six-sided die behaviour only by repeatedly tossing a fair coin. We know

that such an implementation exists [KY76], but we aim to find such a solution automat-

ically by modeling the die as a DTMC and by using an MDP to model all the possible

110

coin-implementations with a given maximum number of states, including six absorbing final

states to model the outcomes. In the MDP, we associate with the states, a set of possible

nondeterministic actions, each of them choosing two states as successors with equal prob-

ability 1/2. Then, each scheduler corresponds to a particular implementation. Our goal is

to check whether there exists a scheduler that induces a DTMC over the MDP, such that

repeatedly tossing a coin simulates die-rolling with equal probabilities for all possible out-

comes:

∃ˆσ.∀ˆs(ˆσ).∃ˆs′(ˆσ).

(cid:16)

initˆs ∧ initˆs′

(cid:17)

⇒

6
(cid:94)

(cid:16)

ℓ=1

P(

(die = ℓ)ˆs) = P(

(die = ℓ)ˆs′)

(cid:17)

(7.5)

7.4.2 Results and Discussions

All of our experiments in Section 7.4.1 were run on a MacBook Pro with a 2.3GHz quad-

core i7 processor with 32GB of RAM. The results are presented in Table 7.1. For our first

case study TA, described in subsection 7.4.1, models and analyses information leakage in the

modular exponentiation algorithm. We experimented with up to four bits for the encryption

key (hence, m ∈ {2, 4, 6, 8}). The specification checks whether there is a timing channel for

all possible schedulers, which is the case for the implementation in modexp.

Our second case study PW, described in subsection 7.4.1 handles the verification of

password leakage through the string comparison algorithm. Here, we experimented with

m ∈ {2, 4, 6, 8}.

In our third case study TS, described in subsection 7.4.1, we assume two concurrent

processes. The first process decrements the value of a secret h by 1 as long as the value

is still positive, and after this, it sets a low variable ℓ to 1. A second process just sets the

value of the same low variable ℓ to 2. The two threads run in parallel; as long as none of

them terminates. A fair scheduler chooses for each CPU cycle the next executing thread.

This opens a probabilistic thread scheduling channel and leaks the value of h. We compare

observations for executions with different secret values h1 and h2 (denoted as h = (h1, h2)).

111

Case Study

Running time(s)
SS
N O

O

SE
N

Total
N

O

#SMT
variables

N

O

#op #st #tr

TA

PW

TS

PC

m = 2
m = 4
m = 6
m = 8

m = 2
m = 4
m = 6
m = 8

5
114
1721
12585

5
207
3980
26885

2
18
140
952

2
26
331
2636

< 1 < 1
1
20
45
865
TO 426

< 1 < 1
1
40
1099
41
TO 364

2
5
19
134
2586
185
TO 1378

3
6
27
247
5079
372
TO 3000

h = (0, 1)
h = (0, 15)
h = (4, 8)
h = (8, 15)
h = (10, 20)

< 1 < 1
8
3
8
19

60
12
60
186

< 1 < 1
1607 < 1
17 < 1
1606 < 1
1
13707

s=(0)
s=(0,1)
s=(0..2)
s=(0..3)
s=(0..4)
s=(0..5)
s=(0..6)

277
822
1690
4631
7353
10661
13320

10
13
15
16
22
19
18

5
1996
5
5808
5
TO
7
TO
TO 21
TO 61
TO 41

1
1667
29
1666
13893

2273
6630
TO
TO
TO
TO
TO

1
8
3
8
20

15
18
20
23
43
80
59

8088
50460
175728
388980

8088
68670
274540
657306

1379
34335
12369
34335
52695

21220
21220
21220
21220
21220
21220
21220

2520
14940
51184
131220

2520
20230
79660
221130

441
8085
3087
8085
13095

6780
6780
6780
6780
6780
6780
6780

14

14

28

44

24
60
112
180

24
70
140
234

7
35
21
35
45

20
20
20
20
20
20
20

46
136
274
460

46
146
302
514

13
83
48
83
108

188
340
494
648
802
956
1110

Table 7.1: Experimental results and comparison. TA: Timing attack. PW: Password leak-
age. TS: Thread scheduling. PC: Probabilistic conformance. TO: Timeout. N: Prototype
presented in [ ´ABBD20b]. O: HyperProb,. SE: SMT encoding. SS: SMT solving. #op:
Formula size (number of operators). #st: Number of states. #tr: Number of transitions.

There is an interesting relation between the data for TS. Both the encoding and running

time for the experiment are proportional to the higher value in the tuple h.

Our last case study PC, described in subsection 7.4.1, is on probabilistic conformance.

The input is a DTMC modelling a fair 6-sided die as well as an MDP whose actions model

single fair coin tosses with two successor states each. We are interested in finding a scheduler

that induces a DTMC that simulates the die outcomes using a fair coin. Given a fixed state

space, we experiment with different numbers of actions. In particular, we started from the

implementation in [KY76] and for the state space of the die section of the protocol, we

added all the possible nondeterministic transitions from the first state to all the other states

(denoted s = 0), from the first and second states to all the others (s = 0, 1), and, similarly

scaled it stepwise to include transitions from all states to all others (s = 0 . . . 6). Each time,

112

we were not only able to satisfy the formula, but also obtain the witness corresponding to

the scheduler satisfying the property.

In our previous prototypical implementation [ ´ABBD20b], due to the encoding of all

formulas for all composed states, both the encoding and SMT solving times were significantly

higher. Hence, we opted for a timeout for cases where the timing did not seem practically

useful. For TA, PW, and PC, we used a timeout of 10000s for the SMT solving.

7.5 Summary

This work brought together the concepts defined in the last chapters into one concrete

tool. Here we discussed in detail the construction and working of our tool HyperProb which

is the only existing model checker for probabilistic hyperproperties. We have elaborated on

the grammar used, how we have defined our models in PRISM language, and described how

the tool should be run. As part of future work, we intend to build a graphical user interface

for the tool to make it easy to use and accessible.

113

Chapter 8

Efficient Probabilistic Model
Checking for Relational Reachability

Following from the previous chapters, the model checking problem for HyperPCTL is

shown to be undecidable, in general, and decidable for memoryless schedulers. The exact

computational complexity grows in the polynomial hierarchy with each scheduler quantifier

alternation.

Intuitively, the computational difficulty of HyperPCTL model checking stems

from the fact that the decision problem is indeed a synthesis problem to find schedulers

that satisfy relational reachability conditions (i.e., comparison between the probabilities of

reachability of certain states). Our SMT-based verification procedure in HyperProb, although

exhautive for memoryless and deterministic schedulers, suffers from severe scalability issues.

This severely limits the scope of application of algorithms.

On close inspection, many applications, especially in the domain of information-flow

security and privacy, fall within certain specific fragments and do not need the full power of

the logic. Thus, in this chapter, we aim to answer the following research question:

Is it possible to identify fragments of probabilistic hyper temporal logics that cover

a wide range of applications (e.g., information-flow security) and devise efficient and

effective fragment-specific model checking algorithms?

We believe such algorithms will have considerable impact on the accessibility of model check-

114

ing for probabilistic hyperproperties. With this motivation, we focus on fragments of Hyper-

PCTL capable of expressing popular quantitative information-flow policies. These fragments

essentially involve quantitative relational reachability reasoning to determine whether quan-

tified policies and computations reaching certain observations under those schedulers yield

equal probability. We investigate the following simple but natural fragments of HyperPCTL

and propose algorithms that solve model checking decision problems for them:

• (1σ1s) Does there exist a scheduler σ, such that under this scheduler, the probability

of reaching a public observation a from a state s is the same as the probability of

reaching a public observation b from s?

• (1σ2s) Does there exist a scheduler σ, such that under this scheduler, the probability

of reaching an observation a on a public channel from a state s1 is the same as the

probability of reaching an observation b on a public channel from a state s2?

• (2σ2s) Do there exist schedulers σ1 and σ2, such that the probability of reaching a

from s1 under σ1 is the same as the probability of reaching b from s2 under σ2?

We provide a PTIME algorithm for synthesizing randomized memoryful schedulers for

the fragment (1σ2s) or (2σ2s) fragment. For the (1σ1s) fragment, we provide a bounded

model checking algorithm viewing MDPs as transformers of distributions of states. We also

demonstrate the effectiveness of our algorithms by conducting several case studies, including

timing side-channel attacks in the modular exponentiation algorithm, probabilistic attacks

in thread scheduling, and violations of secrecy due to scheduling attacks. We observe huge

performance improvement (orders of magnitude speed-up) as compared to the general SMT-

based technique proposed in [D ´ABB22].

8.1 Preliminaries

In addition to the general concepts used across the chapters, here we introduce a few

new relevant terminologies. We use Distr(V ) to denote the set of all discrete probability

115

distributions (µ) over the finite set V such that µ : V → [0, 1], and (cid:80)

v∈V µ(v) = 1. For

paths in MDPs, we use pref(π, i) := s0α0s1 . . . αi−1si to denote the prefix of length i.

We define the convex addition [Qua23, p.71] of two schedulers σ, σ′ ∈ ΣM with weight

λ ∈ [0, 1] as the scheduler [σ⊕λσ′] that initially chooses to behave as σ with probability λ and

with the remaining probability chooses to behave as σ′. Formally, for π = s0α0s1 . . . αn−1sn ∈

PathsM

fin and α ∈ Act we let

[σ ⊕λ σ′](π)(α) :=

σ(π)(α) · λ · p⟨σ|π⟩ + σ′(π)(α) · (1 − λ) · p⟨σ′|π⟩
λ · p⟨σ|π⟩ + (1 − λ) · p⟨σ′|π⟩

(8.1)

where

p⟨σ|π⟩ :=

n−1
(cid:89)

i=0

σ(pref(π, i))(αi)

is the probability that σ was chosen given that we have taken path π so far.

A DTMC D can be viewed as a transformer of distributions. For some i ∈ N, we define

µD,s0
i

∈ Distr(S) as the distribution after i steps from state s0 ∈ S. Formally,

µD,s0
i

(s) := Pr D

s0({π ∈ PathsD

s0 | π(i) = s}), for s ∈ S.

(8.2)

One can formalize these policies in the temporal logic HyperPCTL [D ´ABB22] as follows:

(1σ1s) ∃ˆσ. ∀ˆs(ˆσ). ιinit ˆs =⇒ P(

aˆs) = P(

bˆs)

(1σ2s) ∃ˆσ. ∀ˆs1(ˆσ)∀ˆs2(ˆσ). (ι1

init ˆs1

∧ ι2

init ˆs2

) =⇒ P(

aˆs1) = P(

bˆs2)

(2σ2s) ∃ˆσ1∃ˆσ2. ∀ˆs1(ˆσ1)∀ˆs2(ˆσ2). (ι1

init ˆs1

∧ ι2

init ˆs2

) =⇒ P(

aˆs1) = P(

bˆs2)

In the rest of this chapter, for each fragment, we only consider MDPs where for every state

variable ˆs (or ˆsi) there exists a unique initial state labelled with ιinit (or ιinit

i, respectively).

8.2 Model checking algorithm for (2σ2s) fragment

Consider the model in Fig. 8.1. It shows a fragment of the MDP used to model a side-

channel leak using the execution timing of a program. Each of the sub-MDPs represents

116

{init1}

s0

s1 {init2}

1
2

s3

α

1

Sink

α
1
2

1
2

s2

β

1
2

α

1
2

1
2

s6

α
1
2

1
2

1

{a}

s8

α

α
1
3

2
3

s4

β

1
2

α

2
3

1
3

s5

α

1
2

1

s7

α

1
2

1
2

1

{b}

s9

α

Sink

Figure 8.1: (Partial) MDP modelling information leakage via side-channel.

the scheduling between two threads - one where the actual password checking takes place,

and the other is an attacker thread that runs parallel to the first thread and checks for its

termination. The difference lies in the probability of choosing the threads in two different

schedules. Ideally, starting from the initial states labelled ιinit

1 and ιinit

2, we should be able

to reach the target states labelled a and b, respectively, with equal probability. Formally,

∃ˆσ1∃ˆσ2. ∀ˆs1(ˆσ1)∀ˆs2(ˆσ2). (ιinit

1
ˆs1 ∧ ιinit

2

ˆs2) =⇒ P(

aˆs1) = P(

bˆs2)

(8.3)

In this case, neither σa nor σa achieves probability equal to σb or σb. However, random-

ization can be introduced to combine these schedulers to satisfy the equality requirement.

Alg. 17 shows our model-checking algorithm for the (2σ2s) fragment. The algorithm

takes as input an MDP and a (2σ2s) HyperPCTL formula specified by the labels of the distinct

initial and target states. If a satisfying pair of schedulers exists, the algorithm generates as

output a pair of (memoryful) probabilistic schedulers. The algorithm has a polynomial run

time in the size of the MDP M. More precisely, it individually model checks M for each

target and then makes a linear pass over the resulting memoryless schedulers. The core idea

revolves around combining the minimizing and maximizing schedulers for each target and is

based on [Qua23, p.70-75].

117

Algorithm 17: PTIME model-checking algorithm for the (2σ2s) fragment.
init ∈ L(s1), ι2

Input: M: MDP with two initial states s1, s2 ∈ S with ι1

init ∈ L(s2),

φ: (2σ2s) formula

Output: Whether M |= φ and, if yes: Witnesses for ˆσ1, ˆσ2

1 Function Main(M, φ):
a := maxσ Pr σ
2
a, b, b and σa, σb, σb: Analogously
if I := [a, a] ∩ [b, b] = ∅ then

s1(

4

3

a), and σa := arg maxσ Pr σ

s1(

a)

5

6

7

8

9

10

return False

else

Choose some c ∈ I
λa := c−a
a−a , λb := c−b
b−b
σ1 := [σa ⊕λa σa], σ2 := [σb ⊕λb σb]
return True, (σ1, σ2)

We now describe the algorithm in detail and parallelly illustrate it with the MDP in

Fig. 8.1. We intend to verify the property formalized above on M and, if it holds, return a

pair of schedulers as witnesses. In line 2, we calculate the maximum reachability probability

a for reaching states labelled with a from the relevant initial state and extract the corre-

sponding scheduler in σa (this can be done by standard MDP model checking algorithms).

For our example MDP, the maximum probability a = 1

4 can be obtained from the DTMC
induced (shown in Fig. 8.2b) by σa on Fig. 8.1. We analogously calculate the maximum

and minimum reachability probability (b and b, respectively) to states labelled b from the

other initial state. The corresponding schedulers are extracted as σb and σb. For initial state

init2 the minimum probability to reach b, b = 1

12, can be obtained using DTMC induced

(as shown in Fig. 8.2c) by σb, and the maximum probability b = 2

9 can be obtained using

DTMC induced (as shown in Fig. 8.2d) by σb.

The intersection of the ranges [a, a] and [b, b] determines the existence of satisfying sched-

ulers (line 4). If such schedulers can exist, we choose a random value in the intersection

(line 7) and construct the schedulers using the convex addition of the respective minimal

and maximal schedulers (line 9). In our example, neither of the minimum or maximum sched-

ulers for the respective targets can satisfy the requirement of equal reachability. However,

118

{init1}

s0

1
2

s2

1
2

1
2

1
2

s6

1
2

1
2

1

s3

Sink

{a}

s8

1

s0

{init1}
1
2

1
2

s2

s3

1
2

1
2

1

{a}

s8

1

Sink

(a) DTMC induced by σa.

(b) DTMC induced by σa.

{init2}

s0

1
3

s2

1
2

1
2

1
2

s6

1
2

2
3

1

s3

Sink

{b}

s8

1

{init2}

s0

1
3

s2

2
3

1
3

2
3

1

s3

{b}

s8

1

Sink

(c) DTMC induced by σb.

(d) DTMC induced by σb.

Figure 8.2: DTMCs induced by different schedulers on MDP in Fig. 8.1.

since the ranges [ 1

8, 1

4] and [ 1

12, 2

Assuming c = 1

6, σ1 should choose σa with probability λa = 1

2

3. Similarly, σ2 should choose σb with probability λb = 3

9] are intersecting, we can generate a randomized scheduler.
3 and choose σa with probability
5 and choose σb with probability 2
5.

The correctness of Alg.17 follows from [Qua23, Le. 3.8]. It runs in polynomial time since

we can calculate the minimizing and maximizing schedulers in polynomial time and make a

linear pass through them.

119

Theorem 8.2.1. The model-checking problem for the fragment (2σ2s) over general
schedulers is in PTIME.

Remarks. This algorithm would also work if we allowed s1 = s2. Note that we focus on

reachability objectives here, but the algorithm can be extended to any temporal operator

since we are individually model checking each temporal objective and combining the obtained

schedulers. Hence, the algorithm only needs to be adapted in line 2 to allow computing

minimizing and maximizing schedulers for any temporal operator. Extending to general

HyperPCTL formulas with nested probability operators is, however, not straightforward: For

path formulas with nested probability operators, we cannot employ techniques like value

iteration to compute the maximizing and minimizing schedulers. We can adapt Alg. 17 for

the (1σ2s) fragment by computing a scheduler σ that behaves like σ1 on paths starting with

s1 and like σ2 otherwise.

8.3 Distribution-Based Approach for the (1σ1s) Frag-

ment

In this section, we regard MDPs as transformers of distributions and view (1σ1s) as a

distribution-based objective. Recall that the PTIME algorithm for the (2σ2s) fragment (17)

does not apply to the (1σ1s) fragment because we have to evaluate two separate probability

operators under the same scheduler. This is also the reason why a distribution-based view

only makes sense for the (1σ1s) fragment but not for (1σ2s) or (2σ2s): Distribution-based

objectives can only relate probabilities in a single computation tree, but in the latter frag-

ments, we compare probabilities of two independent computation trees.

Inspired by this

view, we present a bounded model-checking algorithm for this fragment.

Our inputs include the MDP model M under consideration, the specification φ of the

form (1σ1s) with target states labelled with a, b, and the maximum allowed size of the

explored path k. As before, we have a fixed initial state for the model. The algorithm

120

unfolds the MDP step-by-step as a distribution transformer and checks for the equality of

reachability of the targets. The algorithm can be summarized as follows:

• Backward value iteration: We use value iteration to track backward reachability from

the target states to each state in the M. This helps us to draw an initial conclusion

about the existence or absence of the scheduler. The intersection of the minimum and

maximum backward reachability values determines if all or none of the schedulers can

satisfy the property at hand. In either of these cases we can exit early without further

inspection.

• Guided Forward scheduler search: In case we cannot exit early, we begin a recursive

depth-first search unrolling of M by exploring an action combination (a set of actions

comprised of one action choice per state in the current state distribution). The choice

of action combination to be explored is guided by the possible reachability along the

path as calculated from the backward reachability values.

Alg. 18 describes the initial setup for the algorithm along with early exit conditions. For

each target label, we separately calculate the backward transition matrix for k steps and

keep track of the minimum (V l and V r for the left and right target label, respectively) and

maximum reachability (V l and V r for the left and right target label, respectively) across all

actions for each state. Based on these values for the initial state, we can determine if the

property is satisfied for all schedulers (line 4) or no scheduler (line 6). If we cannot exit early,

we create our initial state distribution to begin searching for a satisfying scheduler following

Alg. 19. The state distribution consists of information about the states (hereafter, referred

to as composed states) we are currently visiting and the probability with which we reached

that state. Internally, each composed state stores the identifier (name) of the original state

in M, and the probability with which we have reached the left (Pl) and right (Pr) target

state along our path used to visit this state in the current state distribution. For example,

initially, this state distribution comprises only the initial state, and we assign the whole

121

Algorithm 18: Model-checking algorithm for (1σ1s) fragment

Input: M = (S, Act, P, AP, L): MDP model, k: step bound, init: initial state of

M, φ: (1σ1s) formula.

Output: Scheduler satisfying M |= φ, if found, else null.

1 Function Main(M, k, init, φ):

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

V l, V l := GetBackwardValueIteration(M, a, k)
V r, V r := GetBackwardValueIteration(M, b, k)
if V l(init), V l(init), V r(init), V r(init) are equal then
return True: All schedulers satisfy the property

else if [V l(init), V l(init)] (cid:84) [V r(init), V r(init)] = ∅ then

return False: No such scheduler exists

if a ∈ L(init) then

Pl := 1

else

Pl := 0

if b ∈ L(init) then

Pr := 1

else

Pr := 0

scheduler := ForwardSchedulerSearch(M, 1, k, V l, V r, [{init, Pl, Pr} : 1], a, b)
// Assuming we combine V l and V l in V l, and V r and V r in V r
return scheduler

probability mass of 1 to it as shown in line 16. We additionally remember the reachability

to target states in the initial states as shown in lines 11, 15.

We begin our guided depth-first scheduler search in the recursive method shown in Alg. 19.

If we have already reached path length k (line 2), we check for the equality between the total

reachability of left and right targets across all states in the current distribution (this can

be modified to check whether the values differ by at most ϵ). Based on this we can either

conclude we have found a scheduler (line 3) and start backtracking to collect the actions

taken along this path (line 13), or that no such scheduler is possible in this path (line 5) and

backtrack to explore other options (line 11). For cases that have not reached the k bound,

we would ideally want to generate the set of all combinations of actions for the states in the

current distribution stdist (line 6) and explore for equality. However, to optimize the process,

we use Alg. 20 to filter action combinations that cannot lead to equality in reachability

122

Algorithm 19: Forward Scheduler Search

Input: M: MDP model, i: current iteration, k: step bound V l, V r: Backward

reachability to target states, stdist: distribution of states in the current step
along with the values for Pl, Pr reached in the last i − 1 steps, a: left target
label, b: right target label.

Output: Scheduler satisfying P(

a) = P(

b) in k-steps, if found, else null.

1 Function ForwardSchedulerSearch(M, i, k, V l, V r, stdist, a, b):
st.Pl == (cid:80)
2 if i == k and (cid:80)
return [ ]

st.Pr then

st∈stdist

st∈stdist

// empty scheduler

3
4 else
5

return null

6 priority queue := CheckNextActions(cid:0)M, V l, V r, stdist, a, b(cid:1)

// Naive approach: priority queue := GetAllNextActions(cid:0)M, stdist, a, b(cid:1)

7 while priority queue is not empty do
8

curract, currstate := priority queue.pop()
scheduler := ForwardSchedulerSearch(cid:0)M, i + 1, k, V l, V r, currstate, a, b(cid:1)
if scheduler is null then

9

10

11

12

13

14

15

Go to line 7

else if |scheduler| ≥ 0 then

Append curract to scheduler
return scheduler

return null

values. We then explore each action combination in the filtered set in a depth-first manner

(line 8).

In Alg. 20, we utilize the backward reachability values to determine if a specific action

combination is a good candidate. For a naive exhaustive search among action combinations,

line 3 would have sufficed. However, we can use the values calculated in the backward value

iteration step to determine the minimum and maximum reachability values from a particular

state. This can help us filter out action combinations that do not have an intersecting range

of reachability values. For each new set of actions for the current state distribution (line 4),

we generate its corresponding one-step next state distribution (line 5). We calculate the

minimum and maximum reachability for each state in this new distribution based on whether

we have already reached the target states in the history of states used to reach this current

state: (1) In case we did encounter the target states, we store their value in the Pl and Pr

123

Algorithm 20: Check Next Action

Input: M: MDP model, V l, V r: Backward reachability to left and right target

states, stdist: current state distribution, a: left target label, b: right target
label.

Output: Prioritized list of action combinations for all states in stdist.

1 Function CheckNextAction(M, V l, V r, stdist, a, b):
2

priority queue := [ ], f lag := true, actold := [ ], stold := [ ]
act dist = ×st∈stdistAct(st.name)
for αdist ∈ act dist do

newdist := CreateStateDistribution(stdist, stold, M, αdist, actold, f lag, a, b)
f lag := false, vl := 0, vl := 0, vr := 0, vr := 0
for s ∈ newdist do

if s.Pl > 0 then

[vl, vl]+ = [s.Pl, s.Pl]

else

[vl, vl]+ = [stdist[s] ∗ V l(s.name), stdist[s] ∗ V l(s.name)]

// Set [vr, vr] analogously

if s.Pl > 0 and s.Pr > 0 for all states in newdist then

Add (αdist, newdist) to priority queue with priority 1

else if [vl, vl] ⊆ [vr, vr] (or vice versa) then

Add (αdist, newdist) to priority queue with priority 2

else if [vl, vl] ⊂ [vr, vr] (or vice versa) then

Add (αdist, newdist) to priority queue with priority 3

actold := αdist, stold := newdist

return priority queue

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

section of the composed state and use them as the final reachability values along this path

(line 9), (2) otherwise, we calculate the minimum and maximum values using the probability

value of the state in the current distribution (stdist) and the values from the backward

reachability matrices (line 11). As we go through this filtering, we further prioritize these

action combinations based on these values for the new state distribution produced: (1) In

case we have encountered both the target states along the paths leading to all the states in

the current distribution (line 12), we explore this action with the highest priority as further

distribution transformation would add no more information. (2) In case the range of values

for reaching one target state is a subset of the range for the other target (line 14), explore

this option with second priority as there is at least one common value that we can surely

124

reach. (3) In case the range of value for reaching one target state is a proper subset of the

range for the other target (line 16), we explore this option with the least priority.

Algorithm 21: Create State Distribution

Input: stdist: current state distribution, stold: state distribution created by actold on
stdist, M: MDP model, αdist: set of actions choices we are currently
processing, f lag: denoting if we are generating a new state distribution or
changing an existing one, a: left target label, b: right target label.

Output: new stdist: state distribution by applying αdist on stdist.

1 Function CreateStateDistribution(stdist, stold, M, αdist, actold, f lag, a, b):
2

if f lag == true then

// Generating new state distribution
new stdist := [ ]
for i ∈ [ |stdist| ] do

succs := {s′ | P(stdist[i].name, αdist[i], s′) > 0}
for s′ ∈ succs do

if s′ ∈ {st.name|st ∈ new stdist} then

new stdist[s′]+ = stdist[i][val] ∗ P(stdist[i].name, αdist[i], s′)

else

new st := {name : s′, Pl : 0, Pr : 0}
if stdist[i].Pl is 0 and a ∈ L(s′) then

new st.Pl = stdist[i][val] ∗ P(stdist[i].name, αdist[i], s′)

else if stdist[i].Pl > 0 then

new state.Pl = stdist[i].Pl

// Set Pr value for the composed state analogously
Append (new state : val) to new stdist

else

// Modifying existing state distribution
for i ∈ {index | actold[index] ̸= αdist[index]} do

Remove successors of actold[i] and adjust val in stold
Add successors of αdist[i] in stold

return new stdist

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

In Alg. 21, we elaborate on the creation of the state distribution by applying αdist on stdist.

Note that this state distribution consists of composed states of the form (name, Pl, Pr), and

that the corresponding probability mass accumulated when reaching these states along the

path is referred to as val. As denoted by the f lag in line 2, when we call this method for the

first time on stdist, we collect the successors of each state in the distribution (line 5). When

creating the distribution, we either create a new corresponding composed state (line 10) and

125

update the corresponding reachability to target values along the path (lines 11-15), or we

update the value of the corresponding composed state that already exists in the distribution

(line 8). Once we have created an initial distribution for stdist, we only focus on the actions

that have changed between the old action combination actold considered in the previous call

and the currently considered αdist (line 17). To this effect, we remove the states or change

the value of the probability for the states associated with the action from actold (line 18) and

then add the new successor or modify its value in the distribution as required (line 19).

8.4 Experimental Evaluation

In this section, we give a brief overview of the different case studies and properties we

use to evaluate the effectiveness of our algorithms. Our case studies are motivated by the

models evaluated in [D ´ABB22]. We further discuss the experimental results and compare

their pros and cons with existing solutions.

8.4.1 Case Studies

Thread scheduling (TS)

while h > 0 do {h ← h − 1}; l ← 2

||

l ← 1

Consider the two threads above where h is a secret input and l is the publicly observable

output. Under uniform scheduling, we would be able to observe the different values of l

with different probabilities (l = 2 with a higher probability). This difference in observation

probability will increase as the value of h increases. We verify two different properties for

this model.

• We allow multiple different scheduling probabilities and want to synthesize randomized

schedulers such that starting from different values of h, we can observe a specific value

of l with equal probability. Formally,

TS − 2σ2s: ∃ˆσ1∃ˆσ2. ∀ˆs1(ˆσ1)∀ˆs2(ˆσ2). (h1
ˆs1

̸= h2
ˆs2

) =⇒ P(

(l = 1)ˆs1) = P(

(l = 1)ˆs2

126

1 v o i d mexp ( ) {

2

3

4

5

6

7

8

9

10

c = 0 ; d = 1 ;
w h i l e ( i >= 0 ) {

i = k ;

i = i −1;
c = c ∗ 2 ;
d = ( d∗d ) % n ;
( b ( i ) = 1 ) {
i f

c = c +1;
d = ( d∗ a ) % n ;

}

}

11
12 }
13 t = new Thread ( mexp ( ) ) ;
14 j = 0 ; m = 2 ∗ k ;
15 w h i l e ( j < m & ! t . s t o p )
16 j ++;

Figure 8.3: Modular exponentiation.

• We allow multiple probabilistic scheduling distributions and want to synthesize a sched-

uler such that, starting from a specific h value we can observe (l = 1) and (l = 2) with

equal probability. Formally,

TS − 1σ1s: ∃ˆσ. ∀ˆs(σ). hˆs =⇒ P(

(l = 1)ˆs) = P(

(l = 2)ˆs)

Side-channel timing attack (TA)

For security purposes, we want to be able to avoid opening a channel for side-channel

leaks based on the execution timing of a program. Consider the modular exponentiation

algorithm in RSA (method mexp shown in Fig. 8.3). The absence of an else block allows for

the opening of a side-channel timing leak. An attacker can run a thread parallel to the main

thread to infer the execution time of the methods (loosely considered similar to the number

of execution steps in which mexp terminates).

We aim to ensure we can verify the existence of scheduler(s) that provide the same

distribution over observable outputs, independent of the secret bit value used to calculate

it. This ensures similarity in the execution time of the program, thus avoiding disclosure of

information about the secret input. We experiment with two different properties.

127

1 i n t

s t r c m p ( c h a r ∗ r ) {

2

3

4

5

6

7

8

9

10

11
12 }

c h a r ∗ s = ‘ Bg\ $4 \ 0 ’ ;
i = 0 ;
w h i l e ( s [ i ] ! = ‘ \ 0 ’ ) {

i f ( r [ i ]== ‘\0 ’
r e t u r n 0 ;
i ++;

}
i f ( r [ i ]== ‘\0 ’)
r e t u r n 1 ;
r e t u r n 0 ;

| |

s [ i ] ! = r [ i ] )

Figure 8.4: String comparison.

• We want to synthesize a pair of schedulers (σ1, σ2) starting at states with different b

values and terminating with the same value for a specific bit of the counter j in the

parallel thread. We use Alg. 17 to generate this pair of randomized schedulers.

TA − 2σ2s: ∃ˆσ1∃ˆσ2. ∀ˆs1(ˆσ1)∀ˆs2(ˆσ2). (b1
ˆs1

̸= b2
ˆs2

) =⇒ P(

(j = 1)ˆs1) = P(

(j = 1)ˆs2)

• For the same secret b value, we want to verify if all schedulers can produce different

publicly observable outputs with the same probability. We utilize Alg. 18 to verify this

property.

TA − 1σ1s: ∃ˆσ. ∀ˆs(σ). bˆs =⇒ P(

(j = 1)ˆs) ̸= P(

(j = 2)ˆs)

Password leakage (PW)

Consider the code snippet in Fig. 8.4 of a leaky string-matching implementation. The

code allows a side channel leakage as it returns false at the first encounter with a non-

matching character between the input and secret value. This can provide indirect infor-

mation to the attacker about the length of the actual secret string. We can use a counter

running parallel to this code (similar to TA) to observe the difference in execution time. We

experiment with the following properties:

• We want to verify the existence of a scheduler such that we are to observe the same

128

value of the counter with equal probability when beginning with different secret values.

PW − 2σ2s: ∃ˆσ1∃ˆσ2. ∀ˆs1(ˆσ1)∀ˆs2(ˆσ2). (b1
ˆs1

̸= b2
ˆs2

) =⇒

P(

counter1ˆs1) = P(

counter1ˆs2)

• We want to verify if, for a secret value, the probability of observing different bits of

the counter is not the same.

PW − 1σ1s − 1: ∃ˆσ. ∀ˆs(σ). bˆs =⇒ P(

counter1ˆs) ̸= P(

counter0ˆs)

• Changing PW-1σ1s-1 slightly, we want to verify if we can find a scheduler where the

probability of reaching different counters for the same secret value is the same.

PW − 1σ1s − 2: ∃ˆσ. ∀ˆs(σ). bˆs =⇒ P(

counter1ˆs) = P(

counter2ˆs)

TaskShuffler (TF)

During task scheduling in Real-Time Operating Systems (RTOS), fixed models of schedul-

ing can give rise to faster execution times and easy detection of fault by examining minor

perturbations. However, this predictability can give rise to security issues. Determinism in

the scheduling process gives rise to predictability and can provoke attackers to use the timing

properties of the scheduler to design targeted attacks. Taskshuffler [YMCS16] presents an

algorithm to allow randomly scheduling lower priority tasks in a queue while ensuring higher

priority tasks still make their deadline.

pi
5
8
20

ei
1
2
3

di Vi
4
5
3
8
4
20

τ0
τ1
τ2

Table 8.1: Parameters used for scheduling three tasks τ0, τ1, τ2.

Consider the parameters in Table 8.1; for each of the tasks τ0, τ1, τ2, ei is the worst-case

execution time, di is the relative deadline, pi is the inter-arrival time of successive job releases

by the task. The subscript of each task name represents its priority, i.e., τ0 has a higher

priority than τ1 which in turn has a higher priority than τ2. Using (pi, ei, di), the algorithm

129

calculates Vi, the maximum amount of time, for each task such that (1) a lower priority task

can be scheduled before the current task, and (2) the current task does not miss its deadline.

We refer the readers to the original paper [YMCS16] for details on the algorithm.

We experiment with the following variation of noninterference:

• We want to verify if the probability of creating two distinct schedules, where all tasks

finish their executions, is equal. Here init refers to the initial state of the scheduling

where the respective variables in the algorithm are initialized with the parameters

in the table 8.1. We differentiate the schedules using doneW ithV0 and doneW ithV1

denoting that the tasks finished without using all the time permitted by V0 and V1

respectively.

TF − 1σ1s: ∃ˆσ. ∀ˆs(σ). initˆs =⇒ P(

doneW ithV0 ˆs) = P(

doneW ithV1 ˆs)

• Another interesting property (although not a hyperproperty) is to verify if all jobs

meet their deadline under this protocol. Formally, we can represent this as below. We

set each job’s counter to it ei and count backward at each step of its execution. We

use allDone to denote the state where all three jobs have exhausted their ei counter.

TF − prop: ∀ˆσ. ∀ˆs(σ). initˆs =⇒ P(

allDoneˆs) = 1

8.4.2 Experiments and Results

We have implemented our prototypes for Alg. 17 and Alg. 18 in Python. Our input

models are in the PRISM language [KNP11] and we utilize the model labels to describe our

specifications (in terms of initial and target states). For Alg. 17, we have used the model

checking capabilities of Stormpy [stob] and combined the minimum and maximum schedulers

generated. For Alg. 18, we have utilized Stormpy to parse and store the MDP models. To

evaluate the effectiveness of our algorithms, we compare our results with [DABB21]. Note

that we are not necessarily generating the same type of schedulers - [DABB21] generates

memoryless deterministic schedulers, and our algorithms generate memoryful randomized

130

Case
Study

Secret
Scaled on

Model Size

Scheduler HyperProb Alg. 17
#st #tran #act Possible? Time(s) Time(s)

TA-2σ2s

TS-2σ2s

PW-2σ2s

1 bit
3 bits
5 bits
8 bits
32 bits
50 bits

(h1, h2)=(1,0)
(h1, h2)=(15,0)
(h1, h2)=(20,10)
(h1, h2)=(50,3)

1 bit
3 bits
5 bits
8 bits
32 bits

24
112
264
612
8580
20604

7
35
45
106

24
140
352
850
12610

46
274
694
1684
25156
60904

17
115
150
362

30
154
374
884
12740
30704

11
67
87
208

46
302
782
1922
29186

30
182
462
1122
16770

✓
✓
✓
✓
✓
✓

✓
×
✓
×

✓
✓
✓
✓
✓

0.65
18.80
523.65
TO
TO
TO

0.10
1.37
4.39
22.67

0.61
28.31
1132.56
TO
TO

0.047
0.051
0.048
0.054
0.107
0.152

0.049
0.043
0.049
0.044

0.039
0.046
0.057
0.072
0.147

Table 8.2: Experimental results for scheduler generation using Alg. 17. #st: number of
states in the model, #tran: number of transitions in the model, #act: number of actions
in the model.

or memoryful deterministic schedulers. However, the comparison based on execution time

seems appropriate to showcase the scalability of our algorithms. The column for scheduler

possible notes the actual existence of a scheduler satisfying the corresponding property. All

experiments were done on a MacBook Pro with a 2.3GHz i7 processor and 32GB of RAM.

The execution times are in seconds. We considered a timeout (TO) of 5000s.

Table 8.2 shows the results for our experiments using Alg. 17 and our three case studies

described earlier. We can generate (if possible) randomized schedulers efficiently and the

current size of our models does not pose any challenge to this approach compared to the

results of HyperProb [DABB21]. The efficiency of the implementation is mainly attributed

to the strength of model checking and scheduler generation of Storm. The incorporation of

randomization positively impacts the performance of model checking for this fragment by

orders of magnitude. Note that the two compared approaches generate different types of

schedulers, however, they always agree on the existence of a scheduler. For example, TS-

2σ2s for (h1, h2) = (15, 0) does not produce a scheduler for both the approaches. Evidently,

131

Case
Study

TA-1σ1s

TS-1σ1s

PW-1σ1s-1

PW-1σ1s-2

Secret

Scheduler HyperProb
Scaled on #st #tran #act Possible? Time(s)

Model Size

Alg. 18
k Time(s)

1 bit
3 bits
5 bits
8 bits

h1 = 1
h1 = 15
h1 = 30
h1 = 50

1 bit
3 bits
5 bits
8 bits

3 bit
5 bits

25
113
265
613

8
36
66
106

25
141
353
851

141
353

54

48
276
696
1686

19
117
222
362

48
304
748
1924

304
748

89

31
155
375
885

12
68
128
208

31
183
463
1123

183
463

77

✓
✓
✓
✓

×
×
×
×

✓
✓
✓
✓

×
×

✓

0.77
18.91
408.96

7
11
15
TO 21

0.16
1.66
7.78
53.09

6
19
34
54

0.71
30.83
1201.00

6
10
14
TO 20

43.04
1230.00

0.32

11
15

10

0.156
1.059
6.657
335.000

0.068
0.568
1.776
4.684

0.402
1.297
3.927
12.840

TO
TO

0.40

TF-1σ1s

default

Table 8.3: Experimental results for scheduler generation using Alg. 18. #st: number of
states in the model, #tran: number of transitions in the model, #act: number of actions
in the model.

Alg. 17 outperforms HyperProb by orders of magnitude. In cases where a scheduler cannot

be found, one does not exist, meaning that the formula is falsified by both HyperProb and

Alg. 17.

Table 8.3 shows the results for our experiments using Alg. 18. Compared to [DABB21],

the strength of our approach lies in two aspects: (1) An initial backward value iteration

helps decide cases where all or none of the schedulers can satisfy the property as can be

seen in the case of TS-1σ1s, and (2) the verification focuses on one set of actions at a time

as a possible candidate to pursue in the distribution transformer; this makes it possible

to produce a scheduler quicker than [DABB21], where the whole model has to be encoded

before solving. This is evident in the cases TA-1σ1s and PW-1σ1s-1. However, due to

the inherent hardness of the general problem, this might not always be effective as seen in

PW-1σ1s-2. Both algorithms considered in this fragment are exhaustive searches across all

possible schedulers of their class.

[DABB21] encodes this scheduler set before attempting

to solve the encoding. Thus, although time-consuming, it still produces a result. Since our

132

algorithm considers one action combination at a time for exploration, it can benefit from

further optimization in this aspect.

We verified an aspect of the correctness of TaskShuffler stated using TF-prop in 0.22

seconds using HyperProb. Since it does not fall under the fragment we consider in this

chapter, we did not execute it with the algorithms in the context here.

8.5 Summary

The general model checking problem for HyperPCTL is undecidable. Based on our ex-

ploration across different applications, fragments of the language for which we can create

scalable algorithms exist. In this work, we focused on identifying specific fragments of Hyper-

PCTL that are expressive enough to cover interesting security properties while being simple

enough to find scalable algorithms for them. We presented two different such algorithms -

a convex addition of minimum and maximum schedulers to produce randomized schedulers,

and a distribution transformation-based algorithm to produce deterministic schedulers. We

have provided comparison of our results with existing approaches to show the scalability of

our approaches.

133

Chapter 9

Lightweight Verification of
Hyperproperties

In the last few chapters, we discussed the challenges of model checking probabilistic

hyperproperties. In this chapter, we focus on an approximate statistical approach to model

check probabilistic hyperproperties.

9.1

Introduction

In the last decade, researchers have proposed several adaptations of classical tem-

poral

logics to specify hyperproperties in a formal and systematic way. Examples in

the non-probabilistic setting are HyperLTL [CFK+14] and its asynchronous variant A-

HLTL [BCBS21]. HyperLTL extends LTL [Pnu77] with explicit quantification over paths

that allows to express relations among execution traces from independent system’s runs. Re-

cent works in [FRS15, HSB21, HBFS23a] provide exhaustive and bounded model-checking

algorithms for HyperLTL. For probabilistic hyperproperties, there are two main specifica-

tion languages: HyperPCTL [ ´AB18, D ´ABB22], which quantifies over schedulers and argues

over computation trees, and Probabilistic HyperLogic (PHL) [DFT20] which adds quanti-

fiers for schedulers and reasons about traces. In both contexts, these approaches face two

main challenges: scalability and the need for an explicit model. Scalability is, in particu-

lar, critical: (1) HyperLTL model checking is EXSPACE-complete [BF18], (2) HyperPCTL

134

and A-HLTL model checking are in general undecidable with decidable fragments in EXS-

PACE [BCBS21, D ´ABB22], (3) PHL model checking is in general undecidable with decidable

fragments (reduce to HyperCTL∗) in NSPACE [DFT20, FRS15]. This complexity obstacle

has been a major motivation for the development of alternative approaches to handle the

problem. One possible approach is to provide an approximate result with certain statistical

guarantees, termed SMC. SMC is an approximate model checking method that is subject to

a small probability of drawing a wrong conclusion [LLT+19, LL20, LDB10]. The main idea

is to simulate finitely many traces of a model and conduct hypothesis testing to conclude

if there is enough evidence that the model satisfies or violates the property, subjecting to

a small probability of drawing a wrong conclusion. Such simulation-based approaches have

two main advantages: first, we can use them to approximate the probability of satisfying

the desired property in a model of considerable size, which we would be otherwise unable to

verify exhaustively; second, we can apply them to black-box systems for which we are unable

to access the inner model. This approach is also intuitive as it can terminate early for cases

where it has already found enough evidence for violation. Consider, a case where a property

is required to hold for all traces. In this case, we should not be able to see a violation even if

we simulate just one trace. Given these advantages, we want to study its application to verify

hyperproperties. Another challenge, in terms of verification, is the handling of nondetermin-

ism. When modeling systems, we have to take into consideration the uncertainty that can

arise due to incomplete details, involvement of unknown agents, or noise, in general. From

a verification perspective, we need to be able to argue that a property holds under any such

possibility of nondeterministic uncertainty. Both HyperPCTL and A-HLTL model checking

have the capability of reasoning over nondeterminism, however, the high complexity in their

model checking solutions stems from the need for scheduler synthesis.

In this work, we chose to model systems as MDP to effectively express uncertainty in

systems using nondeterminism and randomization. PLASMA [LS14] is a model checker

that uses a memory-efficient sampling of schedulers [DLST15] to conduct simulation-based

135

statistical analysis. In this work, we extend PLASMA’s capability to include the verification

of linear, bounded hyperproperties over systems modeled as MDP. Our method orchestrates

well-established methods from the SMC community for the analysis of an expressive model

class in light of bounded HyperLTL properties. The result is a scalable, lightweight verifi-

cation approach which is the first of its kind to handle this combination of model class and

property. We have added and experimented with an extension that supports using recorded

traces or requesting simulation of black-box components on-the-fly for hyperproperty verifi-

cation. This opens the door to utilizing our approach for applications in cases where explicit

modeling is not possible or error-prone. For evaluation, we present a diverse set of scal-

ing benchmarks that raise the demand for this expressive model class and property type.

We have selected systems that allow for verification of properties such as noninterference,

side-channel information leak, opacity, and anonymity. The systems under inspection range

from classical examples including dining cryptographers, to examples taken from robotics

path planning and real code snippets. The state space of the resulting models varies in the

order of magnitude from tens to hundreds of billions involving tens to thousands of non-

deterministic actions. Our experimental evaluation indicates good performance on systems,

unperturbed by the size of the state space. To summarize, our main contributions are:

1. To the best of our knowledge, we provide the first SMC approach for the verification

of unquantified and bounded HyperLTL properties involving nondeterminism.

2. We extend the model checker PLASMA by this class of properties. Furthermore, we

add capabilities to execute black-box verification.

3. We showcase the general applicability with an extensive evaluation of our method on

various scalable case studies taken from different domains.

136

9.2 Preliminaries

In this section, we first elaborate on the syntax and semantics of the logic we use to

specify our properties. This is followed by a brief description of the Sequential Probability

Ratio Test which is a standard method in statistical hypothesis testing that forms the core

of our approach.

9.2.1 HyperLTL

HyperLTL [CFK+14] is the extension of linear-time temporal logic (LTL) that allows the

expression of temporal specifications involving relations between multiple paths. Each state

in the path is observed as a set of atomic propositions that hold in that state. HyperLTL

involves the evaluation of specifications over these propositions. An arbitrary path variable

π is used to refer to individual paths that can be generated by the model. Contrary to LTL,

each proposition aπ is associated with a path variable π denoting the path on which it should

be evaluated.

Syntax

We focus on unquantified and bounded HyperLTL defined by the grammar below.

φ := aπ | ¬φ | φ ∧ φ |

φ | φ U ≤kφ

• a ∈ AP is an atomic proposition that evaluates to true or false in a state;

• π is a random path variable from an infinite supply of such variables Π;

•

,

≤k, ≤k, and U ≤k are the ‘next’, ‘finally’, ‘global’, and ‘until’ temporal operators,

respectively,

• k ∈ N is the path length within which the operator has to be evaluated.

Following are the connectives defined as syntactic sugar:

true ≡ aπ ∨ ¬aπ, φ ∨ φ′ ≡ ¬(¬φ ∧ ¬φ′), φ ⇒ φ′ ≡ ¬φ ∨ φ′,

≤k φ ≡ trueU ≤kφ,

≤k φ ≡

137

¬ ≤k ¬φ. We denote U ≤∞,

≤∞, and ≤∞ or the unbounded temporal operators by U,

,

and , respectively. In our work, we consider only the bounded fragment of HyperLTL such

that for all temporal operators (except

), we evaluate the result on finite fragments of the

simulated traces.

Semantics

The path evaluation function V : Π → S ω assigns each path variable π, a concrete path

of the labeled DTMC Below we consider the semantics of HyperLTL,

V |= aπ

iff a ∈ L(cid:0)V (π)[0](cid:1)

V |= ¬φ

iff V ̸|= φ

V |= φ1 ∧ φ2

iff V |= φ1 and V |= φ2

V |= φ

iff V (1) |= φ

V |= φ1U ≤kφ2 iff there exists i ∈ [0, k], V (i) |= φ2

and for all j ∈ [0, i), V (j) |= φ1

where V (i) is the i-shift of path assignment V defined by V (i)(π) = (V (π))(i). For example,

the formula V |= aπ1

1 U kaπ2

2 means that a1 holds on the path V (π1) until a2 holds on the path

V (π2) in k steps.

9.2.2 Sequential Probability Ratio Test

We use Wald’s SPRT [Wal45]. The idea is to continue sampling until we are either able

to conclude or we have exhausted a user-provided sampling budget. Assuming we want to

verify if a property φ holds on our model M with probability greater than and equal to θ,

i.e., PM(φ) ≥ θ. To use SPRT in this case, we add an indifference region around our bound

to create two distinct and flexible hypothesis tests [AP18]. For a given indifference region ϵ,

we define p0 = θ + ϵ and p1 = θ − ϵ. Our resultant hypotheses are,

H0 : PM(φ) ≥ p0 H1 : PM(φ) ≤ p1

(9.1)

138

Using these newly created bounds, we define the following probability ratios,

ratiot =

p1
p0

ratiof =

1 − p1
1 − p0

(9.2)

We define an indicator function 1(T |= φ) ∈ {0, 1} that returns 1 if the trace T satisfies

the property φ, and returns 0 otherwise. When evaluating φ on a set of sampled traces

{T1, . . . , Tn}, we accumulate ratiot if 1(T |= φ) = 1 and ratiof otherwise. Assuming, we

have sampled N traces, the final product of the truth value corresponds to

pratio =

N
(cid:89)

i=1

(p1)1(Ti|=φ)(1 − p1)1(Ti|=¬φ)
(p0)1(Ti|=ϕ)(1 − p0)1(Ti|=¬ϕ)

(9.3)

We iteratively calculate this ratio until the exit condition is met. To restrict the error in the

estimation of the probability θ, we specify error probabilities α as the maximum acceptable

probability of incorrectly rejecting a true H0, and β as the maximum acceptable probability

of accepting a false H0. The boundary error ratios can be defined as A = β/(1 − α) and

B = (1−β)/α. To reach a conclusion, we accept H0 if pratio ≤ A, and accept H1 if pratio ≥ B.

The case for specifications with PM(φ) ≤ θ is similar except we use the reciprocals of ratiot

and ratiof .

9.3 Problem Formulation

HyperLTL allows explicit quantification over traces, allowing the user to express whether

they want their specification to hold across all paths associated with a path variable or in at

least one of those paths. Along with the added expressiveness, this formulation distends ex-

isting challenges - (1) While checking a specification across all possible sets of paths provides

a robust verification result, it is considerably expensive, making it impractical as we scale to

models with larger state spaces. (2) Most real-life systems involve uncertainties in the form

of randomization, nondeterminism, or partial observability. Consequently, this raises a need

to express that, for instance, a fraction of the paths of the system satisfy the specification.

139

To handle the above challenges, we propose a practical formulation for expressing unquan-

tified and bounded HyperLTL formulas for models that involve both probabilistic choices and

nondeterminism. We quantify over the path variables by associating a probabilistic bound

denoting the proportion of the set of traces that should satisfy a given specification. We can

express that a specification is almost always likely or highly unlikely by adjusting the bound

of the probability p to p ≥ 1 or p ≤ 0, respectively. Intuitively, almost always likely can

be considered as a weaker counterpart of ∀ (forall) quantification, and highly unlikely can

be considered as a weaker counterpart of ¬∃ (existential) quantification over path variables.

Note that these limits our expression of HyperLTL formulas with quantifier alternation in

any capacity, and we leave that as an aspect worth exploring in future works.

Consider an MDP M and an unquantified, bounded HyperLTL formula φ that contains

path variables (π1, . . . , πm). We consider tuples of m schedulers to simulate m traces assigned

to these path variables, i.e., we have a one-to-one correspondence between schedulers and

path variables. We are interested in checking if there exists a combination of schedulers that

can satisfy the HyperLTL specification φ on our model within a given probability bound.

Formally, this can be expressed as,

∃σ1∃σ2 . . . ∃σm PM(V |= φ) ∼ θ

(9.4)

where θ ∈ [ϵ, 1 − ϵ] to allow an indifference region for hypothesis testing (see 9.2.2), σi are

schedulers of M, V (πi) is the path drawn from the DTMC Mσi for i ∈ [n] which is induced

by σi on MDP M, and ∼ ∈ {≥, ≤}. Note that we can involve multiple models to yield paths

for each scheduler σi. For properties where we want to check if a given specification holds

across all scheduler combinations, we negate our specification to re-formulate the problem as

in Eq. 9.4. Since we adopt a statistical model checking algorithm, it is worth noting that we

cannot directly observe if a specification holds for all cases, thus, we utilize this approach

to check if we can satisfy its negation. We elaborate on this in Section 9.4.

140

9.4 Approach

We utilize the advantages of SMC to verify hyperproperties by answering our model

checking problem using hypothesis testing, specifically SPRT, as described in Sec. 9.2.2.

The overall approach involves the sampling of schedulers and traces from one (or more)

given MDP, monitoring the satisfaction of the property on these traces, and determining

if we have gathered enough evidence to reach a concrete verdict for the property. In this

section, we explain the concepts and parameters involved in finding the result of this test

such that we can directly use it to answer our model checking question.

9.4.1 Scheduler sampling

One of the main challenges when verifying MDP is the generation of schedulers for ver-

ification.

It stems from the complexity involved in the storage of history to resolve non-

determinism in the current state. We utilize the lightweight scheduler sampling feature of

PLASMA [LS14]. This approach avoids the explicit storage of schedulers by using uniform

PRNG to resolve non-determinism and hashes as seeds for the PRNG. In the following, we

will give an intuition of the approach inbuilt in PLASMA and how we have extended it to

argue about hyperproperties.

PRNG forms the core of the smart sampling algorithm of PLASMA. Given a set of

possible action choices and sufficient runs of the number generator, they allow the generation

of statistically independent numbers that are uniformly distributed across a specified range.

They are uniquely mapped to their seed values intsch, ensuring the reproduction of the same

value when the generator is provided with the same seed. Note that we can use PRNG to

identify individual schedulers but cannot identify specific schedulers. Furthermore, since the

seeds only initialize the PRNG, using problem-specific information (e.g., about the property)

during the generation of the seed does not allow related schedulers.

Each state of the MDP is internally represented as a concatenation of the bits representing

the values of the atomic propositions that are true at that state. A sequence of states can

141

be represented by concatenating their bit sequences. The sum of the bits of such a sequence

of numbers intt, which is an integer, represents a trace. Concatenation of intsch and intt

can be then used to uniquely identify both a scheduler and a trace. PLASMA generates

a hash with this concatenated value which represents the history of both the scheduler and

the trace and is used as a seed to resolve the next nondeterministic choice. PLASMA

uses an efficient iterative hash using modular arithmetic that ensures efficient storage of the

possible schedulers mapping the comparatively large set of schedulers to a smaller set of

integers with a low probability of collision. For more details on this, we refer the reader

to [DLST15]. Once the nondeterministic choice is resolved at a state, PLASMA uses an

independent PRNG to uniformly choose a successor state from the ones available under the

chosen action. This is concatenated with the trace before generating the next hash for the

nondeterministic resolution.

When working with hyperproperties, we would need to consider a tuple of schedulers

and traces. In this aspect, we can either simulate traces from the same MDP using different

schedulers, use different schedulers for each MDP, or a combination of both. We define a

scheduler tuple σ ⊂ Σm as a tuple of schedulers sampled from the set of possible schedulers

allowed by our MDP and m is the number of scheduler quantifiers in our specification as

shown in Eq. 9.4. We define a trace tuple as a tuple of traces sampled from our model based

on the tuple of schedulers. Thus, ωσ represents the trace tuple ω, sampled from the DTMC

induced by the scheduler tuple σ. For simplicity, we consider a one-to-one correspondence

between our schedulers and MDP. We define an indicator function 1(ωσ |= φ) ∈ {0, 1} that

returns 1 if the trace tuple ωσ satisfies the hyperproperty φ, and returns 0 otherwise.

The aim is to verify the satisfaction of the given specification under all or some combi-

nation of nondeterministic choices in our system. Since a scheduler represents a concrete

resolution of nondeterminism across the system, our problem is transformed to that of find-

ing a scheduler tuple that satisfies our specification in the form of the described hypothesis

in Eq. 9.4. Intuitively, SMC considers the proportion of the sampled trace tuples that in-

142

dividually satisfy the property to estimate the true satisfaction probability in the overall

model. To bind the errors in the estimation, the algorithm uses precision and user-provided

error margins.

For the case where we want to conclude all scheduler tuples satisfy the property, we

negate the property and try to find a scheduler tuple that satisfies this negated property.

The falsity of this property makes our original property true. For the case where we want to

search if there exists a scheduler tuple, we pose the hypothesis directly. However, in this case,

a false result does not necessarily guarantee the absence of a witness to the specification;

it suggests that our algorithm was unable to find such a scheduler tuple within the given

budget, error, and precision bound. Note that we cannot derive the exact scheduler tuple

(we get the traces generated but not the reduced DTMC) due to the black-box nature of our

sampling. We can only reason about its existence or absence within the given budget.

9.4.2 Implementation

In this section, we discuss the handling of the hypothesis testing of H0 as shown in Eq. 9.1

in detail. The case for H1 (also shown in Eq. 9.1) is similar except we use the reciprocals

of ratiot and ratiof . As shown in Alg. 22, we begin by initializing the necessary parameters

(line 2). For conducting sequential hypothesis testing on large systems, we need an additional

bound to represent the maximum limit of resources we want to spend on this verification.

To this end, PLASMA utilizes the concept of a user-provided budget. Following the idea

described in [DLST15], the algorithm automatically distributes the budget to determine the

number of schedulers and trace tuples the verification should consider as described in the

previous section. We generate a set of scheduler tuples Σ and create a mapping to store

which scheduler should be used to produce which trace (deriving this information from the

input specification).

In lines 3-4, for each scheduler tuple, we use the internal simulator

to simulate the traces as specified by the mapping. In the case of multiple initial states,

we allow the choice of traces with the same or different initial states. This reduces extra

143

Algorithm 22: Hypothesis testing on Hyperproperties

Input : MDP model: M, spec: φ, Hypothesis H0 : PM(V |= φ) ≥ θ

α, β: desired type I, type II errors,
Nmax : simulation budget, ϵ: indifference region.

Output: Success: There exists a satisfying scheduler tuple,

No success: Could not find a satisfying scheduler tuple,
Inconclusive: No conclusive scheduler tuple was found

▷ Initializes N , M, p0, p1, A, B, k, ratiot, ratiof

1 Function Main(M, φ, α, β, Nmax , ϵ)
2

initialize()
Σ ← {M tuples of k randomly chosen seeds}
∀σ ∈ Σ, ∀i ∈ {1, ..., N } : ωσ
R ← {(σ, n)|σ ∈ Σ ∧ N ∋ n = (cid:80)N
if canEarlyAccept(R) then
Accept H0 and exit

i ← simulate(M, φ, σ)
i=1 1(ωσ
i |= φ)}

Σ ← {σ ∈ Σ|R(σ) > 0}, M ← |Σ| + 1
if |Σ| = 0 then

Quit: No suitable scheduler-tuple found

while |Σ| > 1 do

initializeSchedulerBasedBounds()
foreach σ ∈ Σ, i ∈ {1, . . . , |Σ|} do

ratioi ← 1
foreach j ∈ {1, . . . , N } do

if simulate(M, φ, σ) |= φ then

▷ Remove null schedulers

▷ Initializes αM , βM , AM , BM

ratio ← ratio · ratioT ; ratioi ← ratio · ratioT

else

ratio ← ratio · ratioF ; ratioi ← ratio · ratioF

if ratioi ≤ AM or ratio ≤ A then

Accept H0 and quit: scheduler found

else if ratioi ≥ BM then

Quit iteration for σ: Scheduler tuple rejected

if All schedulers were rejected then

Quit: No scheduler found in given budget

Σ ← filter(Σ)

▷ Keep only the best-performing scheduler tuples

Inconclusive: There exists a scheduler that was neither accepted nor rejected

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

sub-formulas on the property to decide on initial states and allows us to verify the property

only on relevant trace samples.

In line 5, we use a custom model checker that we have

implemented in PLASMA to verify linear, bounded HyperLTL properties on sets of traces

sent as input. We further allow n-ary boolean operations by extending the general idea of

AND, OR, XOR, etc., to reduce the length of input property the user has to provide.

144

In line 6 of the algorithm, we compare the ratio generated using Eq. 9.3 against error

boundary A to check if we have already found enough witnesses to accept our null hypothesis

H0. We do not check against boundary B because the absence of a satisfying scheduler in

this initial phase does not ensure that the possibility of finding such a scheduler is zero. It

hints at the need for further sampling. In line 8, we filter out the null schedulers, i.e., for

which none of the trace tuples satisfied the property. Since we are looking for a scheduler

tuple to satisfy the property with positive probability, null schedulers cannot definitely be

our best search options. For each scheduler tuple in this filtered set, we again sample N

trace tuples. We essentially conduct multiple independent hypothesis tests, one for each

scheduler tuple. Hence, similar to [DLST15], we modify the error for each scheduler to

√
αM = 1 − M

√
1 − α, βM = 1 − M

1 − β to account for the error correction needed. In the

initial phase (lines 3-10), the idea was to check if we can satisfy the boundaries A, B using

the truth value of all trace tuples sampled, irrespective of its scheduler. In the rest if the

algorithm, we check if we can individually accept or reject any scheduler tuple, alongside the

global check for satisfaction across all sampled trace tuples. Since our trace tuples return

an overall true/false for the whole tuple, the error bound for each scheduler tuple would not

change when we are working with alternation-free hyperproperties instead of trace properties.

In lines 17 and 19, we re-calculate pratio (as in Eq. 9.3), both for each scheduler tuple and

for all the sampled trace tuples. As we encounter a satisfying tuple of traces, our overall

pratio decreases as ratiot is a value less than one in this case and with each non-satisfying

trace tuple, it increases. If the ratio obtained over all sampled traces across all schedulers is

reduced below A or its scheduler counterpart is below AM , we either have found a scheduler

tuple that satisfies the property or over all the sampled trace tuples, we have found enough

evidence to conclude that our hypothesis H0 is satisfied.

At the end of the iteration over the scheduler tuples, we can quit the test if all our

scheduler tuples are rejected, or proceed to the next iteration with only the best scheduler

tuples. We rearrange our scheduler tuples in an ascending order based on the number of

145

trace tuples that satisfied φ. Since we are aiming to find a scheduler tuple that satisfies φ

with a probability greater than θ, we only keep the first half of rearranged scheduler tuples,

ensuring that we are looking only at the schedulers that have a higher chance of exceeding

the bound. If our evaluation reaches line 27, the set Σ would contain one scheduler which

we were neither able to accept nor reject, reaching an inconclusive decision about H0 within

the given budget and precision margins. This inconclusive result would indicate we have to

retry the experiment with a higher simulation budget and/or different precision and error

margins for further scrutiny.

Convergence

The algorithm will always terminate in a finite number of iterations as we eliminate

half of our candidate scheduler tuples at each iteration. However, it may not have found a

satisfying scheduler tuple within that boundary. For an MDP M and property φ, we want to

find a good scheduler tuple, i.e., one that satisfies φ with probability p ≥ θ − ϵ. Assuming

we have |§| possible scheduler tuples, and |§g| good schedulers, we use P : § → [0, 1] to denote

the probability with which a scheduler tuple satisfies φ. If we sample M scheduler tuples

and N trace tuples per scheduler tuple, the probability of sampling a trace tuple from a good

scheduler tuple that satisfies φ is,

(cid:32)

(cid:18)

1 −

1 −

|§g|
|§|

(cid:19)M(cid:33)

(cid:32)

(cid:18)

1 −

1 −

(cid:80)

σ∈§g
|§g|

(cid:19)N (cid:33)

Pσ

(cid:124)

(cid:123)(cid:122)
good scheduler tuple

(cid:125)

(cid:124)

(cid:123)(cid:122)
trace satisfies φ

(cid:125)

(9.5)

We aim to maximize the value of this probability by optimizing the values of M and N ,

across which the budget Nmax is the total number of sampling we want to permit. Since we

need to find schedulers that satisfy the property with probability at least θ, we set N = ⌈ 1

θ ⌉.

This ensures that we spend our sampling budget verifying scheduler tuples that have a higher

probability of satisfying our property. For example, if our θ is 0.25, N = 4. If we want to

146

check for our specification to be ≥ θ, any scheduler that satisfies at least 1 of the 4 sampled

traces should be a good candidate for a good scheduler. In case we want to check for our

specification to be ≤ θ, finding such good schedulers would help us reject the hypothesis

easily. We allocate the rest of the budget (such that N · M ∼ Nmax ) to sample scheduler

tuples, thus, we set M = ⌈θNmax ⌉. We have experimented with various values of budget,

adjusting them based on the expected accuracy of our results.

9.5 Case Studies

In this section, we discuss case studies to show the applicability and scalability of our

approach. One of the main advantages of statistical model checking lies in the fact that we

do not necessarily need access to the underlying model to verify a system. This allowed us to

utilize our approach on sets of traces generated from black-box sources. We have separated

our case studies into two sections elaborating on the models of the grey-box (where we have

access to the underlying model) and black-box (where we just have access to a set of traces

generated by different schedulers) examples.

9.5.1 Grey-box verification

Group Anonymity in Dining Cryptographers (DC)

We explored the dining cryptographers problem [Cha88] from the perspective of how it is

designed to maintain anonymity. In this model, three cryptographers go out for dinner and at

the end, want to figure out who paid the bill (their manager or one of them) while respecting

each other’s privacy. The protocol proceeds in two stages:(1) each cryptographer flips a coin

and only informs the cryptographer on their right of the outcome (head or tail), (2) the

cryptographers consider both the coin tosses that they know of, to declare agree in case the

tosses were the same, or disagree otherwise. However, in the case of the cryptographer who

actually paid, they would declare the opposite conclusion.

Given an odd number of cryptographers, we should have an odd number of agrees if the

147

manager pays the bill, an even number of agrees if one of the cryptographers paid, and vice

versa for an even number of cryptographers. We want to verify if there is any information

leakage depending on which cryptographer pays.

In the model, we nondeterministically

choose who pays the bill and the order in which the cryptographers toss their coin. In case

one of the cryptographers pays in both traces, we expect the parity of coins at the end to

be the same. As described in [BP09], the order of coin toss should not affect the anonymity

in the protocol. This good behaviour can be expressed as a hyperproperty,

(cid:16) (cid:95)

φDC =

Cpay iπ1

∧

(cid:95)

Cpay iπ2

(cid:17)

=⇒

i∈(1,2,3)

i∈(1,2,3)

(done ∧ (c1 ⊕ c2 ⊕ c3))π1

(cid:94)

(done ∧ (c1 ⊕ c2 ⊕ c3))π2

(9.6)

For the correctness of the model, we should not be able to find a scheduler that satisfies the

bad behaviour ¬φDC with positive probability, thus, we design the hypothesis as,

∃σ1.∃σ2. PM(V ⊭ φDC ) > 0

(9.7)

We expect this property to be false for an odd number of cryptographers and true for even, as

our model should ensure anonymity. We experimented with both unbiased and biased coins

in the model to check if that affects the parity of agreement. The main challenge for this

study was the size of the models as shown in Table 9.1. Existing exhaustive approaches would

take considerable memory and execution time to verify this model. Hence, an approximate

approach like SMC has its utility here.

Noninterference in path planning (RNI)

Consider the grid in Fig. 9.1 which represents two robots moving across a two-dimensional

plane subdivided (discretized) into n × n cells. The robots can nondeterministically choose

to move to their neighbouring cells unrestricted (up, down, left, or right) unless it is blocked

by the grid boundaries. However, with a certain error probability, the chosen target cell is

148

Figure 9.1: Two robots attempting to reach the same goal.

not reached and instead, the robot stays in its current cell. The grid can hence, be modeled

as an MDP where each state models a grid cell. Note that we do not restrict or force any

specific strategy for the movement of these robots. Thus, each scheduler corresponds to a

specific strategy that defines how the robot moves across the grid. We consider the case

where two robots (R1 and R2) are placed in opposite corners of the grid and aim to reach

the goal state at the center of the grid. Assume R1 is our robot of interest and R2 is an

intruder. Motivated by the idea in [DFT20], a specification of interest would be to check if

the plan of R1 to reach the goal is affected by the plans of R2.

We design the hypothesis as the negation of the required property. Hence, we want

to determine if there exists any such scheduler tuple where the movement of R1 would be

similar but R1 fails to reach the goal in one of them. The unquantified HyperLTL formula

is as follows,

φRNI = (cid:0)actR1 π1 = actR1 π2

(cid:1) (cid:94)

(¬goalR1 π1UgoalR2 π1) ⊕ (¬goalR1 π2UgoalR2 π2)

(9.8)

For any arbitrary probability p, we design our specification as,

∃σ1.∃σ2. PM(V ⊨ φRNI ) > p

(9.9)

149

Figure 9.2: Grid divided into regions to ensure opacity.

Current state opacity (CSO)

Consider the grid in Fig. 9.2 where we use only one robot on the grid, which starts

from any of the starting states labeled S and aims to reach the opposite corner labeled G.

The gray boxes represent obstacles. Instead of analysing reachability, we are interested in

analysing opacity similar to [WNP20]. Opacity requires that an unauthorized user should not

be able to realize the current state of the system. In the context of a robot, opacity ensures

privacy is preserved as the robot moves across the grid. An observer gets an observation

corresponding to each movement of the robot. Note that we have divided the grid into three

regions (blue: near initial, red: between obstacles, green: near goal ) which would generate

the same observation even when the robot is in a different position.

Current state opacity specifically states that while starting from the same initial state

(here: either of the lower left corners marked in blue), it is still feasible to move across the

grid using different paths that can produce the same observation. By different paths, we

refer to cases where the actual positions of the robot are different due to the execution of

different actions (up, down, left, right). This would mean that an intruder should not be

able to guess the exact location by merely gathering observations about the movement of

150

the robot. We can express this formally as,

φCSO = (cid:0)start π1 ∧ start π2)

(cid:94)

¬ ≤k(actπ1 = actπ2)

(cid:94)

≤k(region π1 = region π2)

(9.10)

where act encodes the action taken by the robot on the grid and region denotes the corre-

sponding region observed. We want to check if any such combination of schedulers exists

that satisfies the current state opacity with respect to a given threshold. This is expressed

as,

∃σ1.∃σ2. PM(V |= φCSO ) > p

(9.11)

9.5.2 Black-box verification

We use the example of a side-channel timing attack on a password checker as a black-box

case study. We consider several password checkers that vary in the expected amount of

information leaked by observing the execution time, resulting from different input guesses.

We ran our password checkers on a microcontroller and considered numerical passwords of

length 10 as input.

Following the approach described in Section 9.4.1, a scheduler is represented as a seed

for a pseudo-random number generator. For a black-box model, the model checker calls

a python script with one parameter (the scheduler seed) as an input. This seed is used

by the model to resolve nondeterminism internally via a pseudo-random number generator.

Internally, the script uses the scheduler seed to create a random password guess. Here, we

assume the password guess and the actual password is of the same length to simplify code

run on the microcontroller. The number of correct digits of the generated password guess

is saved and the password is forwarded to the microcontroller via the serial interface over

USB. The execution time of the microcontroller together with the number of correct bits are

returned by the Python script and the out-stream is parsed and interpreted by the model

checker.

151

We convert numerical return values (rounded to a predefined level of precision), e.g.,

the number of correct digits (cd ) or the execution time (et) to traces whose length of con-

secutive symbols of a type reflects those values. For instance, the returned pair of values

execution time=4, correct digits=1 would be converted into the trace

{et, cd } → {et} → {et} → {et} → {} → . . .

Leakage of information from an unsafe password checker can be obtained by observing the

execution times for several inputs.

Intuitively, if the checking of a password with more

consecutive correct digits (in the front) takes longer than a password with fewer correct

digits, observing the execution time for multiple guesses should allow guessing the correct

password. To formalize this, we use the specification of unwanted behavior

φTAM = (

(cd π1 ∧ ¬cd π2) ∧ (et π1 ∧ ¬et π2)) ⊕ (

(cd π2 ∧ ¬cd π1) ∧ (et π2 ∧ ¬et π1))

(9.12)

Consider the example of a password checker that leaks information (BB-L) in Listing 9.1.

In contrast, a simple, safe approach (BB-S) checks the whole password without the option

of an early return as in Line 6 and thus always produces the same execution time regardless

of the correctness of the guess g.

1 b o o l checkPassword ( S t r i n g g ) {
i ;

2

i n t
f o r ( i =0;
{

}
r e t u r n t r u e ;

i < g . l e n g t h ( ) ; ++i )

i f ( g [ i ] ! = s e c r e t [ i ] )
r e t u r n f a l s e ;

3

4

5

6

7

8
9 }

Listing 9.1: Possible leaky password checker (BB-L).

Additionally, we can also add padding to obfuscate actual execution timing.

In our

experiments, we consider a random delay (BB-*R) between 0 and 10 microseconds or a fixed

152

delay (BB-*F) of 2 microseconds. We want to check, for an arbitrary probability p, whether

a combination of schedulers exists, such that bad behaviour, i.e., information leakage can be

derived. This is expressed as,

∃σ1.∃σ2. PM(V |= φTAM ) > p

(9.13)

9.6 Experimentation/Evaluation

The model details of our grey-box case studies have been reported in Table 9.1. Exper-

imental results for our case studies have been summarized in Table 9.2. The parameters in

Table 9.2 refer to the number of schedulers (#sch) and traces (#tr ) that were sampled as

determined by our algorithm, and the length of the trace (k ) as determined by the user based

on knowledge about the model. We separately report the time required for the sampling of

the scheduler (Sim) and trace tuples and the time required to verify (Ver ) the hyperproperty

on them. Reported timing data is the average over 10 runs. Note that in our evaluation we

do not compare our results to the existing model checkers for linear hyperproperties as they

cannot handle probabilistic models with non-determinism.

9.6.1 Black-box verification

Experiments were run on an Intel® Core™ i7 (6x3.30 GHz) with 32Gb RAM, the pass-

word checkers ran on an esp32 micro-controller to alleviate variance in timing due to process

scheduling. To obtain results with higher precision, we execute using multiple parameter con-

figurations - the size of the indifference region (ϵ), the satisfying probability (θ), and sampling

budget (Nmax ). The error probabilities α, β were kept at 0.01 for the whole experiment. Re-

sults and running times for the most accurate runs are shown in Table 9.2, where different

variants of password checkers (see also Table 9.5) are referenced by their labels.

In total, we have run over 480 combinations of parameters to synthesize accurate results.

Table 9.2 lists the results of parameter configurations that maximize the probability of sat-

153

isfying the property without being inconclusive to give an estimate on a worst-case scenario.

In case the property could be satisfied in the majority of the 10 runs, we show results for

two configurations: one leading to a large observed probability and a second one that used

a higher budget and smaller indifference region which, thus, can be expected to be more

precise.

From the results, we can observe that for safe password checking the tested variants with

no padding (S), fixed padding (SF), and random padding (SR) do not allow information

leakage about the correctness of the password guess via correlation of the observed execution

time. In contrast to this, the experiments with a leaky password generator with a fixed or

no padding scheme (LF, L) allow correlating execution time and correctness of passwords.

Note that in most cases the created guesses had only zero to one correct digit, as we did not

implement adversarial strategies to guess larger parts of the password.

9.6.2 Grey-box verification

Experiments were run on an Intel® Core™ i7 (4x2.3 GHz) with 32Gb RAM. We ran

experiments on each of the described case studies by scaling them across the different pa-

rameters involved. However, due to space constraints, we report cases that are sufficient to

show the scalability and robustness of our approach.

The DC component in the tables 9.2 and 9.1 corresponds to the verification of the dining

cryptographers protocol described in Section 9.5.1. Our specification φDC should not hold

for an odd number of cryptographers and should hold for even ones. We have scaled the

model over #c = {4, 7, 8, 15} and witnessed the expected results. We used a constant budget

of 5000 for all the cases reported. We used models directly from PRISM [KNP11] and were

able to verify them without alterations. We experimented with both biased and unbiased

coins. The result produced was the same proving that the biases of the coins do not affect

the outcome of the protocol. The experiment for the biased coin scenario used the exact

same parameters as reported and yielded similar execution times for both scenarios.

154

CS

Param. #States #Transitions #Actions

c = 4
c = 7
c = 8
c = 15

n = 3
n = 5
n = 10

n = 10
n = 20
n = 30

DC

RB
Grid
Fig.9.1

RB
Grid
Fig.9.2

2598
328760
1687113
1011

1034
12346
256926

200
800
1800

6864
1499472
8790480
1012

4888
77152
1852972

1440
6080
13920

5448
1186040
6952248
9 × 1011

2444
38576
926486

720
3040
6960

Table 9.1: Model details of grey-box case studies.

The RNI section in the tables 9.2 and 9.1 corresponds to noninterference case study. We

have scaled our grid for N = {3, 5, 10}. We verify the existence of scheduler tuples that fail

to satisfy noninterference with probability bounds of {0.1, 0.2, 0.5} with a budget of 2000.

The trace lengths have been increased in proportion to the grid sizes. We have experimented

on arbitrary trace lengths which have been adjusted as we increase the grid size. As we do

not specify any smart movement strategy for the robots, these results are based on possible

random walks the robots can make on the grid. The interesting observation here is the

difference in execution time based on the parameters. The cases for θ ≤ 0.1 seem to be

challenging, given the current grid and budget, resulting in an inconclusive result; for n = 10

our experiment ran for more than 2 days hinting at an inconclusive result within the given

budget. This is expected as we are challenging the algorithm to find a scheduler with a

very low probability (between 0 and 0.1) given the large search space. For θ ≥ 0.2, we are

often able to find our target scheduler tuples in the initial sampling phase, leading to short

execution time due to early exit. This is mainly because we are looking for a scheduler

across a wider range of probability (between 0.2 and 1). Using θ ≥ 0.5 becomes challenging

when scaling the model (with the same budget for comparison) due to the growing number

of possible scheduler tuples, and the lack of any specific strategy that finds traces where

both the robots are aiming to reach the goal. Thus, finding a scheduler with probability on

155

Case
Study

Specification

Result

Parameters

#sch #tr

k

Time [sec]
Sim. Ver.

BB

DC

RNI

CSO

False
False
False
True
True
True
Undec
False

Pr (V ⊨ φTAM ) ≥ 0.1 ± 0.01 # S
Pr (V ⊨ φTAM ) ≥ 0.1 ± 0.01 # SF
Pr (V ⊨ φTAM ) ≥ 0.1 ± 0.01 # SR
Pr (V ⊨ φTAM ) ≥ 0.3 ± 0.1 # L
Pr (V ⊨ φTAM ) ≥ 0.25 ± 0.01 # L
Pr (V ⊨ φTAM ) ≥ 0.15 ± 0.1 # LF
Pr (V ⊨ φTAM ) ≥ 0.1 ± 0.01 # LF
Pr (V ⊨ φTAM ) ≥ 0.1 ± 0.01 # LR
Pr (V ⊭ φDC ) ≥ 0.1 ± 0.01
True
(#c = 4)
Pr (V ⊭ φDC ) ≥ 0.01 ± 0.001 (#c = 4)
True
Pr (V ⊭ φDC ) ≥ 0.1 ± 0.01
False
(#c = 7)
Pr (V ⊭ φDC ) ≥ 0.01 ± 0.001 (#c = 7)
False
Pr (V ⊭ φDC ) ≥ 0.1 ± 0.01
True
(#c = 8)
Pr (V ⊭ φDC ) ≥ 0.01 ± 0.001 (#c = 8)
True
Pr (V ⊭ φDC ) ≥ 0.1 ± 0.01
(#c = 15) False
Pr (V ⊭ φDC ) ≥ 0.01 ± 0.001 (#c = 15) False
Pr (V ⊨ φRNI ) ≤ 0.1 ± 0.01
Undec
(n = 3)
Pr (V ⊨ φRNI ) ≥ 0.2 ± 0.01
True
(n = 3)
Pr (V ⊨ φRNI ) ≥ 0.5 ± 0.01
True
(n = 3)
Pr (V ⊨ φRNI ) ≤ 0.1 ± 0.01
Undec
(n = 5)
Pr (V ⊨ φRNI ) ≥ 0.2 ± 0.01
True
(n = 5)
Pr (V ⊨ φRNI ) ≥ 0.5 ± 0.01
Undec
(n = 5)
Pr (V ⊨ φRNI ) ≥ 0.2 ± 0.01
True
(n = 10)
Pr (V ⊨ φRNI ) ≥ 0.5 ± 0.01
(n = 10) Undec
Pr (V ⊨ φCSO ) ≤ 0.05 ± 0.001 (n = 10) Undec
Pr (V ⊨ φCSO ) ≥ 0.3 ± 0.01
True
(n = 10)
Pr (V ⊨ φCSO ) ≤ 0.7 ± 0.01
(n = 10) True
Pr (V ⊨ φCSO ) ≤ 0.05 ± 0.001 (n = 20) Undec
Pr (V ⊨ φCSO ) ≥ 0.3 ± 0.01
True
(n = 20)
Pr (V ⊨ φCSO ) ≤ 0.7 ± 0.01
True
(n = 20)
Pr (V ⊨ φCSO ) ≤ 0.05 ± 0.001 (n = 30)
True
Pr (V ⊨ φCSO ) ≥ 0.3 ± 0.01
True
(n = 30)
Pr (V ⊨ φCSO ) ≤ 0.7 ± 0.01
True
(n = 30)

400
400
400
1201
1001
601
400
400

500
50
500
50
500
50
500
50

200
400
1000
200
400
1000
400
1000

150
900
2100
150
900
2100
150
900
2100

10 80
10 80
10 80
4 80
4 80
7 80
10 80
10 80

10 20
100 20
10 25
100 25
10 30
100 30
10 65
100 65

108.00
93.10
92.80
102.00
85.50
92.00
90.00
88.70

1.60
1.40
2.70
2.60
2.60
2.70
4.50
5.10

385.00
10 10
3.80
5 10
2 10
210.00
10 26 2999.00
5 26
38.17
2 26 1243.00
173.65
5 80
10.4k
2 80

20 30
4 30
2 30
20 45
4 45
2 45
20 55
4 55
2 55

84.00
0.70
0.93
376.00
2.41
1.74
511.00
7.97
2.45

0.09
0.09
0.07
0.10
0.01
0.09
0.08
0.08

0.50
0.40
0.30
0.60
0.80
0.70
1.80
1.90

0.30
0.20
0.15
0.19
0.33
0.67
0.87
1.38

0.82
0.17
0.25
0.41
0.34
0.41
0.35
0.29
0.32

Table 9.2: Data from experimentation. #sch: number of scheduler-tuples sampled, #tr:

number of trace tuples sampled per scheduler tuple,

k: length of traces sampled. α = β = 0.01.

the higher end (between 0.5 - 1) is not always possible in the given budget and indifference

regions.

During our experiments on the opacity case study (CSO), we added a sub-formula to

156

Figure 9.3: DC with n = 4 (Pr ≥ 0.1 ± 0.01).

φCSO to check if the robot reaches the goal in both traces. Given that we do not enforce

any smart movement strategy on the movement of the robot, it usually makes a random

walk in the grid often looping in a few states for a long time. Consequently, the probability

of the robot reaching the goal is highly unlikely. We checked the probability of satisfying

our specification against {0.05, 0.3, 0.7}. We used a budget of 3000 for all versions of this

experiment and increased the trace length in proportion to the increase in the grid size.

The plots in Figs. 9.3, 9.4 depict convergence results, where each line in a graph shows

how the value of ratio changes across a single algorithm run. In each of the plots, the red

line represents the ratio A in Alg. 22 which serves as our exit condition. In Fig. 9.3, we use

the sampling budget of 2000 to calculate the ratio. At the end of this phase, if the ratio is

below A, we can declare that we have found enough evidence for a concrete result of the

specification being satisfied as shown in lines labeled experiment 2 and 3. For the case of

experiment 1, we could not reach a concrete conclusion in the initial round, as the line can

be seen to be well above A. We were required to enter the main algorithm loop and required

a few more samples (∼ 75) to reach the same concrete conclusion.

157

0500100015002000Number of Samples1014710126101051084106310421021100Ratio (in log scale)Experiment 1Experiment 2Experiment 3Acceptance ratio(a) RNI with n = 4(Pr ≥ 0.2 ± 0.01).

(b) RNI with n = 4 (Pr ≥ 0.5 ± 0.01).

Figure 9.4: Plots showing the change in ratio based on sampling across schedulers.

The robotics case plotted in Fig. 9.4a shows that we were able to get a concrete result in

the initial sampling for all three cases. We plotted an undecidable case in Fig. 9.4b. Note

that in experiment 2 we were able to get a concrete result in the initial sampling round;

158

025050075010001250150017502000Number of Samples103510301025102010151010105100Ratio (in log scale)Experiment 1Experiment 2Experiment 3Acceptance ratio0500100015002000250030003500Number of Samples103100103106109101210151018Ratio (in log scale)Experiment 1Experiment 2Experiment 3Acceptance ratioin experiment 1, we were able to reach a concrete result in the main algorithm loop after

intensive sampling within the chosen schedulers; and in experiment 3, we could neither find

an accepting scheduler nor reject all schedulers, leading to an inconclusive result. This

supports the results of undecidability that the algorithm returned. The main reason can be

traced back to the fact that we did not specify any strategy for the robots, thus, sampling

across random walks of the robot.

9.7 Summary

We presented a probabilistic formulation of bounded, unquantified HyperLTL and pro-

vided an SMC approach to verify them over MDP. To handle nondeterminism, our approach

leverages the smart sampling algorithm presented in [DLST15], extending it to reason about

hyperproperties. We have implemented our approach as an extension of PLASMA [LS14]

adding new capabilities to perform black-box verification and demonstrating the scalability

of our approach in several case studies with large state spaces. This work aimed to showcase

that SMC is a feasible solution for cases where exhaustive or bounded model checking is

unable to provide us with any insight. In future directions, we would like to extend support

for quantifier alternations for paths (as in HyperLTL) and scheduler tuples, as the current

approach can only handle existential scheduler tuples and limits our applicability to a wider

variety of security properties.

159

Chapter 10

Related Work

In this chapter, we summarize the inexhaustive line of work that has influenced our work.

We begin at the origins of the trace logic, their evolution to accommodate different types of

requirements, and the model checking methods used for them. Thereafter, we discuss related

works pertaining to the specific problems handled in each specific chapter in this thesis.

10.1 Discrete and Continuous Time Logics

Temporal logic essentially introduces an ordering among events or a ‘sense’ of time in the

system without actually including time. In Linear Temporal Logic (LTL) [Pnu77] argues over

all linear time finite traces. It is the extension of propositional logic with temporal operators.

An orthogonal attempt was developed as Computation Tree Logic (CTL) [BAMP81] to argue

about branching time logic. Later CTL* [EH83] was developed to subsume both LTL and

CTL. LTL and CTL have become standards in the model checking community.

Initially, to verify properties on probabilistic, concurrent finite-state programs, LTL was

used. The aim was to check if an LTL property holds in a Markov chain ‘almost always’.

The initial work on model checking of probabilistic programs and their optimizations was

presented in [HSP83], [Var85], [CY88]. This motivated the development of Probabilistic

Computation Tree Logic (PCTL) [HJ94]. The idea was to ensure soft bounds into properties

instead of ensuring something always happens. For e.g., ‘The system is up 98% of the time’

160

cannot be expressed in LTL or CTL. So, PCTL includes an explicit inclusion of probabilistic

operators in the logic and can be used for both quantitative verification along with the

qualitative answer provided by LTL or CTL. This was followed by [CY95], [DA98] where

the complexity of probabilistic logics for both linear and branching time logics have been

explored in terms of DTMCs.

In [BdA95], the authors extend the logic to include the

concept of satisfaction of probability operators to accommodate non-deterministic choices

and describe model checking algorithms for such systems. The complexity of their algorithm

is linear in the size of the state-space of the model.

In the context of continuous time systems,

[ASSB96] defined the logic Continuous

Stochastic Logic (CSL) to describe properties and proved that the verification problem is

decidable. In [BKH99], the authors provide a model checking algorithm for CSL by reducing

the problem to solving set of linear equations and Volterra integrals. They provide an ap-

proximate symbolic method for solving the integrals using MTDDs (multi-terminal decision

diagrams).

Binary Decision Diagrams Taking a step back, there have been several works on using

Binary Decision Diagrams and they brought in a major breakthrough in efficiently solving

model checking problems. In [Bry86], Bryant came up with the idea of using acyclic graphs

to represent Boolean functions. This brought a major breakthrough as a range of works

utilized this procedure for model checking. The complexity of the binary operations on

functions was proportional to the product of the graphs. The author describes the utility as

“... the performance of a program based on our algorithms when processing a sequence of

operations degrades slowly, if at all” [Bry86]. However, the major shortcoming of the seminal

work was that the size of the graph depended on the ordering in which the input variables

from the systems are used to build the graph. And finding the ordering is in itself a co-NP

problem. BDDs brought in the idea of symbolic model checking. The main idea of symbolic

model checking [BCM+92] [McM93] involves conversion of the state space into their symbolic

161

representations like BDDs which, when given a property, can be traversed and labeled to

check for inconsistencies. One of the main shortcomings of model checking has always been

the state-explosion problem. BDD representation has been a huge progress in handling this

problem. However, BDD was designed to handle Boolean representations. Multi-Terminal

BDD (MTBDD) [CFZ96] was introduced to handled real values. In [BCHG+97], the authors

used BDDs to represent properties, MTBDDs to represent Markov Chains, and utilized the

algorithm in [HJ94]. In [KNP00] the authors published the first experience of working with

the model checker PRISM [KNP01] that used MTBDDs. PRISM has been widely used

across domains to exhibit the power of probabilistic model checking to verify large models

with probabilistic properties.

10.2 Hyperproperties

In a series of work [FGM04], [McL96], [TA05], several important security properties were

hinted to not fall under the general category of trace properties.

In [CS08], the authors

formalized the concept of Hyperproperties. The paper elaborated on the difference between

trace and hyperproperties, possible application fields, and their topological characterization.

This brought on a whole new field of research as the different logics available for trace

properties had to be re-imagined as their hyper-components.

In [CFK+14], the authors

introduced HyperLTL and HyperCTL* to express linear and branching time hyperlogics,

and in [FRS15] the authors introduced the model checking algorithm for the same.

There has been a lot of recent progress in verifying [FMSZ17, FHT18, CFST19, Fin21,

PT18] and monitoring [AB16, FHST19, BSB17, BSS18, FHST18a, SSSB19, HST19] HyperLTL

specifications. A growing set of tools support HyperLTL, including the model checker MCHy-

per [FRS15, CFST19], the satisfiability checkers EAHyper [FHS17], MGHyper [FHH18],

explicit-state chceker for arbitrary quantifier called AutoHyper [BF23]. The model checking

problem of HyperLTL is in general undecidable. Hence, work has been done on fragment

specific approximate model checking solutions for the same. In [HSB21] the authors provided

162

a bounded model checking solution by reducing the problem to quantified Boolean formula

and solving it for a solution. HyperQB [HBS24] is the QBF-encoding based model checker

applying this theory. In [AMTZ21] the authors have explored augmented barrier certificates

for verifying HyperLTL for cyber-physical systems.

In [WNP20], the authors specifically

use HyperLTL to express optimal robotics path planning requirements in HyperLTL. In the

context of run-time verification, in [FHST17] the authors studied the monitoring problem for

HyperLTL under different combination of quantifiers involved, in [AB16] the authors studied

the verification of k-safety in HyperLTL, in [BSS18] the authors combined static analysis and

runtime verification to monitor HyperLTL properties, in [CFH+21] the authors studied the

enforcement of universally quantified HyperLTL, and in [FHST18b] the authors revealed the

development of the tool RVHyper for runtime-verification of HyperLTL hyperproperties. For

real-valued signals, [NKJ+17] introduced HyperSTL and [BDF+22] introduced HyperSTL*.

Additionally, all of the above approaches reason about their logic in synchronous settings.

Recently, there has been efforts to extend these languages to accommodate for asynchronous

settings [BPS21], [BCB+21], [GMOO21], [HBFS23b], [BBST24], [GMOO24] that adds more

expressivity along with additional challenges in their model checking solutions. This can be

interpreted as a way to handle nondeterminism in non-probabilistic systems.

10.3 Parameter Synthesis

A parametric DTMC [Daw04, LMST07] is a special class of Markov models, where

some the transition probabilities (or rates) are not known a-priori and are parameter-

dependent. These models can be adopted to analyze systems with stochastic uncertainty

due to the impossibility to measure certain quantities (e.g., fault rates, packet loss ratios,

etc.). In [Daw04], Daws proposed an approach to express the probability to reach the target

state as a rational function with the domain in the parameter space. This symbolic ap-

proach has been exploited in several model checking algorithms for parametric probabilistic

Markov chains [HHZ11, BGK+11, JCV+14, P ´AJ+15, DJJ+15, QDJ+16] and efficiently imple-

163

mented in PARAM1 and PARAM2 tools [HHZ11]. There are also type-theoretic approaches

(e.g., [SA19]) for synthesizing protocols for differential privacy, which is out of scope of our

target problem. The parameter synthesis problem consists in exploiting the generated ratio-

nal function to find the parameter values that would maximize or minimize the probability

to reach the target state [BGK+11]. The price to pay for these techniques is the increasing

complexity of the rational functions in the presence of large models [LMST07], causing the

parameter synthesis to be also very computationally expensive. However, the introduction

of new efficient heuristics [JCV+14, P ´AJ+15, DJJ+15, QDJ+16, CDP+17, ABC+18, SJK19]

has helped to alleviate this problem by supporting the parameter synthesis for quite large

models. For example, PROPhESY [DJJ+15] supports incremental automatic parameter

synthesis for parametric Markov chains w.r.t. reachability properties expressed in PCTL and

it exploits SMT techniques to determine safe and unsafe regions of the parameter space.

In contrast with PROPhESY, our approach enables the parameter synthesis for the richer

class of formal specifications defined by ReachHyperPCTL, a fragment of HyperPCTL. It is

worth pointing out that other approaches based on parameter lifting [QDJ+16, CDP+17],

although scalable, cannot be adapted to our framework. For example, the work in [QDJ+16]

propose to: (a) relax the dependency among the parameters by introducing free variables,

(b) replace parametric transitions by nondeterministic choices of extremal values, and (c)

analyze the resulting parameter-free Markov decision process by computing lower and upper

bounds on probabilities of regions in the parameter space. This approach is restricted to

work only for probabilistic properties with an eventually operator, while our ReachHyperPCTL

supports other operators such as the until. Furthermore, ReachHyperPCTL allows to compare

the value of two probabilistic operators and this feature makes it infeasible to use param-

eter lifting that provides probability intervals.

In [JJK22] the authors provide a detailed

comparison of the state-of-the-art concepts and solutions in the field of parameter synthesis.

In [J ´AH+19] the authors focus on research questions related to coverage of the parameter

space i.e., if all values in a region satisfy the property, finding regions and do or do not

164

satisfy the property, and finding approximate solution to the previous question to cover a

larger fraction of possible values.

10.4 Model Checking of Probabilistic Hyperproperties

In this context, it is important to note that the work in [DFT20] independently addresses

the problem of incorporating reasoning over nondeterminism similar to chapter 4.The authors

propose the temporal logic PHL. Similar to HyperPCTL, PHL also allows quantification over

schedulers, but path quantification of the induced DTMC is achieved by using HyperCTL∗.

Both the works show that the model checking problem is undecidable for the respective

logics. The difference, however, is in our approaches to deal with the undecidability result,

which leads to two complementary and orthogonal techniques. For both logics the problem

is decidable for non-probabilistic memoryless schedulers. We provide an SMT-based verifica-

tion procedure for HyperPCTL for this class of schedulers. The work in [DFT20] presents two

methods for proving and for refuting formulas from a fragment of PHL for general memo-

ryful schedulers. The authors propose an over-approximate and another under-approximate

automata-based model checking algorithms for the alternation-free n-safety fragment of their

logic PHL on n self-composed systems. The scheduler synthesis step is the main challenge

in this work.The two papers offer disjoint case studies for evaluation. In [WNBP21], the

authors provide a scalable solution to tackle probabilistic hyperproperties using statistical

model checking focusing on HyperPCTL∗. The idea is to use sequential probability ratio tests

with multi-dimensional indifference regions. This allows verification of nested probability op-

erators. However, this work does not allow non-determinism in the models. In asynchronous

settings [GD ´A+23] extends HyperPCTL to argue over novel stutter schedulers which allows

scheduler synthesis in systems that allows stuttering in states. The general complexity of

the model checking problem for probabilistic hyperproperties has motivated numerical solu-

tions for DTMCs [ZC ´AB22], and deductive pruning of strategies generated by abstraction

refinement [AB ˇC+23] for MDPs. Both these methods handle their respective fragments of

165

the logic efficiently.

There has been tremendous effort in language-based techniques for dealing with proba-

bilistic information-flow, specifically differential privacy (e.g., [BEH+17, BGHP16, BKOB13,

BGG+16]), but those techniques are not in the same context as the problem we deal with

across this dissertation. An interesting direction was pursed in [KVAK10] where the authors

viewed MDPs as distribution transformers which was capable of expressing properties in-

volving comparison of values in a set of states with another set of states at the same time.

This view is intuitively connected to hyperproperties. Recently, [ACMZ23, AGV18] have

used this view of MDPs as distribution transformers and worked with distribution-based

objectives for MDPs. For safety properties that can be specified as closed convex stochastic

polytopes, the authors show that the existential safety problem for MDPs is PTIME-complete

and that the universal safety problem is coNP-complete [AGV18]. This perspective has mo-

tivated our algorithm idea in chapter 8.

10.5 Statistical Model Checking

The hardness of the general model checking problem for quantitative properties has

motivated exploration of approximate solutions that find a balance between scalability and

accuracy within a specified confidence bound. The literature mentioned for hyperproperties

above suffer from the challenges of scalability, inability to handle probabilistic systems, or

lack of support for nondeterminism.

To verify hyperproperties in probabilistic systems there are two main families of ap-

proaches proposed in the literature: exact methods [ ´AB18, ´ABBD20b, D ´ABB22, DABB21]

and approximated ones [WNBP21, DP22]. Note that the specification language used in

these works differs from our specification language in chapter 9. Exact methods exploit the

underlying Markov chain structure of the probabilistic system to be verified for computing

precisely (numerically) and for comparing the probabilities of satisfying temporal logic for-

mulas of multiple and independent sequences of sets of states.

166

SMC has been explored to solve problems across different domains for analysing dy-

namic software architectures [CQT+16], performing security risk assessments using attack-

defence trees [GHL+16], verifying cyber-physical systems [CZ11], validation of biochemical

reaction models [Zul15], etc. Verification of bounded LTL for MDP has been proposed us-

ing SMC [HMZ+12] and has shown promising results. Extensive tool support exists for

SMC on trace properties with respect to discrete-event modelling [BCLS13], priced timed

automata [BDL+12], probabilistic model checking [KNP11, KZH+11, You05], black-box sys-

tems [GD08]. Statistical verification of probabilistic hyperlogics has been proposed for

HyperPCTL∗ [WNBP21, DP22], for continuous Markov chains [WZBP19], and for real-

valued signals [AHL+22]. However, none of these works reason about models involving

nondeterminism.

167

Chapter 11

Conclusion and Future Work

In the previous chapters, we have defined a generalized logic to express probabilistic

hyperproperties on Markov models, discussed the complexity involved in their model checking

problem, and explored approximate solutions to model check fragments of the logic that can

still be used to express important information flow and security related properties.

11.1 Summary

In Chapter 3, the parameter synthesis problem takes as input a parametric discrete-time

Markov chain and a probabilistic hyperproperty, and asks for valid parameter values for

which the induced discrete-time Markov chain satisfies the probabilistic hyperproperty. Our

synthesis algorithm works in two steps. In the first step, it computes symbolic conditions

for satisfying the formula, involving rational functions on the set of parameters.

In the

second step, it identifies regions of satisfying parameter configurations by decomposing the

domain of parameter configurations and exploring smaller regions in which either all or none

of the configurations lead to the satisfaction of the input formula. We demonstrated how

our algorithm works on several examples: randomized response, probabilistic conformance,

probabilistic noninterference, and dining cryptographers.

In Chapter 4, we investigated the problem of specifying and model checking probabilistic

hyperproperties of Markov decision processes (MDPs). Our study is motivated by the fact

168

that many systems have a probabilistic nature and are influenced by nondeterministic actions

of their environment. We extended the temporal logic HyperPCTL for DTMCs [ ´AB18] to the

context of MDPs by allowing formulas to quantify over schedulers. This additional expressive

power leads to the undecidability of the HyperPCTL model checking problem on MDPs,

but we also showed that the undecidable fragment becomes decidable for non-probabilistic

memoryless schedulers.

Indeed, all applications discussed here only require this type of

scheduler.

In Chapter 5, we focus on the decidable fragment of our logic and provide proof of de-

cidability by reducing the SAT problem to our model checking problem. We propose an

SMT-encoding based algorithm to model check the single-scheduler logic. The algorithm is

sound and complete and can provide a counterexample or witness to the specified hyper-

property. We provide evaluation results on a range of case studies involving security and

conformance.

In Chapter 6, we studied probabilistic hyperproperties with rewards. To this end, we

extended the temporal hyperlogic HyperPCTL with reward operators that associate quantified

computation trees with interrelated accumulated rewards. We also proposed an SMT-based

algorithm for model checking these formulas for MDPRs. We have created a prototypical

implementation and used it to analyse a few case studies.

In Chapter 7, we introduced HyperProb, a fully automated tool for model checking prob-

abilistic hyperproperties expressed in the temporal logic HyperPCTL for DTMCs and as well

as MDPs. We accumulated our algorithms from the previous chapters and combined them

into a push-button automated model checker for the decidable fragment of HyperPCTL with

rewards. We allow additional optimizations which scale better when compared to our pre-

vious implementations. We provide details of the inner workings of HyperProb along with

details on how to use the tool.

In Chapter 8, we concentrated on the problem of model checking fragments of the tem-

poral logic HyperPCTL for verification of probabilistic hyperproperties. We showed that these

169

fragments can specify interesting and useful probabilistic information-flow security policies

through various case studies. These fragments essentially involve quantitative relational

reachability reasoning to determine whether quantified policies and computations reach cer-

tain observations with equal probability. We proposed two algorithms to solve the model

checking problem for different fragments and scheduler classes. We also demonstrated the

effectiveness of our algorithms by conducting several case studies and showing orders of mag-

nitude speed-up, compared to the state of the art.

In Chapter 9, we explored a statistical model checking (SMC) approach for universally

quantified probabilistic hyperproperties for a fragment of HyperPLTL, allowing argument

over nondeterminism. The algorithm uses a sequential probability ratio test to provide

model checking results that scale well with a reduction in accuracy within a user-defined

confidence bound. We extended the prominent SMC tool PLASMA to handle this fragment

of hyperproperties. We showcased the effectiveness of our approach on case studies of huge

state space size, well beyond the capacity of existing solutions.

11.2 Future Work

Based on the current line of research my future work can be considered in two main

directions:

• Depth-wise, I want to explore further techniques to make more scalable tool support for

probabilistic hyperproperties available to the community. We have explored distinct

approaches to the model checking problem across different fragments and I intend to

incorporate them in HyperProb to enable it to run multiple model checking solutions

based on the type of specification. In the next section, I discuss possible future exten-

sions or optimizations that can be incorporated into our current algorithms.

• Breadth-wise, I want to explore more complex applications where our logic might be

helpful in specifying requirements for verification or synthesis.

170

11.2.1 Improvements in logic and model checking algorithms

HyperPCTL, with its extensions mentioned in this dissertation can be used to express com-

plex system properties. In terms of logic, it still has gaps such as lack of support for structural

constraints over schedulers and underlying DTMCs, and its inability to handle partial ob-

servability. Structural constraints can be used to filter out or limit the ‘good candidates’ for

schedulers of specific during synthesis. Additionally, it lacks support for reasoning over par-

tially observable systems. In reality, most systems are partially observable due to the lack of

range of observability or restricted permissibility of the observer. There exists a long line of

work in probabilistic model checking under partial observation [NPZ15], [AC20], [BPQR15],

[WJW+17], etc. When dealing with such systems, we require synthesis of observation-based

schedulers that agree on choosing the same action in similarly observed states. Intuitively,

this can be expressed as a hyperproperty where based on the comparison of observable out-

puts of two states, we restrict the choice of actions allowed in such paths. The prominent

model checkers for quantitative properties PRISM [KNP11] and STORM [HJK+22] come

with support for partially observable models.

With reference to chap. 8, there are some specific open problems. First, the complexity

of model checking for memoryful deterministic schedulers for all considered fragments re-

mains open. It would also be interesting to further examine the relationship between (1σ1s)

and distribution-based objectives, both from an algorithmic and a complexity perspective:

Can we employ algorithmic approaches for distribution-based objectives to solve our queries,

and can we adapt complexity results to our setting? We also plan to explore more complex

fragments of HyperPCTL like ones that need the until operator along with the comparison of

probabilities. Such formulas appear in reasoning about probabilistic causality with applica-

tion in information-flow security.

From the algorithm perspective, when reasoning over multiple traces, we often utilize

self-composition in cases of universally quantified specifications. This can be optimized by

utilizing the symmetry occurring in the two models. On the other hand, one of the main

171

bottlenecks in the algorithm is the controller synthesis step. Our statistical-based approach

(in chap. 9) was scalable essentially because we were able to handle the resolution of non-

determinism efficiently. Abstraction refinement-based approach [AB ˇC+23] has shown great

promise in this regard. Our current work in chap. 8 has utilized a guided exhaustive search.

Although encouraging, there is ample scope for improvement in this aspect by exploiting

structural intricacies in the model. We currently do not gather any further information

when exploring a scheduler that is not a good candidate for our specification. We can reduce

the number of explored actions here by eliminating the possibility of exploring redundant

action choices based on feedback from an already explored action combination. For statis-

tical model checking, the limitation currently is in its inability to handle the alternation of

quantifiers. But this extension is not trivial. For example, for a ∀ schedulers. ∃ scheduler

property, the result of the SMC algorithm might not be accurate since we are not in reality

exploring all possibilities. However, we still might be able to use over-and-under approxima-

tion over the set of schedulers to provide a result within a specified confidence bound.

11.2.2 Applications

On another front, the latest fast-track progress in the field of machine learning has

brought into focus the need for augmenting reasoning capabilities to large-scale machine

learning models. Probabilistic programming [vdMPYW21], [BKS20] has surfaced as the

general language represented in machine learning. Although the idea might not be new

[SPH84], [LMOW08], [PZ93], there has been a renewed interest in expanding reasoning over

probabilistic programming languages [Kat15], [DLHM18], [TT23], [SBK+23] by extending

existing program verification concepts to accommodate for probabilistic languages. This

poses a scope for probabilistic hyperproperties to expand on these works to express specifi-

cations beyond functional correctness in programs. Recently, the core program correctness

concept of Hoare triples has been extended to Hyper-Hoare triples [DM23] to argue about se-

curity and privacy-related hyperproperties like fairness, robustness, noninterference, etc., in

172

program semantics. Equivalence checking, which in itself is a form of conformance property

and hence, a hyperproperty, of programs among different probabilistic languages can help in

defining methods for easy and fast translation of solutions. Note that these extensions might

not be trivial as the models cannot be always represented as Markov models due to their

complexity. Hence, verification of probabilistic programs would require verification using

theorem proving instead of solely model checking.

Another important application is causality.

It is the relationship between cause and

effect. Due to the randomness involved in practical systems, it makes sense to interpret

the relationship in terms of probability. Trace properties in terms of PCTL have been used

[KM09] to find causes of events. However, using this process we can only argue about

each possible cause individually. Introducing the concept of hyperproperties into causality

would help us take this argument a step further to explore how relationships between causes

affect the occurrences of events. This would have applications in early diagnosis of diseases

using gene patterns, understanding financial markets, figuring out sustainable steps to slow

down climate change, etc. An associated challenge when exploring causality is the size of

the models used. Realistic models of the systems we aim to verify (human body or gene

structure, financial markets, climate cycle, etc.) are huge and cannot be limited to small-size

models as they will take away important aspects, causing a loss of the actual relationships

we want to observe. Our current exhaustive solutions explore all possible cases in a model to

prove its correctness. The general complexity of the problem hints at the need to find better

approximate solutions, based on our current theory, to find a balance between the accuracy

of our solutions and the time taken to find them.

173

BIBLIOGRAPHY

[AB16]

[ ´AB18]

[ ´ABBD20a]

[ ´ABBD20b]

[ABC+18]

[AB ˇC+23]

Shreya Agrawal and Borzoo Bonakdarpour. Runtime verification of k-safety
hyperproperties in HyperLTL. In 2016 IEEE 29th Computer Security Foun-
dations Symposium (CSF), pages 239–252, Lisbon, June 2016. IEEE.

E. ´Abrah´am and B. Bonakdarpour. HyperPCTL: A temporal logic for prob-
abilistic hyperproperties. In Proceedings of the 15th International Conference
on Quantitative Evaluation of Systems (QEST), pages 20–35, 2018.

E. ´Abrah´am, E. Bartocci, B. Bonakdarpour, and O. Dobe. Parameter synthe-
sis for probabilistic hyperproperties. In Proceedings of the 23rd International
Conference on Logic for Programming, Artificial Intelligence and Reasoning
(LPAR), pages 12–31, 2020.

Erika ´Abrah´am, Ezio Bartocci, Borzoo Bonakdarpour, and Oyendrila Dobe.
Probabilistic hyperproperties with nondeterminism. In Dang Van Hung and
Oleg Sokolsky, editors, Automated Technology for Verification and Analysis,
Lecture Notes in Computer Science, pages 518–534, Cham, 2020. Springer
International Publishing.

Sebastian Arming, Ezio Bartocci, Krishnendu Chatterjee, Joost-Pieter Ka-
toen, and Ana Sokolova. Parameter-independent strategies for pmdps via
pomdps. In Proc. of the 15th Int. Conf. on Quantitative Evaluation of Sys-
tems (QEST’18), volume 11024 of LNCS, pages 53–70. Springer, 2018.

Roman Andriushchenko, Ezio Bartocci, Milan ˇCeˇska, Francesco Pontiggia,
and Sarah Sallinger. Deductive controller synthesis for probabilistic hyper-
properties. In Nils Jansen and Mirco Tribastone, editors, Quantitative Eval-
uation of Systems, pages 288–306, Cham, 2023. Springer Nature Switzerland.

[AC20]

Eric Atkinson and Michael Carbin. Programming and reasoning with partial
observability. Proc. ACM Program. Lang., 4(OOPSLA), nov 2020.

[ACMZ23]

S. Akshay, Krishnendu Chatterjee, Tobias Meggendorfer, and Dorde Zikelic.
MDPs as Distribution Transformers: Affine Invariant Synthesis for Safety
In Constantin Enea and Akash Lal, editors, Computer Aided
Objectives.
Verification - 35th International Conference, CAV 2023, Paris, France, July
17-22, 2023, Proceedings, Part III, volume 13966 of Lecture Notes in Com-
puter Science, pages 86–112. Springer, 2023.

[ADDN17]

Aws Albarghouthi, Loris D’Antoni, Samuel Drews, and Aditya V. Nori.
Fairsquare: Probabilistic verification of program fairness. Proc. ACM Pro-
gram. Lang., 1(OOPSLA), oct 2017.

[AGV18]

S. Akshay, Blaise Genest, and Nikhil Vyas. Distribution-based objectives for
Markov Decision Processes. CoRR, abs/1804.09341, 2018.

174

[AHL+22]

[AMTZ21]

[AP18]

[ASSB96]

[BAMP81]

Shiraj Arora, Ren´e Rydhof Hansen, Kim Guldstrand Larsen, Axel Legay, and
Danny Bøgsted Poulsen. Statistical model checking for probabilistic hyper-
properties of real-valued signals.
In Owolabi Legunsen and Grigore Rosu,
editors, Model Checking Software, page 61–78, Cham, 2022. Springer Interna-
tional Publishing.

Mahathi Anand, Vishnu Murali, Ashutosh Trivedi, and Majid Zamani. For-
In Proceedings of
mal verification of hyperproperties for control systems.
the Workshop on Computation-Aware Algorithmic Design for Cyber-Physical
Systems, CAADCPS ’21, page 29–30, New York, NY, USA, 2021. Association
for Computing Machinery.

Gul Agha and Karl Palmskog. A survey of statistical model checking. ACM
Trans. Model. Comput. Simul., 28(1), jan 2018.

Adnan Aziz, Kumud Sanwal, Vigyan Singhal, and Robert Brayton. Verify-
ing continuous time markov chains. page 269–276, Berlin, Heidelberg, 1996.
Springer Berlin Heidelberg.

Mordechai Ben-Ari, Zohar Manna, and Amir Pnueli. The temporal logic of
branching time.
In Proceedings of the 8th ACM SIGPLAN-SIGACT Sym-
posium on Principles of Programming Languages, POPL ’81, page 164–176,
New York, NY, USA, 1981. Association for Computing Machinery.

[BBG08]

C. Baier, N. Bertrand, and M. Gr¨oßer. On decision problems for probabilistic
B¨uchi automata. In Proc. of FOSSACS’08, pages 287–301, 2008.

[BBGK12]

Christel Baier, Tom´as Br´azdil, Marcus Gr¨oßer, and Anton´ın Kucera. Stochas-
tic game logic. Acta Informatica, 49(4):203–224, 2012.

[BBST24]

[BCB+21]

[BCBS21]

Alberto Bombardelli, Laura Bozzelli, C´esar S´anchez, and Stefano Tonetta.
Unifying asynchronous logics for hyperproperties, 2024.

Jan Baumeister, Norine Coenen, Borzoo Bonakdarpour, Bernd Finkbeiner,
and C´esar S´anchez. A temporal logic for asynchronous hyperproperties. In
Alexandra Silva and K. Rustan M. Leino, editors, Computer Aided Verifica-
tion, pages 694–717, Cham, 2021. Springer International Publishing.

J. Baumeister, N. Coenen, B. Bonakdarpour, and B. Finkbeinerand C.
S´anchez. A temporal logic for asynchronous hyperproperties. In Proceedings
of the 33rd International Conference on Computer-Aided Verification (CAV),
pages 694–717, 2021.

[BCHG+97] C. Baier, E. Clarke, V. Hartonas-Garmhausen, M. Kwiatkowska, and
M. Ryan. Symbolic model checking for probabilistic processes. In P. Degano,
R. Gorrieri, and A. Marchetti-Spaccamela, editors, Proc. 24th International
Colloquium on Automata, Languages and Programming (ICALP’97), volume
1256 of LNCS, page 430–440. Springer, 1997.

175

[BCLS13]

[BCM+92]

[BdA95]

[BDF+22]

[BDL+12]

[BEH+17]

[BF18]

[BF23]

[BGG+16]

[BGHP16]

[BGK+11]

Benoˆıt Boyer, Kevin Corre, Axel Legay, and Sean Sedwards. Plasma-lab: A
flexible, distributable statistical model checking library. In Quantitative Eval-
uation of Systems: 10th International Conference, QEST 2013, Buenos Aires,
Argentina, August 27-30, 2013. Proceedings 10, pages 160–164. Springer,
2013.

J. R. Burch, E. M. Clarke, K. L. McMillan, D. L. Dill, and L. J. Hwang.
Symbolic model checking: 1020 states and beyond. Information and Compu-
tation, 98(2):142–170, 1992.

Andrea Bianco and Luca de Alfaro. Model checking of probabilistic and non-
deterministic systems. In P. S. Thiagarajan, editor, Foundations of Software
Technology and Theoretical Computer Science, page 499–513, Berlin, Heidel-
berg, 1995. Springer Berlin Heidelberg.

Sebastian Biewer, Rayna Dimitrova, Michael Fries, Maciej Gazda, Thomas
Heinze, Holger Hermanns, and Mohammad Reza Mousavi. Conformance Re-
lations and Hyperproperties for Doping Detection in Time and Space. Logical
Methods in Computer Science, Volume 18, Issue 1, January 2022.

Peter Bulychev, Alexandre David, Kim Gulstrand Larsen, Marius Mikuˇcionis,
Danny Bøgsted Poulsen, Axel Legay, and Zheng Wang. Uppaal-smc:
arXiv preprint
Statistical model checking for priced timed automata.
arXiv:1207.1272, 2012.

G. Barthe, T. Espitau, J. Hsu, T. Sato, and P.-Y. Strub. *-liftings for dif-
ferential privacy. In Proceedings of the 44th International Colloquium on Au-
tomata, Languages, and Programming (ICALP), pages 102:1–102:12, 2017.

Borzoo Bonakdarpour and Bernd Finkbeiner. The complexity of monitoring
hyperproperties. In 2018 IEEE 31st Computer Security Foundations Sympo-
sium (CSF), pages 162–174, 2018.

Raven Beutner and Bernd Finkbeiner. Autohyper: Explicit-state model
checking for hyperltl. In Sriram Sankaranarayanan and Natasha Sharygina,
editors, Tools and Algorithms for the Construction and Analysis of Systems,
pages 145–163, Cham, 2023. Springer Nature Switzerland.

G. Barthe, M. Gaboardi, B. Gr´egoire, J. Hsu, and P.-Y. Strub. Proving
differential privacy via probabilistic couplings.
In Proceedings of the 31st
Annual ACM/IEEE Symposium on Logic in Computer Science (LICS), pages
749–758, 2016.

G. Barthe, M. Gaboardi, J. Hsu, and B. C. Pierce. Programming language
techniques for differential privacy. SIGLOG News, 3(1):34–53, 2016.

Ezio Bartocci, Radu Grosu, Panagiotis Katsaros, C. R. Ramakrishnan, and
Scott A. Smolka. Model repair for probabilistic systems. In Proc. TACAS
2011, volume 6605 of LNCS, pages 326–340, 2011.

176

[BK08]

[BKH99]

Christel Baier and Joost-Pieter Katoen. Principles of Model Checking (Rep-
resentation and Mind Series). The MIT Press, 2008.

Christel Baier, Joost-Pieter Katoen, and Holger Hermanns. Approximative
In Jos C. M.
symbolic model checking of continuous-time markov chains.
Baeten and Sjouke Mauw, editors, CONCUR’99 Concurrency Theory, page
146–161, Berlin, Heidelberg, 1999. Springer Berlin Heidelberg.

[BKOB13]

G. Barthe, B. K¨opf, F. Olmedo, and S. Z. B´eguelin. Probabilistic relational
reasoning for differential privacy. ACM Transactions on Programming Lan-
guages and Systems (TOPLAS), 35(3):9:1–9:49, 2013.

[BKS20]

[BP09]

Gilles Barthe, Joost-Pieter Katoen, and Alexandra Silva. Foundations of
probabilistic programming. Cambridge University Press, 2020.

Romain Beauxis and Catuscia Palamidessi. Probabilistic and nondeterminis-
tic aspects of anonymity. Theoretical Computer Science, 410(41):4006–4025,
2009.

[BPQR15]

Simon Busard, Charles Pecheur, Hongyang Qu, and Franco Raimondi. Rea-
soning about memoryless strategies under partial observability and uncondi-
tional fairness constraints. Information and Computation, 242:128–156, 2015.

[BPS21]

[Bry86]

[BSB17]

[BSS18]

Laura Bozzelli, Adriano Peron, and C´esar S´anchez. Asynchronous extensions
of hyperltl. In 2021 36th Annual ACM/IEEE Symposium on Logic in Com-
puter Science (LICS), pages 1–13, 2021.

Bryant. Graph-based algorithms for boolean function manipulation. IEEE
Transactions on Computers, C-35(8):677–691, 1986.

N. Brett, U. Siddique, and B. Bonakdarpour. Rewriting-based runtime verifi-
cation for alternation-free HyperLTL. In Proc. of the 23rd Int. Conf. on Tools
and Algorithms for the Construction and Analysis of Systems (TACAS’17),
volume 10206 of LNCS, pages 77–93. Springer, 2017.

B. Bonakdarpour, C. S´anchez, and G. Schneider. Monitoring hyperproper-
In Proc. of the
ties by combining static analysis and runtime verification.
8th Leveraging Applications of Formal Methods, Verification and Validation
(ISoLA’18), volume 11245 of LNCS, pages 8–27. Springer, 2018.

[CARa]

CARL. https://ths-rwth.github.io/carl/.

[CARb]

CARL PARSER. https://github.com/ths-rwth/carl-parser/.

[CDP+17]

M. Ceska, F. Dannenberg, N. Paoletti, M. Kwiatkowska, and L. Brim. Precise
parameter synthesis for stochastic biochemical systems. Acta Informatica,
54(6):589–623, 2017.

177

[CFH+21]

[CFK+14]

Norine Coenen, Bernd Finkbeiner, Christopher Hahn, Jana Hofmann, and
Yannick Schillo. Runtime enforcement of hyperproperties. In Zhe Hou and
Vijay Ganesh, editors, Automated Technology for Verification and Analysis,
page 283–299, Cham, 2021. Springer International Publishing.

Michael R. Clarkson, Bernd Finkbeiner, Masoud Koleini, Kristopher K.
Micinski, Markus N. Rabe, and C´esar S´anchez. Temporal logics for hyper-
properties. In David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Klein-
berg, Friedemann Mattern, John C. Mitchell, Moni Naor, Oscar Nierstrasz,
C. Pandu Rangan, Bernhard Steffen, Madhu Sudan, Demetri Terzopoulos,
Doug Tygar, Moshe Y. Vardi, Gerhard Weikum, Mart´ın Abadi, and Steve
Kremer, editors, Principles of Security and Trust, volume 8414, pages 265–
284. Springer Berlin Heidelberg, Berlin, Heidelberg, 2014.

[CFST19]

[CFZ96]

N. Coenen, B. Finkbeiner, C. S´anchez, and L. Tentrup. Verifying hyperlive-
ness. In Proc. of CAV’19, pages 121–139, 2019.

Edmund M Clarke, Masahiro Fujita, and Xudong Zhao. Multi-terminal bi-
nary decision diagrams and hybrid decision diagrams. In Representations of
discrete functions, pages 93–108. Springer, 1996.

[Cha88]

David Chaum. The dining cryptographers problem: Unconditional sender
and recipient untraceability. Journal of Cryptology, 1(1):65–75, Jan 1988.

[Chu67]

Kai Lai Chung. Markov chains. Springer-Verlag, New York, 1967.

[CQT+16]

[CS08]

[CS10]

[CY88]

[CY95]

[CZ11]

Everton Cavalcante, Jean Quilbeuf, Louis-Marie Traonouez, Flavio Oquendo,
Thais Batista, and Axel Legay. Statistical model checking of dynamic software
architectures.
In Software Architecture: 10th European Conference, ECSA
2016, Copenhagen, Denmark, November 28–December 2, 2016, Proceedings
10, pages 185–200. Springer, 2016.

Michael R. Clarkson and Fred B. Schneider. Hyperproperties. In 2008 21st
IEEE Computer Security Foundations Symposium, page 51–65, Jun 2008.

M. R. Clarkson and F. B. Schneider. Hyperproperties. Journal of Computer
Security, 18(6):1157–1210, 2010.

C. Courcoubetis and M. Yannakakis. Verifying temporal properties of finite-
state probabilistic programs. In [Proceedings 1988] 29th Annual Symposium
on Foundations of Computer Science, pages 338–345, 1988.

Costas Courcoubetis and Mihalis Yannakakis. The complexity of probabilistic
verification. J. ACM, 42(4):857–907, jul 1995.

Edmund M. Clarke and Paolo Zuliani. Statistical model checking for cyber-
physical systems. In Tevfik Bultan and Pao-Ann Hsiung, editors, Automated
Technology for Verification and Analysis, page 1–12, Berlin, Heidelberg, 2011.
Springer Berlin Heidelberg.

178

[DA98]

Luca De Alfaro. Formal verification of probabilistic systems, 1998.

[DABB21]

Oyendrila Dobe, Erika Abraham, Ezio Bartocci, and Borzoo Bonakdarpour.
Hyperprob: a model checker for probabilistic hyperproperties.
In Formal
Methods: 24th International Symposium, FM 2021, Virtual Event, November
20–26, 2021, Proceedings 24, pages 657–666. Springer, 2021.

[D ´ABB22]

Oyendrila Dobe, Erika ´Abrah´am, Ezio Bartocci, and Borzoo Bonakdarpour.
Model checking hyperproperties for markov decision processes. Information
and Computation, 289:104978, 2022.

[Daw04]

[DFT20]

[Dij75]

[DJJ+15]

[DJKV17]

[DLHM18]

[DLST15]

[DM23]

C. Daws. Symbolic and parametric model checking of discrete-time Markov
chains. In Proc. of the First Int. Conf. on Theoretical Aspects of Computing
(ICTAC’04), volume 3407 of LNCS, pages 280–294. Springer, 2004.

Rayna Dimitrova, Bernd Finkbeiner, and Hazem Torfah. Probabilistic hy-
perproperties of markov decision processes.
the
18th International Symposium on Automated Technology for Verification and
Analysis, volume 12302 of LNCS, pages 484–500. Springer, 2020.

In Proc. of ATVA 2020:

E. J Dijkstra. Guarded commands, nondeterminancy and formal derivations
of programs. 1975.

Christian Dehnert, Sebastian Junges, Nils Jansen, Florian Corzilius, Matthias
Volk, Harold Bruintjes, Joost-Pieter Katoen, and Erika ´Abrah´am. Prophesy:
A probabilistic parameter synthesis tool. In Proceedings of the 27th Inter-
national Conference on Computer-Aided Verification (CAV), pages 214–231,
2015.

C. Dehnert, S. Junges, J.-P. Katoen, and M. Volk. A Storm is coming:
A modern probabilistic model checker.
In Proc. of the 29th Int. Conf. on
Computer Aided Verification (CAV’17), volume 10427 of LNCS, pages 592–
600. Springer, 2017.

Saikat Dutta, Owolabi Legunsen, Zixin Huang, and Sasa Misailovic. Testing
probabilistic programming systems.
In Proceedings of the 2018 26th ACM
Joint Meeting on European Software Engineering Conference and Symposium
on the Foundations of Software Engineering, ESEC/FSE 2018, page 574–586,
New York, NY, USA, 2018. Association for Computing Machinery.

Pedro D’argenio, Axel Legay, Sean Sedwards, and Louis-Marie Traonouez.
Smart sampling for lightweight verification of markov decision processes. Int.
J. Softw. Tools Technol. Transf., 17(4):469–484, aug 2015.

Thibault Dardinier and Peter M¨uller. Hyper hoare logic:(dis-) proving pro-
gram hyperproperties (extended version). arXiv preprint arXiv:2301.10037,
2023.

179

[dMB08]

[DMNS16]

[DN11]

[doc]

[DP22]

[DR14]

[DSB+23]

[DW ´A+22]

[EH83]

[FBT13]

[FGM04]

[FHH18]

L. M. de Moura and N. Bjørner. Z3: An efficient SMT solver. In Tools and
Algorithms for the Construction and Analysis of Systems (TACAS), pages
337–340, 2008.

C. Dwork, F. McSherry, K. Nissim, and A. D. Smith. Calibrating noise
to sensitivity in private data analysis. Journal of Privacy Confidentiality,
7(3):17–51, 2016.

Rocco De Nicola. Process Algebras, pages 1624–1636. Springer US, Boston,
MA, 2011.

docker. Docker. https://www.docker.com/get-started.

Spandan Das and Pavithra Prabhakar. Bayesian statistical model checking
for multi-agent systems using hyperpctl*, 2022.

Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential
privacy. Found. Trends Theor. Comput. Sci., 9(3–4):211–407, aug 2014.

Oyendrila Dobe, Stefan Schupp, Ezio Bartocci, Borzoo Bonakdarpour, Axel
Legay, Miroslav Pajic, and Yu Wang. Lightweight verification of hyperprop-
In ´Etienne Andr´e and Jun Sun, editors, Automated Technology for
erties.
Verification and Analysis, pages 3–25, Cham, 2023. Springer Nature Switzer-
land.

Oyendrila Dobe, Lukas Wilke, Erika ´Abrah´am, Ezio Bartocci, and Borzoo
Bonakdarpour. Probabilistic hyperproperties with rewards. In NASA Formal
Methods: 14th International Symposium, NFM 2022, Pasadena, CA, USA,
May 24–27, 2022, Proceedings, pages 656–673. Springer, 2022.

E. Allen Emerson and Joseph Y. Halpern. ”sometimes” and ”not never” revis-
ited: On branching versus linear time (preliminary report). In Proceedings of
the 10th ACM SIGACT-SIGPLAN Symposium on Principles of Programming
Languages, POPL ’83, page 127–140, New York, NY, USA, 1983. Association
for Computing Machinery.

Narges Fallahi, Borzoo Bonakdarpour, and S´ebastien Tixeuil. Rigorous per-
formance evaluation of self-stabilization using probabilistic model checking.
In Proc. of SRDS’13: the 32nd IEEE Int. Conf. on Reliable Distributed Sys-
tems, pages 153–162. IEEE Computer Society, 2013.

Riccardo Focardi, Roberto Gorrieri, and Fabio Martinelli. Classification of
security properties (part ii: Network security). 2004.

B. Finkbeiner, C. Hahn, and T. Hans. MGHyper: Checking satisfiability
of HyperLTL formulas beyond the ∃∗∀∗ fragment. In Proc. of the 16th Int.
Symp. on Automated Technology for Verification and Analysis (ATVA’18),
volume 11138 of LNCS, pages 521–527. Springer, 2018.

180

[FHS17]

[FHST17]

[FHST18a]

[FHST18b]

B. Finkbeiner, C. Hahn, and M. Stenger. EAHyper: Satisfiability, implication,
and equivalence checking of hyperproperties. In Proc. of the 29th Int. Conf.
on Computer Aided Verification (CAV’17), volume 10427 of LNCS, pages
564–570. Springer, 2017.

Bernd Finkbeiner, Christopher Hahn, Marvin Stenger, and Leander Tentrup.
Monitoring hyperproperties.
In Shuvendu Lahiri and Giles Reger, editors,
Runtime Verification, page 190–207, Cham, 2017. Springer International Pub-
lishing.

B. Finkbeiner, C. Hahn, M. Stenger, and L. Tentrup. RVHyper: A run-
time verification tool for temporal hyperproperties. In Proc. of the 24th Int.
Conf. on Tools and Algorithms for the Construction and Analysis of Systems
(TACAS’18), volume 10806 of LNCS, pages 194–200. Springer, 2018.

Bernd Finkbeiner, Christopher Hahn, Marvin Stenger, and Leander Tentrup.
Rvhyper: A runtime verification tool for temporal hyperproperties. In Dirk
Beyer and Marieke Huisman, editors, Tools and Algorithms for the Construc-
tion and Analysis of Systems, page 194–200, Cham, 2018. Springer Interna-
tional Publishing.

[FHST19]

B. Finkbeiner, C. Hahn, M. Stenger, and L. Tentrup. Monitoring hyperprop-
erties. Formal Methods in System Design (FMSD), 54(3):336–363, 2019.

[FHT18]

[Fin21]

B. Finkbeiner, C. Hahn, and H. Torfah. Model checking quantitative hyper-
properties. In Proc. of the 30th Int. Conf. on Computer Aided Verification
(CAV’18), volume 10981 of LNCS, pages 144–163. Springer, 2018.

Bernd Finkbeiner. Model checking algorithms for hyperproperties (invited
paper). In Fritz Henglein, Sharon Shoham, and Yakir Vizel, editors, Verifica-
tion, Model Checking, and Abstract Interpretation, pages 3–16, Cham, 2021.
Springer International Publishing.

[FMSZ17]

B. Finkbeiner, Ch. M¨uller, H. Seidl, and E. Zalinescu. Verifying Security
Policies in Multi-agent Workflows with Loops. In Proc. of CCS’17, 2017.

[FRS15]

[GD08]

[GD ´A+23]

Bernd Finkbeiner, Markus N. Rabe, and C´esar S´anchez. Algorithms for model
In Daniel Kroening and Corina S.
checking HyperLTL and HyperCTL*.
P˘as˘areanu, editors, Computer Aided Verification, Lecture Notes in Computer
Science, pages 30–48. Springer International Publishing, 2015.

David R. Gilbert and Robin Donaldson. A monte carlo model checker for
probabilistic ltl with numerical constraints. 2008.

Lina Gerlach, Oyendrila Dobe, Erika ´Abrah´am, Ezio Bartocci, and Borzoo
Bonakdarpour. Introducing asynchronicity to probabilistic hyperproperties.
In Nils Jansen and Mirco Tribastone, editors, Quantitative Evaluation of Sys-
tems, pages 47–64, Cham, 2023. Springer Nature Switzerland.

181

[GHL+16]

Olga Gadyatskaya, Ren´e Rydhof Hansen, Kim Guldstrand Larsen, Axel
Legay, Mads Chr Olesen, and Danny Bøgsted Poulsen. Modelling attack-
defense trees using timed automata.
In Formal Modeling and Analysis of
Timed Systems: 14th International Conference, FORMATS 2016, Quebec,
QC, Canada, August 24-26, 2016, Proceedings 14, pages 35–50. Springer,
2016.

[GM82]

J. A. Goguen and J. Meseguer. Security policies and security models.
IEEE Symp. on Security and Privacy, pages 11–20, 1982.

In

[GMB17]

M. Guarnieri, S. Marinovic, and D. Basin. Securing databases from proba-
bilistic inference. In Proc. of CSF’17, pages 343–359, 2017.

[GMOO21]

Jens Oliver Gutsfeld, Markus M¨uller-Olm, and Christoph Ohrem. Automata
and fixpoints for asynchronous hyperproperties. Proc. ACM Program. Lang.,
5(POPL), jan 2021.

[GMOO24]

Jens Oliver Gutsfeld, Markus M¨uller-Olm, and Christoph Ohrem. Deciding
asynchronous hyperproperties for recursive programs. Proc. ACM Program.
Lang., 8(POPL), jan 2024.

[HBFS23a]

Tzu-Han Hsu, Borzoo Bonakdarpour, Bernd Finkbeiner, and C´esar S´anchez.
Bounded model checking for asynchronous hyperproperties.
CoRR,
abs/2301.07208, 2023.

[HBFS23b]

Tzu-Han Hsu, Borzoo Bonakdarpour, Bernd Finkbeiner, and C´esar S´anchez.
Bounded model checking for asynchronous hyperproperties.
In Sriram
Sankaranarayanan and Natasha Sharygina, editors, Tools and Algorithms for
the Construction and Analysis of Systems, pages 29–46, Cham, 2023. Springer
Nature Switzerland.

[HBS24]

[Her90]

[HHZ11]

[Hit21]

Tzu-Han Hsu, Borzoo Bonakdarpour, and C´esar S´anchez. Hyperqb: A qbf-
based bounded model checker for hyperproperties, 2024.

Ted Herman. Probabilistic self-stabilization. Information Processing Letters,
35(2):63–67, 1990.

Ernst Moritz Hahn, Tingting Han, and Lijun Zhang. Probabilistic reachability
for parametric Markov models. STTT, 13(1):3–19, 2011.

Christopher Hitchcock. Probabilistic Causation. In Edward N. Zalta, editor,
The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stan-
ford University, Spring 2021 edition, 2021.

[HJ94]

Hans Hansson and Bengt Jonsson. A logic for reasoning about time and
reliability. Formal Aspects of Computing, 6(5):512–535, Sep 1994.

182

[HJK+22]

[HMZ+12]

[HSB21]

[HSP83]

[HST19]

[III92]

[IJ90]

[J ´AH+19]

[JCV+14]

[JJK22]

[Kat15]

Christian Hensel, Sebastian Junges, Joost-Pieter Katoen, Tim Quatmann,
and Matthias Volk. The probabilistic model checker storm. Int. J. Softw.
Tools Technol. Transf., 24(4):589–610, aug 2022.

David Henriques, Jo˜ao G. Martins, Paolo Zuliani, Andr´e Platzer, and Ed-
mund M. Clarke. Statistical model checking for markov decision processes.
In 2012 Ninth International Conference on Quantitative Evaluation of Sys-
tems, pages 84–93, 2012.

Tzu-Han Hsu, C´esar S´anchez, and Borzoo Bonakdarpour. Bounded model
checking for hyperproperties.
In Jan Friso Groote and Kim Guldstrand
Larsen, editors, Tools and Algorithms for the Construction and Analysis of
Systems, page 94–112, Cham, 2021. Springer International Publishing.

Sergiu Hart, Micha Sharir, and Amir Pnueli. Termination of probabilistic
concurrent program. ACM Trans. Program. Lang. Syst., 5(3):356–380, jul
1983.

C. Hahn, M. Stenger, and L. Tentrup. Constraint-based monitoring of hyper-
properties. In Proc. of the 25th Int. Conf. on Tools and Algorithms for the
Construction and Analysis of Systems (TACAS’19), volume 11428 of LNCS,
pages 115–131. Springer, 2019.

J. W Gray III. Toward a mathematical foundation for information flow secu-
rity. Journal of Computer Security, 1(3-4):255–294, May 1992.

A. Israeli and M. Jalfon. Token management schemes and random walks
yield self-stabilizing mutual exclusion. In Cynthia Dwork, editor, Proceedings
of the Ninth Annual ACM Symposium on Principles of Distributed Computing
(PODC), pages 119–131, 1990.

Sebastian Junges, Erika ´Abrah´am, Christian Hensel, Nils Jansen, Joost-Pieter
Katoen, Tim Quatmann, and Matthias Volk. Parameter synthesis for markov
models. CoRR, abs/1903.07993, 2019.

Nils Jansen, Florian Corzilius, Matthias Volk, Ralf Wimmer, Erika ´Abrah´am,
Joost-Pieter Katoen, and Bernd Becker. Accelerating parametric probabilistic
verification. In Proc. QEST 2014, volume 8657 of LNCS, pages 404–420, 2014.

Nils Jansen, Sebastian Junges, and Joost-Pieter Katoen. Parameter Synthe-
sis in Markov Models: A Gentle Survey, pages 407–437. Springer Nature
Switzerland, Cham, 2022.

Joost-Pieter Katoen. Probabilistic programming: A true verification chal-
lenge.
In Bernd Finkbeiner, Geguang Pu, and Lijun Zhang, editors, Au-
tomated Technology for Verification and Analysis, pages 1–3, Cham, 2015.
Springer International Publishing.

183

[KM09]

[KNP00]

[KNP01]

[KNP11]

Samantha Kleinberg and Bud Mishra. The Temporal Logic of Causal Struc-
tures.
In Proceedings of the 25th Conference on Uncertainty in Artificial
Intelligence (UAI), Montreal, Quebec, June 2009.

M. Kwiatkowska, G. Norman, and D. Parker. Verifying randomized dis-
tributed algorithms with prism.
In Proc. Workshop on Advances in Veri-
fication (Wave’2000), Jul 2000.

M. Kwiatkowska, G. Norman, and D. Parker. Prism: Probabilistic symbolic
model checker.
In P. Kemper, editor, Proc. Tools Session of Aachen 2001
International Multiconference on Measurement, Modelling and Evaluation of
Computer-Communication Systems, page 7–12, Sep 2001.

M. Kwiatkowska, G. Norman, and D. Parker. PRISM 4.0: Verification of
In G. Gopalakrishnan and S. Qadeer, ed-
probabilistic real-time systems.
itors, Proc. 23rd International Conference on Computer Aided Verification
(CAV’11), volume 6806 of LNCS, pages 585–591. Springer, 2011.

[Kri63]

S. Kripke. Semantical Consideration on Modal Logic, page 83–94. 1963.

[KVAK10]

[KY76]

[KZH+11]

Vijay Anand Korthikanti, Mahesh Viswanathan, Gul Agha, and YoungMin
Kwon. Reasoning about mdps as transformers of probability distributions.
In 2010 Seventh International Conference on the Quantitative Evaluation of
Systems, pages 199–208, 2010.

D. Knuth and A. Yao. Algorithms and Complexity: New Directions and Recent
Results, chapter The complexity of nonuniform random number generation.
Academic Press, 1976.

Joost-Pieter Katoen, Ivan S Zapreev, Ernst Moritz Hahn, Holger Hermanns,
and David N Jansen. The ins and outs of the probabilistic model checker
mrmc. Performance evaluation, 68(2):90–104, 2011.

[lar]

lark. LARK. https://lark-parser.readthedocs.io/.

[LDB10]

[LL20]

Axel Legay, Benoˆıt Delahaye, and Saddek Bensalem. Statistical model check-
In Howard Barringer, Ylies Falcone, Bernd Finkbeiner,
ing: An overview.
Klaus Havelund, Insup Lee, Gordon Pace, Grigore Ro¸su, Oleg Sokolsky, and
Nikolai Tillmann, editors, Runtime Verification, page 122–135, Berlin, Hei-
delberg, 2010. Springer Berlin Heidelberg.

Kim G. Larsen and Axel Legay. 30 years of statistical model checking. In
Tiziana Margaria and Bernhard Steffen, editors, Leveraging Applications of
Formal Methods, Verification and Validation: Verification Principles - 9th In-
ternational Symposium on Leveraging Applications of Formal Methods, ISoLA
2020, Rhodes, Greece, October 20-30, 2020, Proceedings, Part I, volume 12476
of Lecture Notes in Computer Science, pages 325–330. Springer, 2020.

184

[LLT+19]

[LMOW08]

[LMST07]

[LS14]

[McL96]

[McM93]

[NKJ+17]

[NPZ15]

[NSH13]

[P ´AJ+15]

Axel Legay, Anna Lukina, Louis Marie Traonouez, Junxing Yang, Scott A
Smolka, and Radu Grosu. Statistical model checking.
In Computing and
software science: state of the art and perspectives, pages 478–504. Springer,
2019.

Axel Legay, Andrzej S Murawski, Jo¨el Ouaknine, and James Worrell. On
automated verification of probabilistic programs. In Tools and Algorithms for
the Construction and Analysis of Systems: 14th International Conference,
TACAS 2008, Held as Part of the Joint European Conferences on Theory
and Practice of Software, ETAPS 2008, Budapest, Hungary, March 29-April
6, 2008. Proceedings 14, pages 173–187. Springer, 2008.

R. Lanotte, A. Maggiolo-Schettini, and A. Troina. Parametric probabilistic
transition systems for system design and analysis. Formal Aspects of Com-
puting, 19(1):93–109, 2007.

Axel Legay and Sean Sedwards. On statistical model checking with plasma.
In The 8th International Symposium on Theoretical Aspects of Software En-
gineering, 2014.

John McLean. A general theory of composition for a class of” possibilistic”
properties. IEEE Transactions on Software Engineering, 22(1):53–67, 1996.

Kenneth L McMillan. Symbolic model checking. In Symbolic Model Checking,
pages 25–60. Springer, 1993.

Luan Viet Nguyen, James Kapinski, Xiaoqing Jin, Jyotirmoy V. Deshmukh,
and Taylor T. Johnson. Hyperproperties of real-valued signals. In Proceedings
of the 15th ACM-IEEE International Conference on Formal Methods and
Models for System Design, MEMOCODE ’17, page 104–113, New York, NY,
USA, 2017. Association for Computing Machinery.

Gethin Norman, D. Parker, and Xueyi Zou. Verification and control of par-
tially observable probabilistic real-time systems. In International Conference
on Formal Modeling and Analysis of Timed Systems, 2015.

T. M. Ngo, M. Stoelinga, and M. Huisman. Confidentiality for probabilistic
multi-threaded programs and its verification.
In Proc. of ESSoS’13, pages
107–122, 2013.

S. Pathak, E. ´Abrah´am, N. Jansen, A. Tacchella, and J.-P. Katoen. A greedy
approach for the efficient repair of stochastic models. In Proc. of the 7th Int.
NASA Formal Methods Symp. (NFM’15), volume 9058 of LNCS, pages 295–
309. Springer, 2015.

[Pnu77]

Amir Pnueli. The temporal logic of programs. In 18th Annual Symposium on
Foundations of Computer Science (sfcs 1977), pages 46–57, 1977.

185

[PT18]

[Put90]

[pyc]

[PZ93]

[QDJ+16]

[Qua23]

[RD94]

Adrien Pommellet and Tayssir Touili. Model-checking hyperltl for pushdown
systems. In Mar´ıa del Mar Gallardo and Pedro Merino, editors, Model Check-
ing Software, pages 133–152, Cham, 2018. Springer International Publishing.

Martin L Puterman. Markov decision processes. Handbooks in operations
research and management science, 2:331–434, 1990.

pycarl. pycarl. https://moves-rwth.github.io/pycarl/.

Amir Pnueli and Lenore D Zuck. Probabilistic verification. Information and
computation, 103(1):1–29, 1993.

T. Quatmann, C. Dehnert, N. Jansen, S. Junges, and J.-P. Katoen. Parameter
synthesis for Markov models: Faster than ever. In Proc. of the 14th Int. Symp.
on Automated Technology for Verification and Analysis (ATVA’16), volume
9938 of LNCS, pages 50–67. Springer, 2016.

Tim Quatmann. Verification of Multi-Objective Markov Models. PhD thesis,
RWTH Aachen University, 2023.

Rajeev Alur and David L. Dill. A theory of timed automata. Theoretical
Computer Science, 126:183–235, 1994.

[Rei85]

W. Reisig. Petri Nets: An Introduction, volume 4. Springer, 1985.

[SA19]

[SBK+23]

[SJK19]

[Smi03]

[SPH84]

[SSSB19]

C. Smith and A. Albarghouthi. Synthesizing differentially private programs.
Proc. of the ACM on Programming Languages (PACMPL’19), 3(ICFP):94:1–
94:29, 2019.

Philipp Schr¨oer, Kevin Batz, Benjamin Lucien Kaminski, Joost-Pieter Ka-
toen, and Christoph Matheja. A deductive verification infrastructure for
probabilistic programs. Proc. ACM Program. Lang., 7(OOPSLA2), oct 2023.

J. Spel, S. Junges, and J.-P. Katoen. Are parametric Markov chains mono-
tonic?
In Proc. of the 17th Int. Symp. on Automated Technology for Ver-
ification and Analysis (ATVA’19), volume 11781 of LNCS, pages 479–496.
Springer, 2019.

Geoffrey Smith. Probabilistic noninterference through weak probabilistic
bisimulation. In Proceedings of the 16th IEEE Computer Security Founda-
tions Workshop (CSF), pages 3–13, 2003.

Micha Sharir, Amir Pnueli, and Sergiu Hart. Verification of probabilistic
programs. SIAM Journal on Computing, 13(2):292–314, 1984.

S. Stucki, C. S´anchez, G. Schneider, and B. Bonakdarpour. Gray-box mon-
itoring of hyperproperties.
In Proc. of the 3rd World Congress on Formal
Methods (FM’19), volume 11800 of LNCS, pages 406–424. Springer, 2019.

186

[stoa]

[stob]

[TA05]

[TT23]

STORM: A tool for the analysis of systems involving random or probabilistic
phenomena. http://www.stormchecker.org/index.html.

stormpy. STORMPY. https://moves-rwth.github.io/stormpy/.

Tachio Terauchi and Alex Aiken. Secure information flow as a safety problem.
In International Static Analysis Symposium, pages 352–367. Springer, 2005.

Joseph Tassarotti and Jean-Baptiste Tristan. Verified density compilation for
a probabilistic programming language. Proc. ACM Program. Lang., 7(PLDI),
jun 2023.

[Uck12]

S. L. Uckelman. Arthur prior and medieval logic. 188(3):349–366, 2012.

[Var85]

Moshe Y. Vardi. Automatic verification of probabilistic concurrent finite state
programs. 26th Annual Symposium on Foundations of Computer Science (sfcs
1985), pages 327–338, 1985.

[vdMPYW21] Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, and Frank Wood.

An introduction to probabilistic programming, 2021.

[Wal45]

A. Wald. Sequential Tests of Statistical Hypotheses. The Annals of Mathe-
matical Statistics, 16(2):117 – 186, 1945.

[Way20]

Hillel Wayne. Hypermodeling hyperproperties, Jan 2020.

[WJW+17]

[WNBP21]

[WNP20]

[WZBP19]

Leonore Winterer, Sebastian Junges, Ralf Wimmer, Nils Jansen, Ufuk Topcu,
Joost-Pieter Katoen, and Bernd Becker. Motion planning under partial ob-
servability using game-based abstraction. In 2017 IEEE 56th Annual Con-
ference on Decision and Control (CDC), pages 2201–2208, 2017.

Yu Wang, Siddhartha Nalluri, Borzoo Bonakdarpour, and Miroslav Pajic.
Statistical model checking for hyperproperties. In IEEE Computer Security
Foundations Symposium, pages 1–16, Dubrovnik, Croatia, 2021.

Yu Wang, Siddhartha Nalluri, and Miroslav Pajic. Hyperproperties for
robotics: Planning via hyperltl. In 2020 IEEE International Conference on
Robotics and Automation (ICRA), pages 8462–8468, 2020.

Yu Wang, Mojtaba Zarei, Borzoo Bonakdarpour, and Miroslav Pajic. Statis-
tical verification of hyperproperties for cyber-physical systems. ACM Trans-
actions in Embedded Computing Systems, 18(5):92–, 2019.

[WZBP20]

Y. Wang, M. Zarei, B. Bonakdarpour, and Miroslav Pajic. Probabilistic
conformance for cyber-physical systems. CoRR, abs/2008.01135, 2020.

[YMCS16]

Man-Ki Yoon, Sibin Mohan, Chien-Ying Chen, and Lui Sha. Taskshuffler:
A schedule randomization protocol for obfuscation against timing inference
attacks in real-time systems. In 2016 IEEE Real-Time and Embedded Tech-
nology and Applications Symposium (RTAS), pages 1–12, 2016.

187

[You05]

[ZC ´AB22]

[ZM03]

H˚akan LS Younes. Ymer: A statistical model checker. In Computer Aided
Verification: 17th International Conference, CAV 2005, Edinburgh, Scotland,
UK, July 6-10, 2005. Proceedings 17, pages 429–433. Springer, 2005.

Eshita Zaman, Gianfranco Ciardo, Erika ´Abrah´am, and Borzoo Bonakdar-
pour. Hyperpctl model checking by probabilistic decomposition.
In Mau-
rice H. ter Beek and Rosemary Monahan, editors, Integrated Formal Methods,
pages 209–226, Cham, 2022. Springer International Publishing.

Steve Zdancewic and Andrew C. Myers. Observational determinism for con-
current program security.
In 16th IEEE Computer Security Foundations
Workshop (CSFW-16 2003), 30 June - 2 July 2003, Pacific Grove, CA, USA,
page 29. IEEE Computer Society, 2003.

[Zul15]

Paolo Zuliani. Statistical model checking for biological applications. Interna-
tional Journal on Software Tools for Technology Transfer, 17:527–536, 2015.

188