Human-in-the-loop systems play a pivotal role in numerous safety-critical applications, ensuring both safety and efficiency in complex operational environments. However, these systems face a significant challenge stemming from the inherent variability in human performance, influenced by factors such as workload, fatigue, task learning, expertise, and individual differences. Therefore, effective management of human cognitive resources is paramount in designing efficient human-in-the-loop systems.To address this challenge, it is critical to design robust and adaptive systems capable of continuously adapting models of human performance, and subsequently providing tailored feedback to enhance it. Effective feedback mechanisms play a pivotal role in improving the overall system performance by optimizing human workload, fostering skill development, and facilitating efficient collaboration among individuals within diverse human teams, each with their unique skill sets and expertise.In this dissertation, the primary focus lies in exploring optimal and game-theoretic approaches for feedback design to enhance system performance, particularly in scenarios where humans are integral components. We begin by studying the problem of optimal fidelity selection for a human operator servicing a stream of homogeneous tasks, where fidelity refers to the degree of exactness and precision while servicing the task. Initially, we assume a known human service time distribution model, later relaxing this assumption. We design a human decision support system that recommends optimal fidelity levels based on the operator's cognitive state and queue length. We evaluate our methods through human experiments involving participants searching for underwater mines.We extend the optimal fidelity selection problem by incorporating uncertainty into the human service-time distribution. This extension involves the development of a robust and adaptive framework that accurately learns the human service-time model and adapts the policy while ensuring robustness under model uncertainty. However, a major challenge in designing adaptive and robust systems arises from the conflicting objectives of exploration and robustness. To mitigate system uncertainty, an agent must explore high-uncertainty state space regions, while robust policy optimization seeks to avoid these regions due to poor worst-case performance. To address this trade-off, we introduce an efficient Deterministic Sequencing of Exploration and Exploitation (DSEE) algorithm for model-based reinforcement learning. DSEE interleaves exploration and exploitation epochs with increasing lengths, resulting in sub-linear cumulative regret growth over time.In addition to cognitive resource management, enhancing human performance can also be achieved through tutoring for skill development. In this context, we study the impact of evaluative feedback on human learning in sequential decision-making tasks. We conduct experiments on Amazon Mechanical Turk, where participants engage with the Tower of Hanoi puzzle and receive AI-generated feedback during their problem-solving. We examine how this feedback influences their learning and skill transfer to related tasks. Additionally, we explore computational models to gain insights into how individuals integrate evaluative feedback into their decision-making processes.Lastly, we expand our focus from a single human operator to a team of heterogeneous agents, each with diverse skill sets and expertise. Within this context, we delve into the challenge of achieving efficient collaboration among heterogeneous team members to enhance overall system performance. Our approach leverages a game theoretic framework, where we design utility functions to incentivize decentralized collaboration among these agents.
Read
