PERFORMANCE EVALUATION OF HVAC-CONNECTED OCCUPANCY SENSOR SYSTEMS By Yiyi Chu A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Civil Engineering—Doctor of Philosophy 2022 ABSTRACT PERFORMANCE EVALUATION OF HVAC-CONNECTED OCCUPANCY SENSOR SYSTEMS By Yiyi Chu Occupancy is highly unpredictable and depends on occupants’ schedules and their interactions with building systems. Occupancy sensor systems have been deployed in buildings for many years, and many research studies have been conducted that use a range of sensor modalities for occupancy sensing and counting. However, no comprehensive review of occupancy sensor system reliability has been compiled. In addition, there is not currently a universal methodology and metrics to evaluate and report occupancy sensor systems’ reliability. There have also been increasing studies on the implementation of occupant-based controls, especially energy savings evaluation in typical office building models. However, although there are many universities throughout the U.S., prototypical academic building models do not currently exist for use in evaluating energy saving potential. To address these research gaps, in this research, a review of the literature on occupancy sensor systems was completed to develop a comprehensive list of influential variables that may impact occupancy sensor system reliability. Next a survey was developed and distributed to a diversity of stakeholders to obtain a list of the most important factors that may influence occupancy sensor system performance. Then, a methodology was developed to assess the reliability of occupancy sensor systems in residential buildings in a controlled laboratory environment. This includes both “typical” testing, evaluating how reliable and accurate an occupancy sensor system is over time in a typical residential building environment, and “failure” testing, identifying individual influential variables that impact performance. The developed methodology was then implemented to evaluate a novel occupancy detection sensor system’s reliability. For typical testing, results show that on average, the overall accuracy of the tested sensor system ranged from 62.4% to 76.4%. For the failure testing, the number of occupants, presence of large objects, presence of interior light sources, and number of doors were identified as not influential, while lighting level, location of occupants, additional door in the entry/exit area, and having the TV on are variables determined to impact the sensor system performance. Furthermore, the U.S. DOE reference medium office building model was used as the basis to develop typical academic building models. The model was rezoned to add new spaces based on the space type and functional use data collected from 293 academic buildings across five U.S. universities of different sizes. Four types of typical academic building models were then identified using clustering methods. These include typical “Office-dominated”, “Laboratory-dominated”, “Study room-dominated”, and “Mixed-use” academic building models. Occupant-based controls were then added to the model to evaluate the potential energy savings of these developed models. Results show that among all these four typical academic building models, in ASHRAE Climate Zone 5, the total annual HVAC energy savings ranges from 35% to 51% under “Occupancy presence” scenarios, and a further energy saving increase (3-9%) from “Occupancy presence” scenarios to “Occupancy counting” scenarios. The proposed methodology for evaluating the reliability of occupancy sensor systems presents an opportunity for use as a standardized method to evaluate residential occupancy sensor systems that currently does not exist. This work also provides typical academic building models with integrated occupancy schedules which can be used to evaluate energy saving measures, and aid building designers and operators in making informed decisions in applying appropriate control strategies to optimize building energy systems, as well as predict energy use and demand. I dedicate this dissertation to my family and my beloved husband, Zhengyu for their constant support and unconditional love. I love you all dearly. iv ACKNOWLEDGEMENTS There are many who helped me along the way on this journey. I want to take a moment to thank them. Firstly, I would like to express my sincere gratitude to my advisor Dr. Kristen Cetin for her invaluable advice, dedicated support, and guidance during my PhD study. She continuously provided encouragement and was always willing and enthusiastic to assist in any way she could during the research project. Her insightful feedback pushed me to sharpen my thinking and brought my work to a higher level. I could not have imagined having a better advisor and mentor for my PhD study. Besides my advisor, I would like to thank the rest of my PhD committee, Professors Peter Savolainen, Joerg Petrasch, and Weiyi Lu for providing valuable comments and suggestions in general. My gratitude extends to Professors Nizar Lajnef from Michigan State University, Senem Velipasalar from Syracuse University, Gregor Henze from University of Colorado Boulder, Joshua R. Smith from University of Washington, Soumik Sarkar from Iowa State University, Janusz L. Konrad from Boston University, and Robert Karlicek from Rensselaer Polytechnic Institute for their assistance and collaboration at every stage of the research project. My sincere thanks also go to my colleagues and friends, Debrudra Mitra, Soham Vanage, and Hao Dong for providing valuable suggestions and support. Special thanks to Debrudra Mitra for all his patient help in the experimental lab and field work through my PhD work. A special thanks to my colleague Ryan Gallagher for his timely assistance in providing technical and logistic support in lab and field work at MSU. Additionally, I would like to thank my colleagues, Maria Milan, Emily Kawka, Sam Evans, Matthew Roberts, Don Madhawajayaweera, Zachary Ruddick, v Brady Berg, Ceren Aydin, Mehdi Bulduk, Celso Santos, Javed Miandad, Wasif Naqvi, Md Fyaz Sadiq, Marzia Ashraf, Oyendrila Dobe, Tanner Herald, Hamed Bolandi, Talal Salem, Xuyang Li, and Peng Chen for providing help in the lab work. I would also like to thank my teammates, Burak Kakillioglu from Syracuse University, Mohamad Katanbaf and Ali Saffari from UW, Sin Yong Tan from ISU, Hao Lu from RPI, Mertcan Cokbas from BU, for their patience in collaboration at every stage of the research project. And companion from my colleagues and friends, Arna Ganguly, Ritam Ganguly, Jin Yan, Yanhuang Hua, Mengying Zhang, Fan Zhou, Weizhuo Shi, Chao Wang, Chi Zhan, Jing Zheng, Zheng Li, and Conniee Fan. Special thanks to the financial support from the Advanced Research Projects Agency‐ Energy (ARPA‐E) (DE‐AR0001256 and DE-AE0001316), U.S. Department of Energy. I am also grateful for the Graduate Grant-In-Aid awarded by ASHRAE. I also want to say thank you to IIBEC for the sponsorship of attending IIBEC International Convention & Trade Show. I would also like to thank for the fellowships and travel funds provided by the Department of Civil and Environmental Engineering, the College of Engineering and the Graduate School at Michigan State University. Finally, I would like to thank my parents and grandparents for their caring and unconditional love. I would not have made it this far without them. Special thanks to my beloved husband, Zhengyu, and my parents-in-law who all have been supportive and caring. Zhengyu has been a true and great supporter and has unconditionally loved me during my good and bad times. He has also provided invaluable suggestions throughout this journey and my career path. These past several years have not been an easy ride, both academically and personally. I truly thank Zhengyu for always standing by my side, even when I was irritable and depressed. vi TABLE OF CONTENTS LIST OF TABLES .......................................................................................................................... x LIST OF FIGURES ..................................................................................................................... xiii CHAPTER 1 – INTRODUCTION ................................................................................................. 1 1.1 Research Motivation .......................................................................................................... 1 1.2 Research Objectives and Research Questions ................................................................. 10 1.2.1 Objective 1: Identify the most important variables that impact the reliability of occupancy sensor systems .............................................................................................. 11 1.2.2 Objective 2: Develop a standard method to evaluate the reliability of several state- of-art occupancy sensor systems .................................................................................... 12 1.2.3 Objective 3: Typical academic building energy model development and energy saving evaluation ............................................................................................................ 13 1.3 Research Organization ..................................................................................................... 15 REFERENCES ............................................................................................................................. 17 CHAPTER 2 – INFLUENTIAL VARIABLES IMPACTING THE RELIABILITY OF BUILDING OCCUPANCY SENSOR SYSTEMS: A SYSTEMATIC REVIEW AND EXPERT SURVEY....................................................................................................................................... 22 2.1 Abstract ............................................................................................................................ 22 2.2 Introduction...................................................................................................................... 22 2.3 Literature Review ............................................................................................................ 27 2.3.1 Radio frequency-based sensors ............................................................................. 31 2.3.2 Sound wave-based sensors .................................................................................... 39 2.3.3 Infrared-based sensors ........................................................................................... 42 2.3.4 Vision-based sensors ............................................................................................. 43 2.3.5 Sensor fusion ......................................................................................................... 45 2.4 Expert Survey on Most and Least Important Influential Variables ................................. 49 2.5 Survey Results ................................................................................................................. 50 2.5.1 Most important and least important variables by category ................................... 51 2.5.2 Most important variables across all variable categories ........................................ 56 2.6 Conclusions and Future Work ......................................................................................... 59 APPENDIX ................................................................................................................................... 62 REFERENCES ............................................................................................................................. 66 CHAPTER 3 – DEVELOPMENT AND TESTING OF A PERFORMANCE EVALUATION METHODOLOGY TO ASSESS THE RELIABILITY OF OCCUPANCY SENSOR SYSTEMS IN RESIDENTIAL BUILDINGS ................................................................................................. 77 3.1 Abstract ............................................................................................................................ 77 3.2 Introduction...................................................................................................................... 78 3.3 Evaluation Methodology for Sensor System Reliability ................................................. 80 3.3.1 Typical testing ....................................................................................................... 81 3.3.2 Failure testing ........................................................................................................ 86 3.3.3 Ground truth .......................................................................................................... 88 vii 3.3.4 Laboratory Setup ................................................................................................... 88 3.3.5 Performance metrics .............................................................................................. 89 3.4 Experimental Results: Case Study ................................................................................... 91 3.4.1 Sensor System Description .................................................................................... 91 3.4.2 Test bed ................................................................................................................. 91 3.4.3 Data collection ....................................................................................................... 94 3.5 Results.............................................................................................................................. 95 3.5.1 Typical testing results ............................................................................................ 95 3.5.2 Failure testing results ........................................................................................... 100 3.5.2.1 Non-influential individual variables.......................................................... 101 3.5.2.2 Influential individual variables.................................................................. 103 3.6 Conclusions.................................................................................................................... 109 APPENDICES ............................................................................................................................ 111 APPENDIX A: Typical occupancy schedules and activity profiles .................................... 112 APPENDIX B: Typical testing results with associated typical occupancy schedules and activity profiles .................................................................................................................... 116 APPENDIX C: Supplemental variable testing results ......................................................... 120 REFERENCES ........................................................................................................................... 121 CHAPTER 4 – TYPICAL ACADEMIC BUILDING ENERGY MODEL DEVELOPMENT AND ENERGY SAVING EVALUATION OF OCCUPANT-BASED CONTROL................. 124 4.1 Abstract .......................................................................................................................... 124 4.2 Introduction.................................................................................................................... 125 4.3 Methodology .................................................................................................................. 128 4.3.1 Typical space types and distribution in academic buildings ............................... 128 4.3.2 Typical academic building models ...................................................................... 134 4.3.3 Typical occupancy schedule in academic buildings ............................................ 140 4.3.4 Occupant-based control Strategies ...................................................................... 143 4.3.4.1 Temperature setback ................................................................................. 143 4.3.4.2 Ventilation setback .................................................................................... 144 4.4 Results............................................................................................................................ 146 4.4.1 Baseline energy consumption comparison .......................................................... 146 4.4.2 Occupant-based control – Ventilation Setback ................................................... 147 4.4.2.1 Zone-level ventilation setback .................................................................. 147 4.4.2.2 System-level ventilation setback ............................................................... 150 4.4.3 Energy savings..................................................................................................... 151 4.5 Conclusions.................................................................................................................... 153 APPENDICES ............................................................................................................................ 156 APPENDIX A: Space types of assignment for typical academic building model .............. 157 APPENDIX B: Detailed floor plans and whole building geometry for typical academic building model ..................................................................................................................... 160 REFERENCES ........................................................................................................................... 163 CHAPTER 5 – CONCLUSIONS AND FUTURE WORK ........................................................ 166 5.1 Conclusions and Contributions ...................................................................................... 166 5.1.1 Influential variables impacting the reliability of building occupancy sensor systems ......................................................................................................................... 166 viii 5.1.2 A standard methodology to test the reliability of various occupancy sensor systems ......................................................................................................................... 168 5.1.3 Typical Academic Building Energy Model Development and Energy Saving Evaluation of Occupant-based Control ........................................................................ 170 5.2 Limitations ..................................................................................................................... 171 5.2.1 Influential variables impacting the reliability of building occupancy sensor systems ......................................................................................................................... 171 5.2.2 A standard methodology to test the reliability of various occupancy sensor systems ......................................................................................................................... 172 5.2.3 Typical Academic Building Energy Model Development and Energy Saving Evaluation of Occupant-based Control ........................................................................ 172 5.3 Future Work ................................................................................................................... 173 5.3.1 Sensor Reliability and Performance Improvement.............................................. 173 5.3.2 Standardized Performance metrics for reliability evaluation .............................. 173 5.3.3 Typical academic building models improvement................................................ 173 ix LIST OF TABLES Table 1 Most common types of occupancy sensor types used in single sensor and sensor fusion applications in residential and commercial buildings ........................................................ 30 Table 2 Influential variables evaluated in recent literature using Doppler sensors for occupancy sensing ........................................................................................................................ 33 Table 3 Influential variables evaluated in recent literature using RFID sensor systems for occupancy sensing ........................................................................................................................ 35 Table 4 Influential variables evaluated in recent literature using Wi-Fi sensor systems for occupancy sensing ........................................................................................................................ 37 Table 5 Influential variables evaluated in recent literature using Bluetooth/BLE-based sensor systems for occupancy sensing .......................................................................................... 39 Table 6 Influential variables evaluated in recent literature using sound wave-based sensor systems for occupancy sensing ..................................................................................................... 41 Table 7 Influential variables evaluated in recent literature using infrared-based sensor systems for occupancy sensing ..................................................................................................... 43 Table 8 Influential variables evaluated in recent literature using vision-based sensor systems for occupancy sensing ..................................................................................................... 45 Table 9 Influential variables evaluated in recent literature using sensor fusion systems for occupancy sensing ........................................................................................................................ 47 Table 10 Compiled list of influential variables from literature review and expert stakeholder feedback..................................................................................................................... 48 Table 11 Most and least important variables for residential buildings, and those never chosen as most/least important ..................................................................................................... 54 Table 12 Most and least important variables for commercial buildings, and those never mentioned as most/least important................................................................................................ 55 Table 13 Most important variables for residential and commercial buildings ............................. 59 Table 14 Five typical household types and associated occupancy profiles .................................. 82 Table 15 1-person household with day work profile .................................................................... 83 Table 16 Typical testing results with “1-person household with day absence profile” using “Test Duration” ................................................................................................................... 96 x Table 17 Performance evaluation metric calculation for typical testing results analysis ............. 99 Table 18 Tested variables and associated testing levels ............................................................. 100 Table 19 Testing scenarios and results of non-influential individual variables.......................... 102 Table 20 Testing scenarios and results of “Lighting level” and “Motion level” variables......... 104 Table 21 Testing scenarios and results of “Location of occupants”, “Use of robots”, and “presence of pets” variables ........................................................................................................ 105 Table 22 Testing scenarios and results of additional variables, including “Presence of another door in Entry/Exit area”, and “TV On” ......................................................................... 108 Table 23 Sche2: 1-person household with night absence profile ............................................... 112 Table 24 Sche3: 1-person household with stay home profile ..................................................... 113 Table 25 Sche4: 2-person household where both with day absence profile ............................... 114 Table 26 Sche5: 2-person household where one with day absence profile and the other with stay home profile ................................................................................................................ 115 Table 27 Typical testing results with “1-person household with night absence profile” ........... 116 Table 28 Typical testing results with “1 person with stay home profile” ................................... 117 Table 29 Typical testing results with “2-person where both with day absence profile” ............ 118 Table 30 Typical testing results with “2-person where 1 with day work profile and other one with stay home profile” ........................................................................................................ 119 Table 31 Supplemental testing results for individual variable -- “Presence of robot” with the initial Occupied/Unoccupied state of the test space.............................................................. 120 Table 32 Colleges and universities, by CCIHE size category and associated characteristics, from which, building and space use data were utilized ..................................... 129 Table 33 Typical academic building types based on K-means clustering .................................. 131 Table 34 Mean percent (by area) of functional space use type in the four defined typical academic buildings...................................................................................................................... 133 Table 35 Mean of FICM space types as a percent (by area) of total available space in the four defined typical academic buildings ..................................................................................... 133 Table 36 Space types under each category and space area distribution in each typical academic building type ............................................................................................................... 136 xi Table 37 Space types of assignment on each floor for the Office-dominated typical academic building model ............................................................................................................ 138 Table 38 Assumptions of space types used in LBNL Occupancy Simulator and associated occupancy density from ASHRAE Standard 62.1-2019............................................................. 142 Table 39 Occupant-based control strategies for typical academic building models (Pang et al., 2020) ................................................................................................................................. 145 Table 40 Space types assignment on each floor for the laboratory-dominated typical academic building model ............................................................................................................ 157 Table 41 Space types assignment on each floor for the study room-dominated typical academic building model ............................................................................................................ 158 Table 42 Space types assignment on each floor for the mixed-use typical academic building model ............................................................................................................................ 159 Table 43 Most important variables for residential and commercial buildings ........................... 167 xii LIST OF FIGURES Figure 1 Summary of the most commonly used occupancy sensors .............................................. 4 Figure 2 Diagram of challenges associated with the reliability and energy savings evaluation of occupancy sensor systems....................................................................................... 10 Figure 3 Schematic diagram of research objectives ..................................................................... 11 Figure 4 Research Organization by Published and Submitted Papers (Note: (Chu et al. 2021, 2022) are references for these papers) ............................................................................... 16 Figure 5 Frequency of occupancy sensor modalities used in recent literature from 2003 to 2020, for occupancy detection and counting, including (a) types of single sensor system modalities and sensor fusion, and (b) types of single sensor system modalities, with sensor fusion subdivided into single sensor system modalities with other environmental sensors ........................................................................................................................................... 29 Figure 6 (a) Count value (Method 1) and (b) weightage value (Method 2) for most important influential variables for residential building applications. (Note: Red columns indicate where more than half of responses include this variable for Method 1) ........................ 52 Figure 7 (a) Count value (Method 1) and (b) weightage value (Method 2) for most important influential variables for commercial building applications. (Note: Red columns indicate where more than half of responses include this variable for Method 1) ........................ 53 Figure 8 Weightage value for variables considered as the most important in (a) residential and (b) commercial buildings based on the final survey question ................................................ 57 Figure 9 Average ranking and standard deviation of variables considered as the most important in (a) residential buildings and (b) commercial buildings based on the final survey question ............................................................................................................................. 58 Figure 10 Occupancy profile and location data ............................................................................ 84 Figure 11 Residential laboratory used for testing of occupancy sensor systems .......................... 93 Figure 12 Functional space use in the four defined typical academic buildings, including the percent of building area used for classroom, laboratory, office, and study areas ................. 132 Figure 13 Floor plans and building geometry for office-dominated typical academic building model, including (a) 1st floor; (b) 2nd floor; (c) 3rd floor; (d) whole building geometry ..................................................................................................................................... 139 Figure 14 The example daily occupancy profiles of (a) Classroom, (b) Office, (c) Meeting room and (d) Others, for typical academic building model on both weekday and weekend ..... 143 xiii Figure 15 Baseline annual energy consumption comparison among four types of typical academic building models in ASHRAE Climate Zone 5A (Chicago) ........................................ 147 Figure 16 Zone minimum primary air flow rate (m3/s) in example spaces: (a) Office -- Perimeter_bot_GA_OF_1; (b) Classroom -- Perimeter_bot_Classroom_1; (c) Meeting room -- Perimeter_bot_Meeting_RM; (d) Others -- FirstFloor_Corridor_1. (Note: B = Baseline; P = Occupancy presence sensing; C = Occupancy counting sensing) ...................... 148 Figure 17 Zone outdoor air flow rate (m3/s) in example spaces: (a) Office -- Perimeter_bot_GA_OF_1; (b) Classroom -- Perimeter_bot_Classroom_1; (c) Meeting room -- Perimeter_bot_Meeting_RM; (d) Others -- FirstFloor_Corridor_1 ............................. 149 Figure 18 System outdoor air flow rate (kg/s) in example VAV_1 system of (a) Typical Office-dominated; (b) Typical Laboratory-dominated; (c) Typical Study room-dominated; and (d) Typical Mixed-use building models ............................................................................... 151 Figure 19 Energy saving achieved through Occupant-based Control for typical academic building as compared to the baseline buildings with no OBC in ASHRAE Climate Zone 5A (Note: lighter colors indicate “Occupancy presence” scenarios; darker colors indicate “Occupancy counting” scenarios; _P = occupancy presence scenarios; _C = Occupancy counting scenarios; OD=office-dominated; LD=laboratory-dominated; SD=study room- dominated; M=mixed-use) .......................................................................................................... 153 Figure 20 Detailed floor plans and whole building geometry for laboratory-dominated typical academic building model, including (a) 1st floor plan; (b) 2nd floor plan; (c) 3rd floor plan ..................................................................................................................................... 160 Figure 21 Detailed floor plans and whole building geometry for study room-dominated typical academic building model, including (a) 1st floor plan; (b) 2nd floor plan; (c) 3rd floor plan ..................................................................................................................................... 161 Figure 22 Detailed floor plans and whole building geometry for mixed-use typical academic building model, including (a) 1st floor plan; (b) 2nd floor plan; (c) 3rd floor plan....... 162 xiv CHAPTER 1 – INTRODUCTION 1.1 Research Motivation Energy consumption throughout the world has significantly increased over the past 20 years, as the population has increased, as well as the use of energy-consuming technologies. Building energy use accounts for approximately 40% of the total primary energy consumption in the U.S. and the E.U., and 27.3% in China (Cao et al., 2016). For end uses in buildings located these regions, space heating and cooling energy make up a large portion of total energy consumption in the residential (46%) and commercial (36%) sectors (Cao et al., 2016). Moving forward, it is projected that total energy demand from heating and cooling will increase in cooling- dominant and less-developed regions (Cao et al., 2016). Therefore, to reduce emissions and energy use overall, in order to reduce climate change impacts and improve grid operations, it is important to develop energy saving approaches to help reduce heating and cooling energy use in buildings. The energy consumption of buildings and their systems are primarily dependent on outdoor weather conditions, building and system characteristics, and the occupants who utilize them. Data on these parameters is necessary to monitor performance, assess inefficiencies, and identify opportunities for improvements. Among these, weather data is generally ubiquitous and easier to obtain and/or collect using local or regional ground-based weather station or satellite-derived data. Many studies have used such data to demonstrate the relationship between building energy consumption and climate parameters (e.g., Hadley 1993), with outdoor temperature typically being most important. However, even though this information is generally readily available, weather is not a variable that can be adjusted to improve building performance. For individual buildings, data on building systems and their characteristics are relatively available, including assessors and other 1 publicly available information, energy code requirements, and data from the building owners/operators. There are many recent studies focused on the use of this information to assess and improve building performance. For example, for building envelope characteristics, Menyhart and Krarti (2017) applied Dynamic Insulation Materials (DIMs) to replace the traditional, static insulation and found that DIMs can achieve heating and cooling energy savings ranging from 7% to 42%. For energy-consuming building systems, including the heating, ventilation, and air conditioning (HVAC) system, appliances, equipment, and other miscellaneous loads, a broad range of studies have focused on assessing the impact of these systems on energy performance have also been investigated. For example, Anand et al. (2019) investigated the relationship of occupancy with plug and lighting loads energy consumption for several spaces of an institutional building floor, and deep neural network was used to estimate possible energy savings with a rule- based energy-use behavior. Energy efficiency standards for residential and commercial equipment have also been found to be a major source of energy saving in U.S. (U.S. DOE, 2018). Data, however, on the third above-mentioned influential factor, occupants, is more challenging to collect and less readily available as compared to the other two. Buildings are strongly influenced by the occupants that use them and their energy-consuming behaviors (Hong et al., 2016). There are significant opportunities to better collect and use occupancy related data to inform building controls, to target energy use reductions and efficiency improvements. When there are less occupants, ventilation requirements are lower (ASHRAE 62.1, 2019); when there are no occupants, thermal comfort requirements no longer need to be met (ASHRAE 55, 2017). Therefore, occupancy sensing technologies used in buildings can help to significantly reduce HVAC loads (Jung et al., 2019). 2 To collect accurate occupancy information, such as occupant presence, counting, location, and tracking (Teixeira et al., 2010), occupancy recognition technologies have been developed and studied in the recent decades. Existing technologies for occupancy information extraction in buildings can generally be classified into two categories: single sensor systems and sensor fusion technology. Single sensor systems utilize one or multiple sensors with the same type to infer occupancy information, while sensor fusion methods combine more than one type of occupancy sensors to compensate the disadvantages of each type of sensor systems. Many of the single sensor technologies have been used to predict the presence and counting of occupants in buildings. Based on a review of 26 recent papers on occupancy sensor systems, the most common types (see Figure 1), are PIR (passive infrared) (Liao et al., 2010), and CO2 (Jiang et al., 2016) sensors, followed by RFID (Liao et al., 2012) and Wi-Fi-based sensors (Zou et al., 2017). Another important, but less commonly used sensor technology is video (Erickson et al., 2009), which is often used to collect ground truth data for occupancy studies. The benefits of the use of single sensor systems is the simplicity of the sensor deployment and data collection, however, edge cases could happen when only one type of sensor is used, for example, the motion sensor could not detect people when a person is static, causing false positive, where sensor fusion technologies could help add more capabilities. 3 16 14 12 # of Papers 10 Detection (7) Counting (19) 8 6 4 2 0 Figure 1 Summary of the most commonly used occupancy sensors Due to the emergence of machine learning and the big data industry, as well as the significant reductions in cost and size of equipment needed to store data and run computational algorithms, data-driven models have been increasingly used in research to help improve the prediction accuracy of various technologies. These are of particular importance and benefit to the second type of occupancy sensor system - sensor fusion technology. Recent studies using sensor fusion (e.g., Yang et al., 2014; Ryu et al., 2016) have developed low-cost, non-intrusive occupancy sensors systems to predict occupancy in buildings. The data collected from different sensors is pre- processed and fed into a training model. Next, the resulting model is used to predict occupancy (presence or count) based on the provided inputs. In general, the more inputs and diverse data collected as input into the model, the more accurate the models become (Yang et al., 2018). The challenge with occupants and occupancy data, however, is that occupants are inherently unpredictable, varying both from person to person, and from building type to building type. Factors such as interior building layout and geometry, lighting levels and other physical parameters, depending on the type of sensor(s) used, can significantly impact the ability of a 4 system to detect occupants (Gennarelli et al., 2016; Labeodan et al., 2013; Wang et al., 2017; Yun et al., 2014). This impact varies depending on the type of sensor system used and its corresponding algorithms. Similarly, occupant characteristics, including their physical features and movements, their preferences, and their occupant-building interactions with energy-consuming appliances and systems in buildings, make the accurate detection of occupants challenging due to limited predictability (Yoshino et al., 2017; Yan et al., 2017). The relative importance of these occupant characteristics, similar to physical building characteristics, is also dependent on the sensor system considered. Among the recent literature, while some studies have considered multiple different factors in evaluating their influence on occupancy sensor system performance, a comprehensive survey gauging of the relative importance of these diverse factors on the performance of occupancy sensor systems has not been conducted. In addition, in considering such factors can vary substantially by building type and occupant characteristics, while both single sensor and sensor fusion methods have improved significantly in recent years, these technologies are not 100% accurate. The methodology used to evaluate performance, and the metric used to report it are not uniform across the literature. Sensors can fail under various scenarios and edge cases. For occupancy presence sensors, this can be either as a false positive reading (i.e., the sensor system indicates there is an occupant when there is not), or false negative (i.e., the sensor system does not register an occupant when there is one present). For occupancy counting, the failure would be to incorrectly count the number of occupants in a space. A false positive would result in systems being on (e.g., HVAC, lighting) that do not need to be on since no occupants are present; a false negative would result in a potentially thermally uncomfortable (HVAC) or dark space (lighting). For occupancy counting, an incorrect reading of the number of occupants would result in ventilation rates that could be too high or low for the 5 number of occupants present. In summary, occupancy, while challenging to detect with as strong an accuracy as weather and building characteristics, is of significant importance if the goal of a building is to be both energy efficient and comfortable for the occupants that use it. However, without an established method to evaluate the performance of these sensor systems, it is not possible to objectively evaluate and compare the performance of multiple sensor systems. Currently there is no standard method to evaluate the reliability of occupancy sensors. To address these two challenges, the first focus area of this research is on the development of a universal evaluation methodology to test reliability of occupancy sensor systems in residential buildings. Beyond reliability of these sensor systems, among the most important features is their function to support energy savings in buildings. Currently, occupancy sensors detecting the presence/non-presence of people in a space are often used in commercial and in some cases residential spaces to determine whether to turn on/off the lights. In the 2018 International Energy Conservation Code, for some types of buildings occupancy sensors connected to lighting are required (IECC 2018). These sensors are often directly interfaced and integrated with the lighting systems. However, in this research, lighting-connected occupancy sensors are not the focus as this is a more widely used application with significant research. More recently, occupancy sensors, including both occupancy presence and occupancy counting sensors are being considered for HVAC control applications. In residential buildings, smart thermostats with built-in occupancy sensors have been increasing in use (Moon et al., 2011). These thermostats generally include a “home” and “away” mode, depending on whether the sensor detects movement or not. In “away” mode, occupancy is considered to be gone from the home and thus the setpoint temperature is adjusted for the single-zone HVAC system to unoccupied mode, to reduce consumption. Many thermostats, however, do not yet have these occupancy sensors integrated into them. There are 6 various energy saving evaluation methods conducted using modeling, or laboratory/field testing methods related to occupancy information. For example, Jain et al. (2013) examined the impact that information representation has on energy consumption behavior by conducting a one-month empirical study with 39 participants in an urban residential building. Wang et al. (2020) explored a co-simulation platform to assess energy saving impact and economic benefits of occupancy driven thermostats in a residential building. EnergyPlus was integrated into the co-simulation platform to evaluate energy consumption and indoor air temperatures. Qin and Pan (2020) analyzed the energy use of high-rise residential buildings in subtropical climate and examined the impacts of different energy saving measures for developing strategies for achieving very low- energy high-rise buildings by using EnergyPlus software. In commercial buildings, HVAC systems are generally more complex than residential buildings since there are typically multiple thermal zones with a variety of space types. In addition, modern commercial buildings require mechanical ventilation per energy code requirements, thus outdoor air is required to maintain acceptable levels of indoor air quality. The energy consumption associated with ventilation accounts for approximately 50% of the total HVAC energy use in commercial buildings in U.S. (CBECS, 2012). Based on ASHRAE Standard 62.1-2019 (ASHRAE 2019), the outdoor air flow depends on the number of occupants in commercial buildings, thus adjusting outdoor air intake based on real-time number of occupants could significantly improve energy saving. In addition, GPC 36 (High Performance Sequences of Operation for HVAC Systems) within ASHRAE (ASHRAE, 2018) in recent years has also included consideration for occupancy sensing technologies. Given these recent developments in state-of-the art controls, including GPC there is a need to evaluate the level of energy savings that can be achieved in commercial buildings using such controls. 7 There are two main methods which can be used to evaluate energy savings, including laboratory/field testing, and energy simulation. For energy modeling, while some recent efforts within the past year have considered office building types (i.e., Pang et al., 2020), none have considered academic buildings, which is a primary building type of interest to ARPA-E/DOE and this research (ARPA-E SENSOR FOA, 2017). In academic buildings there are significant transient populations, thus there is a significant opportunity for energy savings from occupancy sensor systems. However, currently there is no established prototypical building model for academic building types which is able to be used to evaluate energy savings from occupancy sensors. The U.S. Department of Energy Prototype (U.S. DOE, 2018) and Reference (Deru et al., 2011) building models were created, in part, to represent the most common types of buildings and their associated energy-impacting characteristics. These can be used with the energy simulation engine, EnergyPlus (Crawley et al., 2000), to assess the building energy performance of such typical building types in different climate zones, for different age buildings (Deru et al., 2011) and/or buildings built to different energy codes (U.S. DOE, 2018). Despite the substantial effort made to validate these and other building energy models, there is, however, typically a discrepancy between the predicted building energy use, and the energy used during actual operation. This is due in part to the unpredictable nature of occupants and their energy-related impacts. Various related studies have been conducted to improve the occupant-related energy efficiency in buildings. In particular, occupant-based control has been the subject of increasing research interest in recent years. Zhang et al. (2018) summarized that the energy saving potential from the use of occupant- based controls is 10%-25% for residential buildings, and 5%-30% for commercial buildings. However, the occupant-based control used in existing studies varies substantially from the perspective of occupant sensing, ranging from simple presence-based switching of lighting 8 systems to full model predictive control. Energy savings estimates vary substantially for these technologies as well (Naylor et al., 2018). Commercial buildings can be classified into 15 different types according to their principal activity, as outlined in the CBECS (Commercial Buildings Energy Consumption Survey) dataset (Michaels and Leckey, 2012). Based on these, Commercial Reference building models were developed (DOE Commercial Reference Buildings) to represent nearly 70% of the commercial buildings in the U.S. (Deru et al., 2011), including offices, schools, restaurants, hotels, etc. The reference office building models have been widely used in energy simulation efforts in research projects to assess the potential energy savings of typical office buildings in U.S. Buildings used for educational purposes are among the subgroups of commercial buildings in the U.S. that consume a large amount of HVAC energy (U.S. EIA, 2012). Academic buildings at universities are included among these and can vary substantially from typical office buildings in their space use, occupancy patterns, and energy use. However, despite substantial energy use, there are few studies which have considered occupant-based control in academic buildings. In summary, the second portion of this research focuses on developing typical academic building models and assessing the influence of occupant-based control strategies on energy efficiency improvement in the developed typical academic building models. Figure 2 summarizes the two focus areas of this research and the associated above-mentioned challenges. 9 Figure 2 Diagram of challenges associated with the reliability and energy savings evaluation of occupancy sensor systems 1.2 Research Objectives and Research Questions Based on the above-described challenges, the overarching purpose of this research is to evaluate the reliability of occupancy sensor systems and HVAC energy saving from occupancy sensor systems in residential and commercial buildings. More specifically, the scope of this research includes the development of a systematic test procedure to access the reliability of sensor systems based on a stakeholder consensus on the most important variables to evaluate, development and use of a methodology to evaluate laboratory energy saving from occupancy sensors in buildings, and development of typical academic building models and HVAC energy saving evaluation using occupant-based control. Figure 3 represents the relationship of these aspects of this research and how they relate to the overarching purpose. The details of each objective are described below, where each objective aims to address several major research questions. 10 Figure 3 Schematic diagram of research objectives To develop a systematic testing protocol to evaluate the reliability of the occupancy sensor systems using experimental method, and provide feedback to manufacturers and researchers, this research addresses two major issues identified as Objectives 1 and 2. For the evaluation of energy savings of HVAC systems using occupant-based control, including both the development of typical academic building models, and the implementation of occupancy-based controls for developed typical academic building models, this research focuses on two major and related issues designated as Objective 3a and 3b. 1.2.1 Objective 1: Identify the most important variables that impact the reliability of occupancy sensor systems Occupancy sensor systems are sensitive to different types of variables based on their operational characteristics and types of sensors used. If sensitive to variations in such variables, 11 this would limit an occupancy sensor system’s ability to accurately determine the occupancy of a building. Thus specifically, this objective aims to answer the following research questions: a) For each occupancy sensor/sensor system type, what are the influential variables that are likely to cause failures? b) What is the relative impact of variations in these variables on different occupancy sensors/sensor system types? c) What do diverse stakeholders agree are the most and least important variables for use in a standard methodology of evaluation of occupancy sensor system reliability? d) Are these most important variables the same for residential and commercial building settings? In this research, the comprehensive evaluation of potential influential variables that may cause sensor failures are summarized based on literature review, the results from a workshop with the occupancy sensor stakeholders, and the results of a survey of diverse stakeholders on the most and least important variables that affect occupancy sensor performance for commercial and residential building applications. 1.2.2 Objective 2: Develop a standard method to evaluate the reliability of several state-of-art occupancy sensor systems This objective focuses on the development of a testing protocol to evaluate the robustness of sensor systems in a range of “typical” and “challenging” scenarios, in order to identify areas in need of improvement for the system being tested. The following questions will be addressed in this objective related to the reliability evaluation: 12 a) Should a standard set of variables be used to test all types of occupancy sensor systems, or should variations in different variables be evaluated, depending on the sensor system type? b) What experimental methodology should be used to test the identified variables from Objective 1? c) What evaluation methods should be used to analyze the experimental results? d) What reliability evaluation metric should be used for the occupancy sensor systems? In this research, typical testing and failure testing are defined based on the results of research conducted for Objective 1 which identify the potential sources of issues and/or failures for different sensors. Typical testing is defined as the “typical” conditions that are expected to occur on an everyday basis in the residential or commercial setting. Failure testing is the “challenging” conditions that may be less common and/or more extreme than what might occur under a “typical” scenario (e.g., someone running quickly, very dark/bright lighting conditions, etc.), which are helpful in determining what (extreme) conditions will elicit a system failure. Several experimental design methods are considered, and one is chosen for the method development. These test methods are then used for laboratory testing of the reliability of occupancy sensor systems. 1.2.3 Objective 3: Typical academic building energy model development and energy saving evaluation This objective focuses on the development of a prototypical academic building energy model, which can be used to evaluate energy savings from the use of occupancy sensor system in academic buildings, using occupant-based control (OBC) strategies. Academic buildings are a combination of various space types with different functional uses based on the Postsecondary 13 Education Facilities Inventory and Classification Manual (FICM). The following questions will be addressed: a) What is the typical composition of space types for typical academic buildings? b) How many prototypical academic building models are needed to sufficiently represent typical academic buildings in the U.S.? c) What modifications should be made to existing prototypical building models in order for them to represent the features of typical academic buildings, and to allow for the use of occupancy-based controls? d) What are the potential energy savings for the developed typical academic buildings using occupancy sensor systems to control the HVAC system? To determine typical academic building models, data from different size universities was collected, as designated by the Carnegie Classification of Institutions of Higher Education (CCIHE). This system provides university size categories according to the FTE-based enrollment (FTE = Full-Time Headcount plus 1/3 part-time headcount). Iowa State University data, as well as data from the Texas Higher Education Data (THED) was used. THED provides, under the Facilities Inventory and Audit portal, detailed building, and relevant space information for all Texas-based campuses. This includes the predominant purpose and function of a building and corresponding spaces in that building, the functional use of each room in a building, following the FICM (Facilities Inventory and Classification Manual) code, and the size of each room, and other details. For these universities, FTE-based Enrollment data was obtained from the Integrated Postsecondary Education Data System (IPEDS) under the National Center for Education Statistics. Ultimately, based on this analysis, four universities are chosen from the THED system, in addition 14 to Iowa State University (ISU), to support a diversity of university sizes. To define typical scenarios of space use, the percentage of the area of each space type is used to define the typical academic buildings. A medium office DOE Reference building (U.S. DOE, 2020) was used as the basis of design, then conditioned zones were further divided into different types of rooms with corresponding sizes based on functional uses by using EnergyPlus and Sketchup. The resulting model is used to evaluate energy savings compared to a baseline without the use of OBC. 1.3 Research Organization This research is organized into one review paper, and several journal papers and associated conference papers (Figure 4). The citations for the papers indicated in this figure are listed in Section 1.4 (Chu et al. 2021, 2022). Chapter 2 includes a review of the literature on occupancy sensor systems focusing on influential variables that may cause systems to incorrectly represent occupancy and developed a survey to attain the most influential variables. Chapter 3 proposes a standard performance evaluation methodology to assess the reliability of occupancy sensor systems in residential buildings. Chapter 4 discuss a novel methodology of developing typical academic building models and potential energy savings evaluation based on occupant-based control. The conclusions, limitations, research contributions, and future work are described in Chapter 5. The details of each of the corresponding publications are including the following Chapters 2, 3 and 4. 15 Figure 4 Research Organization by Published and Submitted Papers (Note: (Chu et al. 2021, 2022) are references for these papers) 16 REFERENCES 17 REFERENCES AMERICAN SOCIETY OF HEATING, REFRIGERATING & AIR-CONDITIONING ENGINEERS. "ASHRAE Standard 55: thermal environmental conditions for human occupancy." (2017). AMERICAN SOCIETY OF HEATING, REFRIGERATING & AIR-CONDITIONING ENGINEERS. “ASHRAE 62.1: Ventilation and Acceptable Indoor Air Quality in Residential Buildings.” (2019). Anand, Prashant, David Cheong, Chandra Sekhar, Mattheos Santamouris, and Sekhar Kondepudi. "Energy saving estimation for plug and lighting load using occupancy analysis." Renewable Energy 143 (2019): 1143-1161. Cao, Dai, and Liu. "Building Energy-consumption Status Worldwide and the State-of-the-art Technologies for Zero-energy Buildings during the past Decade." Energy & Buildings 128 (2016): 198-213. Chu, Y., & Cetin, K. (2022). Sensing systems for smart building occupant-centric operation. The Rise of Smart Cities, 431-461. Chu, Y., Mitra, D., O’neill, Z., & Cetin, K. (2022). Influential variables impacting the reliability of building occupancy sensor systems: A systematic review and expert survey. Science and Technology for the Built Environment, 28(2), 200-220. Chu, Y., Mitra, D., & Cetin, K. (2021). Implementation of Occupant-based Control in Typical Academic Buildings. ASHRAE Transactions, 127(1). Chu, Y., Mitra, D., & Cetin, K. Measurement and Verification and Model Validation to Evaluate Energy and Demand Savings from Smart Building Technologies in a Residential, Controlled Laboratory. In Construction Research Congress 2022 (pp. 367-375). Crawley DB, Lawrie LK, Pedersen CO, Winkelmann FC. Energy plus: energy simulation program. ASHRAE journal. 2000 Apr 1;42(4):49-56. Deru, M., Field, K., Studer, D., Benne, K., Griffith, B., Torcellini, P., Liu, B., Halverson, M., Winiarski, D., Rosenberg, M., Yazdanian, M., Huang, J., Crawley, D. (2011). U.S. Department of Energy commercial reference building models of the national building stock. 1-118. DOE, U., 2018. Commercial Prototype Building Models. https://www.energycodes.gov/development/commercial/prototype_models (Accessed on November 24th, 2020). EIA, U., 2012. Commercial Building Energy Consumption Survey (CBECS). https://www.eia.gov/consumption/commercial/ (Accessed on November 24th, 2020). 18 Erickson V L, Lin Y, Kamthe A, et al. Energy efficient building environment control strategies using real-time occupancy measurements[C]. Proceedings of the First ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings. ACM, 2009: 19-24. Gennarelli G, Ludeno G, Soldovieri F. Real-time through-wall situation awareness using a microwave Doppler radar sensor. Remote Sensing. 2016 Aug;8(8):621. Guideline, A. S. H. R. A. E. (2015). 36: High Performance Sequences of Operation for HVAC systems. ASHRAE, Atlanta, GA. Hadley DL. Daily variations in HVAC system electrical energy consumption in response to different weather conditions. Energy and buildings. 1993 Jan 1;19(3):235-47. Hong T, Taylor-Lange SC, D’Oca S, Yan D, Corgnati SP. Advances in research and applications of energy-related occupant behavior in buildings. Energy and buildings. 2016 Mar 15;116:694-702. International Energy Conservation Code, 2018. https://codes.iccsafe.org/content/iecc2018 (Accessed on October 20th, 2020). Jain RK, Taylor JE, Culligan PJ. Investigating the impact eco-feedback information representation has on building occupant energy consumption behavior and savings. Energy and Buildings. 2013 Sep 1;64:408-14. Jiang C, Masood M K, Soh Y C, et al. Indoor occupancy estimation from carbon dioxide concentration[J]. Energy and Buildings, 2016, 131: 132-141. Jung W, Jazizadeh F. Human-in-the-loop HVAC operations: A quantitative review on occupancy, comfort, and energy-efficiency dimensions. Applied Energy. 2019 Apr 1;239:1471- 508. Labeodan T, Maaijen R, Zeiler W. The human behavior: a tracking system to follow the human occupancy. InProceedings of the international conference on cleantech for smart cities and buildings (CISBAT 2013) 2013 Sep (Vol. 4, p. 6). Liao C, Barooah P. An integrated approach to occupancy modeling and estimation in commercial buildings[C]. American Control Conference (ACC), 2010. IEEE, 2010: 3130-3135. Liao C, Lin Y, Barooah P. Agent-based and graphical modelling of building occupancy[J]. Journal of Building Performance Simulation, 2012, 5(1): 5-25. Menyhart, K., & Krarti, M. (2017). Potential energy savings from deployment of Dynamic Insulation Materials for US residential buildings. Building and Environment, 114, 203-218. Michaels J, Leckey T. Commercial Buildings Energy Consumption Survey (CBECS). US Energy Information Administration. 2012. 19 Moon JW, Han SH. Thermostat strategies impact on energy consumption in residential buildings. Energy and Buildings. 2011 Feb 1;43(2-3):338-46. Naylor S, Gillott M, Lau T. A review of occupant-centric building control strategies to reduce building energy use. Renewable and Sustainable Energy Reviews. 2018 Nov 1;96:1-0. Pang, Z., Chen, Y., Zhang, J., O’Neill, Z., Cheng, H., Dong, B.. Nationwide Energy Saving Potential Evaluation for Office Buildings with Occupant-Based Building Controls, 2020. ASHRAE Annual Conference Paper. Qin H, Pan W. Energy use of subtropical high-rise public residential buildings and impacts of energy saving measures. Journal of Cleaner Production. 2020 May 1;254:120041. Ryu S H, Moon H J. Development of an occupancy prediction model using indoor environmental data based on machine learning techniques[J]. Building and Environment, 2016, 107: 1-9. Saving Energy Nationwide in Structures with Occupancy Recognition (SENSOR), 2017. Advanced Research Projects Agency – Energy. https://arpa-e- foa.energy.gov/Default.aspx?Search=DE-FOA-0001737&SearchType= (Accessed on November 24th, 2020). Teixeira, Thiago, Gershon Dublon, and Andreas Savvides. "A survey of human-sensing: Methods for detecting presence, count, location, track, and identity." ACM Computing Surveys5.1 (2010): 59-69. Wang C, Pattawi K, Lee H. Energy saving impact of occupancy-driven thermostat for residential buildings. Energy and Buildings. 2020 Mar 15;211:109791. Wang J, Xiong J, Jiang H, Chen X, Fang D. D-Watch: Embracing “Bad” multipaths for device-free localization with COTS RFID devices. IEEE/ACM Transactions on Networking. 2017 Sep 21;25(6):3559-72. Yan D, Hong T, Dong B, Mahdavi A, D’Oca S, Gaetani I, Feng X. IEA EBC Annex 66: Definition and simulation of occupant behavior in buildings. Energy and Buildings. 2017 Dec 1;156:258-70. Yang J, Pantazaras A, Chaturvedi KA, Chandran AK, Santamouris M, Lee SE, Tham KW. Comparison of different occupancy counting methods for single system-single zone applications. Energy and Buildings. 2018 Aug 1;172:221-34. Yang Z, Li N, Becerik-Gerber B, Orosz M. A systematic approach to occupancy modeling in ambient sensor-rich buildings. Simulation. 2014 Aug;90(8):960-77. Yoshino H, Hong T, Nord N. IEA EBC annex 53: Total energy use in buildings—Analysis and evaluation methods. Energy and Buildings. 2017 Oct 1;152:124-36. 20 Yun J, Lee SS. Human movement detection and identification using pyroelectric infrared sensors. Sensors. 2014 May;14(5):8057-81. Zhang Y, Bai X, Mills FP, Pezzey JC. Rethinking the role of occupant behavior in building energy performance: A review. Energy and Buildings. 2018 Aug 1;172:279-94. Zou H, Jiang H, Yang J, et al. Non-intrusive occupancy sensing in commercial buildings[J]. Energy and Buildings, 2017, 154: 633-643. 21 CHAPTER 2 – INFLUENTIAL VARIABLES IMPACTING THE RELIABILITY OF BUILDING OCCUPANCY SENSOR SYSTEMS: A SYSTEMATIC REVIEW AND EXPERT SURVEY 2.1 Abstract Occupancy information is critical for buildings, as it can substantially impact energy consumption. However, occupancy is highly unpredictable, and depends on occupants’ schedules and their interactions with building systems. Occupancy sensor systems have been deployed in buildings for many years, using a broad range of sensor types, most typically for lighting control. Similarly, many research studies have been conducted that use a range of sensor modalities for occupancy sensing. These studies have assessed reliability considering diverse variables, testing methods, and evaluation metrics. However, no comprehensive review of occupancy sensor system reliability has been compiled. In this research, a review of the literature on occupancy sensor systems is presented, focusing on influential variables that may cause systems to incorrectly represent occupancy. Next, a survey was developed and completed by researchers, practitioners, and other stakeholders, ranking the most influential variables. The results of this effort provide a tiered list of what both literature and experts suggest the most influential factors are on occupancy sensor system performance. Future work includes the development of a standard methodology to evaluate the reliability of different occupancy sensor systems based on these efforts. Keywords: Building occupancy; Occupancy sensor systems; reliability testing; influential variables 2.2 Introduction Energy consumption throughout the world has significantly increased over the past 20 years, as the world population has increased, as has the use of energy-consuming technologies. 22 Building energy use accounts for approximately 40% of the total primary energy consumption in the U.S. and in the E.U., and approximately 27% in China (Cao et al., 2016). For end uses in buildings located these regions, space heating and cooling energy make up a large portion of total energy consumption. In the U.S., heating and cooling consumes approximately 51% of the total energy use in the residential (U.S. EIA, 2019) and 33% in commercial sectors (U.S. EIA, 2019). Moving forward, it is projected that total energy use from heating and cooling will increase in cooling-dominant and less-developed regions throughout the world (Cao et al., 2016). Therefore, in an effort to reduce greenhouse gas (GHG) emissions and energy use overall to limit climate change impacts and improve electric grid operations, it is important to develop energy saving approaches to reduce, in particular, heating and cooling energy use in buildings. The energy consumption of mechanically conditioned buildings and their systems are primarily influenced by outdoor weather conditions, building and system characteristics, and the occupants who utilize them. Data on these parameters are necessary to monitor performance, assess inefficiencies, and identify opportunities for improvements. Weather data are generally easier to obtain and/or collect using local or regional ground-based weather station or satellite- derived data. For individual buildings, data on building systems and their characteristics are sometimes available, most typically through building owners and/or operators, with more data being available if the building has a building management system (BMS) or building automation system (BAS). Occupancy data, however, are more challenging to collect and less readily available as compared to the others. Buildings are strongly influenced by the occupants that use them and their energy- consuming behaviors (Hong et al., 2016). There are significant opportunities to improve the collection and use of occupancy-related data to inform building controls, to target energy use 23 reductions and efficiency improvements (Wang et al., 2017). When there are less occupants in commercial buildings, ventilation requirements are lower (ASHRAE, 2019); when there are no occupants, thermal comfort requirements no longer need to be met (ASHRAE, 2017), thus enabling adjustments to setpoint temperatures without impacting occupants. Therefore, through both mechanisms, occupancy sensing technologies used in buildings can help to significantly reduce HVAC use, also resulting in reduced energy demands of buildings (Jung et al., 2019). To collect accurate occupancy information, such as occupant presence, counting, location, and tracking (Teixeira et al., 2010), occupancy recognition technologies have been developed and studied in the recent decades. Existing technologies for occupancy information extraction in buildings can generally be classified into two categories: single sensor systems and sensor fusion technology. Single sensor systems utilize one or multiple sensors of the same type to infer occupancy information, while sensor fusion methods combine the use of multiple types of sensors and sensing modalities to detect occupancy. The benefit of the use of single sensor systems is the relative simplicity of the sensor deployment and data collection, however, edge cases can occur where the utilized single sensor type determined occupancy incorrectly. For example, a motion-based sensor may not be able to detect occupants who are not moving, causing a false negative, the sensor provides “unoccupied” as its output when there are occupants in the space. Sensor fusion technologies that have multiple sensor modalities can help to compensate for known weaknesses of single sensor systems, by adding other sensor systems that work better where others have weaknesses. These then typically use a data-driven model which is developed based on the data collected from the different sensor system modalities, to determine occupancy. In general, the more inputs and diverse data collected as input into the model, the more accurate the models become (Yang et al., 2018). 24 The challenge with occupants and occupancy data, however, is that occupants are inherently unpredictable. Occupants and their schedules vary both from person to person, and from building type to building type (Gu et al., 2018). While there are “typical” occupancy schedules and activities that represent the most common behaviors (e.g., Mitra et al., 2020), even the most “normal” people can deviate from these typical scenarios, creating edge case occupancy scenarios. In order to create the most reliable occupancy sensor systems possible, they should perform well under both typical scenarios and edge cases, and thus in a range of building scenarios, indoor environmental conditions, and occupancy scenarios. To evaluate if occupancy sensor systems perform well across this range of scenarios, what characterizes “typical” must be defined, such that system performance can be tested and evaluated under these conditions following a standardized methodology. The results from the use of such a methodology would enable comparison of occupancy sensor system performance, to determine their relative reliability. Currently, however, there is no standard method of testing, nor a standard definition of “typical” occupancy scenarios and typical building conditions under which to test. There are occupancy schedules commonly used in energy modeling software (Christensen et al., 2005; Deru et al., 2011), however these define only the occupancy fraction (percent of total occupants in a space), and not their activities, locations, building interactions, and other variables. There are also prototypical building energy models developed to represent typical residential (Mendon et al., 2014) and commercial buildings (Deru et al., 2011). However, when considering these typical buildings for use in defining an appropriate setup for a controlled laboratory environment for sensor system testing and evaluation, translating the buildings’ setup to a space- limited environment for controlled laboratory testing can present challenges. Similarly, edge cases must be defined. Edge cases occur when occupants, building systems or other factors deviate from 25 “typical” conditions. Therefore, to determine what type of normal and edge cases may occur, it is necessary to define what specific variables may influence the performance of occupancy sensor systems, how these variables are likely to vary under normal and extreme scenarios. There are many factors that may influence building occupancy sensor system performance. These can generally be classified into four categories, including building-related, environment- related, occupant-related, and others. For building-related and environment-related characteristics, factors such as interior building layout and geometry, and lighting levels can significantly impact the ability of a system to detect occupants (Gennarelli et al., 2016; Labeodan et al., 2013; Wang et al., 2017; Yun et al., 2014). This impact varies depending on the type of sensor system being evaluated, its corresponding algorithms, and how the sensor system is deployed in a particular location. Similarly, occupant-related characteristics, including their physical features and movements, their preferences, and their occupant-building interactions with energy-consuming appliances and systems can make the accurate detection of occupants challenging (Yoshino et al., 2017; Yan et al., 2017). The relative importance of these occupant characteristics, similar to physical building characteristics, is also dependent on the sensor system considered. Other variables include those that do not fit into the other three categories, such as presence of pets. Among recent literature, while many of sensor testing studies have considered multiple different factors in evaluating their influence on the occupancy sensor system performance, these papers follow different methods and test different sets of variables. The current gap in the literature is that 1) there is no comprehensive assessment gauging the relative influence and importance of these diverse factors on the performance of occupancy sensor systems; 2) there has been no attempt to develop a standard method to evaluate sensor system performance, including defining what influential factors to consider for typical and edge case scenarios. Therefore, to address these 26 challenges, the objectives of this paper are to 1) conduct a comprehensive literature review on the variables that impact occupancy sensor system performance, including what variables are likely to vary under normal and extreme scenarios; 2) obtain expert feedback from occupancy sensor system stakeholders regarding the most and least important variables influencing occupancy sensor performance for commercial and residential building applications, in order to evaluate the relative importance of these variables. The development of a standard methodology to evaluate sensor system performance is the focus of ongoing efforts. The next section describes a general review of the commonly used occupancy sensor systems, to summarize the influential variables that could cause sensor failures. Section 3 presents the expert workshop, expert survey, and their results that were used to identify, based on stakeholder feedback, the most important variables to evaluate for sensor system reliability. 2.3 Literature Review Human-sensing technologies can be divided into five categories: presence, counting, location, tracking, and identification (Teixeira, et al., 2010). From the perspective of building energy performance, whether or not there are people is significant for lighting control in both residential and commercial buildings. For HVAC system control, the presence/non-presence of people is of significance in residential buildings, while how many people in each thermal zone of a building is critical in commercial buildings, particularly for ventilation controls. The specific location, tracking of occupants and their identification have more limited impact on the energy use of a building. Thus, in this research, the main focus among these categories is occupant presence and counting. Many sensor technologies have been used to assess the presence and counting of occupants in buildings. Of the 120 recent papers published between 2003 and 2020 on occupancy sensor 27 systems identified based on keywords search in Google Scholar, such as “occupancy sensor”, “occupancy detection”, and “occupancy counting”, including both single sensor systems and sensor fusion systems, 80 focused on either occupancy detection (presence/non-presence) and occupancy counting (number of people). These are used in this paper. Figure 5 (a) and (b) summarizes the number of papers where each of the occupancy sensor technologies appear. From Figure 5 (a), the most commonly used sensor technologies include both single sensor systems, which are radio frequency-, vision-, infrared-, sound wave-based sensors, and sensor fusion systems. Sensor fusion is generally a combination of one or more of the single sensor systems’ sensing modalities and one or more environmental sensors. Figure 5 (b) further subdivides the sensor modalities’ sensor fusion systems into each of the single sensor systems as well as each of the environmental sensors and provides the number of papers in which each appears. For those papers that use sensor fusion systems, each is included in the count for each of the single sensor systems and environmental sensors. From Figure 5 (b), it is also noted that there are some other types of sensors that were used in sensor fusion methods, such as door sensors, reed switches, and pressure mats, which have not been discussed in detail as a single sensor system in this research since it was not found to be common for these sensor types to be used as a single sensor system, as compared to others. 28 45 40 35 30 # of Papers 25 20 15 10 5 0 (a) 50 45 Sensor fusion Single sensor 40 35 # of papers 30 25 20 15 10 5 0 (b) Figure 5 Frequency of occupancy sensor modalities used in recent literature from 2003 to 2020, for occupancy detection and counting, including (a) types of single sensor system modalities and sensor fusion, and (b) types of single sensor system modalities, with sensor fusion subdivided into single sensor system modalities with other environmental sensors This literature review is discussed by sensor type. A literature review of 80 relevant peer- reviewed research articles on occupancy sensor systems in buildings, focusing on occupancy detection and occupancy counting, resulted in five types of sensor technologies that were most commonly used (Figure 5 (a)), including single sensor systems (radio frequency-, soundwave-, 29 infrared-, vision-based sensors), and sensor fusion, which are discussed herein. For the single sensor systems, the focus is on the potential sensor failures for each type of sensor modality. For sensor fusion technologies, which are typically a combination of one or more of the single sensor systems sensing modalities and environmental sensors, the potential sensor failures include failures caused by not only the above single sensor modality, but also the environmental sensors, as well as the system’s interpretation and use of this information. Table 1 shows each of the types of sensors for each sensor category. Table 1 Most common types of occupancy sensor types used in single sensor and sensor fusion applications in residential and commercial buildings Sensor Category Sensor type Normal Doppler Radar Ultra-wide band (UWB) Radar Radio frequency-based Radio-frequency Identification (RFID) Wi-Fi Bluetooth low energy (BLE) Acoustic Sound wave-based Ultrasonic Infrared-based Active infrared (IR)/Passive infrared (PIR) Vision-based Video/camera Combination of environmental sensors (e.g., temperature, relative Sensor fusion humidity, CO2, VOCs) and the above-listed single sensor types In the following section, a literature review is discussed, and a summary table provided for each of these five sensor technologies, including the type of occupancy information each paper studied (presence/counting), the building type where this occurred (residential/commercial), the sensor type (single/sensor fusion), and the influential variables (building-related, environment- 30 related, occupant-related, others) that either have been tested and shown to impact occupancy detection/counting, or were mentioned as variables that may cause sensor failure. Across these studies, most of the goals are to develop an occupancy sensor system and test it in a controlled environment to evaluate the performance of the system. Several also combined the developed occupancy sensor system with HVAC control to evaluate the level of energy savings that can be achieved with occupancy-based control. Data collection periods, in general, ranged from 1 min to four months, where the shorter ones focused on assessing specific variables, while the longer ones were to assess the performance of the occupancy sensor system over a longer period of time, coving a range of scenarios such as workdays, weekend, and holidays, and different weather conditions. Most of the data collection periods are several days to several weeks in length. Ground truth data (e.g., data from the camera system) was also collected across these studies to be compared with occupancy data attained from occupancy sensors to assess their reliability. Different performance metrics were used in these papers, including accuracy, confusion matrix (true positive, true negative, false positive, and false negative), F-score (calculated based on confusion matrix), and other statistical methods (e.g., mean average error, mean absolute error, root mean square error, normalized root mean square error, standard deviation, and root mean square deviation). 2.3.1 Radio frequency-based sensors Radio frequency-based sensors utilize radio waves as media for signal transmission, ranging from approximately 20 kHz to around 300 GHz (Basnayaka et al., 2017), and detect occupants based on a change in the characteristics of the radio signals. Doppler-shift sensors operate on the principle that radio waves reflected from a moving object will result in a frequency shift that is related to the radial component of the object’s velocity. Generally, such a sensor will 31 register as detecting occupancy as long as it is able to detect motion. There are four main classifications of motion levels, including major (e.g., people walking), minor (e.g., people extending arms), fine (e.g., sitting and typing), and no motion (e.g., sleeping), as defined in National Electrical Manufacturers Association (NEMA) WD 7 Occupancy Motion Sensors Standard (NEMA, 2011). The level of motion that occurs at a particular instance could influence the performance of sensors. At the finer levels of motion, it may be more difficult for such a sensor to detect occupants, and thus may elicit a failure (Yavari et al. 2013). There are two main categories of Doppler sensors, including normal Doppler radar sensors, and ultra-wide band (UWB) radar sensors. Compared to UWB radar sensors, normal Doppler radar sensors can more easily experience interference with other parallel signatures, such as from nearby systems generating electromagnetic interference in the same frequency range (Diraco et al., 2017). UWB sensors, however, use a radio signal with a fractional bandwidth equal to or greater than 0.20 or at a frequency greater than 500 MHz (Wilzeck et al., 2010). UWB also uses the Doppler- shift effect to detect occupants. It is also able to operate over a larger bandwidth and wider range of frequencies, resulting in submillimeter range resolution and high penetration power, supporting detection of small objects even through obstacles (Diraco, et al., 2017). In addition, UWB sensors can penetrate walls and other obstacles with relatively low power consumption compared to normal Doppler radar systems (Kim et al., 2016). Table 2 summarizes recent research using Doppler radar sensors in occupancy sensing technologies. Of the listed studies in Table 2, the influential variables that appeared multiple times in multiple studies are building-related (presence of large metal objects), and occupant-related (motion level, number of occupants, location of occupants, posture of occupants) variables. 32 Table 2 Influential variables evaluated in recent literature using Doppler sensors for occupancy sensing Occupancy Sensor type Reference Building type Influential variables mentioned/tested Information (Yavari et al. 2013) Presence - Motion level of occupants 1. Number of occupants; 2. Multiple occupants walking at similar speeds; Normal Doppler (Yavari et al., 2014) Presence - 3. Non-human periodic motion; Radar 4. Number of sensors; 5. Direction of arrival of sensors. 1. Presence of large metal objects (fan, (Sekine et al., 2012) Presence - microwave, oven, washing machines); 2. Environmental noise. 1. Motion level of occupants; (Kilic et al., 2013) Presence Commercial 2. Range of sensor system; 3. Measurement duration. 1. Location of occupants (Occupants behind (Kim et al., 2013, Yarovoy Presence/ walls); et al., 2006, Ossberger et al., Commercial Location 2. Distance between the sensor and occupants; 2004) 3. Body shape of occupants. Presence/ 1. Number of occupants; (Gulmezoglu et al., 2014) Commercial Tracking 2. Occlusion. 1. Location of occupants (Stationary occupants (Rane et al., 2016, Mabrouk Presence - behind the wall); UWB Doppler radar et al., 2014) 2. Posture of occupants. 1. Wall type (drywall, wooden door, brick wall, load bearing concrete wall); (Singh et al., 2011) Presence Commercial 2. Motion level of occupants; 3. Presence of large metal objects; 4. Posture of occupants. 1. Size/shape of the test space; 2. Number of occupants; (Yavari et al., 2018) Counting Commercial 3. Location of occupants; 4. Location of sensors; 5. Clustering of occupants. 33 Another type of RF sensor is RFID (radio-frequency identification). RFID sensors identity and track tags attached to occupants or other item. An RFID tag receives signals from a nearby RFID reader, then transmits digital data back to the reader. The reader recognizes the received information from each of the tags to decide whether a person is in this space. RFID is an identification technology designed to provide data on the location and the testing occupants itself. RFID sensors have several advantages, including that they do not require line of sight conditions, and that they can have on-board storage capacity, making it more flexible for data collection. RFID sensors are truly real-time, and thus do not have accumulated errors like vision-based sensors. RFID sensors, however, can be highly impacted by noisy environments due to noise interference and the sensitivity patterns of anisotropic antennas (Li et al., 2011). One of the main disadvantages is the needed network density, i.e., a larger number of sensors are needed as compared to other sensor types (Wang et al., 2017). RFID sensor systems require a complex infrastructure of beacon nodes, which can be expensive and cumbersome to install and manage. It also requires calibration once the built environment has been changed, such as chairs or furniture being in different positions. Table 3 provides a list of recent research efforts that have evaluated the use of RFID sensor systems. Of the listed studies in Table 3, the variables that appeared multiple times in multiple papers include occupant-related variables (motion level, number of occupants, location of occupants). 34 Table 3 Influential variables evaluated in recent literature using RFID sensor systems for occupancy sensing Sensor Occupancy Influential variables Reference Building type type Information mentioned/tested 1. Number of occupants; 2. Motion level of occupants; 3. Location of occupants (Li et al., 2012) Counting Commercial (occupants walk close to the boundary of thermal zones); 4. Response time of the sensor. 1. Motion level of occupants; 2. Size/shape of test area; Counting/ (Li et al., 2011) Commercial 3. Number of occupants; Location 4. Location of occupants; 5. Environmental noise. 1. Different locations of sensor (RFID tags such as attached to hat, RFID (Ranjan et al., shoes, belt, shirt, pants, wrists, and Presence Residential 2012) ankles, antennas placed on diff. appliances); 2. Configuration of antennas. (Wang et al., 1. Awareness of occupants wearing Counting Commercial 2017) tags. 1. Location of occupants; 2. Environmental noise; 3. Number of tags; Counting/ (Xu et al., 2021) Commercial 4. Protocol reconfigurability; Location 5. Tag read rate; 6. Allowable operational bandwidth. Note: The “location of occupants” can influence the sensor performance when occupants are at a boundary (e.g., corners), as occupants may not be sensed or detectable by the sensor system in this location. The “size/shape of test area” refers to the size, interior layout, and geometry of a space, which may influence sensor system configuration, number of sensors needed and their ability to cover the test area. A third type of RF sensor uses Wi-Fi communication. Wi-Fi describes a local wireless network that uses radio waves to communicate data, typically originating from the Internet. To be considered Wi-Fi, the radio signal must use the IEEE 802.11 standard (2016) to communicate. Multiple versions of Wi-Fi are defined in the IEEE specifications, including common ones such as 2.4 GHz and 5 GHz frequency radio waves. Wi-Fi infrastructure is widely available in most buildings along with mobile devices (e.g., smart phones, tablets, laptops) with Wi-Fi connectivity 35 carried by occupants. The data packets are transmitted in existing Wi-Fi traffic, and both received signal strength (RSS), and, in some cases, the MAC address of each occupant’s mobile device connected to Wi-Fi can be extracted to estimate occupancy, including either presence/non- presence or number of occupants. The MAC address of each device serves as a unique identifier for each of the occupants. Some of the main advantages of the use of Wi-Fi include the use of a common and widely implemented network, especially in residential and commercial buildings. In addition, the cost of using this technology is negligible in comparison to some other sensor types (Ouf et al., 2017), particularly when Wi-Fi is already available. Disadvantages include that it requires occupants to carry their mobile devices (Zou et al., 2018) and higher power consumption compared to some other sensor types (Mahmoud et al., 2015). In addition, there may be privacy concerns for occupants since the personal MAC addresses and other universally unique identifiers (UUIDs) need to be monitored and collected (Wagner et al., 2018). Table 4 includes a summary of recent studies which have used Wi-Fi-based sensors systems to evaluate occupancy. Of the listed studies, the influential variables that appeared in multiple papers include building-related variables (size/shape of test area, presence of large metal objects), environment-related variables (electromagnetic interference), occupant-related variables (number of occupants), and others (distance between occupants and sensors). 36 Table 4 Influential variables evaluated in recent literature using Wi-Fi sensor systems for occupancy sensing Sensor Occupancy Reference Building type Influential variables mentioned/tested type Information 1. Distance between occupants and sensors; 2. Orientation of the antennas; (Ravichandran 3. Motion level of occupants; Presence Residential et al., 2015) 4. Posture of occupants; 5. Presence of large metal objects (refrigerator, computer monitor, large furniture). 1. Number of occupants; 2. Clustering of occupants; (Xi et al., 2014) Counting Commercial 3. Walking speed of occupants; 4. Spatial distribution of occupants. (Palipana et al., 1. Distance between occupants and Presence Commercial 2016) sensors. 1. Wi-Fi device not turned on by occupants; 2. Phone in sleep mode if not in use; (Wang et al., Wifi- 3. Occupant(s) has multiple devices 2017, Wang et based Counting Commercial (phones, laptops, wireless printers); al., 2018, Ouf et 4. Presence of large metal objects al., 2017) (interior metal separations); 5. Size/shape of test area; 6. Electromagnetic interference; 1. Phone in sleep mode if not in use; (Balaji et al., 2. Occupant leaves space without Counting Commercial 2013) carrying their phone; 3. Electromagnetic interference. 1. Presence of large metal objects; (Lu et al., 2016) Counting Commercial 2. Data collection timestep. (Vasisht et al., Presence Residential 1. Signal strength of sensors 2016) 1. Size/shape of test area; (Petrovic et al., Residential/ Counting 2. Number of occupants; 2018) Commercial 3. Response time of sensor(s) (Vattapparamba Counting Commercial 1. Electromagnetic Interference n et al., 2016) Note: Occupant posture was found to impact the ability of some technologies to detect respiration rates for use in occupancy detection. For moving occupants, the spatial distribution of occupants and their movements was found to influence occupancy count accuracy, where randomly moving and distributed occupants reduced estimate errors. 37 A fourth RF wireless sensor technology is Bluetooth, which uses short-wavelength radio transmissions in the range of 2400-2480 MHz, standardized in IEEE 802.15.1 (2005), to exchange data within short ranges from fixed and mobile devices. Using Bluetooth, smart devices need to be in discoverable mode for an initial registration to be connected. As long as the Bluetooth capability is enabled, there are no subsequent actions needed to change Bluetooth settings. One example is an iBeacon, which uses Bluetooth low-energy (BLE) wireless technology to provide location-based information. There are three main components used in the detection of occupancy using Bluetooth: beacon transmitters, which send uniquely identified beacon packets with a Universally Unique Identifier (UUID), receivers who install a client mobile application on their smartphones to periodically scan signals to detect beacons in a building, and remote servers which gather and implement algorithms to identify whether there is a person in the space based on the information that the client mobile application receives from occupants’ smart phones. The main advantages of the use of Bluetooth sensor technologies include much lower power consumption compared to standard Bluetooth and Wi-Fi devices (Putra et al., 2017). However, a main disadvantage would be the potential interference this system with Wi-Fi, which may disturb the connection if multiple Bluetooth devices are running at the same time. Table 5 summarizes recent research using Bluetooth and/or BLE-based sensors systems to evaluate occupancy. Of the listed studies, the variables that appeared multiple times in multiple papers are building-related variables (size/shape of test area). 38 Table 5 Influential variables evaluated in recent literature using Bluetooth/BLE-based sensor systems for occupancy sensing Sensor Occupancy Building Influential variables Reference type Information type mentioned/tested 1. Cyclic behavior of beacons; (Conte et al., 2014, Commerci 2. Humidity; Presence Corna et al., 2015) al 3. Occupant(s) have multiple devices. 1. Size/shape of test area; (Filippoupolitis et Commerci 2. Location of sensor(s); Presence al., 2016) al 3. Distance between occupants and sensors. BLE- 1. Occupant leaves space without based carrying their phone; Commerci (Shen et al., 2016) Presence 2. Occupants go beyond the al Bluetooth range; 3. Response time of sensor(s). Commerci (Park et al., 2018) Counting 1. Size/shape of test area. al Commerci 1. Size/shape of test area; (Longo et al., 2019) Counting al 2. Number of occupants. 2.3.2 Sound wave-based sensors There are two main types of soundwave-based sensors, including acoustical sensors and ultrasonic sensors. Acoustic wave sensors are named because their detection mechanism is a mechanical or acoustic wave using a piezoelectric material. For this application such sensors detect only human audible sound with frequency of 20 Hz–20 kHz (Launer et al., 2016). Ultrasonic sensors measure distances based on transmitting and receiving ultrasonic signals, which detects these sounds that are inaudible to humans. An ultrasonic sensor is generally made up of piezoelectric material, where the ultrasonic transmitter transmits an ultrasonic wave, when then travels through a medium of air until it is intersected by a material. The wave is reflected back when it detects a person, which can then be detected by the ultrasonic receiver. By analyzing the time and distance that the reflected ultrasonic wave is received, it can infer whether there is a person in the space. Compared to other sensors, the main advantages of the use of sound wave- 39 based sensor technologies is that it can include an increased range of sensitivity for minor movement. However, the disadvantage would be that certain materials absorb sound waves, such as cloth or foam, causing problems when a person is covered in multiple layers of clothing and the sensor would not detect motion consistently, and they are highly sensitive to reflective materials such as glass or plastic (Yavari et al., 2014). Research focused on sound wave-based sensors for occupancy detection is provided in Table 6. Of the listed studies, the variables that appeared multiple times in multiple papers include: building-related (size/shape of test area), occupant-related (number of occupants, motion levels, walking speed of occupants), and other (location of sensors, distance between occupants and sensors, number of sensors) variables. 40 Table 6 Influential variables evaluated in recent literature using sound wave-based sensor systems for occupancy sensing Sensor Occupancy Building Reference Influential variables mentioned/tested type Information type 1. Number of occupants; 2. Location of sensors; (Khan et Counting Commercial 3. Distance between occupants and sensors; al., 2015) 4. Environmental noise; 5. Noise level of occupants. 1. Number of occupants; 2. Location of sensors; (Xu et al., Counting Commercial 3. Presence of non-occupant sounds; 2013) 4. Characteristics of sound (utterance length of sound). 1. Different wall materials (cloth, foam) absorption (Yavari et of sound waves; Presence - al., 2014) 2. Presence of reflective materials (glass, plastic); 3. Motion levels of occupants. 1. Size/shape of test area; (Shih et al., Counting Commercial 2. Performance over time of the sensor; 2015) 3. Number of sensors. 1. Size/shape of test area; (Shih et al., Presence/ 2. Number of occupants; Commercial 2016) Counting 3. Distance between occupant and sensors; 4. Blind spots where sensor cannot see. Acoustic 1. Body shape of occupants (height, width, girth, hand-waist distance, and bounce); 2. Motion level of occupants; 3. Location of sensors; 4. Angle between occupants and sensors; (Khalil et 5. Number of occupants walking simultaneously; Presence Commercial al., 2018) 6. Occupant(s) with a purse; 7. Walking speed of occupants; 8. Occupant in a wheelchair; 9. Location of the door frame (corner); 10. Door size; 11. Number of sensors. 1. Occupant height; 2. Walking speed of occupants; 3. Number of sensors; 4. Posture of occupants; 5. Presence of large metal objects; (Hnat et al., Presence Residential 6. Occupants wearing hats; 2012) 7. Occupants standing in doorways; 7. Movement of furniture over time; 8. Opening and closing of doors and windows; 9. Presence of household objects (bags, laundry baskets). 41 2.3.3 Infrared-based sensors PIR (passive infrared) sensors, which are among the most common types of occupancy sensors used, are passive in that they do not generate or radiate energy for detection purposes. Instead, they function through the detection of infrared radiation emitted by or reflected from objects. Active IR sensors require both an emitter and receiver, where the IR emitter emits a beam of light, facing an in-line receiver. If nothing is in the way, the receiver sees the emitted signal. If the receiver fails to see an IR beam, it detects that a person is between the emitter and the receiver, and therefore present in the monitored area. Low-resolution IR arrays are based on thermopile technology, of which the thermopile element is sensitive to a motionless object. The main advantages of the use of infrared-based sensor technologies is that it is the most commonly used technology for occupancy presence sensing. However, the disadvantage would be that they cannot detect static occupants, and the sensitivity of this sensor type drops off drastically with distance and requires line of sight (Santra et al., 2018). There have been studies focusing on the improvement of motion sensors to detect static occupants. For example, Wu et al., (2019) developed an optical shutter based on a PIR sensor, which indicated an accuracy of 97% in unoccupied and occupied scenarios. Ma et al., (2019) also proposed an active PIR sensing system to actively detect static thermal targets. Research using IR-based sensors for occupancy detection is provided in Table 7. Of the listed studies, the variables that appeared multiple times in multiple papers include occupant- related variables (location of occupants, body shape of occupants, number of occupants, motion level of occupants, clustering of occupants). 42 Table 7 Influential variables evaluated in recent literature using infrared-based sensor systems for occupancy sensing Occupancy Building Influential variables Sensor type Reference Information type mentioned/tested (Dodier et al., Presence Commercial 1. Location of occupants. 2006) 1. Differences among multiple individual sensor; 2. Response time of sensor; (Kuutti et al., 3. Height of sensor; Counting Commercial 2014) 4. Body shape of occupants; 5. Clustering of occupants; 6. Air flow (e.g., from HVAC vents). 1. Number of occupants; 2. Different walking directions of occupants (back and forth); Infrared- (Yun et al., 3. Motion level of occupants; based sensor Presence - 2014) 4. Distance between occupants and sensors; 5. Number of sensors; 6. Location of sensors. (Zappi et al., 1. Number of occupants; Counting Commercial 2007) 2. Clustering of occupants. 1. Location of occupants; (Raykov et al., Counting Commercial 2. Number of occupants; 2016) 3. Data collecting frequency. (Chowdhury 1. Body shape of occupants; Presence - et al., 2016) 2. Location of occupants. (Liu et al., Presence - 1. Motion levels of occupants. 2017) 2.3.4 Vision-based sensors Vision-based sensors include camera and video technologies. These sensors are used to collect video/camera frame data to infer occupancy information. Methods include background subtraction, where the foreground elements are separated out from the background by generating a foreground mask to be used for differentiating frames before and after a person appears, and pattern matching approaches, where Haar-like features and a cascade of classifiers are constructed. Other methods include HOG (Histogram of Oriented Gradients), SVM (Support Vector Machine), 43 SIFT (Scale-invariant feature transform)-inspired features, to detect and locate occupants in a video frame, frame-differencing (subtracting consecutive frames pixelwise), and optical flow (i.e. measuring the motion gradient of each pixel over a number of frames). The advantage of vision- based sensors is that video is considered as the most accurate occupancy sensing technology, and thus is usually used as a ground truth method (Shen et al., 2017). The disadvantage of the traditional RGB cameras is the passive vision problems due to brightness variations, shadows, and occlusions (Kwolek et al., 2015). In addition, the privacy concern is also a concern when using vision-based sensors. Recently, 3D depth-based cameras have been increasingly implemented, which use structured light to measure distances from every point to the camera, such that it can work in a completely dark environment, and the privacy concern could also be addressed by using only the depth data (Diraco et al., 2015, Munir et al., 2017). A summary of the variables that have been tested for vision-based sensor is presented in Table 8. Of the listed studies, the variables that appeared multiple times in multiple papers include building-related (presence of large metal objects), environment-related (lighting levels, presence of direct sunlight), and occupant-related (clustering of occupants) variables. 44 Table 8 Influential variables evaluated in recent literature using vision-based sensor systems for occupancy sensing Sensor Occupancy Influential variables Reference Building type type Information mentioned/tested 1. Lighting levels; 2. Presence of interior lighting sources; 3. Presence of direct sunlight; 4. Age of occupants; (Zou et al., 2017) Counting Commercial 3. Hairstyles of occupants; 4. Posture of occupants; 5. Motion level of occupants; 6. Presence of large metal objects; 7. Clustering of occupants. 1. Presence of large metal objects; 2. Objects located in the shadow; Vision- 3. Blind spots where sensor cannot based (Teizer et al., Presence - see; 2007) 4. Lighting levels; 5. Air temperature; 6. Presence of direct sunlight. (Tomastik et al., 1. Door sizes (Multiple Counting Commercial 2008) entrances/exits). (Ahmad et al., Counting Commercial 1. Number of occupants. 2018) 1. Lighting levels; (Yang et al., 2. Presence of cobweb on the Counting Commercial 2018) camera; 3. Clustering of occupants. Note: Age was found to influence occupancy detection abilities as age influence the shape and size of occupants’ heads used as a target for occupancy detection. 2.3.5 Sensor fusion Sensor fusion methods usually combine the use of environmental sensors, such as temperature, relative humidity, lighting, and CO2, with other types of sensors, such as those discussed in the previous portions of this research, to indicate occupancy information. Compared to sensor systems than only use one sensor type, sensor fusion methods have the advantage of multiple sensing modalities. With the appropriately developed data processing algorithms, this can help to reduce the occurrence of false positives and negatives through the use of multiple sensor 45 systems’ data to check the agreement among sensor data output to largely improve accuracy. However, the limitation of sensor fusion is that models can differ largely in terms of their predictive accuracy; additionally, the model formalism chosen, and its complexity can substantially influence the model accuracy (Hobson et al., 2019). A summary of recent sensor fusion research efforts to detect occupancy is included in Table 9. Of the listed studies, the variables that appeared multiple times in multiple papers include building-related (size/shape of test area), occupant-related (number of occupants), and other (short- term transition, number of sensor types, locations of sensors, response time of sensors) variables. Based on the literature review, a comprehensive list of variables that may impact the occupancy sensor systems is compiled (Table 10). This compiled list was also used to determine variables that should potentially be considered when evaluating the performance of the sensor systems. The number of times each of the variable was discussed in this literature review is also shown in Table 10. It is noted that among the variables that appear multiple times in multiple papers for each of the sensor systems, several variables appear in these lists for two or more of the five sensor system types, including size/shape of test area, presence of large metal objects, motion level of occupants, number of occupants, and location of occupants. The variables were arranged based on the categories they belong to, the order for each variable within each category was random, to limit bias for discussion of stakeholders when ranking the most/least important variables. 46 Table 9 Influential variables evaluated in recent literature using sensor fusion systems for occupancy sensing Sensor Occupancy Building Influential variables Reference Sensor type type Information type mentioned/tested (Hailemari Sensor fusion (CO2, 1. Short term transition (a Commer am et al., Presence current, lighting, person just arrived or left cial 2011) motion, sound) their office). (Ekwevug Sensor fusion (sound, Commer be et al., Counting case temperature, 1. Size/shape of test area. cial 2013) CO2 and motion) 1. Number of sensors; 2. Interrupted Wi-Fi connection; Sensor fusion 3. Loss of electricity; (temperature, relative 4. Physical damage to (Yang et Presence/ Commer humidity, CO2, sensors; al., 2014) Counting cial lighting, motion, door 5. Data corruption; switch) 6. Location of sensors; 7. Size/shape of test area; 8. Presence of large metal objects. 1. Number of occupants; 2. Size/shape of test area; Sensor Sensor fusion 3. Short term transition; fusion (Chen et Commer (temperature, relative 4. Response time of Counting al., 2016) cial humidity, CO2, sensors; VOCs, noise, PIR) 5. Air temperature; 6. Presence of direct sunlight. 1. Size/shape of test area; 2. Noise level of Sensor fusion (door occupants; (Zikos et Presence/ Commer counter, acoustic, 3. Motion level of al., 2016) Counting cial PIR, CO2) occupants; 4. Door sizes; 5. Number of sensor types. Sensor fusion 1. Number of occupants; Residenti (Zikos et (temperature, 2. Short term transition; Presence al/Comm al., 2016) humidity, CO2, 3. Response time of ercial pressure) sensors. Sensor fusion (CO2, 1. Size/shape of test area; (Meyn et Commer Counting PIR, video, access 2. Number of sensor types; al., 2009) cial control) 3. Location of sensors. 47 Table 10 Compiled list of influential variables from literature review and expert stakeholder feedback Category All variables Frequency A1. Glass walls & mirrors (reflective surfaces) 1 A2. Size (length/width) and shape of test area 6 A3. Location/characteristics of windows 0 A4. Number of doors (entrances/exits) 1 A5. Door sizes (i.e., single vs. double, other) 3 A6. Wall, floor, and ceiling color/characteristics 1 Building-related A7. Height of ceiling 0 A8. Building envelope type (e.g., brick, siding, EFIS, batt vs. continuous insulation) 1 A9. Presence of large objects (especially metal objects) within or near a space 9 A10. Electromagnetic interference 1 A11. Ventilation rates 0 **A12. Other building-related variables (please specify) 0 B1. Lighting level (regardless of source of light) (lux) 1 B2. Spectral distribution of light 0 B3. Presence of interior lighting sources (non-overhead) 1 B4. Indoor humidity 1 Environment-related B5. Indoor temperature 2 B6. Mean radiant temperature (indoor) 0 B7. Presence of sunlight - direct 2 B8. Presence of sunlight - diffuse 0 **B9. Other environment-related variables (please specify) 0 C1. Number of occupants (including 0) 10 C2. Age of occupant(s) 1 C3. Metabolic rate 0 C4. Spatial location of occupant(s) 5 C5. Level of motion of occupant(s) 10 C6. Noise level of occupant(s) 3 C7. Clustering of occupants (distance between occupants) 5 C8. Speed of occupant (e.g., walking vs. running) 2 C9. Speed of occupants relative to one another 1 Occupant-related C10. Number of occupants entering/exiting a room at the same time 1 C11. Speed of occupants entering/exiting a room at the same time 0 C12. Presence of occupants in adjacent spaces 0 C13. Clothing color/contrast/patterns (including images on clothing) of occupants 0 C14. Occupant clothing level (clo) 0 C15. Skin color of occupant(s) 0 C16. Body shape of occupant(s) 4 C17. Occupants wearing heavy winter coats with a cold surface temperature 0 **C18. Other occupant-related variables (please specify) 0 D1. Presence of pets 0 D2. Motion characteristics of pets (if present) 0 D3. Size/type of pets 0 D4. Use of robots 0 D5. Presence of mylar balloons/party items 0 D6. Ability to communicate with thermostat 0 Others D7. Initial performance/performance over time 1 D8. Range of devices (related to electromagnetic interference) 0 D9. Repeatability of performance of system 0 D10. Presence of heat sources such as a heat lamp 0 D11. Presence of wheelchairs, strollers, shopping carts, etc. 4 D12. Vibrations in spaces 0 **D13. Other variables (please specify) 0 Note: The content with “**” indicates that these variables are not from the literature review or stakeholder feedback but are shown because in the survey this was an option for participants to provide in the case that they want to suggest other variables. The “0” represents variables that were identified based on discussion with stakeholders. 48 2.4 Expert Survey on Most and Least Important Influential Variables Using the results of the literature review, the influential variables tested in previous literature were compiled into a comprehensive list. An expert workshop was held on June 26, 2019 with a diverse set of 50+ stakeholders from academia, consultants, utility workers, manufacturers, and building owners. Four facilitated discussions were completed during the workshop, the first of which included a discussion of influential variables impacting the reliability of occupancy sensor systems. The question posed during this session is as follows: “Which variables that may affect the performance of a sensor system should be included in a test standard/guideline for occupancy recognition sensor systems? Which are most critical?” The compiled list of influential variables was provided to attendees for discussion. Throughout the discussion, additional variables were suggested by stakeholder attendees, which were added into a final list of potential influential variables (Table 10). The final list of variables was then divided into four categories, including (a) building-related variables, (b) environment- related variables, (c) occupant-related variables, and (d) other variables. Using this list of variables, an online survey was developed to determine which influential variables were “most important” and “least important” for use in the development of a standard method of testing of occupancy sensor systems. The participants chose to answer the questions for residential building applications or for commercial building applications. Next, the participants were presented with the list of variables in Table 10 divided into one question for each of the four categories. For each question, the participants consider one category of variables, and were asked to categorize which they would consider the most and least important in evaluating the reliability performance of occupancy sensor systems. Respondents were limited to the choice of the top 6 most and top 6 least influential variables. Within each of the “most” and “least” influential groups, 49 participants were also asked to rank the order of importance. Participants were not required to select a certain number of variables, however a suggested amount in each category was provided in the survey. Not all variables in each question (each category) were required to be selected, thus the variables that were not selected as most or least important were not used in the survey analysis. Following the four questions on the four variable categories, a final question provided a list of all variables across all categories and asked participants to select the most and least important variables. This allowed for the comparison of responses from the categorial rankings (Questions 1-4) and overall rankings of variables (Question 5). The survey was sent to all ARPA-E SENSOR teams (2017), members of Annex 79 (Occupant-Centric Building Design and Operation) (O’Brien et al., 2018), and all workshop attendees. 54 survey responses were gathered in Fall 2019. 24 expert stakeholder responses were complete responses, including 9 focused on residential buildings, 15 focused on commercial building applications. This number of responses is similar to other expert surveys in terms of number of collected responses from experts (e.g., Constenla et al. 2015). 2.5 Survey Results Two methods were used to evaluate the survey data results. The first method (“Method 1”) was a count of the number of times that participants selected each variable, regardless of how it was ranked among the “most” and “least” important categories. In this case, the higher the count, the more important the variable; and the lower the count, the less important the variable. The second method (“Method 2”) incorporated the rankings of participants’ responses using a weighing factor. In this method, for each of the first four questions, the first six variables selected in the “most” and “least” important variables were considered. In the case of the most important variables, the variable that was ranked first received a weightage factor of 6, the second received a factor of 50 5, etc. The same method was applied for the least important variables. For both methods, the final count or score was summed across all participants. Two methods were used as a way to double check the results, to make sure that results from the two methods used to ask stakeholders to rank these variables are consistent. All variables were divided into different categories and then combined together to see if these two methods identified the same or similar set of important influential variables. Natural breaks in the summed count or score were used to determine the tier of importance of each variable. If the resulting count (from Method 1) for each variable was larger than half of the total responses from the data for that category of variables, then the variables were categorized as Tier 1 (i.e., the most/least important variables). Following Method 2, similar results were obtained. Thus, with this data, the Tier 1 variables were then ordered based on their count. 2.5.1 Most important and least important variables by category Figure 6 (a) shows the results of the count of variables under each category for residential buildings using Method 1. The red columns are the variables where the count value is larger than half of the total responses, which is classified as Tier 1 – the most important variables. Weightage distribution is also provided (Method 2), showing that the variables designated as Tier 1 based on the first method also have higher weightage value, as seen in Figure 6 (b), which shows the results are consistent between these two methods. Figure 7 (a) and (b) present the count and weightage results for commercial buildings. 51 10 9 (a) 8 7 6 Count 5 4 3 2 1 0 A2 A4 A9 A1 A3 A10 A6 A7 A11 A5 A8 A12 B1 B3 B5 B7 B4 B6 B8 B2 B9 C4 C5 C1 C6 C13 C7 C8 C11 C14 C17 C3 C10 C15 C18 C2 C9 C12 C16 D1 D4 D9 D2 D6 D7 D11 D3 D10 D5 D8 D12 Variable 50 40 (b) Weightage 30 20 10 0 A2 A9 A4 A3 A1 A10 A7 A11 A6 A5 A8 A12 B1 B3 B5 B7 B6 B4 B8 B2 B9 C5 C4 C1 C6 C13 C7 C17 C8 C11 C3 C13 C14 C18 C2 C9 C12 C16 D1 D4 D9 D2 D7 D6 D11 D10 D3 D13 D8 D5 D12 Variable Figure 6 (a) Count value (Method 1) and (b) weightage value (Method 2) for most important influential variables for residential building applications. (Note: Red columns indicate where more than half of responses include this variable for Method 1) 52 14 12 (a) 10 Count 8 6 4 2 0 A2 A9 A7 A4 A3 A5 A10 A11 A1 A6 A12 A8 B1 B7 B3 B2 B4 B5 B6 B8 B9 C1 C4 C5 C7 C10 C8 C3 C11 C13 C14 C2 C6 C9 C12 C15 C17 C16 C18 D9 D7 D6 D10 D4 D11 D1 D3 D5 D8 D13 D2 D12 Variable 80 70 (b) 60 Weightage 50 40 30 20 10 0 A2 A9 A7 A4 A11 A3 A5 A10 A1 A12 A6 A8 B1 B7 B3 B2 B5 B4 B6 B8 B9 C1 C4 C5 C7 C10 C8 C3 C11 C14 C13 C2 C6 C9 C13 C15 C17 C12 C16 C18 D9 D7 D6 D10 D11 D4 D1 D3 D5 D8 D13 D2 D12 Variable Figure 7 (a) Count value (Method 1) and (b) weightage value (Method 2) for most important influential variables for commercial building applications. (Note: Red columns indicate where more than half of responses include this variable for Method 1) Table 11 shows the survey results for the most important and least important variables and variables that are never mentioned as most or least important for residential buildings. The order in the table represents the importance of these variables. The other variables not presented here are considered in-between variables, being neither most nor least important. Half of the most important variables are never mentioned as the least important variables, and one third of the least important variables are also never mentioned as the most important variables, which shows the results are consistent with each other. In addition, there is no overlap between the most and least important variables. Table 12 summarizes the results of the most important and least important variables in commercial buildings, parallel to Table 11. 53 Table 11 Most and least important variables for residential buildings, and those never chosen as most/least important Most Important Least Important A2. Size (length/width) and shape of test A8. Building envelope type (e.g. brick, siding, EFIS, area batt vs. continuous insulation) C5. Level of motion of occupant(s) A6. Wall, floor, and ceiling color/characteristics D1. Presence of pets B6. Mean radiant temperature (indoor) B1. Lighting level (regardless of source of B4. Indoor humidity light) (lux) C4. Spatial location of occupant(s) C15. Skin color of occupant(s) C1. Number of occupants (including 0) C2. Age of occupant(s) A9. Presence of large objects (especially C16. Body shape of occupant(s) metal objects) within or near a space B3. Presence of interior lighting sources C3. Metabolic rate (non-overhead) A4. Number of doors (entrances/exits) D5. Presence of mylar balloons/party D4. Use of robots D11. Presence of wheelchairs Never Chosen as Most important Never Chosen as Least Important A5. Door sizes (i.e., single vs. double, other) A2. Size (length/width) and shape of test area A8. Building envelope type (e.g., brick, A12. Other building-related variables (please specify) siding, EFIS, etc..) A12. Other building-related variables B1. Lighting level (regardless of source of light) (lux) (please specify) B9. Other (please specify) C1. Number of occupants (including 0) C2. Age of occupant(s) C4. Spatial location of occupant(s) C9. Speed of occupants relative to one C5. Level of motion of occupant(s) another C12. Presence of occupants in adjacent C8. Speed of occupant (e.g., walking vs. running) spaces D9. Repeatability of performance of system (i.e., does C16. Body shape of occupant(s) it provide the same results consistently) 54 Table 12 Most and least important variables for commercial buildings, and those never mentioned as most/least important Most Important Least Important A8. Building envelope type (e.g., brick, siding, A2. Size (length/width) and shape of test area EFIS, etc.) C1. Number of occupants (including 0) B4. Indoor humidity D9. Repeatability of performance of system (i.e., B5. Indoor temperature does it provide the same results consistently) D7. Initial performance/performance over time (e.g., does it take a while for the system to C6. Noise level of occupant(s) learn/work well) B1. Lighting level (regardless of source of light) C16. Body shape of occupant(s) (lux) C4. Spatial location of occupant(s) C2. Age of occupant(s) A9. Presence of large objects (especially metal D1. Presence of pets objects) within or near a space D8. Range of devices (related to C5. Level of motion of occupant(s) electromagnetic interference) B7. Presence of sunlight - direct D4. Use of robots -- D5. Presence of mylar balloons/party items D11. Presence of wheelchairs, strollers, -- shopping carts, Never Chosen as Most important Never Chosen as Least Important B8. Presence of sunlight - diffuse A2. Size (length/width) and shape of test area C12. Presence of occupants in adjacent spaces A4. Number of doors (entrances/exits) C16. Body shape of occupant(s) C1. Number of occupants (including 0) D2. Motion characteristics of pets (if present) C4. Spatial location of occupant(s) C7. Clustering of occupants (distance between -- occupants) 55 Comparing the most important variables for residential buildings with that of commercial buildings, approximately 2/3 of variables are the same, including the size and shape of test area, lighting level, spatial location of occupant(s), presence of large objects, number of occupants, and the level of motion of occupant(s). Similarly, for the least important variables, approximately half are the same for both residential and commercial buildings, including building envelope type, indoor humidity, age of occupant(s), presence of mylar balloons, and presence of wheelchairs. These variables are either difficult to adjust in an existing building, such as the building envelope type, or in most applications, are a rare occurrence, such as balloons that typically are only present during a party. The presence of wheelchairs is also not common, except in specific building types. Indoor humidity in mechanically controlled commercial buildings is typically controlled within a certain range and is generally a variable that sensor system types are not impacted by. For the other important variables, there are differences between residential and commercial buildings. For example, the presence of pets is important for residential buildings but not for commercial, which makes sense given pets are much more likely to be in a home than a commercial building. Commercial buildings are also more likely to have overhead lighting sources as compared to residential which likely include a broader diversity of lighting sources. The use of robots is also a variable that is also more common in homes, specifically sweeping/vacuum robots. The presence of direct sunlight appears to be ranked more importantly for commercial buildings, likely because there are typically more windows in commercial buildings compared to residential, and that a visually comfortable work environment is important. 2.5.2 Most important variables across all variable categories The last question in the survey asked participants to, regardless of the type/category of variables, identify and rank the most/least important variables. Figure 8 shows the weightage 56 values calculated for each variable based on the ranking of respondents in (a) residential buildings, and (b) commercial buildings, indicating that there are large jumps/breaks in rankings among the variables. Approximately half of the variables were not chosen by respondents and thus not shown. In addition, for residential buildings, for 85% of the variables, their weightage values were less than 10, compared to the five most highly ranked variables with a weightage value larger than 22 (shown in red). For commercial buildings, 92% of variables had a weightage value of less than 15, whereas the four top variables had a weightage value of more than 36. 60 60 (b) (a) 50 50 40 Weightage 40 Weightage 30 30 20 20 10 10 0 A2 D1 C1 B1 C4 A9 C5 0 C1 C7 A2 C4 A4 C5 A5 A9 A7 C10 D4 A6 B7 C3 D7 D10 A3 A5 A7 A3 B1 D7 A6 C3 D4 D10 C8 C10 D2 A4 B3 B6 D5 D6 B4 C9 C12 A12 B6 D2 B4 B7 B5 C9 C17 C6 Variable Variable Figure 8 Weightage value for variables considered as the most important in (a) residential and (b) commercial buildings based on the final survey question The final question in the survey asked responders to indicate, across all variables, regardless of category or type, what variables are most important, in rank order. This was used as a check of the above-mentioned variables which were evaluated by category of variable. Figure 9 shows the average ranking across all responses for each of these variables for both residential (Figure 9 a) and commercial (Figure 9 b) buildings. The variables, where none or only one responder provided a ranking, are not included in the analysis. Among the variables with more than one ranking, if a person did not rate that variable, we assume the remaining unrated variables are the next rank after their last rank. For example, if one person rated four variables 1 to 4, the rest were rated are considered as 5. Error bars indicate standard deviation. From the results, we 57 can see that the standard deviation is similar across the different variables. In addition, the average rankings for the most important variables determined in Figure 8 are smaller compared to the others, indicating that they are more important than others. 6 6 Average ranking (1 is 5 Average Rank (1 is 5 4 4 3 3 most important) most important) 2 2 1 1 0 0 A2 D1 C1 B1 C4 A9 C5 D4 C1 C7 A2 C4 C5 A9 B1 Variable Variable (a) (b) Figure 9 Average ranking and standard deviation of variables considered as the most important in (a) residential buildings and (b) commercial buildings based on the final survey question Considering the top variables from Figure 8, the results of this final survey question are very close to the results shown in Table 11 and 12, with the exception of variable C7, Clustering of occupants, for commercial buildings. As such, the most important variables for standard testing and evaluation of occupancy sensor systems for residential and commercial buildings are summarized in Table 13. 58 Table 13 Most important variables for residential and commercial buildings Building type Most important (Tier 1) A2. Size (length/width) and shape of test area C5. Level of motion of occupant(s) D1. Presence of pets B1. Lighting level (regardless of source of light) (lux) C4. Spatial location of occupant(s) Residential buildings C1. Number of occupants (including 0) A9. Presence of large objects (especially metal objects) within or near a space B3. Presence of interior lighting sources (non-overhead) A4. Number of doors (entrances/exits) D4. Use of robots A2. Size (length/width) and shape of test area C1. Number of occupants (including 0) B1. Lighting level (regardless of source of light) (lux) C4. Spatial location of occupant(s) Commercial buildings A9. Presence of large objects (especially metal objects) within or near a space C5. Level of motion of occupant(s) B7. Presence of sunlight - direct C7. Clustering of occupants (distance between occupants) 2.6 Conclusions and Future Work This paper reviewed existing occupancy sensor technologies and summarized the potential influential variables that may cause sensor failures. Using the results of the literature review, the influential variables tested in previous literature were compiled into a comprehensive list, and additional variables were suggested by stakeholder attendees. Then an expert survey was 59 conducted across a diversity of stakeholders on the most and least important variables impacting occupancy sensor performance based on this comprehensive influential variable list. A final list of most and least important variables was determined for both residential and commercial buildings. The results of this work provide insights for sensor manufacturers on what variables industry stakeholders care about most when assessing the performance of occupancy sensor systems, and also provide initial results for use in considering a standard set of variables by which to test occupancy sensor systems for comparative performance evaluation. The limitations of this work include that there are many different influential variables and combinations of variables that can impact occupancy sensor systems, some of which may be more important in some building scenarios as compared to others. It was not attempted to rank the order of importance for different building types, such as hospitals versus schools. It is anticipated that they may be variations in these ranking among building types, as well as the frequency of occurrence of various variables. However, this effort was an attempt to define the most and least important variables for residential and commercial building applications overall. The stakeholders that participated in the survey were also mostly based in the U.S., which may influence results. Moving forward, as climate change challenges becoming increasingly important to address, significant efforts are and will continue to be made to improve the efficiency of buildings, as one of the largest consumers of energy and electricity. Occupancy sensor systems used to more efficiently control the lighting, HVAC, and other systems in a building represent a strong opportunity for energy savings compared to most building operations today. The results of this work will support the development of a standard methodology to test these systems. By having a standard set of variables that a diversity of representative stakeholders suggests are most important to use, the next step is to develop associated methods of testing to evaluate the performance of 60 occupancy sensor systems, which will help to provide a standard method for comparative performance evaluation, to determine which sensor systems work best and where these systems can be improved. Additionally, it is also important to consider that if a reliability issue is identified for a certain type of occupancy sensor system, further efforts are needed to mitigate these to improve the sensor’s reliability and performance. For example, if a sensor system is determined to be sensitive to natural daylight, it may be recommended to not install this sensor system near exterior windows, or additional changes to the sensor system may be made to reduce the sensitivity to daylight. The reliability challenges, however, vary depending on each sensor type, therefore, the specific methods to address each of these issues are also unique to the sensor system and need to be address individually. 61 APPENDIX 62 Appendix: “Occupancy Sensor System Evaluation Methodologies - Stakeholder Feedback” survey Q4: Please categorize the building-related variables, including those that you believe would be most and least important to test for Occupancy presence sensing systems: A1. Glass walls & mirrors (reflective surfaces) A2. Size (length/width) and shape of test area A3. Location/characteristics of windows A4. Number of doors (entrances/exits) A5. Door sizes (i.e., single vs. double, other) A6. Wall, floor, and ceiling color/characteristics A7. Height of ceiling A8. Building envelope type (e.g., brick, siding, EFIS, batt vs. continuous insulation) A9. Presence of large objects (especially metal objects) within or near a space A10. Electromagnetic interference A11. Ventilation rates A12. Other building-related variables (please specify) Q5: Please provide any additional comments or suggestions for building-related variables (e.g., any additional variables/situations that should be considered) Q6: Please categorize the environment-related variables including those that you believe would be most and least important to test for Occupancy presence sensing systems: B1. Lighting level (regardless of source of light) (lux) B2. Spectral distribution of light B3. Presence of interior lighting sources (non-overhead) B4. Indoor humidity B5. Indoor temperature B6. Mean radiant temperature (indoor) B7. Presence of sunlight – direct B8. Presence of sunlight – diffuse B9. Other environment-related variables (please specify) Q7: Please provide any additional comments or suggestions for environment-related variables (e.g. any additional variables/situations that should be considered): 63 Q8: Please categorize the occupant-related variables including those that you believe would be most and least important to test for Occupancy presence sensing systems: C1. Number of occupants (including 0) C2. Age of occupant(s) C3. Metabolic rate C4. Spatial location of occupant(s) C5. Level of motion of occupant(s) C6. Noise level of occupant(s) C7. Clustering of occupants (distance between occupants) C8. Speed of occupant (e.g. walking vs. running) C9. Speed of occupants relative to one another C10. Number of occupants entering/exiting a room at the same time C11. Speed of occupants entering/exiting a room at the same time C12. Presence of occupants in adjacent spaces C13. Clothing color/contrast/patterns (including images on clothing) of occupants C14. Occupant clothing level (clo) C15. Skin color of occupant(s) C16. Body shape of occupant(s) C17. Occupants wearing heavy winter coats with a cold surface temperature C18. Other occupant-related variables (please specify) Q9: Please provide any additional comments or suggestions for occupant-related variables (e.g., any additional variables/situations that should be considered): Q10: Please choose other variables that you believe would be the most/least important to evaluate for Occupancy presence sensing systems: D1. Presence of pets D2. Motion characteristics of pets (if present) D3. Size/type of pets D4. Use of robots D5. Presence of mylar balloons/party items D6. Ability to communicate with thermostat D7. Initial performance/performance over time (e.g., does it take a while for the system to learn/work well) D8. Range of devices (related to electromagnetic interference) D9. Repeatability of performance of system (i.e., does is provide the same results consistently) D10. Presence of heat sources such as a heat lamp D11. Presence of wheelchairs, strollers, shopping carts, etc. D12. Other variables (please specify) 64 Q11: Please provide any additional comments or suggestions for other variables: Q12: Above you selected important variables for environment, building, occupant, and other categories. Of those that you felt were the most important, please list below the topmost important variables across all categories that should be tested for occupancy presence sensor systems used in residential buildings AND occupancy counting systems used in commercial buildings 65 REFERENCES 66 REFERENCES Ahmad, J., H. Larijani, R. Emmanuel, M. Mannion, A. Javed & A. Ahmadinia. 2018. An intelligent real-time occupancy monitoring system with enhanced encryption and privacy. 2018 IEEE 17th International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC), Berkeley, CA, July 16-18, pp. 524-529. Anand, P., D. Cheong, C. Sekhar, M. Santamouris and S. Kondepudi. 2019. Energy saving estimation for plug and lighting load using occupancy analysis. Renewable Energy, 143, pp.1143-1161. Anderson, D., R.H. Luke, J.M. Keller, M. Skubic, M. Rantz and M. Aud. 2009. Linguistic summarization of video for fall detection using voxel person and fuzzy logic. Computer vision and image understanding, 113(1), pp.80-89. ARPA-E’s SENSOR Program. 2017. Saving Energy Nationwide in Structures with Occupancy Recognition. https://arpa-e.energy.gov/technologies/programs/sensor. ASHRAE. 2017. ANSI/ASHRAE Standard 55-2017, Thermal Environmental Conditions for Human Occupancy. Atlanta: ASHRAE. ASHRAE. 2019. ANSI/ASHRAE Standard 62.1, Ventilation and Acceptable Indoor Air Quality in Residential Buildings. Atlanta: ASHRAE. Balaji, B., J. Xu, A. Nwokafor, R. Gupta and Y. Agarwal. 2013. November. Sentinel: occupancy-based HVAC actuation using existing Wi-Fi infrastructure within commercial buildings. Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems, Roma, Italy, November 11-15, pp. 1-14. Baronti, P., P. Barsocchi, S. Chessa, F. Mavilia and F. Palumbo. 2018. Indoor Bluetooth low energy dataset for localization, tracking, occupancy, and social interaction. Sensors 18(12), pp. 4462. Barsocchi, P., A. Crivello, M. Girolami, F. Mavilia and F. Palumbo. 2017. Occupancy detection by multi-power Bluetooth low energy beaconing. 2017 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Sapporo, Japan, September 18-21, pp. 1-6. Basnayaka, D.A. and H. Haas. 2017. Design and analysis of a hybrid radio frequency and visible light communication system. IEEE Transactions on Communications, 65(10), pp. 4334- 4347. Cao, X., X. Dai and J. Liu. 2016. Building energy-consumption status worldwide and the state-of-the-art technologies for zero-energy buildings during the past decade. Energy and buildings, 128, pp.198-213. Chen, J., H. Chen, and X. Luo. 2019. Collecting building occupancy data of high resolution based on Wi-Fi and BLE network. Automation in Construction, 102, pp.183-194. 67 Chen, Z., C. Jiang, and L. Xie. 2018. Building occupancy estimation and detection: A review. Energy and Buildings, 169, pp.260-270. Chen, Z., M.K. Masood and Y.C. Soh. 2016. A fusion framework for occupancy estimation in office buildings based on environmental sensor data. Energy and Buildings, 133, pp.790-798. Chowdhury, M.F., S.Z. Ali, S. Boual, R. Hopper and F. Udrea. 2016. Development of plasmonic MEMS CMOS infrared sensors for occupancy detection. 2016 46th European Solid- State Device Research Conference (ESSDERC), Lausanne, Switzerland, September 12-15, pp. 97-100. Christensen, C., R. Anderson, S. Horowitz, A. Courtney and J. Spencer. 2006. BEopt (TM) Software for Building Energy Optimization: Features and Capabilities. Report No. TP-550- 39929, National Renewable Energy Laboratory (NREL), Golden, CO. Constenla, D., C. Garcia and N. Lefcourt. 2015. Assessing the economics of dengue: results from a systematic review of the literature and expert survey. Pharmacoeconomics, 33(11), pp.1107-1135. Conte, G., M. De Marchi, A.A. Nacci, V. Rana and D. Sciuto. 2014. BlueSentinel: a first approach using iBeacon for an energy efficient occupancy detection system. BuildSys@ SenSys, Memphis, TN, November 3-6, pp. 11-19. Corna, A., L. Fontana, A.A. Nacci and D. Sciuto. 2015. Occupancy detection via iBeacon on Android devices for smart building management. 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, Germany, March 14-18, pp. 629-632. Deru, M., K. Field, D. Studer, K. Benne, B. Griffith, P. Torcellini, B. Liu, M. Halverson, D. Winiarski, M. Rosenberg and M. Yazdanian. 2011. US Department of Energy commercial reference building models of the national building stock. Diraco, G., A. Leone and P. Siciliano. 2015. People occupancy detection and profiling with 3D depth sensors for building energy management. Energy and Buildings, 92, pp.246-266. Diraco, G., A. Leone and P. Siciliano. 2017. A radar-based smart sensor for unobtrusive elderly monitoring in ambient assisted living applications. Biosensors, 7(4), p.55. Dodier, R.H., G.P. Henze, D.K. Tiller and X. Guo. 2006. Building occupancy detection through sensor belief networks. Energy and buildings, 38(9), pp.1033-1043. DOE, U., 2018. Commercial Prototype Building Models. https://www.energycodes.gov/development/commercial/prototype_models. Ekwevugbe, T., N. Brown, V. Pakka and D. Fan. 2013. Real-time building occupancy sensing using neural-network based sensor network. 2013 7th IEEE International Conference on Digital Ecosystems and Technologies (DEST), Stanford, CA, July 24-26, pp. 114-119. 68 Erickson, V.L., Y. Lin, A. Kamthe, R. Brahme, A. Surana, A.E. Cerpa, M.D. Sohn, and S. Narayanan. 2009. Energy efficient building environment control strategies using real-time occupancy measurements. Proceedings of the First ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings, Berkeley, CA, November 3, pp. 19-24. Filippoupolitis, A., W. Oliff and G. Loukas. 2016. Bluetooth low energy-based occupancy detection for emergency management. 2016 15th International Conference on Ubiquitous Computing and Communications and 2016 International Symposium on Cyberspace and Security (IUCC-CSS), Granada, Spain, December 14-16, pp. 31-38. Gennarelli, G., G. Ludeno and F. Soldovieri. 2016. Real-time through-wall situation awareness using a microwave Doppler radar sensor. Remote Sensing, 8(8), p.621. Gulmezoglu, B., M.B. Guldogan and S. Gezici. 2014. Multiperson tracking with a network of ultrawideband radar sensors based on Gaussian mixture PHD filters. IEEE Sensors Journal, 15(4), pp.2227-2237. Hadley, D.L. 1993. Daily variations in HVAC system electrical energy consumption in response to different weather conditions. Energy and buildings, 19(3), pp.235-247. Hailemariam, E., R. Goldstein, R. Attar and A. Khan. 2011. Real-time occupancy detection using decision trees with multiple sensor types. Proceedings of the 2011 Symposium on Simulation for Architecture and Urban Design, Boston, MA, April 3-7, pp. 141-148. Hnat, T.W., E. Griffiths, R. Dawson and K. Whitehouse. 2012. Doorjamb: unobtrusive room- level tracking of people in homes using doorway sensors. Proceedings of the 10th ACM Conference on Embedded Network Sensor Systems, Ontario, Canada, November 6-9, pp. 309- 322. Hobson, B.W., D. Lowcay, H.B. Gunay, A. Ashouri and G.R. Newsham. 2019. Opportunistic occupancy-count estimation using sensor fusion: A case study. Building and environment, 159, p.106154. Hong, T., S.C. Taylor-Lange, S. D’Oca, D. Yan and S.P. Corgnati. 2016. Advances in research and applications of energy-related occupant behavior in buildings. Energy and buildings, 116, pp.694-702. IEA EBC, 2018. Annex 79 - Occupant-Centric Building Design and Operation. https://annex79.iea-ebc.org/. IEEE. 2005. IEEE 802.15.1-2005, IEEE Standard for Information technology-- Local and metropolitan area networks-- Specific requirements-- Part 15.1a: Wireless Medium Access Control (MAC) and Physical Layer (PHY) specifications for Wireless Personal Area Networks (WPAN). IEEE. 2005. IEEE 802.15.1-2005, IEEE Standard for Information technology-- Local and metropolitan area networks-- Specific requirements-- Part 15.1a: Wireless Medium Access 69 Control (MAC) and Physical Layer (PHY) specifications for Wireless Personal Area Networks (WPAN). Jiang, C., M.K. Masood, Y.C. Soh and H. Li. 2016. Indoor occupancy estimation from carbon dioxide concentration. Energy and Buildings, 131, pp.132-141. Jiefan, G., X. Peng, P. Zhihong, C. Yongbao, J. Ying and C. Zhe. 2018. Extracting typical occupancy data of different buildings from mobile positioning data. Energy and Buildings, 180, pp.135-145. Jung, W. and F. Jazizadeh. 2019. Human-in-the-loop HVAC operations: A quantitative review on occupancy, comfort, and energy-efficiency dimensions. Applied Energy, 239, pp.1471-1508. Kaushik AR. 2006. Characterization of passive infrared sensors for monitoring occupancy pattern. 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, August 30–September 3, pp. 5257-5260. Khalil, N., D. Benhaddou, O. Gnawali and J. Subhlok. 2018. Nonintrusive ultrasonic-based occupant identification for energy efficient smart building applications. Applied Energy, 220, pp.814-828. Khan, M.A.A.H., H.S. Hossain and N. Roy. 2015. Sensepresence: Infrastructure-less occupancy detection for opportunistic sensing applications. 2015 16th IEEE International Conference on Mobile Data Management, Pittsburgh, PA, June 15-18, pp. 56-61. Kilic, Y., H. Wymeersch, A. Meijerink, M.J. Bentum and W.G. Scanlon. 2013. An experimental study of UWB device-free person detection and ranging. 2013 IEEE International Conference on Ultra-Wideband (ICUWB), Sydney, Australia, September 15-18, pp. 43-48. Kim, C., J.Y. Lee, T. Cho, D. Ki, B.H. Cho and J. Yoon. 2013. Multi-target localization of breathing humans. 2013 IEEE International Conference on Ultra-Wideband (ICUWB), Sydney, Australia, pp. 49-54. Kim, N.S. and J.M. Rabaey. 2016. A high data-rate energy-efficient triple-channel UWB- based cognitive radio. IEEE Journal of Solid-State Circuits, 51(4), pp.809-820. Kuutti, J., P. Saarikko and R.E. Sepponen. 2014. Real time building zone occupancy detection and activity visualization utilizing a visitor counting sensor network. 2014 11th international conference on remote engineering and virtual instrumentation (REV), Porto, Portugal, February 26-28, pp. 219-224. Kwolek, B. and M. Kepski. 2015. Improving fall detection by the use of depth sensor and accelerometer. Neurocomputing, 168, pp.637-645. Labeodan, T., R. Maaijen and W. Zeiler. 2013. The human behavior: a tracking system to follow the human occupancy. Proceedings of the international conference on cleantech for smart cities and buildings (CISBAT 2013), Lausanne, Switzerland, September 4-6, Vol. 4, p. 6. 70 Launer, S., Zakis, J.A. and Moore, B.C., 2016. Hearing aid signal processing. Hearing aids, pp.93-130. Li, N., Calis, G. and B. Becerik-Gerber. 2012. Measuring and monitoring occupancy with an RFID based system for demand-driven HVAC operations. Automation in construction, 24, pp.89-99. Li, S., Li, N., B. Becerik-Gerber and G. Calis. 2011. Rfid-based occupancy detection solution for optimizing hvac energy consumption. Proceedings of the 28th International Association for Automation and Robotics in Construction, Seoul, Korea, June 29-July 2, pp. 587-592. Liao, C. and P. Barooah. 2010. An integrated approach to occupancy modeling and estimation in commercial buildings. American Control Conference (ACC), Baltimore, MA, June 30-July 2, pp. 3130-3135. Liao, C., Y. Lin and P. Barooah. 2012. Agent-based and graphical modelling of building occupancy. Journal of Building Performance Simulation, 5(1), pp.5-25. Liu, H., Y. Wang, K. Wang and H. Lin. 2017. Turning a pyroelectric infrared motion sensor into a high-accuracy presence detector by using a narrow semi-transparent chopper. Applied Physics Letters, 111(24), p.243901. Longo, E., Redondi, A.E. and Cesana, M., 2019. Accurate occupancy estimation with WiFi and bluetooth/BLE packet capture. Computer Networks, 163, p.106876. Lu, H., W. Pan, N.D. Lane, T. Choudhury and A.T. Campbell. 2009. SoundSense: scalable sound sensing for people-centric applications on mobile phones. Proceedings of the 7th international conference on Mobile systems, applications, and services, Kraków, Poland, June 22-25, pp. 165-178. Lu, X., H. Wen, H. Zou, H. Jiang, L. Xie and N. Trigoni. 2016. Robust occupancy inference with commodity WiFi. 2016 IEEE 12th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), New York, NY, October 17-19, pp. 1-8. Ma, R., F. Hu and Q. Hao. 2016. Active compressive sensing via pyroelectric infrared sensor for human situation recognition. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(12), pp.3340-3350. Maaspuro, M., 2018. Infrared occupancy detection technologies in building automation-a review. ARPN Journal of Engineering and Applied Sciences, 13(19), pp.8055-8068. Mabrouk, M., S. Rajan, M. Bolic, I. Batkin, H.R. Dajani and V.Z. Groza. 2014. Detection of human targets behind the wall based on singular value decomposition and skewness variations. 2014 IEEE Radar Conference, Cincinnati, OH, May 19-23, pp. 1466-1470. Mahmoud, M.S. and A.A. Mohamad. 2016. A study of efficient power consumption wireless communication techniques/modules for internet of things (IoT) application. 71 Mendon, V.V. and Z.T. Taylor. 2014. Development of Residential Prototype Building Models and Analysis System for Large-Scale Energy Efficiency Studies Using EnergyPlus. Publication No. PNNL-SA-101622), Pacific Northwest National Lab.(PNNL), Richland, WA. Menyhart, K. and Krarti, M., 2017. Potential energy savings from deployment of Dynamic Insulation Materials for US residential buildings. Building and Environment, 114, pp.203-218. Meyn, S., A. Surana, Y. Lin, S.M. Oggianu, S. Narayanan and T.A. Frewen. 2009. A sensor- utility-network method for estimation of occupancy in buildings. Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, Shanghai, China, December 15-18, pp. 1494-1500. Mitra, D., N. Steinmetz, Y. Chu and K.S. Cetin. 2020. Typical occupancy profiles and behaviors in residential buildings in the United States. Energy and Buildings, 210, p.109713. Munir, S., R.S. Arora, C. Hesling, J. Li, J. Francis, C. Shelton, C. Martin, A. Rowe and M. Berges. 2017. Real-time fine grained occupancy estimation using depth sensors on ARM embedded platforms. 2017 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS), Pittsburgh, PA, April 18-21, pp. 295-306. NEMA. 2016. NEMA WD 7- 2011 (R-2016), Occupancy Motion Sensors Standard. Arlington: NEMA. Ni, L.M., Y. Liu, Y.C. Lau and A.P. Patil. 2003. LANDMARC: indoor location sensing using active RFID. Proceedings of the First IEEE International Conference on Pervasive Computing and Communications (PerCom 2003), Fort Worth, TX, March 26-26, pp. 407-415. Ossberger, G., T. Buchegger, E.R.W.I.N. Schimback, A. Stelzer and R. Weigel. 2004. Non- invasive respiratory movement detection and monitoring of hidden humans using ultra wideband pulse radar. 2004 International Workshop on Ultra Wideband Systems Joint with Conference on Ultra-Wideband Systems and Technologies. Joint UWBST & IWUWBS 2004 (IEEE Cat. No. 04EX812), Kyoto, Japan, May 18-21, pp. 395-399. Ouf, M.M., Issa, M.H., Azzouz, A. and Sadick, A.M., 2017. Effectiveness of using WiFi technologies to detect and predict building occupancy. Sustainable buildings, 2, pp.1-10. Palipana, S., P. Agrawal and D. Pesch. 2016. Channel state information based human presence detection using non-linear techniques. Proceedings of the 3rd ACM International Conference on Systems for Energy-Efficient Built Environments, New York, NY, November 16- 17, pp. 177-186. Park, J.Y., T. Dougherty and Z. Nagy. 2018. A Bluetooth based occupancy detection for buildings. Proceedings of Building Performance Analysis Conference and SimBuild, Chicago, IL, September 26-28, pp. 807-814. Pedersen, T.H., Nielsen, K.U. and Petersen, S., 2017. Method for room occupancy detection based on trajectory of indoor climate sensor data. Building and Environment, 115, pp.147-156. 72 Petrovic, T., Echigo, K. and Morikawa, H., 2018. Detecting presence from a WiFi router’s electric power consumption by machine learning. IEEE Access, 6, pp.9679-9689. Putra, G.D., A.R. Pratama, A. Lazovik and M. Aiello. 2017. Comparison of energy consumption in Wi-Fi and bluetooth communication in a Smart Building. 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, January 9- 11, pp. 1-6. Rahman, T., A.T. Adams, R.V. Ravichandran, M. Zhang, S.N. Patel, J.A. Kientz and T. Choudhury. 2015. Dopplesleep: A contactless unobtrusive sleep sensing system using short- range doppler radar. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka, Japan, September 7-11, pp. 39-50. Ranjan, J., Y. Yao, E. Griffiths and K. Whitehouse. 2012. Using mid-range RFID for location based activity recognition. Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, PA, September 5-8, pp. 647-648. Ravichandran, R., E. Saba, K.Y. Chen, M. Goel, S. Gupta and S.N. Patel. 2015. WiBreathe: Estimating respiration rate using wireless signals in natural settings in the home. 2015 IEEE International Conference on Pervasive Computing and Communications (PerCom), St. Louis, MO, March 23-27, pp. 131-139. Rayanchu, S., A. Patro and S. Banerjee. 2011. Airshark: detecting non-WiFi RF devices using commodity WiFi hardware. Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference, Berlin, Germany, November 2-4, pp. 137-154. Raykov, Y.P., E. Ozer, G. Dasika, A. Boukouvalas and M.A. Little. 2016. Predicting room occupancy with a single passive infrared (PIR) sensor through behavior extraction. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany, September 12-16, pp. 1016-1027. Ryu, S.H. and Moon, H.J., 2016. Development of an occupancy prediction model using indoor environmental data based on machine learning techniques. Building and Environment, 107, pp.1-9. Santra, A., Ulaganathan, R.V. and Finke, T., 2018. Short-range millimetric-wave radar system for occupancy sensing application. IEEE sensors letters, 2(3), pp.1-4. Sekine, M., K. Maeno and T. Kamakura. 2012. Detection of human presence by irregularity of doppler signal using multivariate autoregressive model. Proc. 2012 Health Informatics Workshop, Beijing, China, August 12, pp. 1-6. Shen, W. and G. Newsham. 2016. Smart phone-based occupancy detection in office buildings. 2016 IEEE 20th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Nanchang, China, May 4-6, pp. 632-636. 73 Shen, W., Newsham, G. and Gunay, B., 2017. Leveraging existing occupancy-related data for optimal control of commercial office buildings: A review. Advanced Engineering Informatics, 33, pp.230-242. Shih O, Rowe A. Occupancy estimation using ultrasonic chirps. 2015. Proceedings of the ACM/IEEE Sixth International Conference on Cyber-Physical Systems, Seattle, WA, April 14-16, pp. 149-158. Shih, O., P. Lazik and A. Rowe. 2016. AURES: A wide-band ultrasonic occupancy sensing platform. Proceedings of the 3rd ACM international conference on systems for energy-efficient built environments, Palo Alto, CA, November 16-17, pp. 157-166. Singh, S., Q. Liang, D. Chen and L. Sheng. 2011. Sense through wall human detection using UWB radar. EURASIP Journal on Wireless Communications and Networking, 2011(1), pp.1-11. Teixeira, T., G. Dublon and A. Savvides. 2010. A survey of human-sensing: Methods for detecting presence, count, location, track, and identity. ACM Computing Surveys, 5(1), pp.59-69. Teizer, J., C.H. Caldas and C.T. Haas. 2007. Real-time three-dimensional occupancy grid modeling for the detection and tracking of construction resources. Journal of Construction Engineering and Management, 133(11), pp.880-888. Tomastik, R., Y. Lin and A. Banaszuk. 2008. Video-based estimation of building occupancy during emergency egress. 2008 American Control Conference, Seattle, WA, June 11-13, pp. 894- 901. Vasisht, D., Kumar, S. and Katabi, D. 2016. Decimeter-level localization with a single WiFi access point. 13th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 16), Santa Clara, CA, March 16-18, pp. 165-178. Vattapparamban, E., B.S. Çiftler, I. Güvenç, K. Akkaya and A. Kadri. 2016. Indoor occupancy tracking in smart buildings using passive sniffing of probe requests. 2016 IEEE International Conference on Communications Workshops (ICC), Kuala Lumpur, Malaysia, May 23-27, pp. 38-44. Wagner, A., O’Brien, W. and Dong, B., 2018. Exploring occupant behavior in buildings. Wagner, A., O’Brien, W., Dong, B., Eds. Wang, J., J. Xiong, H. Jiang, X. Chen and D. Fang. 2017. D-watch: Embracing “bad” multipaths for device-free localization with cots rfid devices. IEEE/ACM Transactions on Networking, 25(6), pp.3559-3572. Wang, W., J. Chen and T. Hong. 2018. Occupancy prediction through machine learning and data fusion of environmental sensing and Wi-Fi sensing in buildings. Automation in Construction, 94, pp.233-243. 74 Wang, W., J. Chen and X. Song. 2017. Modeling and predicting occupancy profile in office space with a Wi-Fi probe-based Dynamic Markov Time-Window Inference approach. Building and Environment, 124, pp.130-142. Wang, W., J. Chen, G. Huang and Y. Lu. 2017. Energy efficient HVAC control for an IPS- enabled large space in commercial buildings through dynamic spatial occupancy distribution. Applied Energy, 207, pp.305-323. Wang, W., J. Chen, T. Hong and N. Zhu. 2018. Occupancy prediction through Markov based feedback recurrent neural network (M-FRNN) algorithm with WiFi probe technology. Building and Environment, 138, pp.160-170. Wilson, J. and N. Patwari. 2010. See-through walls: Motion tracking using variance-based radio tomography networks. IEEE Transactions on Mobile Computing, 10(5), pp.612-621. Wilzeck, A., M.P. Guirao, E. Dimitrov. UWB Technology and Regulation. https://ecfsapi.fcc.gov/file/60001090750.pdf. Wu, L. and Y. Wang. 2018. A low-power electric-mechanical driving approach for true occupancy detection using a shuttered passive infrared sensor. IEEE Sensors Journal, 19(1), pp.47-57. Xi, W., J. Zhao, X.Y. Li, K. Zhao, S. Tang, X. Liu and Z. Jiang. 2014. Electronic frog eye: Counting crowd using WiFi. IEEE INFOCOM 2014-IEEE Conference on Computer Communications, Toronto, Canada, April 27-May 2, pp. 361-369. Xu, C., S. Li, G. Liu, Y. Zhang, E. Miluzzo, Y.F. Chen, J. Li and B. Firner. 2013. Crowd++ unsupervised speaker count with smartphones. Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, Zurich, Switzerland, September 8-12, pp. 43-52. Xu, G., P. Sharma, X. Hui and E.C. Kan. 2021. 3-D Indoor Device-Free Object Detection by Passive Radio Frequency Identification. IEEE Transactions on Instrumentation and Measurement, 70, pp.1-13. Yan, D., T. Hong, B. Dong, A. Mahdavi, S. D’Oca, I. Gaetani and X. Feng. 2017. IEA EBC Annex 66: Definition and simulation of occupant behavior in buildings. Energy and Buildings, 156, pp.258-270. Yang, J., A. Pantazaras, K.A. Chaturvedi, A.K. Chandran, M. Santamouris, S.E. Lee and K.W. Tham. 2018. Comparison of different occupancy counting methods for single system- single zone applications. Energy and Buildings, 172, pp.221-234. Yang, Z., Li, N., B. Becerik-Gerber and M. Orosz, 2014. A systematic approach to occupancy modeling in ambient sensor-rich buildings. Simulation, 90(8), pp.960-977. Yarovoy, A.G., L.P. Ligthart, J. Matuzas and B. Levitas. 2006. UWB radar for human being detection. IEEE Aerospace and Electronic Systems Magazine, 21(3), pp.10-14. 75 Yavari, E., A. Lee, K. Pang, N.A. McCabe and O. Boric-Lubecke. 2014. Radar and conventional occupancy sensors performance comparison. 2014 Asia-Pacific Microwave Conference, Sendai, Japan, November 4-7, pp. 444-446. Yavari, E., C. Song, V. Lubecke and O. Boric-Lubecke. 2014. Is there anybody in there?: Intelligent radar occupancy sensors. IEEE Microwave Magazine, 15(2), pp.57-64. Yavari, E., H. Jou, V. Lubecke and O. Boric-Lubecke. 2013. Doppler radar sensor for occupancy monitoring. 2013 IEEE Topical Conference on Power Amplifiers for Wireless and Radio Applications, Austin, TX, January 20, pp. 145-147. Yavari, E., X. Gao and O. Boric-Lubecke. 2018. Subject Count Estimation by Using Doppler Radar Occupancy Sensor. 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI. July 17-21, pp. 4428-4431. Yoshino, H., T. Hong and N. Nord. 2017. IEA EBC annex 53: Total energy use in buildings—Analysis and evaluation methods. Energy and Buildings, 152, pp.124-136. Yun, J. and S.S. Lee. 2014. Human movement detection and identification using pyroelectric infrared sensors. Sensors, 14(5), pp.8057-8081. Zappi, P., E. Farella and L. Benini. 2007. Enhancing the spatial resolution of presence detection in a PIR based wireless surveillance network. 2007 IEEE Conference on Advanced Video and Signal Based Surveillance, London, UK, September 5-7, pp. 295-300. IEEE. Zikos, S., A. Tsolakis, D. Meskos, A. Tryferidis and D. Tzovaras. 2016. Conditional Random Fields-based approach for real-time building occupancy estimation with multi-sensory networks. Automation in Construction, 68, pp.128-145. Zou, H., H. Jiang, J. Yang, L. Xie and C. Spanos. 2017. Non-intrusive occupancy sensing in commercial buildings. Energy and Buildings, 154, pp.633-643. Zou, H., Y. Zhou, J. Yang and C.J. Spanos. 2018. Device-free occupancy detection and crowd counting in smart buildings with WiFi-enabled IoT. Energy and Buildings, 174, pp.309- 322. Zou, J., Q. Zhao, W. Yang and F. Wang. 2017. Occupancy detection in the office by analyzing surveillance videos and its application to building energy conservation. Energy and Buildings, 152, pp.385-398. 76 CHAPTER 3 – DEVELOPMENT AND TESTING OF A PERFORMANCE EVALUATION METHODOLOGY TO ASSESS THE RELIABILITY OF OCCUPANCY SENSOR SYSTEMS IN RESIDENTIAL BUILDINGS 3.1 Abstract With the emergence of advanced occupancy sensor technologies to better detect occupancy in buildings, a universal methodology and metrics are required to evaluate and report sensor systems’ reliability and compare the performance across multiple sensor systems. This research presents a methodology to assess the reliability of occupancy sensor systems in residential buildings in a controlled laboratory environment, including both “typical” and “failure” testing scenarios. The developed methodology was then implemented to evaluate a novel occupancy detection sensor system’s reliability. “Typical” testing evaluates the overall accuracy of the sensor system, which suggest how reliable the occupancy sensor system is over time. Results show that on average, the precision and recall are 0.75 and 0.70, indicating similar numbers of false positives and false negatives across the dataset. The overall accuracy of the tested sensor system was 62.4% to 76.4%. Failure testing results indicate whether there are influential variables impacting the sensor performance. For the tested sensor system, the number of occupants, presence of large objects, presence of interior light sources, and number of doors are not influential, while lighting level, location of occupants, additional door in the entry/exit area, and having the TV on are variables determined to impact the sensor system performance. Keywords: Sensor reliability; Standard testing methodology; Typical testing; Failure testing; Performance metrics; Occupancy sensor systems 77 3.2 Introduction Buildings account for approximately 40% of the total energy use in the U.S. (U.S. EIA, 2021). In residential buildings, heating and cooling energy consumption consists, on average, of more than 50% of this energy use among the different energy end uses (RECS, 2015). Therefore, it is of importance to develop energy saving approaches to help reduce heating and cooling energy use in residential buildings. Energy consumption in residential buildings is strongly impacted by occupants and their energy use patterns (Hong et al., 2016). There have been an increasing number of studies focusing on the development of more advanced occupancy sensor systems to better collect occupancy information such that this occupancy related data could be used to inform building controls. For example, Razavi et al. (2019) proposed a genetic programming approach to predict the present and future home-occupancy status of households based on high-frequency meter data. Tan et al. (2022) used an ensemble method to combine sensor data from different data modalities, including environmental sensors (temperature, humidity, and illuminance), image sensors, and acoustic sensors to predict occupancy information in residential buildings. Such controls can help to target opportunities to reduce energy use and improve operational efficiency. For example, occupancy sensors may be connected to lighting systems to detect the presence/non-presence of people to determine when to turn lights on or off (Nagy et al., 2015). More recently, occupancy sensors have been considered for HVAC (heating, ventilation, and air conditioning) control applications. Turley et al. (2020) proposed non-probabilistic occupancy models developed based on occupancy data derived from occupancy sensors incorporated in smart thermostats to provide occupant-based controls in residential buildings in Colorado. These thermostats generally include a “home” and “away” mode, depending on whether the sensor detects movement or not. In “away” mode, 78 occupants are considered to be outside of the home and thus the setpoint temperature is adjusted to unoccupied mode, to reduce energy consumption. These thermostats generally include a “home” and “away” mode, depending on whether the sensor detects movement or not. In “away” mode, occupants are considered to be outside of the home and thus the setpoint temperature is adjusted to unoccupied mode, to reduce energy consumption. Esrafilian-Najafabadi and Haghighat (2021) used deep learning algorithms to provide dynamic estimations of preconditioning time and future occupancy patterns as a function of occupancy, indoor temperature, and weather data to make control decisions in residential buildings. However, occupancy patterns, particularly in residential buildings, can be highly unpredictable, varying significantly from person to person, and across households and building types. In a prior study (Chu et al., 2021a), more than 50 different variables were identified that may impact the ability of an occupancy sensor system to detect occupants. While occupancy detection methods have been the focus of increasing interest in recent years, there are opportunities to improve the accuracy of existing technologies. Sensor systems can incorrectly provide occupancy information under various scenarios and edge cases. For example, for occupancy detection sensors, most typically used in residential buildings, this can be either a false positive reading (i.e., the sensor system indicates there is an occupant(s) when there is not), or false negative (i.e., the sensor system cannot detect an occupant when there is one or more present). However, the methodology used to evaluate reliability of performance, and the metrics used to report this are not uniform across the literature. Without an established standard method to evaluate the performance of these sensor systems, it is not possible to objectively evaluate and compare the performance of multiple sensor systems. To address this challenge, therefore, the focus of this research is on the development of a universal evaluation methodology to test the reliability of 79 occupancy sensor systems in residential buildings. As a case study, the developed testing methodology was implemented to evaluate a novel occupancy detection sensor system in a controlled laboratory environment. This research is organized as follows. Section 2 presents the proposed standard methodology to test the reliability of occupancy sensor systems, while Section 3 describes the novel occupancy detection sensor system developed and evaluated. The case study results of applying the developed reliability evaluation methodology to the occupancy detection sensor system are detailed in Section 4. Conclusions and discussion are provided in Section 5. 3.3 Evaluation Methodology for Sensor System Reliability The proposed methodology for testing the reliability of occupancy sensor systems in residential buildings includes both “typical” testing and “failure” testing. For typical scenarios, the occupancy sensor system is tested with real occupants following occupancy schedules and activity profiles that represent typical conditions in a controlled residential environment, to determine the frequency of failures. The purpose of typical testing is to evaluate the accuracy and reliability of each occupancy sensor system under real occupancy scenarios over time. For the “failure” testing, the performance of the occupancy sensor systems is determined by testing the impact of a range of variables to determine if they are influential or not, using the one variable at a time (OVAT) testing method (Hasan et al., 2016). When a single variable is considered, all other variables are fixed at standard level, while the variable being evaluated is varied across a range of values to assess how it impacts performance. The purpose of this component of testing is to identify which variable(s) may cause sensing failures, and thus influence the performance of the occupancy sensor system. 80 3.3.1 Typical testing Occupancy schedules under typical scenarios were first created, utilizing American Time Use Survey (ATUS) data (BLS 2020) as it collects activity data for people in the United States, including periods they spend at home, in residential buildings. The activity data of participants from the ATUS was then mapped to the presence or absence of occupants in residential buildings. This data was also classified based on occupant characteristics including their age group, whether it is weekday or weekend, and types of households. Three most common types of occupancy profiles, including “Day absence”, “Stay at home”, and “Night absence” profiles, were obtained from our prior study (Mitra et al., 2021), which represents approximately 88% of people in the United States. Based on these most common types of residential occupancy profiles, each was selected for use in representing typical occupancy profiles of residential buildings to evaluate the reliability of occupancy sensors systems in this research. The occupancy schedules from the ATUS-derived data were then used to compare to the raw data in the ATUS database, from which participant’s activity data was selected that most closely matched occupancy profiles. The criteria used for matching was calculated as the participant’s activity data that had the least difference in occupancy fraction as compared to the three previously identified typical individual profiles. The detailed activity information for the selected occupant IDs was then extracted from ATUS dataset, which forms “Sche 1”, “Sche 2” and “Sche 3”. These typical profiles are individual schedules obtained among all types of households based on ATUS data. For the evaluation of the performance of occupancy sensor systems in homes, typical occupancy schedules in households need to be then developed. Based on RECS data (2020), 1-member households and 2-member households account for around 64% of the total amount of 81 households. Therefore, these two types of households were chosen in this work. For 1-member households, the three typical profiles were directly selected. For 2-member households, the three personal schedules were combined to form 2-person household schedules. Among these three typical profiles, the portion of household members following Sche 2 “Night absence” is relatively small, compared to the others (Mitra et al., 2021). Thus, for 2-person households, only “Sche 1” and “Sche 3” were used. “Sche 4” includes two people following “day absence”; and “Sche 5” includes one person following “day absence” and the other following “stay-at-home”. The combined schedule when two people follow “Stay-at-home” was not used in this research since it has less varied occupancy scenarios for use in evaluating sensor reliability. The five typical household profiles are summarized in Table 14. Table 14 Five typical household types and associated occupancy profiles Item Household Type Occupancy Profile Sch1 1-person household "Day absence" Sch2 1-person household "Night absence" Sch3 1-person household "Stay-at-home" Sch4 2-person household "Day absence" for both household members "Day absence" for one person; Sch5 2-person household "Stay-at-home" for the other The detailed activity profiles for these five typical occupancy scenarios were then created for use in evaluating occupancy sensor systems under a range of typical conditions. As an example, Table 14 shows a sample of Sch1 over a 24-hour period, in the column “24-hour duration”, which represents the ATUS-defined schedule across this period. ATUS collects occupancy data from 4 am of one day to 4 am of the following day (e.g., Table 15) with a time interval of 5-minute. These 82 occupancy profiles are shown in Figure 10. All typical occupancy schedules and activity data used are also included as supplemental data, in the Appendix A. Table 15 1-person household with day work profile 24-hr Intermediate Test ATUS Start End duration duration duration activity Location Motion level time time (min) (min) (min) details 4:00 6:00 120 10 10 Sleeping Bedroom None Washing, dressing and 6:00 6:30 30 30 10 Bathroom Minor grooming oneself Eating and Dining 6:30 6:50 20 20 10 Minor drinking room Not in 6:50 18:20 690 10 10 Go to work None home Eating and Dining Minor & 18:20 19:20 60 60 20 drinking room Major Television Living 19:20 22:30 190 190 15 Minor and movies room 22:30 4:00 330 10 10 Sleeping Bedroom None 83 Time Sche1 Sche2 Sche3 Sche4_Person1 Sche4_Person2 Sche5_Person1 Sche5_Person2 0:00 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 Kitchen/ Bedroom Bathroom Living room Not in home Dining room Figure 10 Occupancy profile and location data Motion level was defined by the National Electrical Manufacturers Association (NEMA) WD 7 Occupancy Motion Sensors Standard (NEMA 2016), including four categories based on the activity level, which are major, minor, fine, and no motion. An example of major motion is people walking; an example of minor motion is people extending their arms; an example of fine motion is people sitting and typing; an example of no motion is people sleeping. Each activity was also assigned a specific location or a set of locations in the laboratory space for occupants, who follow these typical profiles. 84 Given that there are a number of activities that occur over long durations of many hours (e.g., “no one at home”, these activities may be shortened to optimize laboratory testing time and utilization. These testing durations proposed are included in the column “test duration”, shown in Table 15. The “Test duration” is a shorter version of sleeping and not at home activity times, since there are rarely variations in occupancy and activities occurring during these periods. The longer time periods were reduced to 10 to 20 minutes. Preliminary testing results suggested that a 10- to 20-minute duration was able to capture the behavior of a sensor system sufficiently to mimic a longer period of time. This shorter duration also enables a more time-efficient testing method which is beneficial given this test method requires human subjects to complete. There are limitations of the use of the “test” duration for extrapolating to the 24-hour duration. Further studies can be implemented in future to investigate this further. The sensor performance is evaluated at 1-min intervals, which means that the ground truth and sensor output data is also collected and compared every 1 minute. This is based on results attained from the expert survey developed in a prior study (Chu et al., 2021a). The last question in this survey asked the time frequency of evaluation of the performance of occupancy sensor systems in residential buildings. A 1-minute frequency was the most commonly recommended. As such, for a 10-minute activity, this will result in ten classification results (occupied/unoccupied) from the sensor system, for use as input in calculating the resulting performance metrics. In order to translate the “test” duration test results into a reliability metric representative of a 24-hour period, the performance of the sensor system across the shortened activity periods must be extrapolated to the initial “24-hour duration” period. To do so, test results were scaled up from actual “Test duration” period to “24-hour duration” period by assuming the results from the “test” duration are proportional to the time for the activity across the “24-hour duration”. 85 3.3.2 Failure testing A recent study (Chu et al., 2021a) conducted a comprehensive literature review on influential variables impacting the performance of occupancy sensor systems. This research compiled a list of these variables, amounting to approximately 50 variables in total. An expert survey was then used to determine what a diversity of stakeholders suggest as the “most important” variables. The resulting most important variables suggested for residential buildings included 10 variables, more specifically, size and shape of the test area, level of motion of occupant(s), presence of pets, lighting level, spatial location of occupant(s), number of occupants, presence of large objects, presence of non-overhead interior lighting sources, number of doors, and use of robots (e.g., robot vacuum). The proposed standard set of variables to be tested are the same across any occupancy sensor system that may be tested following this method. The size and shape of the test area variable means different interior configuration and layouts of a controlled test space. This would also influence how many sensors are needed, and where the sensors are installed for a particular sensor system. For level of motion of occupants, four different motion levels are defined including major, minor, fine and no motion (Chu et al. 2021a). These motion levels are based on the National Electrical Manufacturers Association (NEMA) WD 7 Occupancy Motion Sensors Standard (NEMA 2016). Some sensor systems may have failures when occupants are not moving, such as PIR motion sensors (Kilic et al., 2013). Lighting level indicates a binary variable, which represents lights being on or off, since lighting levels are not typically dimmable in a residential building. Some types of sensors are sensitive to lighting levels and/or certain kinds of light (Yang et al., 2018). Pets, large objects, interior lighting sources, and robots are considered, since they could be mistaken for a person by certain types of occupancy sensor systems. For number of doors, there may be potential failures caused by 86 variations in this variable if sensor systems miss detecting people entering or leaving when there are multiple entry/exit doors. This is particularly the case for door-based occupancy sensor systems, which rely on detection of occupants in these locations to determine occupancy. This set of variables is proposed as a standard set of variables to evaluate all occupancy sensor systems considered. Apart from the above-mentioned most important variables, additional variables may also be included in the evaluation methods. These may be added based on known failures or sensitivities of certain sensors or combination of sensor types, or as suggested by the manufacturer of the sensor system who wishes to evaluate the occupancy sensor system under these conditions. Apart from the above-mentioned most important variables, additional variables may also be included in the evaluation methods. These may be added based on known failures or sensitivities of certain sensors or combination of sensor types, or as suggested by the manufacturer of the sensor system who wishes to evaluate the occupancy sensor system under these conditions. Based on how each variable might influence the sensor performance, different levels of variables are determined. For example, for the variable “motion level”, there are four levels (major, minor, fine and none), indicating that the occupancy sensor system will be tested under these four different motion levels to determine which level will fail the sensor system. For each variable level, at least three trials, under identical conditions, are conducted for each scenario to make sure the testing results are more robust. For the OVAT method, as mentioned above, a standard value must be determined for each of the variables not being tested in a particular scenario. The target for this standard set of values would be the environmental conditions that are most commonly occurring in a residential environment. In this case lights are on, with no pets, large objects, or robots. For spatial location 87 of occupants, this was in the center of the largest main room, which was the living room area. For testing of large objects, non-overhead interior lighting sources, robot, and pets, this also occurred in center. All variables were tested under both the presence of occupants (i.e., the laboratory test home was occupied), and without occupants (i.e., when the space was unoccupied). For each variable, at least three trials, under identical conditions, were conducted for each scenario. 3.3.3 Ground truth Ground truth provides the information that is known to be real or true by direct observation or measurement. In this paper, it refers to the real occupancy information (i.e., whether or not the test space was occupied) compared to the sensor prediction. The ground truth data must be provided to enable the comparison to the sensor output to evaluate the sensor performance. For typical testing, ground truth data was pre-defined/known based on the developed typical occupancy schedules and activity profiles. For failure testing, ground truth data was manually recorded. This data was used to compare with the sensor output to determine sensor performance. The ground truth data was recorded at the same frequency as the sensor data. 3.3.4 Laboratory Setup In order to ensure repeatability of test results, as well as to best mimic conditions experienced by occupancy sensors in residential buildings, standard laboratory conditions are proposed and used. Variables that are untested should be within acceptable limits to reduce potential sensor failures due to these untested variables. These are as follows: 1) Exterior windows are covered with an opaque cover to eliminate exterior lighting sources. Insulated foam boards could be used to cover windows. 2) Light sources are LED. 88 3) Lighting levels are approximately 300 lux at the work plane in the center of the test space(s) following DiLaura et al. (2011). 4) Uniform lighting levels are present throughout the test space. The lighting level is tested initially in each room. As needed supplemental overhead lighting is used to make lighting levels uniform across different spaces. 5) Indoor temperature is 21 +/- 2 C. Indoor temperature is controlled by setting the temperature setpoint using smart thermostat in residential buildings. 6) Relative humidity must be less than 80% +/- 5%, with a targeted value of 30-60%. As suggested by Ramos et al. (2015) the sensing ability may decrease with higher humidity for IR sensors. Based on the ASHRAE Equipment and Systems Handbook (2020), 30% to 60% relative humidity is ideal for human occupancy. 7) Electronics that emit potential interference in a similar frequency band (e.g., microwaves) as the tested sensor system(s) are turned off during testing to avoid electromagnetic interference. 3.3.5 Performance metrics Performance metrics for reliability evaluation of occupancy sensor systems have been discussed in detail in a recent study (Chu et al. 2021b). Among 80 peer-reviewed research articles, accuracy is the most widely used metric, which is the ratio of correct predictions over total number of testing scenarios. Some recent papers have used a confusion matrix to discuss failures. This method separates the counts of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) that occur during testing. Such a method enables a stronger understanding of the types of errors that occur. However, in many cases for prior studies, only an overall accuracy value 89 or error estimation was provided rather than analyzing these four terms in detail. Therefore, in this proposed standard methodology, a confusion matrix, and associated metrics, including precision, recall, F1-score, and accuracy were used to evaluate the “typical” testing results of the occupancy sensor systems. Precision is the percentage of true positive out of all the predicted positives as shown in Equation (1), while recall represents the ratio of true positives to all actual positives as shown in Equation (2). An F1-score is the harmonic mean of precision and recall Equation (3), which combines both false positives and false negatives, and gives precision and recall the same weight (Deng et al. 2016). Precision = TP/(TP+FP) (1) Recall = TP/(TP+FN) (2) F1-score = 2 * (Precision * Recall)/(Precision + Recall) (3) While “failure” testing aims to test the influence of each individual variable on the sensor performance, confusion matrix (TP, TN, FP, FN) is chosen as a standard metrics so that results can be evaluated for each testing scenario. By comparing the sensor output with ground truth (actual occupancy information), a confusion matrix will be provided. TP represents that the sensor output is the same as the ground truth, which is “Occupied”; TN means that the sensor output is “Unoccupied” which is same as the ground truth; FP is that the sensor output is “Occupied” but the space is actually “Unoccupied”; FN means that the sensor output is “Unoccupied” but the space is actually “Occupied”. After testing one variable, for all scenarios, if the prediction result is either TP or TN, it means that the sensor output is correct compared to the ground truth, indicating the variable is not influential since it does not appear to cause the sensor system to fail. Otherwise, if 90 either FP or FN was observed, that means this variable is influential since it may cause the sensor system to fail at certain conditions. 3.4 Experimental Results: Case Study 3.4.1 Sensor System Description The tested sensor system is the MicroCam platform, which is a battery-powered, stand- alone sensor system designed to detect occupancy in residential homes. It contains a local processing unit, a camera, a microphone, a motion sensor, and a Near-IR Band (NIR) Light Emitting Diode (LED). Multiple MicroCam platforms operate concurrently and communicate wirelessly with each other to detect presence of people. This provides more coverage for the detection task that the system is designed for. Advanced algorithms are employed to translate data streams into actionable adjustments to home heating and cooling. The algorithms are implemented and executed locally on the sensor unit making the solution stand-alone, not relying on external computation units or cloud computing. MicroCam platforms are designed to operate under daylight conditions as well as low- light/no-light conditions, and to detect people even when they are static. This system is also designed to differentiate different sources of motion, i.e., motions of pets, robot vacuums, etc. from people. After local data processing, the MicroCam platforms send only a binary occupancy result to the “lead platform”, which is then connected to the HVAC system of a home to adjust setpoints based on occupancy. 3.4.2 Test bed For testing and evaluation of occupancy sensor systems, controlled laboratory tests were conducted using a residential building laboratory. The residential building laboratory facility, 91 located in East Lansing, MI, includes two identical single-story wood-framed residential buildings, one of which was used in this study. This laboratory space includes eight double-glazed windows and two exterior doors. The interior includes white-colored drywall, grey floors and white ceilings, with interior walls consisting of wood studs with drywall on either side. The interior of the laboratory test space is reconfigurable, including movable interior walls to allow for testing by using different interior layouts. In this study, two configurations were chosen, including a two- bedroom and three-bedroom layout. The three-bedroom layout is shown in Figure 11 (a). To create a 2-bedroom layout, the interior wall between bedroom 1 (BR1) and bedroom 2 (BR2) was removed. Overhead LED lighting was installed and evenly distributed on the ceiling. The interior of the test space is furnished to represent a typical residential home configuration. This included a living room, kitchen, dining room, bedroom/bathroom, and storage room. Seven occupancy sensor platforms, including one lead platform (mc04) and six member platforms (mc01, mc03, mc05, mc06, mc07, mc08), were installed on the wall and ceiling per developer’s instructions. Each platform is an independent occupancy sensor component with multiple sensing technologies. Each platform utilizes sensor fusion technology to integrate motion sensors, microphones, and image sensors to detect occupancy. This configuration, as shown in Figure 11 (a), was used to provide sufficient coverage of the entire testing space. The lead platform (mc04) was connected via Wi-Fi and located in BR1. The member platforms communicated with the lead platform using Bluetooth. Platforms mc06 and mc08 were installed such that they faced the entry/exit area for each of the two exterior doors. The platforms mc01, mc03, and mc05 were used to cover each of the three bedrooms, and mc07 was used for the living room, kitchen and dining area. 92 (a) Interior layout showing the three-bedroom configuration and deployment of sensors platforms mc01 through mc08 (b) 1 m x 1 m grid of locations within the laboratory for use in directing occupants where to be throughout testing (Note: BR – Bedroom; K/D – Kitchen/Dining; LR – Living room; SR – Storage room) Figure 11 Residential laboratory used for testing of occupancy sensor systems 93 During testing it was important, for consistency, to ensure that the occupants move about the test space in a consistent manner. Therefore, the interior space was divided into seven different blocks, labeled from A to G, where each letter represents a different location. This included A through C for the three bedrooms, D for the doorway, G for the kitchen and dining area, and E and F are for the living room space. Within each quadrant, a 1 m x 1 m grid was created and labeled on the floor (Figure 11 (b)) The storage room (SR) shown is where all equipment was located, therefore it was not used as a location where occupants could go during testing. People were assigned to different locations, following the occupancy schedule and activity profiles. For example, occupants were moving in quadrant G when they are cooking and eating in the kitchen and dining area. When people were reading, watching TV, and playing video games, they sat on the couch in the living room (Blocks E and F). Bedroom 3 (Block A) is considered to be a bedroom with a bathroom. Bedroom 1 (Block C) and Bedroom 2 (Block B) were also used for sleeping. 3.4.3 Data collection The testing was conducted across a period of several months, from April to July 2021. A web scraper was used to extract occupancy output from the sensor output interface using Home Assistant (2021), an open-source smart home platform. The sensor output collected was a binary classification of “Occupied” or “Unoccupied”. Ground truth data was manually recorded during failure testing and predefined for typical testing based on the developed typical occupancy schedules and activity profiles. 94 3.5 Results 3.5.1 Typical testing results Typical testing was implemented following the five typical occupancy schedules and activity profiles, as detailed in Table 15 and in the Appendix. Table 16 shows an example of the typical testing results with the 1-person household with day absence profile, following the “test duration” testing period. The remaining typical test results are included in Tables 27-30 in the Appendix B. The sensor performance was evaluated at 1-min intervals for these activity durations. The “error time” represents the total time that sensor failures occur for the “test duration”. For example, for the “eating and drinking” activity, the error time is eight minutes out of the 10-minute test duration, which means that there were eight detection errors out of the ten times of detection, or 80% of the time. Following the methods proposed in the methodology section, these errors were extrapolated to the 24-hour duration. From these tables, most failures occurred when occupants were using the computer, or sitting (either on the couch for reading or for eating and drinking). For these cases, the positioning of the sensor platforms greatly affects the performance. For instance, when people are using the computer, they are sitting and their back is towards the camera, causing an occlusion by the chair and thus misdetection. Similarly, as seen in Figure 11 a, the relative positioning of the couch and mc06 is the main reason for misdetection of people sitting on the couch. 95 Table 16 Typical testing results with “1-person household with day absence profile” using “Test Duration” Error ATUS-Based Test Intermediate Error 24-hour Error Ground Time Activity Duration duration Time1 duration Time1 truth (min, detail (min) (min) (min) (min) (min) %) 0 Sleeping 10 10 0 120 0 (0%) Teeth brushing; 0 Shower; Occupied 10 30 0 30 0 (0%) Getting dress Eating and 8 10 20 16 20 16 drinking (80%) 6 No one in home Unoccupied 10 10 6 690 414 (60%) Cook food; Bring food to dinner table; Eating and 2 watching TV; 20 60 6 60 6 (10%) Remove dishes, clean kitchen and Occupied table 0 Watching TV 15 190 0 190 0 (0%) 10 Sleeping 10 10 10 330 330 (100%) 1 Calculated by extrapolation based on error time for “test duration”; see methodology section 96 Performance metrics were then used to evaluate the reliability of the occupancy sensor system over time, including a confusion matrix (TP, TN, FP, and FN), Precision, Recall, F1-score, and accuracy. These metrics were calculated for the “test duration”, for the five typical occupancy schedules and activity profiles (Table 15), then extrapolated to the “24-hour duration”. The last column, “Summary”, represents the overall performance evaluation results combining all five sets of testing results. From Table 17, for the “test duration”, for 1-person households there are more TPs and FNs with Sch3 compared to that of Sch1 and Sch2. This is because Sch3 is the 1-person with a stay-at-home profile where the test duration is longer, thus a greater number of data points is collected, compared to the other two schedules. Since there are more TPs than FNs for Sch3, the highest accuracy was attained for Sch3 in the 1-person household category. For the 2-person household, even though the total number of data points collected is higher for Sch5, there were more FNs compared to that of Sch4, therefore, Sch4 resulted in higher accuracy. When the data is scaled to the “24-hour duration”, the TPs, TNs, FPs and FNs were proportionally increasing according to the time duration when they occurred. As a result, as some schedules have different activities that were scaled, the confusion matrix TPs, TNs, FPs, and FNs are adjusted differently. Following this extrapolation, the overall accuracy with Sch1 and Sch3 decreased by 22% and 20%, respectively, from the “Test duration” to “24-hour duration”, since the FP increased slightly more compared to the others. For Sch2, both TPs and FPs largely increased, but FPs increased a bit more compared to TPs, and FN slightly increased, thus the accuracy with Sch2 decreased by 20%. For Sch4, the TPs and FNs increased a similar amount, with TNs substantially increasing, thus the accuracy has a slight increase of 4%. For Sch5, since 97 the increase in TPs and TNs are similar to that of FPs and FNs, the accuracy is similar, with a slight decrease of 1.6%. There are some scenarios where Precision is 1, which means there are no FPs. Low Precision also appears in Sche1 and Sche2, which are less than 0.5, indicating that there are more FPs in these scenarios. However, there is no one value for Recall, which indicates there are always FNs in all scenarios, and FNs are consistent across all scenarios. On average, the Precision and Recall are 0.75 and 0.70, respectively, which means that there are a similar number of FPs and FNs across the dataset. The overall accuracy of the typical testing is also calculated. For the “Test duration” this was 76.4%, and 62.4% for the “24-hour duration”. 98 Table 17 Performance evaluation metric calculation for typical testing results analysis 1-person household 2-person household Sche3- Sche3- Sche5- Sche5- Overall Sche1 Sche2 Total Sche4 Total Day11 Day21 Day11 Day21 TP 55 46 69 65 235 115 125 102 342 577 TN 4 0 3 0 7 25 0 0 25 32 FP 6 15 0 0 21 0 0 0 0 21 FN 20 14 18 15 67 17 45 38 100 167 Test Duration Precisio 0.90 0.75 1.00 1.00 0.92 1.00 1.00 1.00 1.00 0.96 n Recall 0.73 0.77 0.79 0.81 0.78 0.87 0.74 0.73 0.77 0.78 F1-score 0.81 0.76 0.88 0.90 0.84 0.93 0.85 0.84 0.87 0.86 Accurac 69.4% 61.3% 80.0% 81.3% 73.3% 89.2% 73.5% 72.9% 78.6% 76.4% y TP 398 594 1004 1996 656 875 1531 3527 TN 276 0 3 279 685 0 685 964 FP 414 750 0 1164 0 0 0 1164 24-hr FN 352 96 433 881 99 565 664 1545 Duration Precisio (scaled) 0.49 0.44 1.00 0.63 1.00 1.00 1.00 0.75 n Recall 0.53 0.86 0.70 0.69 0.87 0.61 0.70 0.70 F1-score 0.51 0.58 0.82 0.66 0.93 0.76 0.82 0.72 Accurac 46.8% 41.2% 69.9% 52.7% 93.1% 60.8% 77.0% 62.4% y Note: 1’Day1’ and ‘Day2’ appear when testing was completed across two days to represent one typical schedule. 99 3.5.2 Failure testing results Twelve individual variables were tested, including the standard set of variables (1-10) from the stakeholder survey (Chu et al. 2021) and two additional variables that were identified to be potentially influential to the tested sensor system (11-12). These included the presence of an extra door in the entry/exit area and having the TV on. For (11), this variable was tested because of the known sensitivity of the sensor system to other doors near an entry/exit door. For (12), this was chosen to be tested because when the TV was on, its sounds and pictures could potentially be interpreted by the sensor system as being occupants. All tested variables and their range of values tested are listed in Table 18. Additional trials were added to certain variables when inconsistent results occurred within the first three trials, to collect further data on performance. Table 18 Tested variables and associated testing levels Variables Levels Motion level None Fine Minor Major Number of occupants 1 2 Spatial location of occupants D7 G1 G7 F5 E10 E4 A6 C1 Interior lighting level Off On Presence of large metal objects Non-presence Presence Presence of interior lighting sources Non-presence Presence Number of exterior doors 1 2 Presence of (vacuum) robot Non-presence Presence Presence of pets Non-presence Presence Test area configuration Two-bedroom Three-bedroom During failure testing, each scenario was tested for five minutes continuously. Across this period, data was collected every minute. Based on whether there were state changes across this 5- 100 min period, the resulting value was determined to be a TP, TN, FP or FN. A five-minute period was also used in between tests to allow for the sensor system to respond to changed conditions. In addition, to prevent data loss, simple occupied/unoccupied scenarios were also used as a check to make sure the sensor system was operational before each test. The results of failure testing are reported herein in two parts, including those variables found to be influential and those found not to be influential. 3.5.2.1 Non-influential individual variables Individual non-influential variables included number of occupants, presence of large objects, presence of large interior light sources, and number of doors, as shown in Table 19. In other words, in all scenarios tested, both with and without occupants present in the laboratory test space, all resulting data collected was either TPs or TNs, as shown in the last three columns. The last three columns describe the sensor output, ground truth, and confusion matrix. For number of occupants, scenarios with 1 and 2 occupants were evaluated, however this variable was not found to be impactful. The presence of large objects (2m metal ladder) and the presence of interior light sources (floor lamp) were not found to impact sensor reliability. For number of doors, for 1 door, occupants entered and exited from the same door. For the 2-door scenario, occupants entered from one of the two exterior doors, and then exited from the other. No impact was found for this variable. 101 Table 19 Testing scenarios and results of non-influential individual variables Presence of Interior Number Test Lighting Use of Presence Motion Number Sensor Ground Prediction Occupancy large object lighting Location of variable level robots of pets level of doors Output truth result objects sources occupants 1 1 1 TP 1 1 1 TP Number 1 1 1 TP of Presence On No No No No Major Center 1 2 1 1 TP occupants 2 1 1 TP 2 1 1 TP Yes 0 0 TN Yes 0 0 TN Yes 0 0 TN Non-presence On No No No - - 0 1 No 0 0 TN Presence No 0 0 TN of No 0 0 TN large Yes 1 1 TP objects Yes 1 1 TP Yes 1 1 TP Presence On No No No Major Center 2 1 No 1 1 TP No 1 1 TP No 1 1 TP Yes 0 0 TN Presence Non-presence On No Yes No No - - 0 1 0 0 TN of Yes 0 0 TN interior Yes 1 1 TP light sources Presence On No Yes No No Major Center 2 1 1 1 TP Yes 1 1 TP 1 0 0 TN 1 0 0 TN 1 0 0 TN Non-presence On No No No No - - 0 2 0 0 TN 2 0 0 TN umber of 2 0 0 TN doors 1 1 1 TP 1 1 1 TP 1 1 1 TP Presence On On No No No Major Center 2 2 1 1 TP 2 1 1 TP 2 1 1 TP 102 3.5.2.2 Influential individual variables Lighting level was tested only under non-presence scenarios, as shown in Table 20. This was because with the presence of occupants, other variables are introduced such as motion level, location, and number of occupants. Seven trials were conducted with lights on, where FPs occurred twice. This was found to occur when, prior to testing, occupants exited the door, making the space unoccupied, however the sensor system missed detecting this event thus the output remained occupied. In other words, this does not mean that the sensor system detects occupants when the space is not occupied, but rather when there is an issue with the entry/exit event. With lights off, four of the five trials resulted in FPs. This may be because of light reflection, off some objects present in the test space, or an issue with the entry/exit event not being detected. For motion level, Table 20 presents the testing scenarios and results. Before each test, the occupant exited the space to ensure the sensor system started in an unoccupied state, then the occupant entered the space and maintained the design motion level across each trial period. The results indicate there were no failures for “Major”, “Minor” and “Fine” motion levels, and only one failure for the “None” motion level. For location of occupants, eight locations were used. These locations (Figure 11 b) included in the center of spaces (E10), corners (E4, D7, G1), locations near sensor platforms (G7), and in locations that are anticipated to be challenging for the sensor system to detect people correctly (F5, A6). Results (Table 21) show that the only failure occurred at E10. This may be because of its proximity to the door, causing the sensor system to register that the occupant is leaving the space when they did not. 103 Table 20 Testing scenarios and results of “Lighting level” and “Motion level” variables Presence of Interior Number Lighting Use of Presence Motion Number Ground Prediction Test variable Occupancy large metal lighting Location of Output level robot of pets level of doors truth result objects sources occupants On 0 0 TN On 0 0 TN On 1 0 FP On 0 0 TN On 0 0 TN Lighting On 0 0 TN Non-presence No No No No - - 0 1 level On 1 0 FP Off 1 0 FP Off 1 0 FP Off 0 0 TN Off 1 0 FP Off 1 0 FP Major 1 1 TP Major 1 1 TP Major 1 1 TP Minor 1 1 TP Minor 1 1 TP Motion Minor 1 1 TP level of Presence On No No No No Center 1 1 occupants Fine 1 1 TP Fine 1 1 TP Fine 1 1 TP None 0 1 FN None 1 1 TP None 1 1 TP 104 Table 21 Testing scenarios and results of “Location of occupants”, “Use of robots”, and “presence of pets” variables Presence Presence of Number Test Lighting of large Use of Presence Motion Number Ground Prediction Occupancy interior Location of Output variable level metal robots of pets level of doors truth result lighting occupants objects sources D7 1 1 TP G1 1 1 TP G7 1 1 TP Location F5 1 1 TP of Presence On No No No No Major 1 1 occupants E10 0 1 FN E4 1 1 TP A6 1 1 TP C1 1 1 TP 1 0 FP Non- On No No Yes No NA NA 0 1 1 0 FP presence Use of 1 0 FP robot 1 1 TP Presence On No No Yes No Major Center 1 1 1 1 TP 1 1 TP 1 0 FP Non- On No No No Yes NA NA 0 1 1 0 FP presence Presence 1 0 FP of pets 1 1 TP Presence On No No No Yes Minor Center 2 1 1 1 TP 1 1 TP 105 For use of robots, a robot vacuum was used. For non-presence scenarios, the occupants were in the space initially and turned on the robot, then left the space, keeping robot running for at least five minutes without the presences of occupants. Results show consistent FPs when the space was unoccupied. No failures occurred for presence scenarios. For presence of pets, two rabbits were used, which were placed in an enclosure in the center of the test space and allowed to move freely within this space during testing. Similar to use of robots, there were consistent FPs during non- presence scenarios. This was an unexpected result. After discussions with the developer, supplementary tests were conducted for these scenarios, wherein the system started in an unoccupied state, meaning there were robot vacuum or pets but no occupants in the space. From this supplemental testing, it was determined that when the initial state is unoccupied, the presence of robot vacuum or pets did not cause FPs, indicating that the FPs in the prior tests were not due to robot vacuum or pets being detected as people, but rather due to exit events of the people (who were initially in the space) not being properly registered, which is caused by the continuous motion in the entry/exit area. Testing was also completed a second time for when the sensor system began in an occupied state (i.e., the robot vacuum or pets as well as occupants), then the occupants left the space, while the robot vacuum or pets remained. The results from this testing was consistent with the first set of testing results reported in Table 21. The supplemental results are included in Appendix C. As discussed, several additional variables were also tested beyond those considered to be the “most important.” These include the presence of another door in the entry/exit area, and having the TV on, the results of which are shown in Table 22. Figure 11 shows that the “entry/exit” platform mc06 can “see” both the exterior door and the interior bedroom door. This scenario was tested with an occupant entering and exiting the BR3 door. Results indicated FNs during this 106 scenario, indicating the sensor system interpreted this as the occupant exiting the space when they did not or the person in BR3 was in a blind region. For the TV on scenario, the results indicate consistent FPs occurring when the TV was left on in an unoccupied space. This is likely because the sensor system incorrectly interpreted the TVs picture and/or sounds as being an occupant. This is another scenario, for which supplementary testing was performed. Results suggested that this kind of error can be avoided. 107 Table 22 Testing scenarios and results of additional variables, including “Presence of another door in Entry/Exit area”, and “TV On” Presence Interior Number Test Lighting of large Use of Presence Motion Number Ground Prediction Occupancy lighting Location of Output variable level metal robots of pets level of doors truth result sources occupants objects 0 1 FN 0 1 FN Another 1 1 TP door in Presence On No No No No Major Center 1 1 0 1 FN E/E area 1 1 TP 1 1 TP 0 1 FN 1 0 FP Non- On No No No No NA NA 0 1 1 0 FP presence 1 0 FP TV on 1 1 TP Presence On No No No No Major Center 2 1 1 1 TP 1 1 TP 108 3.6 Conclusions This research proposes a standard evaluation methodology to test the reliability of occupancy sensor systems in residential buildings. The proposed methodology includes both “Typical testing” and “Failure testing”. Typical testing is used to evaluate the reliability of each occupancy sensor system under scenarios that mimic real occupancy scenarios. Failure testing follows one variable at a time (OVAT) methods to test the most important variables based on a stakeholder survey, to determine if these variables impact sensor system reliability. The developed methodology was then implemented for a novel occupancy detection sensor system to test the sensor system’s reliability. The following overall conclusions can be made for this study: (1) Cluster analysis methods were used to analyze American Time Use Survey-derived data, resulting in three main types of occupancy profiles for residential buildings, specifically “Day absence”, “Stay at home”, and “Night absence” profiles. These representative residential occupancy profiles for individuals were then used, in combination, to create five 1-person and 2-person household, representative, 24-hour occupancy and activity scenarios, using activity data from the ATUS database that most closely match these occupancy profiles. (2) The resulting five activity schedules were followed by real occupants in a controlled, residential laboratory environment to evaluate occupancy sensor system performance. Occupancy sensor systems were tested using a shortened “test duration” for all activities, which allowed for all unique activities to occur, but shortening longer duration activities to reduce the need to test continuously for 24 hours. The results were extrapolated to “24- hour duration” results, with the 24-hour results representing the predicted reliability of the sensor system over a typical 1-day period in a residential building. Results show that on 109 average, the Precision and Recall are 0.75 and 0.70, respectively, which means that there are similar number of FPs and FNs across the whole dataset. The overall accuracy of the tested novel system ranged from 62.4% to 76.4%. (3) For failure testing, individual variable testing provided insights as to the impact of a range of levels of individual variables on sensor system performance. Variables were identified for testing based on a both stakeholder feedback, as well as based on known possible sensitivities of the sensor system being evaluated. This resulted in the testing of a total of 12 individual variables. For the tested sensor system, the “Number of occupants”, “Presence of large objects”, “Presence of interior light sources”, and “Number of doors” are not influential, while “Lighting level”, “Location of occupants”, “Another door in the entry/exit area”, and “TV on” variables were determined to impact sensor system performance. (4) Supplementary tests showed that the performance of the evaluated system is affected most by the positioning of the platforms monitoring the entry/exit doors. Individual sensor evaluations showed that the system can differentiate between the sources of motion, i.e. can differentiate between people and robot vacuums and pets. The proposed methodology for evaluating the reliability of occupancy sensor systems presents an opportunity for use as a standardized method to evaluate residential occupancy sensor systems that currently does not exist. The focus on typical scenarios enables the reporting of metrics representing the reliability of the sensor system under typical U.S. household scenarios. This could be used as a comparative measure of performance across sensor systems. The focus on failure testing enables a focus on the potential weaknesses in the sensor system reliability, as well as targets for further testing and development. 110 APPENDICES 111 APPENDIX A: Typical occupancy schedules and activity profiles Table 23 Sche2: 1-person household with night absence profile Start time End time 24-hr Test ATUS activity details Specific location Motion level duration duration (min) (min) 4:00 12:00 480 10 Sleeping Bedroom None 12:00 13:00 60 15 Eating and drinking Dining room Minor & Major 13:00 13:10 10 10 Laundry Bathroom Major 13:10 13:40 30 10 Washing, dressing and grooming oneself Bathroom Minor 13:40 15:30 110 15 Computer use for leisure Living room Minor 15:30 4:00 750 15 Go to work Not in home None 112 Table 24 Sche3: 1-person household with stay home profile 24-hr Test Start time End time duration duration ATUS activity details Specific location Motion level (min) (min) 4:00 9:00 300 10 Sleeping Bedroom None 9:00 9:20 20 10 Washing, dressing and grooming oneself Bathroom Minor 9:20 9:23 3 3 HH & personal mail & messages (except e-mail) Not in home Major 9:23 9:28 5 5 Food and drink preparation Kitchen Major 9:28 9:29 1 1 Eating and drinking Dining room Minor 9:29 11:00 91 10 Reading for personal interest Living room Minor 11:00 12:00 60 16 Television and movies (not religious) Living room Minor 12:00 12:10 10 10 Financial management Living room Minor 12:10 14:30 140 10 Reading for personal interest Living room Minor 14:30 17:00 150 15 Television and movies (not religious) Living room Minor 17:00 17:30 30 10 Reading for personal interest Living room Minor 17:30 18:30 60 15 Television and movies (not religious) Living room Minor 18:30 19:00 30 10 Food and drink preparation Kitchen Major 19:00 19:15 15 10 Eating and drinking Dining room Minor 19:15 23:30 255 15 Television and movies (not religious) Living room Minor 23:30 0:30 60 10 Reading for personal interest Living room Minor 0:30 4:00 210 10 Sleeping Bedroom None 113 Table 25 Sche4: 2-person household where both with day absence profile 24-hr Test Start time End time duration duration ATUS activity details Specific location Motion level (min) (min) 4:00 6:00 120 10 Both are sleeping Bedroom None 6:00 6:30 30 10 Both washing, dressing and grooming oneself Bathroom Minor 6:30 6:40 10 10 1 food preparation other Eating and drinking Kitchen/Dining room Major 6:40 6:50 10 10 Both Eating and drinking Dining room Minor Not in home/Dining 6:50 6:55 5 5 1 left for job, other Eating and drinking None/Minor room 6:55 18:20 685 25 Both left for work Not in home / Not in home/Dining 18:20 18:30 10 5 1 is not in home, other one is Eating and Drinking None/Minor room 18:30 18:45 15 5 1 is food preparation other Eating and drinking Kitchen/Dining room Major 18:45 19:15 30 20 Both Eating and drinking Dining room Minor 1 Eating and drinking and other computer use for Dining room/Living 19:15 19:20 5 5 Minor leisure room 1 Television and Movies and other computer use for 19:20 21:00 100 20 Living room Minor leisure 1 watching Television and movies and other washing, Living 21:00 21:30 30 10 Minor/Major dressing and grooming oneself room/Bathroom 1 watching Television and movies and other Health Living 21:30 21:32 2 2 Minor/Major related selfcare room/Bedroom Living 21:32 22:30 388 20 1 watching Television and movies and other Sleeping Minor/None room/Bedroom 22:30 4:00 270 10 Both Sleeping Bedroom None 114 Table 26 Sche5: 2-person household where one with day absence profile and the other with stay home profile 24-hr Test Start time End time duration duration ATUS activity details Specific location Motion level (min) (min) 4:00 6:00 120 10 Both are Sleeping Bedroom None 6:00 6:20 20 10 Both are Washing, dressing and grooming oneself Bathroom Major 6:20 6:40 20 10 1 interior cleaning and other Washing, dressing and grooming oneself Whole space/Bathroom Major 6:40 7:00 20 10 1 Eating and drinking and other Washing, dressing and grooming oneself Dining room/Bathroom Major 7:00 7:10 10 10 Both are Eating and drinking Dining room Minor 7:10 7:15 10 10 1 interior cleaning and other Eating and drinking Whole space/Dining room Major/Minor 7:15 7:20 5 5 1 interior cleaning and other left for job Whole space/Not in home Major/None 7:20 7:45 25 10 1 in home and doing kitchen and food clean up Kitchen/Not in home Major/None 7:45 7:50 5 5 1 in home and doing interior cleaning Whole space/Not in home Major/None 7:50 8:10 20 10 1 in home and doing Care for animals and pets (not veterinary care) Living room/Not in home Minor/None 8:10 8:30 20 10 1 in home and doing Computer use for leisure (exc. Games) Living room/Not in home Minor/None 8:30 9:30 60 15 1 in home and doing television and movies (not religious) Living room/Not in home Minor/None 9:30 10:30 60 10 1 in home and doing Reading for personal interest Living room/Not in home Minor/None 10:30 10:45 15 5 1 in home and doing Telephone calls Living room/Not in home Minor/None 10:45 11:45 60 10 1 in home and doing Playing games Living room/Not in home Minor/None 11:45 12:45 60 15 1 in home and doing television and movies (not religious) Living room/Not in home Minor/None 12:45 12:55 10 10 1 in home and doing Food and drink preparation Kitchen/Not in home Major/None 12:55 13:25 30 10 1 in home and doing Eating and drinking Dining room/Not in home Minor/None 13:25 15:25 120 20 1 in home and doing television and movies (not religious) Living room/Not in home Minor/None 15:25 15:55 30 10 1 in home and doing Sleeping Bedroom/Not in home None 15:55 16:55 60 15 1 in home and doing television and movies (not religious) Living room/Not in home Minor/None 16:55 18:25 90 15 1 in home and doing Playing games Living room/Not in home Minor/None 18:25 18:30 10 10 1 in home and doing Food and drink preparation Kitchen/Not in home Major/None 18:30 18:35 5 5 1 is doing Food and drink preparation and other Eating and drinking Kitchen/Dining room Major/Minor 18:35 19:05 30 10 Both are Eating and drinking Dining room Minor 19:05 19:30 25 10 1 is Eating and drinking and other Kitchen and food clean-up Kitchen/Dining room Major/Minor 19:30 19:35 5 5 1 is Television and movies and other Kitchen and food clean-up Living room/Kitchen Minor/Major 19:35 20:00 25 15 Both watching television and movies (not religious) Living room Minor 1 watching television and movies and other socializing and communicating 20:00 21:00 60 10 Living room Minor with others 1 watching television and movies and other Washing, dressing and 21:00 21:30 30 10 Living room/Bathroom Minor/Major grooming oneself 1 Computer use for leisure (exc. Games) and other watching television and 21:30 22:00 390 10 Living room Minor movies 22:00 22:20 20 20 1 sleeping and other Washing, dressing and grooming oneself Bedroom/Bathroom None/Major 22:20 4:00 280 10 Both Sleeping Bedroom None 115 APPENDIX B: Typical testing results with associated typical occupancy schedules and activity profiles Table 27 Typical testing results with “1-person household with night absence profile” Test Interme Erro 24-hr Err Error ATUS-Based Activity Ground durat diate r durat or time detail truth ion duration Tim ion Ti (min) (min) (min) e (min) me Sleeping 10 0 10 0 480 0 Heat food; Bring food to dinner table; Eating and watching tv, using 15 0 60 0 60 0 phone; Remove plates and cleaning kitchen and table Occupied Collecting dresses from bedroom and put them in washer for 10 1 10 1 10 1 laundry Shower; 10 0 30 0 30 0 Dressing & grooming Computer use for leisure (exc. 15 13 130 113 110 95 Games) Unoccupi Left for job 15 0 15 0 750 0 ed 116 Table 28 Typical testing results with “1 person with stay home profile” ATUS-Based Test Error Intermediate 24-hr Ground Error Error Activity duration time duration duration truth Time Time detail (min) (min) (min) (min) Sleeping 10 0 10 0 300 0 Mouthwash; Occupied Shower and others; 10 0 20 0 20 0 Dressing Outside house: HH & personal mail & Unoccupied 3 0 3 0 3 0 messages (except e- mail) Food and drink 5 0 5 0 5 0 preparation Eating and 1 0 1 0 1 0 drinking Reading for 10 9 91 82 91 82 personal interest Television and movies (not 16 0 60 0 60 0 Occupied religious) Financial 10 0 10 0 10 0 management Reading for 10 9 140 126 140 126 personal interest Television and movies (not 15 0 150 0 150 0 religious) Reading for 10 5 30 15 30 15 personal interest Television and movies (not 15 0 60 0 60 0 religious) Bring stuff from refrigerator and preprocess; Using range, cook 10 0 30 0 30 0 food; Bring all cooked Occupied food to table Eating and 10 0 15 0 15 0 drinking Television and movies (not 15 0 255 0 255 0 religious) Reading for 10 0 60 0 60 0 personal interest Sleeping 10 10 30 30 210 210 117 Table 29 Typical testing results with “2-person where both with day absence profile” Err Test Intermedi Err 24-hr Err or Ground durati ate or durati or ATUS-Based activity detail time truth on duration Tim on Tim (mi (min) (min) e (min) e n) Sleeping 10 2 10 2 120 24 Mouth wash; Taking shower; Occupie 10 0 30 0 30 0 Dressing; Grooming. d Food preparation 10 0 10 0 10 0 Eating and drinking 15 0 15 0 15 0 Unoccup Left for work 25 0 25 0 685 0 ied Having snacks while waiting for 5 0 10 0 10 0 other person Heating and bringing food to table 5 0 15 0 15 0 Eating, discussing and using phone 20 0 30 0 30 0 Eating and drinking Occupie 5 0 5 0 5 0 Television and Movies d 20 15 100 75 100 75 Taking shower; 10 0 30 0 30 0 Getting dressed for sleep Health related self-care 2 0 2 0 2 0 Sleeping 20 0 73 0 388 0 Note: Only one of the two activity profiles is provided in the “Activity detail” column 118 Table 30 Typical testing results with “2-person where 1 with day work profile and other one with stay home profile” Err Test or Intermed Err 24-hr Err Grou durati tim iate or durati or ATUS-Based activity detail nd on e duration Tim on Tim truth (min) (mi (min) e (min) e n) Sleeping 10 0 10 0 120 0 Mouthwash; Shower; Dressing. 10 0 20 0 20 0 Arrange bedroom 10 0 20 0 20 0 Making food for breakfast; 10 1 20 2 20 2 Bring food to table, start eating Eating and drinking 10 0 10 0 10 0 Interior cleaning 10 0 10 0 10 0 Kitchen and food clean up 10 0 25 0 25 0 Interior cleaning 5 0 5 0 5 0 Clean plates for pet and bring food for them; 10 7 20 14 20 14 Playing with pets Computer use for leisure (exc. Games) 10 10 20 20 20 20 Television and movies (not religious) 15 15 60 60 60 60 Reading for personal interest 10 0 60 0 60 0 Telephone calls 5 0 15 0 15 0 Playing games 10 0 60 0 60 0 Occup Television and movies (not religious) 15 0 60 0 60 0 ied Food and drink preparation 10 2 10 2 10 2 Eating and drinking 10 10 30 30 30 30 Television and movies (not religious) 20 9 120 54 120 54 Sleeping 10 0 30 0 30 0 Television and movies (not religious) 15 0 60 0 60 0 Playing games 15 0 90 0 90 0 Food and drink preparation 10 5 10 5 10 5 Eating together and discussing, use mobile 10 0 30 0 30 0 Bring used plates from table to kitchen, start 10 3 25 8 25 8 cleaning the kitchen Kitchen and food clean-up 5 1 5 1 5 1 Watching television and movies (not religious) 15 11 25 18 25 18 Socializing and communicating with others 10 0 60 0 60 0 Taking shower; 10 0 30 0 30 0 Dressing for sleep Sleeping 10 9 65 59 390 351 Note: Only one of the two activity profiles is provided in the “Activity detail” column 119 APPENDIX C: Supplemental variable testing results Table 31 Supplemental testing results for individual variable -- “Presence of robot” with the initial Occupied/Unoccupied state of the test space Prese Inter nce ior Use Prese # Gro Test Light of Mot # of Predic Occupanc Initial light of nce Loca of Out und variab ing large ion occup tion y State ing rob of tion do put trut le level metal level ants result sour ots pets ors h objec ces ts 1 0 FP Occupi 1 0 FP ed Prese 1 0 FP nce Non- Ye On No No No NA NA 0 1 of presence s 0 0 TN robot Unocc 0 0 TN upied 0 0 TN 120 REFERENCES 121 REFERENCES “Home Assistant.” https://www.home-assistant.io/. Accessed on 12 Nov 2021. Chu, Y., Cetin, K.S. (2021b) Sensing Systems for Smart Building Occupant-Centric Operation. The Rise of Smart Cities: Advanced Structural Sensing and Monitoring Systems (In press). Chu, Y., Mitra, D., O’Neill, Z. and Cetin, K., (2021a). Influential variables impacting the reliability of building occupancy sensor systems: A systematic review and expert survey. Science and Technology for the Built Environment, 28(2), 200-220. Deng, X., Liu, Q., Deng, Y., & Mahadevan, S. (2016). An improved method to construct basic probability assignment based on the confusion matrix for classification problem. Information Sciences, 340, 250-261. DiLaura, D. L., Houser, K., Mistrick, R., & Steffy, G. R. (2011). The lighting handbook: reference and application. EIA, U., 2015. Residential Energy Consumption Survey (RECS). https://www.eia.gov/consumption/residential/ (Accessed on October 20th, 2021). Esrafilian-Najafabadi, M., & Haghighat, F. (2021). Occupancy-based HVAC control using deep learning algorithms for estimating online preconditioning time in residential buildings. Energy and Buildings, 252, 111377. Hasan, H.A., Abdullah, S.R.S., Kofli, N.T. and Yeoh, S.J., 2016. Interaction of environmental factors on simultaneous biosorption of lead and manganese ions by locally isolated Bacillus cereus. Journal of Industrial and Engineering Chemistry, 37, pp.295-305. Hong T, Taylor-Lange SC, D’Oca S, Yan D, Corgnati SP. Advances in research and applications of energy-related occupant behavior in buildings. Energy and buildings. 2016 Mar 15;116:694-702. Kilic, Y., H. Wymeersch, A. Meijerink, M.J. Bentum and W.G. Scanlon. 2013. An experimental study of UWB device-free person detection and ranging. 2013 IEEE International Conference on Ultra-Wideband (ICUWB), Sydney, Australia, September 15-18, pp. 43-48. Mitra, D., Chu, Y. and Cetin, K., 2021. Cluster analysis of occupancy schedules in residential buildings in the United States. Energy and Buildings, 236, p.110791. Moon JW, Han SH. Thermostat strategies impact on energy consumption in residential buildings. Energy and Buildings. 2011 Feb 1;43(2-3):338-46. Nagy, Z., Yong, F. Y., Frei, M., & Schlueter, A. (2015). Occupant centered lighting control for comfort and energy efficient building operation. Energy and Buildings, 94, 100-108. 122 NEMA. 2016. NEMA WD 7- 2011 (R-2016), Occupancy motion sensors standard. Arlington: NEMA. Ramos, T., Dedesko, S., Siegel, J. A., Gilbert, J. A., & Stephens, B. (2015). Spatial and temporal variations in indoor environmental conditions, human occupancy, and operational characteristics in a new hospital building. PLoS One, 10(3), e0118207. Razavi, R., Gharipour, A., Fleury, M., & Akpan, I. J. (2019). Occupancy detection of residential buildings using smart meter data: A large-scale study. Energy and Buildings, 183, 195-208. Ruparathna, R., Hewage, K., & Sadiq, R. (2016). Improving the energy efficiency of the existing building stock: A critical review of commercial and institutional buildings. Renewable and sustainable energy reviews, 53, 1032-1045. Table of Contents 2020 ASHRAE Handbook—HVAC Systems and Equipment. (n.d.). Www.ashrae.org. https://www.ashrae.org/technical-resources/ashrae-handbook/table-of- contents-2020-ashrae-handbook-hvac-systems-and-equipment. Tan, S. Y., Jacoby, M., Saha, H., Florita, A., Henze, G., & Sarkar, S. (2022). Multimodal sensor fusion framework for residential building occupancy detection. Energy and Buildings, 111828. Turley, C., Jacoby, M., Pavlak, G., & Henze, G. (2020). Development and evaluation of occupancy-aware HVAC control for residential building energy efficiency and occupant comfort. Energies, 13(20), 5396. U.S. Bureau of Labor Statistics, American Time Use Survey. http://www.bls.gov/tus/#data (Accessed on 3rd January 2022). U.S. Energy Information Administration - EIA - independent statistics and analysis. Annual Energy Outlook 2021. (n.d.). Retrieved February 26, 2022, from https://www.eia.gov/outlooks/aeo/. Yang, J., A. Pantazaras, K.A. Chaturvedi, A.K. Chandran, M. Santamouris, S.E. Lee and K.W. Tham. 2018. Comparison of different occupancy counting methods for single system- single zone applications. Energy and Buildings, 172, pp.221-234. 123 CHAPTER 4 – TYPICAL ACADEMIC BUILDING ENERGY MODEL DEVELOPMENT AND ENERGY SAVING EVALUATION OF OCCUPANT-BASED CONTROL 4.1 Abstract Studies on occupant-based controls have been increasing over the past decade because of the significant energy saving potentials they can offer in buildings. Many of these studies focus on office buildings. However, there are very few studies that focus on academic buildings, which represent a significant number of buildings in the U.S., and a substantial amount of energy consumption. The space use, occupant types, and energy-use patterns in academic buildings at universities and colleges can vary substantially from other prototypical building types. The objective of this paper is thus to develop typical academic building models and then use these models to evaluate the energy savings potential of implementing occupant-based control (OBC) in the developed models using EnergyPlus. The U.S. DOE reference office building model was modified and rezoned to add new space types to represent the typical characteristics and space use compositions of academic buildings based on the space type and functional use data collected from 293 academic buildings across five diverse U.S. universities/college. Four types of typical academic building models were then determine based on clustering analysis, including “Office- dominated”, “Laboratory-dominated”, “Study room-dominated”, and “Mixed-use” academic building models. The occupancy schedules were then updated to include stochastic occupancy schedules representing academic building use. Next, the baseline and proposed models were built, of which a fixed setpoint schedule and minimum outdoor air flowrate were used for the baseline model. The proposed model uses OBC, resetting the temperature schedule and minimum outdoor air flow schedule based on occupancy, following the recommendations of ASHRAE 90.1, and 62.1. Results show that there is significant energy saving potential for academic buildings with the 124 implementation of OBC, including an HVAC energy savings of 35% to 51% under “Occupancy presence” scenarios, and a further energy saving increase (3-9%) from “Occupancy presence” scenarios to “Occupancy counting” scenarios. The results of this work also provide typical academic building models with integrated occupancy schedules which can be used to evaluate other energy saving measures, and aid building designers and operators in making informed decisions in applying appropriate control strategies to optimize building energy systems, as well as predict energy use and demand. Keywords: Typical Academic Building Models; Occupant-based Control; Energy Saving Evaluation 4.2 Introduction Building energy consumption accounts for approximately 40% of the total primary energy consumption in the U.S. (Robinson et al., 2017). In commercial buildings, heating and cooling energy consumption takes up the largest portion, which is more than 30% of this energy use, among all different energy end uses (U.S. EIA 2021). Therefore, in an effort to reduce energy use overall, it is important to develop energy saving approaches to help reduce heating and cooling energy use in commercial buildings. In modern commercial buildings in the U.S., mechanical ventilation that brings in outdoor air is required by energy codes to maintain acceptable levels of indoor air quality. Based on the Commercial Building Energy Consumption Survey (U.S. EIA 2021), energy consumption associated with ventilation accounts for approximately 50% of total HVAC energy use in commercial buildings in U.S. Per ASHRAE Standard 62.1-2019 (2019), the minimum required outdoor airflow is dependent on the number of occupants in commercial buildings as well as the size of the space being ventilated. With the emergence of various occupancy sensor systems to 125 better predict real-time presence/non-presence and/or number of occupants in commercial buildings, there are significant opportunities to save energy by applying occupancy-based controls. In addition, consideration of occupancy sensing technologies has also been included in ASHRAE Guideline 36 - High Performance Sequences of Operation for HVAC Systems (ASHRAE, 2018). Given these recent developments in state-of-the art controls, there is a need to evaluate the level of energy savings that can be achieved using such controls. Commercial building reference models (Deru et al., 2011) have been created to represent the most common types of commercial buildings and their energy use patterns. Existing studies mainly focus on evaluating the energy saving potential from OBC for typical office buildings using energy simulation tools. However, these prototypical building models use simplified perimeter and core zoning methods that are not conducive to accurately modeling room based OBC. Various studies have been conducted to improve typical office building models. For example, Im et al. (2019) updated the small and medium office prototype models to include multiple new space types, which enables the updated models to be more flexible in applying OBC by space types. Pang et al. (2021) also proposed a detailed zoning plans for medium and large office reference building models and implemented OBC, of which the potential energy savings was determined to be up to 45%. Buildings used for educational purposes are among the subgroups of commercial buildings in the U.S. that consume a large amount of HVAC energy (U.S. EIA, 2021). Academic buildings at colleges and universities are included among these. These can vary substantially in their space use, occupancy patterns, and resulting energy use. However, despite their substantial energy use, there are no established prototypical building models for academic building types, nor any that can be used to evaluate energy savings potential from occupancy sensors. Given the intermittency in 126 which many spaces in academic buildings are used, this suggests substantial potential for energy savings from occupancy-based control. Therefore, the objective of this research is the development of typical academic building energy models that can be used to evaluate the potential energy savings of occupant-based controls. In an initial study by the authors (Chu et al., 2020) two simplified typical academic building models were developed using five conditioned zones (4 perimeter and 1 core zone) on each of the three stories of the building. The occupancy schedules used for each zone were based on typical characteristics and space use compositions of academic buildings from (Mitra et al., 2019). However, using this simplified version of occupancy modeling and zoning, the energy savings from the use of occupancy-based control likely is underestimated. This is because each zone modeled includes multiple space types, thus if one of the spaces within a particular zone becomes unoccupied, while the others in the same zone are still be occupied, the entire zone is still considered occupied. As such, an improved method to model each space type is to consider each space as a single zone, such that each space may be controlled separately based on the occupancy in that zone. To achieve this, this research first proposes a data-driven method, based on academic building data, to determine the typical space distribution in academic buildings. This research then develops zoning for each floor by assigning a variety of space types and their areas, where each space type is considered as a thermal zone. Sketchup OpenStudio plug-in was used to build the geometry of the typical academic building models, and EnergyPlus was used to complete the building energy simulation. The remainder of this research is organized as follows. Section 2 introduces the methodology for creating academic building energy models, including developing typical space distributions, energy model geometry, occupancy schedules, and occupancy-based controls. 127 Section 3 presents the energy saving estimates from the assigned controls for the typical academic building models developed in this paper. Section 4 summarizes the conclusion of this paper. 4.3 Methodology 4.3.1 Typical space types and distribution in academic buildings To create prototypical academic building models, first the typical space distributions of U.S. academic buildings was determined, based on data collected from multiple universities. The Postsecondary Education Facilities Inventory and Classification Manual (FICM) (Cyros and Korb, 2006) was used for assigning space classifications. Within FICM, there are 10 major space use categories of assignable space and 3 major space use categories of non-assignable space. In this context, “assignable” space means all spaces which are assigned to, or available for assignment to an occupant(s) or specific use, including classrooms, laboratories, offices, study areas, special use space, general use space, support rooms, health care, residential and unclassified space. “Non- assignable” space indicates the spaces cannot be assigned to specific use but are still necessary for the general operation of a building. This includes building service areas, circulation areas, and mechanical areas. The existing space type codes that colleges and universities use have the same categories and the same coding; however, they can have different subcategories of space types under each main space type category. The Carnegie Classification of Institutions of Higher Education (CCIHE) (2018) provides the college/university size categories according to FTE-based enrollment (FTE = Full-Time Headcount plus 1/3 part-time headcount) of the college/university. Based on 2018 CCIHE data, 60% of the 4,323 institutions in the U.S. are four-year institutions. In this research, given the potentially substantial differences in how two-year and four-year institutions operate, this study is limited to only four-year institutions. In order to consider a diversity of university sizes, data was 128 collected from all buildings at five U.S. universities and colleges, each under a different CCIHE size class. Data was obtained from these universities through their facilities management organizations and through publicly available data (THED 2020). The collected data includes detailed building and room information, including the predominant purpose and function of each building, the corresponding spaces, their sizes, and their functional use based on FICM. For the studied colleges and universities, FTE-based enrollment data was obtained from the Integrated Postsecondary Education Data System (IPEDS) (2019) under the National Center for Education Statistics. Table 32 summarizes the five universities with different sizes and associated information. The last column indicates the number of academic buildings at each college/university used for the determination of typical space distributions in this research. Table 32 Colleges and universities, by CCIHE size category and associated characteristics, from which, building and space use data were utilized CCIHE 2018 FTE-based # of Academic Four-year Institutions Class Enrollment Buildings Very small Jarvis Christian College <1000 5 Small Sul Ross State University 1000 to 3,000 19 Medium Lamar University 3,000 to 10,000 42 Texas A & M University-College Large > 10,000 190 Station Iowa State University 37 Among the 293 selected academic buildings, the total area of classroom, laboratory, office, and study facilities makes up more than 50% of the total space area, for approximately 188 academic buildings. Therefore, in this research, we mainly focus on the academic buildings with office, classroom, laboratory, and study room being used as the main occupied spaces. Those buildings like health care facility and special uses were not considered here. Based on this 129 assumption, we defined functional use spaces, which indicate the mostly used spaces, including offices, classrooms, laboratories, and study rooms. In order to determine the percentage of each space type, interquartile range was used to detect outliers for special use, general use, supporting facilities, health care, residential, non- assigned spaces, and other spaces. Using this method, the interquartile range (IQR) of each data is calculated first, then the IQR is multiplied by 1.5, which is then used as a threshold above which data points are considered outliers. Next the outliers are removed, and the mean value of the new data was used as the percentage of each space, including special use (0.32%), general use (2.38%), supporting facilities (0.44%), health care (0%), residential (0%), non-assigned spaces (0%), and others (25.5%). After that, we use 100% to subtract the sum of these spaces other than functional use spaces to attain the percentage of the functional use spaces (71.36%). According to the functional use spaces data, the majority of spaces in these buildings are either office, laboratory, or study room spaces, amounting to more than 70%. A “study-room” is defined as a room or area used by individuals to study at their convenience (FICM, Cyros and Korb, 2006). Since different occupant activities occur due to different functional use space types, this results in varied energy use and savings potential. As such, different typical academic building models are needed to represent different functional uses. First, K-mean clustering (Warne and Ganorkar, 2015) was used to segment the space distribution and composition data from the selected academic building into groups and group the data with similar distributions into one group. In this case, K=5 was used since there are four different functional use space types, and the buildings with 100% office space should be defined as an office building, which would use the existing prototypical office building models. After the clusters of building types (by space use distribution) 130 were attained, the space distribution and composition data were manually checked within each group to confirm buildings appeared to be in the correct cluster. The resulting five groups of buildings include 100% office, study room-dominated, office- dominated, laboratory-dominated, and mixed-use. In these categories “dominated” means that more than 70% of the total space use is associated with that space use type, while the remaining space is for other purposes. Mixed-use indicates that the distribution of functional space types is more evenly distributed compared to those “dominated” space types and no space type accounts for more than 50% of the total space use. Table 33 shows the distribution of the typical academic building types after our classification. Note that the 100% office building type was not developed further as it represents a typical office building, for which a detailed building model already exists (Deru et al 2011). From Table 33, after eliminating the 100% office building type, we can see that the laboratory-dominated academic building group takes up the largest percentage of these four typical academic building models, followed by office-dominated. Mixed-use and study room- dominated academic buildings only account for approximately 12% of the four typical academic building types. Table 33 Typical academic building types based on K-means clustering Building type # of academic buildings Percentage 100% Office 67 22.9% Office-dominated 86 29.3% Laboratory-dominated 114 38.9% Mixed-use 15 5.1% Study room-dominated 11 3.8% 131 For each typical academic building type from Table 33, Figure 12 shows the box plot of functional use space distribution. After removing the outliers using the same interquartile range rule as mentioned above, the mean value of each space type was determined. The distribution of the functional use spaces is shown in across the 293 buildings (Figure 12), which presents the mean percent (by area) of functional space use type in the four defined typical academic buildings. Multiplying the total percentage of the functional area use (71.36%), as calculated before, by the percentage of each functional use space type in Figure 12, we attained the space distribution across the ten assignable spaces and non-assignable space as defined in FICM (Cyros and Korb, 2006), which can be seen in Table 35. Others refers to the spaces that don’t fall into any of the categories defined in FICM (Cyros and Korb, 2006). Figure 12 Functional space use in the four defined typical academic buildings, including the percent of building area used for classroom, laboratory, office, and study areas 132 Table 34 Mean percent (by area) of functional space use type in the four defined typical academic buildings Office- Laboratory- Study room- Mixed-use dominated dominated dominated Classroom 9.74% 3.38% 0.06% 42.28% Laboratory 20.14% 73.57% 7.17% 20.64% Office 69.39% 22.96% 19.92% 34.50% Study room 0.73% 0.09% 72.85% 2.59% Table 35 Mean of FICM space types as a percent (by area) of total available space in the four defined typical academic buildings Office- Laboratory- Study room- Category from FICM Mixed-use dominated dominated dominated Classroom 6.95% 2.41% 0.04% 30.17% Laboratory 14.37% 52.50% 5.12% 14.73% Office 49.52% 16.39% 14.21% 24.62% Study room 0.52% 0.07% 51.99% 1.85% Functional use 71.36% 71.36% 71.36% 71.36% Special use 0.32% 0.32% 0.32% 0.32% General use 2.38% 2.38% 2.38% 2.38% Supporting facility 0.44% 0.44% 0.44% 0.44% Health care - - - - Residential - - - - Non-assigned space - - - - Other 25.50% 25.50% 25.50% 25.50% Total 100% 100% 100% 100% 133 4.3.2 Typical academic building models Next the building energy model was developed in which the space types and occupancy schedules are to be applied. The medium office DOE Reference Building model (Deru et al 2011), which is a three-story building with a total area of 4,982 m2, was used as a starting point for the academic building models. This is based on the finding that the average college/university building is 4,496 m2, which is closest to the medium size office DOE Reference Building model, based on the 2018 CBECS (Commercial Buildings Energy Consumption Survey) data. In the initial model, there are five zones on each floor, including four perimeter zones and one core zone. The gross window-wall ratio (WWR) is 33%. The HVAC system is a VAV (variable air volume) system with reheat. This model was then used to create the four academic building models, as defined above. Based on the total area of the building and percentage space distribution (Table 35), the total area of each space type was calculated (Table 36). Each space type was then further divided into sub-categories, for example, the “classroom” space type was defined to include “classrooms” and “seminar rooms” to represent the space types under that category according to their functionality, as shown in the second column of Table 36. The subcategory area fractions were attained from Im et al. (2019), which indicates the space types and area fractions for major zones of updated small and medium office models, and from Mitra et al. (2020), which explored the area distribution of different academic spaces, including staff/faculty office, conference, seminar room, graduation student office, and classrooms. After determining the area of each space type throughout the building, the spaces must be assigned to each floor. When assigning space types to each floor, the typical unit area of a space type was first studied. For example, the area of a typical office. This was defined based on various resources, such as the Whole Building Design Guide (2021), which provides the floor area of a 134 typical space for a variety of space types. The Time-Saver Standards for Building Types (De Chiara, 2001) was also reviewed, which also provides typical size of different space types for various building types, including educational buildings. The selected typical space type areas are summarized in the third column of Table 36. 135 Table 36 Space types under each category and space area distribution in each typical academic building type Selected typical Office- Laboratory- Study room- Mixed- Category from FICM Space types unit space area dominated dominated dominated use (m2) (m2) (m2) (m2) (m2) Classrooms 72.46 CLASSROOM 346 120 2 1503 Seminar rooms 55.74 Research labs 117.06 LABORATORIES 716 2615 255 734 Teaching labs 62.71 Staff offices 13.94 Faculty offices 13.94 OFFICES 2,467 816 708 1,227 Graduate student offices 27.87 Conference rooms1 44.96 STUDY FACILITIES Study rooms 62.71 26 3 2,590 92 SPECIAL USE Special use 16.00 16 16 16 16 1 Meeting rooms 70.61 GENERAL USE Lounge/ Recreation 18.58 119 119 119 119 Food facilities 11.15 SUPPORTING FACILITIES Active storage2 14.86 22 22 22 22 Corridor - OTHERS Stairway 5.30 1,270 1,270 1,270 1270 Restroom 13.01 1 Note: A conference space is typically equipped with tables and chairs, and is used by a specific organizational unit or office area; Meeting Rooms are used for general purposes such as community or campus group meetings and not associated with a particular department 2 Note: Active storage is an area for use as a part of routine operation of a business. This suggests that occupants will visit this space routinely. This is in contrast to inactive storage which may be visited less frequently. 136 For each typical academic building type, a combination of different space types was assigned to each floor (50 m x 33 m) based on the total area of each space type and typical unit space area. For all three floors, perimeter zones are 5 m in width, and core zone is 35 m x 18.3 m. The corridor in between the perimeter zones and core zone has a width of 2.5 meters. The perimeter zones and core zones were further assigned with different space types to represent each typical academic building type. Dividing the total area of a certain space type on each floor by the typical unit space area, the number of each space types on that floor was then obtained. A summary of the final assignment of space types, the number of rooms for each space type and associated area on each floor for the Office-dominated typical academic building model is presented in Table 37 as an example. The summary for the Laboratory-dominated, Study room-dominated, and Mixed-use academic building models are listed in Table 40-42 in Appendix A. 137 Table 37 Space types of assignment on each floor for the Office-dominated typical academic building model Area of 1st floor 2nd floor 3rd floor Category from Office- Space types Number of Area Number of Area Number of Area FICM dominated rooms (m2) rooms (m2) rooms (m2) Classroom 2 145 2 145 - - CLASSROOM 346 Seminar - - - - 1 56 Research lab 1 117 1 117 3 351 LABORATORIES 716 Teaching lab 2 125 - - - - Staff office - - 15 209 - - Faculty office - - - - 12 167 OFFICES 2467 Graduate student 25 697 18 502 13 362 office Conference - - 6 270 6 270 STUDY 26 Study room 1 26 - - - - FACILITIES SPECIAL USE 16 Special use - - - - 1 16 Meeting room 1 71 - - - - Lounge- GENERAL USE 119 1 38 - - - - Recreation Food facilities - - - - - - SUPPORTING 22 Active storage 1 23 - - - - FACILITIES Corridor 1 291 1 290 1 290 Stairway 1 15 1 15 1 15 OTHERS 1270 Lobby 1 105 1 105 1 105 Restroom 1 15 1 15 1 15 138 Sketchup (2019) and OpenStudio Sketchup plug-in (2020) were then used to draw the geometry of the typical academic building models. Figure 13 shows the floor plans of the office- dominated typical academic building model, including detailed space types, and the building geometry. The floor plans for the other three typical academic building types are summarized in Figure 20-22 in Appendix B. Figure 13(d) presents the 3-D geometry in SketchUp for the typical office-dominated academic building model. (a) (b) (c) (d) Figure 13 Floor plans and building geometry for office-dominated typical academic building model, including (a) 1st floor; (b) 2nd floor; (c) 3rd floor; (d) whole building geometry 139 The geometry was then transferred to EnergyPlus (2020), which was used as the energy simulation tool in this paper. An Energy Management System (EMS) in EnergyPlus was used to apply occupant-based control using the python package Eppy (2020). 4.3.3 Typical occupancy schedule in academic buildings To represent the stochastic nature of occupancy in academic buildings, several methods were used to develop occupancy schedules representative of the space types in these buildings. The LBNL Occupancy Simulator (Chen et al., 2016) was developed to simulate the typical occupancy schedules in commercial buildings. Given that academic buildings have many space types similar to that of commercial buildings and are generally considered one of the subcategories of commercial buildings, this tool was used to aid in the generation of stochastic occupancy schedules. There are, however, only three space types defined in the LBNL Occupancy Simulator, including Office, Meeting Room and Others, yet there is a broader diversity of space types in typical academic building models. Therefore, for the remaining space types not covered, alternative methods and/or assumptions were used. Table 38 summarizes the assumptions for all space types and associated occupancy density for each space type according to ASHRAE Standard 62.1 (2019). The Classroom has similar intermittent occupancy pattern to a Meeting room since there are 1-hour or 1.5-hour classes that occur several times a week in a classroom. However, there is no design population in the Meeting room defined in the LBNL Occupancy Simulator because people who may come to a meeting have already been assigned to offices or classrooms. In addition, the occupant density in classrooms is generally much higher compared to other space types, thus requiring larger amounts of outdoor air for ventilation, and therefore occupancy-based controls are likely to have a more substantial positive impact on energy use. 140 Therefore, the classroom space type was first treated as an Open office, which is the same concept of an office defined in LBNL simulator, but with a lower occupant density compared to a private office. This is to reflect the design population in the classrooms and the whole building since the LBNL simulator calculates the design population for each space through dividing the area of the space by the occupancy density of that space. This schedule was then updated to be a Meeting room schedule because of the similar intermittent occupancy pattern. Seminar was similar to the classroom, but since there is a very small portion of spaces that are seminar rooms, it was considered to be a Meeting room. Research lab, Graduate student office, and Study room spaces were assumed to be Open office with occupancy densities defined based on ASHRAE Standard 62.1. Teaching lab spaces were defining using Meeting room. Staff and faculty offices were assumed to be Private office. Conference and Meeting rooms were treated as Meeting room. The other spaces, including Lounge/Recreation, Food facilities, Active storage, Corridor, Lobby, Stairway, and Restroom were considered as Other space type. 141 Table 38 Assumptions of space types used in LBNL Occupancy Simulator and associated occupancy density from ASHRAE Standard 62.1-2019 Assumption in LBNL Occupancy Occupancy density2 Space types Simulator (m2/person) Classroom Meeting room/Open office 2.86 Seminar room Meeting room 2 Research lab Open office 4 Teaching lab Meeting room 4 Staff office Private office 20 Faculty office Private office 20 Graduate student office Open office 131 Conference Meeting room 2 Study room Open office 10 Meeting room Meeting room 2 Lounge/Recreation Other - Food facilities Other - Active storage Other - Corridor Other - Lobby Other - Stairway Other - Restroom Other - 1 Occupancy density value is from LBNL Occupancy Simulator. 2 Occupancy density values from ASHRAE Standard 62.1 The LBNL Occupancy Simulator assumes that there are no people working on weekends, including Saturday and Sunday. However, this is not always the case in academic buildings, as students may go to campus to do work and/or study on weekends. To reflect this situation, a new occupant type was added, named “Student”. This was added into the occupant types for Office and Research lab space types, with 10% student attendance among all different occupancy types in the building. The example daily occupancy profiles of each space type, including Classroom, Office, 142 Meeting Room and Others, for the typical academic building models on both weekdays and weekends are presented in Figure 14. 1 1 Classroom_Weekday Office_Weekday Office_Weekend Classroom_Weekend 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0:00:00 0 0:00:00 1:10:00 2:20:00 3:30:00 1:10:00 2:20:00 3:30:00 4:40:00 5:50:00 7:00:00 4:40:00 5:50:00 7:00:00 8:10:00 9:20:00 10:30:00 8:10:00 9:20:00 10:30:00 11:40:00 12:50:00 14:00:00 11:40:00 12:50:00 14:00:00 15:10:00 16:20:00 17:30:00 15:10:00 16:20:00 17:30:00 18:40:00 19:50:00 21:00:00 18:40:00 19:50:00 21:00:00 22:10:00 23:20:00 22:10:00 23:20:00 (a) Office (b) Classroom 1 1 Meeting room_Weekday Others_Weekday Meeting room_Weekend Others_Weekend 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0:00:00 0 0:00:00 1:10:00 2:20:00 3:30:00 1:10:00 2:20:00 3:30:00 4:40:00 5:50:00 7:00:00 4:40:00 5:50:00 7:00:00 8:10:00 9:20:00 10:30:00 8:10:00 9:20:00 10:30:00 11:40:00 12:50:00 14:00:00 11:40:00 12:50:00 14:00:00 15:10:00 16:20:00 17:30:00 15:10:00 16:20:00 17:30:00 18:40:00 19:50:00 21:00:00 18:40:00 19:50:00 21:00:00 22:10:00 23:20:00 22:10:00 23:20:00 (c) Meeting room (d) Others Figure 14 The example daily occupancy profiles of (a) Classroom, (b) Office, (c) Meeting room and (d) Others, for typical academic building model on both weekday and weekend 4.3.4 Occupant-based control Strategies 4.3.4.1 Temperature setback According to ASHRAE Standard 90.1 (2019), Occupied-Standby Controls are recommended to be implemented to help support reduced energy use. Following this, active heating and cooling setpoints should be setback at least 0.5 °C when a space is in occupied-standby mode (scheduled to be occupied but there are no occupants in the space). In this research a 1°C temperature setback was applied. 143 4.3.4.2 Ventilation setback Occupant-based ventilation setbacks were also included. These include zone-level and system-level ventilation setbacks (Pang et al., 2020), according to the Ventilation Rate Procedure in the current version of ASHRAE Standard 62.1 (2019). For zone-level control, the breathing zone outdoor air flow rate, Vbz, zone outdoor air flow rate Voz, and zone minimum primary air flow, Vpz-min, are dynamically reset based on the occupancy information. For the system-level, the system minimum outdoor air intake is also dynamically reset based on real-time occupancy information. The system ventilation efficiency Ev is also reset based on detailed methods related to occupancy. Table 39 summarizes the occupant-based control strategies for energy saving analysis of typical academic building models (Pang et al., 2020). It is noted that in Table 2, there are four scenarios considered: Baseline, Temperature setback, Occupancy sensing, and Occupancy counting. The “Baseline” is the model which calculates the energy consumption of an academic building without any occupant-based controls; the “Temperature setback” model includes zone- level temperature setbacks, which decrease/increase the zone setpoint temperature by 1 ℃ depending on if the system is in heating or cooling mode; the “Occupancy presence” model includes the use of occupancy detection technology such that the air handling unit (AHU) outdoor air intake is able to be adjusted based on the presence and non-presence of occupants; the “Occupancy counting” model includes occupancy counting technology enabling ventilation setbacks to be applied depending on the real time number of occupants. All occupant-related variables that require adjustment are summarized in Table 39 with detailed calculations from ASHRAE Standard 62.1. 144 Table 39 Occupant-based control strategies for typical academic building models (Pang et al., 2020) Control Strategy Attribute Baseline Temperature setback Occupancy presence Occupancy counting Zone population 𝑃𝑧 = 𝑃(𝑧𝑑𝑒𝑠𝑖𝑔𝑛) Same as baseline Same as baseline 𝑃𝑧' = 𝑃(𝑧𝑎𝑐𝑡𝑢𝑎𝑙) If Occupied, Same as If Occupied, Breathing zone Vbz=Rp∗𝑃𝑧+Ra∗Az Same as baseline baseline; Vbz'=Rp∗𝑃_𝑧'+R_a∗A_z; outdoor air flow If Unoccupied, Vbz=0 If Unoccupied, V_bz=0 Zone air Ez = constant: Distribution Ez = 1 for cooling, Same as baseline Same as baseline Same as baseline Zone effectiveness Ez = 0.8 for heating level If Occupied, Same as If Occupied, Zone outdoor Voz=Vbz/Ez Same as baseline baseline; V_oz'=V_bz'/E_z; air flow If Unoccupied, Voz=0 If Unoccupied, V_oz=0 If Occupied, Same as If Occupied, V_(pz- Zone minimum baseline; min)'=V_oz'∗1.5; V(pz-min) =Voz∗1.5 Same as baseline Venti- primary air flow If Unoccupied, V(pz-min) If Unoccupied, V_(pz-min) lation =0 =0 setback System 𝑉𝑜𝑢 = ∑ 𝑉𝑏𝑧 If the zone is Occupied, Minimum 𝑉𝑜𝑢 ′ = ∑ 𝑉𝑏𝑧′ 𝐴𝑙𝑙𝑍𝑜𝑛𝑒𝑠 Same as baseline Same as baseline; Uncorrected 𝐴𝑙𝑙𝑍𝑜𝑛𝑒𝑠 If Unoccupied, Vbz=0 outdoor air intake Uncorrected system outdoor D= Ps∕Pzsum air fraction: Xs=Vou'/Vps X_s=V_ou''/V_ps System System Ev=0.88∗D+0.22 for Zone outdoor air fraction: Z_(d,i)=V_(oz,i)/V_(dz,i) Ventilation Same as baseline level D<0.6 Z(d,i)=V(oz,i)/V(dz,i) Z_(d,i_max)=Max(Z_(d,i)) efficiency Ev=0.75 for D≥0.6 Z(d,imax)=Max(Z(d,i)) E_v''=1+X_s-Z_(d,i_max) Ev'=1+Xs-Z(d,imax) System Minimum Vot= Vou/Ev Same as baseline Vot'= Vou'/Ev' V_ot''= V_ou''/E_v'' outdoor air intake Temperature OBC Temperature No 1℃ 1℃ 1℃ setback setback 145 4.4 Results 4.4.1 Baseline energy consumption comparison Chicago (ASHRAE Climate Zone 5A) was chosen as the climate zone for assessment of the energy savings potential for comparison. This is because Chicago is in a cold climate zone where heating is dominant. In this scenario, there will be more energy saving potential in winter with the use of OBC when less outdoor air in required based on a smaller number of occupants. The four typical academic building models developed have different occupancy profiles due to different composition of space types, resulting in different energy consumption. Figure 15 includes the baseline energy consumption each of the four typical academic building models. It is noted that the Mixed-use typical academic building model energy consumption is largest compared to the other three typical academic building models. This may be because that the classroom space type accounts for the largest portion of all spaces in the Mixed-use model. In addition, the maximum number of occupants is the largest; it also has the largest average real-time number of occupants in the building. This is followed by is the typical Laboratory-dominated academic building model which is slightly higher than that of the typical Office-dominated academic building model. This is likely because the portion of laboratory space type in the typical Laboratory-dominated model is higher than the percentage of office space type in the typical Office-dominated model. The occupancy density is also slightly lower in the office space types compared to that in laboratory spaace types. The Study room-dominated model consumes the least energy because it has the lowest portion of classroom, laboratory, and office, and the occupancy density in study area is not as high as in the classroom and laboratory spaces. 146 900000 Office-dominated 800000 Laboratory-dominated Study room-dominated Energy Consumption (kWh) 700000 Mix-use 600000 500000 400000 300000 200000 100000 0 Cooling Heating Total:HVAC Figure 15 Baseline annual energy consumption comparison among four types of typical academic building models in ASHRAE Climate Zone 5A (Chicago) 4.4.2 Occupant-based control – Ventilation Setback 4.4.2.1 Zone-level ventilation setback The zone minimum primary air flow Vpz_min was reset based on occupancy information under both “Occupancy presence” and “Occupancy counting” scenarios. Figure 16 shows the zone minimum primary air flow profiles in four different space types, including office, classroom, meeting room and others, in typical Office-dominated academic building model. Zones on the first floor were selected to represent these four space types, which inlude “Perimeter_bot_GA_OF_1”, “Perimeter_bot_Classroom_1”, “Perimeter_bot_Meeting_RM”, and “FirstFloor_Corridor_1”, respectively. The name of the zones are the names used in EnergyPlus. The notation after each of the zone names represents the three different scenarios, where “_B” indicates the “Baseline” scenario; “_P” indicates the “Occupancy Presence” scenario; and “_C” represents the “Occupancy Counting” scenario. 147 Zone Minumu primary air Zone Minumu primary air flow (m3/s) flow (m3/s) 0 0.05 0.1 0.15 0.2 0.25 0 0.01 0.02 0.03 0.04 0.05 01/05 01:00:00 01/05 01:00:00 01/05 08:00:00 01/05 15:00:00 01/05 09:00:00 01/05 22:00:00 01/05 17:00:00 01/06 05:00:00 01/06 01:00:00 01/06 12:00:00 01/06 09:00:00 Figure 16 Zone minimum primary air flow rate (m3/s) in example spaces: (a) Office -- 01/06 19:00:00 01/06 17:00:00 01/07 02:00:00 01/07 01:00:00 01/07 09:00:00 01/07 09:00:00 (c) 01/07 16:00:00 (a) 01/07 17:00:00 Perimeter_bot_Meeting_RM_B 01/07 23:00:00 01/08 01:00:00 01/08 06:00:00 Perimeter_bot_GA_OF_1_B 01/08 09:00:00 01/08 13:00:00 01/08 20:00:00 01/08 17:00:00 01/09 03:00:00 01/09 01:00:00 01/09 10:00:00 01/09 09:00:00 01/09 17:00:00 01/09 17:00:00 01/09 24:00:00 01/10 01:00:00 01/10 07:00:00 01/10 09:00:00 Perimeter_bot_Meeting_RM_P 01/10 14:00:00 Perimeter_bot_GA_OF_1_P 01/10 17:00:00 01/10 21:00:00 01/11 01:00:00 01/11 04:00:00 01/11 09:00:00 01/11 11:00:00 01/11 18:00:00 01/11 17:00:00 Perimeter_bot_GA_OF_1; (b) Classroom -- Perimeter_bot_Classroom_1; (c) Meeting room -- 148 Zone Minumu primary air Zone Minumu primary air flow (m3/s) flow (m3/s) 0 0.05 0.1 0.15 0 0.02 0.04 0.06 0.08 0.1 01/05 01:00:00 01/05 01:00:00 01/05 08:00:00 01/05 08:00:00 01/05 15:00:00 01/05 15:00:00 01/05 22:00:00 01/05 22:00:00 01/06 05:00:00 01/06 05:00:00 01/06 12:00:00 01/06 12:00:00 01/06 19:00:00 01/06 19:00:00 01/07 02:00:00 01/07 02:00:00 01/07 09:00:00 Perimeter_bot_Meeting_RM; (d) Others -- FirstFloor_Corridor_1. (Note: B = Baseline; P = 01/07 09:00:00 01/07 16:00:00 (d) 01/07 16:00:00 (b) 01/07 23:00:00 FirstFloor_Corridor_1_B Perimeter_bot_Classroom_1_B 01/07 23:00:00 01/08 06:00:00 in the other three typical academic building models are similar to Figure 16. 01/08 06:00:00 01/08 13:00:00 01/08 13:00:00 01/08 20:00:00 01/08 20:00:00 01/09 03:00:00 01/09 03:00:00 01/09 10:00:00 01/09 10:00:00 01/09 17:00:00 01/09 17:00:00 01/09 24:00:00 FirstFloor_Corridor_1_P 01/09 24:00:00 01/10 07:00:00 01/10 07:00:00 01/10 14:00:00 Perimeter_bot_Classroom_1_P 01/10 14:00:00 01/10 21:00:00 01/10 21:00:00 01/11 04:00:00 01/11 04:00:00 01/11 11:00:00 01/11 11:00:00 01/11 18:00:00 01/11 18:00:00 compared to “Office” and “Others” space types. The zone minimum primary air flow rate profiles “Classroom” and “Meeting room” have more vacancy. As a result, the zone minimum primary air scenarios, while for “Occupancy Counting” scenario, the zone minimum primary air flow Occupancy presence sensing; C = Occupancy counting sensing) flow in these space types decreases, thus these areas have a larger potential energy savings, decreases further with a decrease in number of occupants. Among the four different space types, zero when the space is unoccupied under both “Occupancy Presence” and “Occupancy Counting” From Figure 16, it can be seen that compared to the baseline, the Vpz_min decreases to Figure 17 shows the zone outdoor air flow rate profiles in the same four example spaces, including Perimeter_bot_GA_OF_1, Perimeter_bot_Classroom_1, Perimeter_bot_Meeting_RM, and FirstFloor_Corridor_1, respectively. From this figure, the zone outdoor air flow rate decreases from the “Baseline” scenario to “Occupancy Presence” scenario. It decreases further for the “Occupancy Counting” scenario during the daytime while the space is occupied, compared to the “Occupancy Presence” scenario. Similar to the zone minimum primary air flow rate, the zone outdoor air flow rate decreases with a larger vacancy period in the “Classroom” and “Meeting room” space types compared to others. The zone outdoor air flow rate profiles in the other three typical academic building models are similar to Figure 17. 0.05 PERIMETER_BOT_GA_OF_1_B PERIMETER_BOT_CLASSROOM_1_B 0.2 Zone outdoor air flow rate (m3/s) PERIMETER_BOT_GA_OF_1_P Zone outdoor air flow rate (m3/s) PERIMETER_BOT_CLASSROOM_1_P 0.04 0.15 0.03 0.1 0.02 0.01 0.05 0 01:00:00 0 01:00:00 10:00:00 19:00:00 04:00:00 10:00:00 19:00:00 04:00:00 13:00:00 22:00:00 07:00:00 13:00:00 22:00:00 07:00:00 16:00:00 01:00:00 10:00:00 16:00:00 01:00:00 10:00:00 19:00:00 04:00:00 13:00:00 19:00:00 04:00:00 13:00:00 22:00:00 07:00:00 16:00:00 22:00:00 07:00:00 16:00:00 01:00:00 10:00:00 19:00:00 01:00:00 10:00:00 19:00:00 01/05 01/05 01/05 01/06 01/06 01/05 01/05 01/05 01/06 01/06 01/07 01/07 01/08 01/06 01/06 01/07 01/07 01/08 01/08 01/08 01/09 01/09 01/09 01/08 01/08 01/09 01/09 01/10 01/10 01/11 01/11 01/09 01/10 01/10 01/11 01/11 01/11 01/11 (a) (b) PERIMETER_BOT_MEETING_RM_B FIRSTFLOOR_CORRIDOR_1_B 0.12 PERIMETER_BOT_MEETING_RM_P 0.1 FIRSTFLOOR_CORRIDOR_1_P Zone outdoor air flow rate (m3/s) Zone outdoor air flow rate (m3/s) 0.1 0.08 0.08 0.06 0.06 0.04 0.04 0.02 0.02 0 0 01:00:00 01:00:00 10:00:00 19:00:00 04:00:00 10:00:00 19:00:00 04:00:00 13:00:00 22:00:00 07:00:00 13:00:00 22:00:00 07:00:00 16:00:00 01:00:00 10:00:00 16:00:00 01:00:00 10:00:00 19:00:00 04:00:00 13:00:00 19:00:00 04:00:00 13:00:00 22:00:00 07:00:00 16:00:00 22:00:00 07:00:00 16:00:00 01:00:00 10:00:00 19:00:00 01:00:00 10:00:00 19:00:00 01/05 01/05 01/05 01/05 01/06 01/06 01/06 01/05 01/05 01/06 01/06 01/06 01/07 01/07 01/08 01/08 01/07 01/07 01/08 01/08 01/08 01/09 01/09 01/09 01/10 01/08 01/09 01/09 01/09 01/10 01/10 01/11 01/11 01/11 01/10 01/11 01/11 01/11 (c) (d) 3 Figure 17 Zone outdoor air flow rate (m /s) in example spaces: (a) Office -- Perimeter_bot_GA_OF_1; (b) Classroom -- Perimeter_bot_Classroom_1; (c) Meeting room -- Perimeter_bot_Meeting_RM; (d) Others -- FirstFloor_Corridor_1 149 4.4.2.2 System-level ventilation setback For the system-level ventilation setback, the system VAV_1, the air handler servicing the first floor, is used to illustrate the system outdoor air flow profiles under all scenarios. The systems servicing the second (VAV_2) and third floor (VAV_3) are similar to VAV_1. Figure 18 summarizes the outdoor air flow rate for VAV_1 under the “Baseline”, “Occupancy presence” and “Occupancy Counting” scenarios in all typical academic building models. “OD” represents “Office-dominated”; “LD” represents “Laboratory-dominated”; “SD” represents “Study room-dominated”; “M” represents “Mixed-use”. Figure 18 shows that the outdoor air mass flow rate substantially decreases from the “Baseline” to “Occupancy presence” scenario regardless of if the space is occupied or unoccupied. For the “Occupancy counting” scenario, the outdoor air flow rate further decreases based on the actual, real-time number of occupants in each space. 150 System outdoor air flow rate System outdoor air flow rate (kg/s) (kg/s) 0 0.5 1 1.5 2 0 0.5 1 1.5 2 01/05 01:00:00 01/05 01:00:00 01/05 08:00:00 01/05 08:00:00 01/05 15:00:00 01/05 15:00:00 01/05 22:00:00 01/05 22:00:00 01/06 05:00:00 01/06 05:00:00 OD_VAV_1_B 01/06 12:00:00 SD_VAV_1_B 01/06 12:00:00 01/06 19:00:00 01/06 19:00:00 01/07 02:00:00 01/07 02:00:00 4.4.3 Energy savings 01/07 09:00:00 01/07 09:00:00 01/07 16:00:00 01/07 16:00:00 (c) 01/07 23:00:00 (a) 01/07 23:00:00 01/08 06:00:00 01/08 06:00:00 typical academic buildings. 01/08 13:00:00 Figure 18 System outdoor air flow rate (kg/s) in example VAV_1 system of (a) Typical 01/08 13:00:00 01/08 20:00:00 SD_VAV_1_P 01/08 20:00:00 01/09 03:00:00 01/09 03:00:00 OD_VAV_1_P 01/09 10:00:00 01/09 10:00:00 01/09 17:00:00 01/09 17:00:00 01/09 24:00:00 01/09 24:00:00 01/10 07:00:00 01/10 07:00:00 01/10 14:00:00 01/10 14:00:00 01/10 21:00:00 01/10 21:00:00 01/11 04:00:00 01/11 04:00:00 01/11 11:00:00 01/11 11:00:00 SD_VAV_1_C 01/11 18:00:00 01/11 18:00:00 OD_VAV_1_C 151 System outdoor air flow rate System outdoor air flow rate (kg/s) (kg/s) 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 3 01/05 01:00:00 01/05 01:00:00 01/05 08:00:00 01/05 08:00:00 Office-dominated; (b) Typical Laboratory-dominated; (c) Typical Study room-dominated; 01/05 15:00:00 01/05 15:00:00 01/05 22:00:00 01/05 22:00:00 01/06 05:00:00 M_VAV_1_B 01/06 05:00:00 01/06 12:00:00 01/06 12:00:00 01/06 19:00:00 LD_VAV_1_B 01/06 19:00:00 01/07 02:00:00 01/07 02:00:00 01/07 09:00:00 01/07 09:00:00 01/07 16:00:00 01/07 16:00:00 (d) 01/07 23:00:00 (b) 01/07 23:00:00 01/08 06:00:00 01/08 06:00:00 01/08 13:00:00 01/08 13:00:00 01/08 20:00:00 01/08 20:00:00 M_VAV_1_P 01/09 03:00:00 01/09 03:00:00 LD_VAV_1_P 01/09 10:00:00 01/09 10:00:00 01/09 17:00:00 01/09 17:00:00 01/09 24:00:00 01/09 24:00:00 01/10 07:00:00 01/10 07:00:00 01/10 14:00:00 01/10 14:00:00 01/10 21:00:00 01/10 21:00:00 01/11 04:00:00 M_VAV_1_C 01/11 04:00:00 01/11 11:00:00 01/11 11:00:00 01/11 18:00:00 01/11 18:00:00 LD_VAV_1_C and (d) Typical Mixed-use building models energy savings ranges from 35%-52% for “Occupancy presence” scenarios across all types of controls, for typical academic buildings. For ASHRAE Climate Zone 5A, the total annual HVAC applying the proposed occupant-based control, especially occupant-related ventilation setback temperature and zone- and system-level ventilation setbacks for all types of typical academic The overall energy savings achieved through the use of occupant-based control using buildings are shown in Figure 19. This figure shows that significant energy savings is possible by Among “Occupancy presence” scenarios, the typical Mixed-use academic building achieves the highest HVAC energy savings, which may be because it has the largest portion of classroom spaces, with the highest occupancy density. The larger the design population is, the larger the amount of energy savings is that can be achieved when the space is unoccupied or when there is a smaller number of occupants utilizing this pace. This is followed by the typical Office- dominated model, then the typical Laboratory-dominated model. The typical Study room- dominated academic building model achieves the least energy savings, but it is still a significant amount, which is around 35%. The same ranking is also observed when comparing cooling and heating energy savings separately for the “Occupancy presence” scenarios. However, the heating energy savings is much higher compared to the cooling energy savings. This is weather dependent, and is most likely because the climate zone considered is Chicago, where heating requirements are greater due to the cold climate. When comparing “Occupancy presence” and “Occupancy counting”, there is an increase in the energy savings, ranging from 3% to 10%. Among the four typical academic building models under “Occupancy counting” scenarios, the energy savings ranking is similar to that of “Occupancy presence”. The main difference is that energy savings of typical Laboratory-dominated academic building models are similar to that of typical Office-dominated academic building models. This may be because laboratory space type was assumed as Open Office, therefore, a similar occupancy profiles were considered for both typical Office-dominated and Laboratory-dominated academic building models. From “Occupancy presence” scenarios to “Occupancy counting” scenarios, the largest increase in energy savings was observed for the typical Laboratory-dominated model, which is around 9%, indicating 152 that the typical Laboratory-dominated academic building models have larger variations in occupancy profiles when the space is occupied. 70% OD_P LD_P SD_P M_P OD_C LD_C SD_C M_C 60% 50% Energy saving (%) 40% 30% 20% 10% 0% Cooling Heating Total:HVAC Figure 19 Energy saving achieved through Occupant-based Control for typical academic building as compared to the baseline buildings with no OBC in ASHRAE Climate Zone 5A (Note: lighter colors indicate “Occupancy presence” scenarios; darker colors indicate “Occupancy counting” scenarios; _P = occupancy presence scenarios; _C = Occupancy counting scenarios; OD=office-dominated; LD=laboratory-dominated; SD=study room- dominated; M=mixed-use) 4.5 Conclusions Building energy consumption is highly impacted by occupants and their energy use patterns, therefore, there are significant opportunities for energy efficiency improvement by applying occupant-based controls in buildings. This research focuses on, first, defining what a typical academic building is and how it is structured, as currently there no prototypical academic building model available. It then uses this to assess the energy savings potential of the use of occupancy sensors in such buildings. The space type and functional use data was collected from 293 academic buildings across five U.S. universities of different sizes was used to define typical space distribution and 153 composition for use in typical academic building models. Four types of typical academic building models were then defined, which are typical Office-dominated, Laboratory-dominated, Study room-dominated, and Mixed-use academic building models, using clustering. Next the medium size office DOE Reference building model was used as a baseline, as it is similar to that of academic buildings in size, then re-zoned to represent the typical characteristics and space use compositions of academic buildings, to develop the four typical academic building models. The occupancy schedules were initially built using LBNL Occupancy Simulator with some additional modifications. The baseline and proposed models when then built, of which a fixed setpoint schedule and minimum outdoor air flowrate were used for the baseline model. The proposed model utilizes occupant-based controls, which resets the temperature schedule and minimum outdoor air flow schedule under both “Occupancy presence” and “Occupancy counting” scenarios. Baseline energy consumption were first compared across the four building models, indicating that the energy savings evaluation for each of the buildings should be separated since different academic building models have different energy patterns due to different mixes of space functional uses. After applying the occupant-based controls, results show that there is significant energy saving potential for the proposed typical academic building models with the implementation of occupant-based controls. Among all these four typical academic building models, the total HAVC energy savings ranges from 35% to 51% under “Occupancy presence” scenarios. An additional energy saving increase (3~9%) from “Occupancy presence” scenarios to “Occupancy counting” scenarios is also achieved. The results of this work provide typical academic building models with integrated occupancy schedules which can be used to evaluate energy saving measures, and aid building designers and operators in making informed decisions 154 in applying appropriate control strategies to optimize building energy systems, as well as predict energy use and demand. We also noted that the energy savings could vary depending on different variables, such as building characteristics, space distributions, and occupancy schedules, among others. Future efforts should consider a sensitivity analysis to investigate how much influence the variations of these variables could have on building energy savings. 155 APPENDICES 156 APPENDIX A: Space types of assignment for typical academic building model Table 40 Space types assignment on each floor for the laboratory-dominated typical academic building model Area of 1st floor 2nd floor 3rd floor Category from FICM Laboratory Space types Number of Area Number of Area Number of Area -dominated rooms (m2) rooms (m2) rooms (m2) Classroom 1 72 - - - - CLASSROOM 120 Seminar - - - - 1 56 Research lab 4 492 7 940 8 1014 LABORATORIES 2615 Teaching lab 2 126 - - - - Staff office - - 2 166 - - Faculty - - - - 1 166 office OFFICES 816 Graduate 4 392 - - - - student office Conference - - 2 130 - - STUDY FACILITIES 26 Study room 1 23 - - - - SPECIAL USE 16 Special use - - - - - - Meeting 1 71 - - - - room Lounge- GENERAL USE 119 1 38 - - - - Recreation Food - - - - - - facilities SUPPORTING Active 22 1 23 - - - - FACILITIES storage Corridor 1 291 1 290 1 290 Stairway 1 15 1 15 1 15 OTHERS 1270 Lobby 1 105.00 1 105 1 105 Restroom 1 15 1 15 1 15 157 Table 41 Space types assignment on each floor for the study room-dominated typical academic building model Area of 1st floor 2nd floor 3rd floor Category from Study Space types Number of Area Number of Area Number of Area FICM room- dominated rooms (m2) rooms (m2) rooms (m2) Classroom / / / / / / CLASSROOM 2 Seminar / / / / / / LABORATORIE Research lab 1 122 / / / / 255 S Teaching lab 2 126 / / / / Staff office / / 2 189 / / Faculty office / / / / 1 166 OFFICES 708 Graduate student 4 215 / / / / office Conference / / / / 1 135 STUDY 2,590 Study room 5 642 9 1048 8 935 FACILITIES SPECIAL USE 16 Special use / / / / / / Meeting room 1 71 / / / / Lounge/ GENERAL USE 119 1 38 / / / / Recreation Food facilities / / / / / / SUPPORTING 22 Active storage 1 23 / / / / FACILITIES Corridor 1 290 1 290 1 290 Stairway 1 15 1 15 1 15 OTHERS 1,270 Lobby 1 105 1 105 1 105 Restroom 1 15 1 15 1 15 158 Table 42 Space types assignment on each floor for the mixed-use typical academic building model Area of 1st floor 2nd floor 3rd floor Category from Mixed- Space types Number of Area Number of Area Number of Area FICM use rooms (m2) rooms (m2) rooms (m2) Classroom 4 442 5 661 4 433 CLASSROOM 1,503 Seminar / / / / 1 56 Research lab 1 122 1 122 2 382 LABORATORIES 734 Teaching lab 2 126 / / / / Staff office / / 4 319 / / Faculty office / / / / 2 301 OFFICES 1,227 Graduate student 4 392 / / / / office Conference 1 135 / / STUDY 92 Study room 1 23 / / 1 65 FACILITIES SPECIAL USE 16 Special use / / / / / / Meeting room 1 71 / / / / Lounge/ GENERAL USE 119 1 38 / / / / Recreation Food facilities / / / / / / SUPPORTING 22 Active storage 1 23 / / / / FACILITIES Corridor 1 290 1 290 1 290 Stairway 1 15 1 15 1 15 OTHERS 1,270 Lobby 1 105 1 105 1 105 Restroom 1 15 1 15 1 15 159 APPENDIX B: Detailed floor plans and whole building geometry for typical academic building model (a) (b) (c) Figure 20 Detailed floor plans and whole building geometry for laboratory-dominated typical academic building model, including (a) 1st floor plan; (b) 2nd floor plan; (c) 3rd floor plan 160 (a) (b) (c) Figure 21 Detailed floor plans and whole building geometry for study room-dominated typical academic building model, including (a) 1st floor plan; (b) 2nd floor plan; (c) 3rd floor plan 161 (a) (b) (c) Figure 22 Detailed floor plans and whole building geometry for mixed-use typical academic building model, including (a) 1st floor plan; (b) 2nd floor plan; (c) 3rd floor plan 162 REFERENCES 163 REFERENCES ASHRAE. ANSI/ASHRAE Standard 62.1-2019. Ventilation and Acceptable Indoor Air Quality. Atlanta, GA: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.:2019. ASHRAE. ANSI/ASHRAE Standard 90.1-2019. Energy Standard for Buildings Except Low- Rise Residential Buildings. Atlanta, GA: American Society of Heating, Refrigerating and Air- Conditioning Engineers, Inc.:2019. ASHRAE. ASHRAE Guideline 36-2018. High Performance Sequences of Operation for HVAC Systems. Atlanta, GA: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.:2018. Chen, Y., Luo, X., & Hong, T. (2016). An agent-based occupancy simulator for building performance simulation. Chu, Y., Mitra, D., & Cetin, K. (2021). Implementation of Occupant-based Control in Typical Academic Buildings. ASHRAE Transactions, 127(1). Cyros, K. L., & Korb, R. (2006). Postsecondary Education Facilities Inventory and Classification Manual (FICM): 2006 Edition. NCES 2006-160. National Center for Education Statistics. De Chiara, J. (2001). Time-saver standards for building types. McGraw-Hill Professional Publishing. Deru, M., Field, K., Studer, D., Benne, K., Griffith, B., Torcellini, P., ... & Crawley, D. (2011). US Department of Energy commercial reference building models of the national building stock. EIA, U.S., 2018. Commercial Buildings Energy Consumption Survey (CBECS). Ellis, P. G., Torcellini, P. A., & Crawley, D. (2008). Simulation of energy management systems in EnergyPlus. EnergyPlus. (n.d.). Retrieved January 30, 2022, from https://energyplus.net/. Im, P., New, J. R., & Bae, Y. (2019). Updated OpenStudio Small and Medium Office Prototype Models. Oak Ridge National Lab.(ORNL), Oak Ridge, TN (United States). Macumber, D. (2020, October 23). Release sketchup plugin V1.1.0 · openstudiocoalition/ openstudio-sketchup-plugin. GitHub. Retrieved January 30, 2022, from https://github.com/openstudiocoalition/openstudio-sketchup-plugin/releases/tag/v1.1.0. Mitra, D., Chu, Y., & Cetin, K. (2020). Development of Typical Occupant Profiles in Academic Buildings in the United States. ASHRAE Transactions, 126(2). 164 National Institute of Building Sciences. (n.d.). WBDG (Whole Building Design Guide). WBDG. Retrieved January 23, 2022, from https://www.wbdg.org/. Pang, Z., Chen, Y., Zhang, J., O'Neill, Z., Cheng, H., & Dong, B. (2020). Nationwide HVAC energy-saving potential quantification for office buildings with occupant-centric controls in various climates. Applied Energy, 279, 115727. Robinson, C., Dilkina, B., Hubbs, J., Zhang, W., Guhathakurta, S., Brown, M. A., & Pendyala, R. M. (2017). Machine learning approaches for estimating commercial building energy consumption. Applied energy, 208, 889-904. Shulman, L. S. (2001). The Carnegie classification of institutions of higher education (CCIHE). Menlo Park: Carnegie Publication. SketchUp Pro 2019. SketchUp. (n.d.). Retrieved January 30, 2022, from https://www.sketchup.com/. Texas Higher Education Data. THED: Home. (n.d.). Retrieved January 23, 2022, from http://www.txhighereddata.org/. The Carnegie Classification of Institutions of Higher Education ®. Carnegie Classifications | Home Page. (n.d.). Retrieved January 23, 2022, from https://carnegieclassifications.iu.edu/. Warne, P. P., & Ganorkar, S. R. (2015). Detection of diseases on cotton leaves using K-mean clustering method. International Research Journal of Engineering and Technology (IRJET), 2(4), 425-431. Welcome to EPPY's documentation! Welcome to eppy's documentation! - eppy 0.5.44 documentation. (n.d.). Retrieved January 30, 2022, from https://pythonhosted.org/eppy. Zhao, H. X., & Magoulès, F. (2012). A review on the prediction of building energy consumption. Renewable and Sustainable Energy Reviews, 16(6), 3586-3592. 2017–18 Integrated Postsecondary Education Data System (IPEDS) Methodology Report. The Integrated Postsecondary Education Data System. (n.d.). Retrieved January 23, 2022, from https://nces.ed.gov/ipeds/. 165 CHAPTER 5 – CONCLUSIONS AND FUTURE WORK The research presented in this dissertation has led to an expansion of knowledge on the influential variables that impact the reliability of occupancy sensor systems, a standard methodology to test the reliability of various occupancy sensor systems, and the development of typical academic building models and their use to assess the energy savings potential of the use of these technologies. The research findings provide a novel methodology for evaluating the performance of occupancy sensor systems and for developing and accessing technology-based energy savings potential in typical academic building models. 5.1 Conclusions and Contributions Based on the results in this dissertation, the following conclusions can be drawn: 5.1.1 Influential variables impacting the reliability of building occupancy sensor systems Existing occupancy sensor technologies were reviewed and summarized to attain a comprehensive list of the potential influential variables that may cause sensor failures along with stakeholder discussion. Then an expert survey was conducted across a diversity of stakeholders on the most and least important variables impacting occupancy sensor performance based on this comprehensive influential variable list. This resulted in a final list of most and least important variables for assessing energy for both residential and commercial buildings. The most important variables are summarized in Table 43. The results of this work provide insights for sensor manufacturers on what variables industry stakeholders consider to be most important when assessing the performance of occupancy sensor systems. This also provides initial results for use in considering a standard set of variables by which to test occupancy sensor systems for comparative performance evaluation. The unique contribution of this work is the comprehensive 166 influential variable list that might influence occupancy sensor performance based on literature review, and list of the most important variable list for residential and commercial buildings attained from an expert survey. Table 43 Most important variables for residential and commercial buildings Building type Most important (Tier 1) A2. Size (length/width) and shape of test area C5. Level of motion of occupant(s) D1. Presence of pets B1. Lighting level (regardless of source of light) (lux) C4. Spatial location of occupant(s) Residential buildings C1. Number of occupants (including 0) A9. Presence of large objects (especially metal objects) within or near a space B3. Presence of interior lighting sources (non-overhead) A4. Number of doors (entrances/exits) D4. Use of robots A2. Size (length/width) and shape of test area C1. Number of occupants (including 0) B1. Lighting level (regardless of source of light) (lux) C4. Spatial location of occupant(s) Commercial buildings A9. Presence of large objects (especially metal objects) within or near a space C5. Level of motion of occupant(s) B7. Presence of sunlight - direct C7. Clustering of occupants (distance between occupants) 167 5.1.2 A standard methodology to test the reliability of various occupancy sensor systems A standard evaluation methodology was proposed to test the reliability of occupancy sensor systems in residential buildings, including both “Typical testing” and “Failure testing”. “Typical testing” is used to evaluate the reliability of each occupancy sensor system under scenarios that mimic real occupancy scenarios. “Failure testing” follows one variable at a time (OVAT) methods to test the most important variables based on a stakeholder survey (see previous section), to determine if these variables impact sensor system reliability. The proposed methodology for evaluating the reliability of occupancy sensor systems presents an opportunity for use as a standardized method to evaluate the reliability of residential occupancy sensor systems that currently does not exist. The focus on typical scenarios enables the reporting of metrics representing the reliability of the sensor system under typical U.S. household scenarios. This could be used as a comparative measure of performance across sensor systems. The focus on failure testing enables the ability to assess the potential weaknesses in sensor system reliability, which can provide valuable information for sensor developers to adapt their design to avoid sensor failures from the impact of various influential variables, thus improving sensor performance beyond that which could be assessed from typical scenario testing only. Standardized performance metrics were also proposed in this research for both the “typical” and “failure” testing, which are summarized from a state-of-art literature review. Both a detailed confusion matrix and associated metrics were provided, instead of only the use of overall accuracy for “typical” testing evaluation. A case study was completed to verify the feasibility of the proposed methodology. The developed methodology was implemented for a novel occupancy detection sensor system to test the sensor system’s reliability. For “typical testing”, results show that on average, the Precision 168 and Recall are 0.75 and 0.70, respectively, which means that there are similar number of false positives (FPs) and false negatives (FNs) across the dataset. The overall accuracy of the tested novel system ranged from 62.4% to 76.4%. For “failure testing”, individual variable testing provided insights as to the impact of a range of levels of individual variables on sensor system performance. For the tested sensor system, the “Number of occupants”, “Presence of large objects”, “Presence of interior light sources”, and “Number of doors” are not influential, while “Lighting level”, “Location of occupants”, “Another door in the entry/exit area”, and “TV on” variables were determined to impact sensor system performance. Supplementary tests showed that the performance of the evaluated systems is affected most by the positioning of the platforms monitoring the entry/exit doors. Individual sensor evaluations showed that the system can differentiate between the sources of motion, i.e., can differentiate between people and robot vacuums and pets. The unique contributions of this work include the proposed a novel methodology to evaluate the reliability of occupancy sensor systems, including both typical testing and failure testing, in residential buildings. The proposed methodology for evaluating the reliability of occupancy sensor systems presents an opportunity for its use as the basis for development of a standardized method to evaluate residential occupancy sensor systems, including existing sensor systems as well as those being developed and commercialized. Within this methodology, the focus on typical scenarios enables the reporting of metrics representing the reliability of the sensor system under typical U.S. household scenarios. This could be used as a comparative measure of performance across sensor systems. The focus on failure testing enables the ability to identify potential weaknesses in the sensor system reliability, as well as targets for further testing and development. In summary, this effort could help establish and form standards that define a standard 169 way to evaluate occupancy sensor systems’ performance to push forward the applications of various occupancy sensor technologies. With an understand of a testing metric indicating the performance of various sensors systems, this should help encourage those that may be skeptical of performance to realize their potential. 5.1.3 Typical Academic Building Energy Model Development and Energy Saving Evaluation of Occupant-based Control A methodology was proposed to develop typical academic building models. First, the space type and functional use data collected from 293 academic buildings across five U.S. universities covering different sizes of universities was used to attain the typical space distribution and composition in typical academic building models. Four types of typical academic building models were then determined based on cluster analysis, which include typical Office-dominated, Laboratory-dominated, Study room-dominated, and Mixed-use academic building models. Next, the medium size office DOE Reference building model was used as the basis to develop the academic building models in EnergyPlus. It was re-zoned to represent the typical characteristics and space use compositions of academic buildings, to develop the four typical academic building models. The stochastic occupancy schedules were built based initially on the LBNL Occupancy Simulator. Next the baseline and proposed models were created, of which a fixed setpoint schedule and minimum outdoor air flowrate were used for the baseline model. The proposed models utilize the occupant-based controls, which resets the temperature schedule and minimum outdoor air flow schedule under both “Occupancy presence” and “Occupancy counting” scenarios. Baseline energy consumptions were first compared, indicating that the energy savings evaluation should be separated since different academic building models have different energy 170 patterns due to different space functional uses. After applying the occupant-based controls, results show that there are significant energy saving potentials for the proposed typical academic building models with the implementation of occupant-based controls. Among all these four typical academic building models, the total HVAC energy savings ranges from 35% to 51% under “Occupancy presence” scenarios, compared to baseline energy consumptions, and a further energy saving increase (3 - 9%) from “Occupancy presence” scenarios to “Occupancy counting” scenarios, resulting in an overall energy savings potential for “Occupancy counting” of 38-56%. The developed typical academic building models with integrated occupancy schedules can be used as a comparative measure of energy savings across different energy saving measures, and aid building designers and operators in making informed decisions in applying appropriate control strategies to optimize building energy systems, as well as predict energy use and demand. The unique contributions of this work include proposing a novel methodology to define the space distribution and composition for typical academic buildings and classify the typical academic buildings into four different categories, including typical office-dominated, laboratory-dominated, study-room dominated, and mixed-use academic buildings, which could be used for defining typical academic building models. 5.2 Limitations 5.2.1 Influential variables impacting the reliability of building occupancy sensor systems There are many different influential variables and combinations of variables that can impact occupancy sensor systems, some of which may be more important in some building scenarios as compared to others. It was not attempted to rank the order of importance for different building types, such as hospitals versus schools. It is anticipated that they may be variations in these ranking among building types, as well as the frequency of occurrence of various variables. 171 However, this effort was an attempt to define the most and least important variables for residential and commercial building applications overall. The stakeholders that participated in the survey were also mostly based in the U.S., which may influence results. 5.2.2 A standard methodology to test the reliability of various occupancy sensor systems When evaluating the “Typical testing” results, given that there are a number of activities that occur over long durations of many hours (e.g., “unoccupied”), shortened testing durations were proposed to optimize laboratory testing time. The longer time periods were generally reduced to 10 to 20 minutes. Preliminary testing results suggested that a 10- to 20-minute duration was able to capture the behavior of a sensor system sufficiently to mimic a longer period of time. This shorter duration also enables a more time-efficient testing method which is beneficial given this test method requires human subjects to complete. However, there are limitations of the use of the “test” duration for extrapolating to the 24-hour duration that could benefit from further testing as a part of future work. 5.2.3 Typical Academic Building Energy Model Development and Energy Saving Evaluation of Occupant-based Control The typical space distribution and composition for typical academic building models were attained from 293 academic buildings across five U.S. universities covering different sizes of universities. Assumptions were made for the percentage of each space types based on the existing studies, however, there are not any standards or guidelines to use as a reference. In addition, the occupancy schedules were developed, in part, based on the LBNL occupancy simulator, which may also have some limitations. 172 5.3 Future Work 5.3.1 Sensor Reliability and Performance Improvement After the standard methodology for reliability testing, it is also important to consider that if a reliability issue is identified for a certain type of occupancy sensor system, further efforts are needed to mitigate these to improve the sensor’s reliability and performance. For example, if a sensor system is determined to be sensitive to natural daylight, it may be recommended to not install this sensor system near exterior windows, or additional changes to the sensor system may be made to reduce the sensitivity to daylight. 5.3.2 Standardized Performance metrics for reliability evaluation Performance metrics were summarized from a state-of-art literature review. Both a detailed confusion matrix and associated metrics were provided, instead of only the use of overall accuracy for “typical” testing evaluation. However, it is noted that there is not currently a standard that suggests a certain level of performance using these metrics merits a sensor system would “pass”. This could be an opportunity for future work. 5.3.3 Typical academic building models improvement 293 academic buildings across five U.S. universities covering different sizes of universities were used to attain the typical space distribution and composition in typical academic building models. In the future, more university data could be collected to evaluate and further improve the typical academic building models. We also noted that the energy savings could vary depending on different variables, such as building characteristics, space distributions, and occupancy schedules. Future efforts would 173 benefit from a sensitivity analysis to investigate how much influence the variations of these variables would have on building energy savings. 174