By Harshavardhan Vijay Kalbhor A THESIS Submitted to Michigan State University i n partial fulfilment of the requirements f or the degree of Construction Management Master of Science 2019 ABSTRACT ALIGNMENT BETWEEN INTENSITY OF RISK AND LEVEL OF COLLABORATION IN PARTNERED ARCHITECTURE, ENGINEERING AND CONSTRUCTION PROJECTS: A QUANTITATIVE APPROACH TO TEST IMPACTS ON PROJECT PERFORMANCE OUTCOMES B y Harshavardhan Vijay Kalbhor Risk is a typical characteristic of Architecture, Engineering, and Construction (AEC) projects ; the intensity of which is influenced by factors such as the dynamic nature of project elements (e.g., fragmented multi - disciplinary project teams), interactions among these elements, and lack of clear project goals. Project Partnering is a project delivery practice , adopting which, two or mo re organizations commit to harboring an environment of collaboration in a structured approach, with the intention of achieving optimum shared project performance goals (e.g., reduced costs, delays). Project management theory and practice both endorse that as the intensity of risk in a project increases, a higher level of collaboration among the multi - disciplinary project teams is desirable in order to achieve optimal project performance outcomes. However , a theoretical gap exists in providing empirical rein forcement supporting this assertion . The goal of this study is to conduct an empirical examination of the impact of the association between intensity of risk and adopted level of collaboration on performance outcomes of AEC projects. Th is study investigate d 127 partnered projects from the United States completed between 2010 and 2018 . Literature study , exploratory data analysis , and coding were employed to develop models to assess the study variables ( r isk intensity , level of collaboration, fit between them and performance outcomes ) and assign them to projects in the data set. The n on - parametric Kruskal - Wallis test was used to compare performance outcome data across different risk - collaboration fit categories and results are presented accordingly. Copyright by HARSHAVARDHAN VIJAY KALBHOR 2019 iv ACKNOWLEDGEMENTS This thesis is a direct result of the confidence placed in me by Dr. Sinem Mollaoglu, committee chair and my advisor. Had it not been for the opportunity presented by her to explore the world of research, I would never have realized my potential for the same. For this, I will be forever grateful. In the same breath, I must recognize Dr. Angelo Garcia, my first friend in the United States, research partner and brother, who epitomize s the motto of our research team just keep swimming! I am thankful to the International Partnering Institute (IPI) for su pporting this research effort. I would like to thank my parents, Vijay and Sadhana, for the autonomy they provided to make my own choices . I have come to recognize the rarity of that privilege. I would like to acknowledge my younger sibling , Vaishnavi, for her silent but instrumental support. I would like to thank Gayatri, for being the constant now through this portion of our journey as well . I would like to thank the other committee members Dr. Syal, for his wise demeanor and pragmatic opinions; and Dr. Ikpe, for facilitating my introduction to the seemingly daunting world of statistics. The patience and constructive criticism of the committee has been instrumental in shaping this thesis. I will owe any further mastery in Construction Management to the foundation laid by the knowledgeable professors at MSU Dr. Moham ed El - Gafy, Dr. Tariq Abdelhamid, Prof. Tim Mro zowski, Dr. George Berghorn, and Prof. Joseph Maguire. I would also like to acknowledge the administrative staff, particularly Jill Selke , for being welcoming and super - helpful so that students like me could focus on the academics. Last , but not the least; I would like to acknowledge my peers, the CM class of 2018 Animesh, VJ, Ashwini, Shayonne, Lukas and Hannah; and my roommates Bhushan, Nitesh and Saad, for their treasured camaraderie. I also cannot forget Hetal, Hari and Tanvi, my family away from family , who literally provided a roof for me to stay and showed me how to navigate this huge campus. v I apologize if I have missed exp ressing my gratitude for anyone who has touched my life up until my journey to this destination. Thank you for being a part . Hope we keep crossing paths and reminisce our wonderful time here at MSU . Go G reen! G o White! vi TABLE OF CONTENTS LIST OF TABLES ................................ ................................ ................................ ................................ ........... viii LIST OF FIGURES ................................ ................................ ................................ ................................ ............ x CHAPTER 1 INTRODUCTION ................................ ................................ ................................ ..................... 1 1.1 Backgrou nd ................................ ................................ ................................ ................................ ... 1 1.2 Need Statement ................................ ................................ ................................ ............................ 2 1.3 Research Scope ................................ ................................ ................................ ............................. 3 1.4 Research Goa ls and Objectives ................................ ................................ ................................ ..... 4 1.5 Overview of Research Methodology ................................ ................................ ............................. 5 1.6 Deliverables and Outcomes ................................ ................................ ................................ .......... 7 1.7 ................................ ................................ ................................ .............................. 7 CHAPTER 2 LITERA TURE REVIEW ................................ ................................ ................................ ............. 9 2.1 Introduction ................................ ................................ ................................ ................................ .. 9 2.2 Risk in AEC Projects ................................ ................................ ................................ ....................... 9 2.2.1 Defining Risk ................................ ................................ ................................ .......................... 9 2.2.2 Risk Identification and Classification ................................ ................................ ................... 10 2.2.3 Risk Intensity and its Assessment ................................ ................................ ....................... 14 2.2.4 Approaches to Risk Management ................................ ................................ ....................... 19 2.3 Collaborative Practices in the AEC Industry ................................ ................................ ................ 21 2.3.1 Collaborative Information and Communication Technologies (ICTs) ................................ . 21 2.3.2 Collaborative Project Delivery Methods ................................ ................................ ............. 23 2.3.3 Collaborative Project Delivery Practices ................................ ................................ ............. 25 2.4 Partnering in AEC Projects ................................ ................................ ................................ .......... 26 2.4.1 Background and Definition of Partnering ................................ ................................ ........... 26 2.4.2 Partnering Elements and Tools ................................ ................................ ........................... 2 7 2.4.3 Partnering Level and its Assessment ................................ ................................ .................. 30 CHAPTER 3 RESEARCH METHODOLOGY ................................ ................................ ................................ 36 3.1 Introduction ................................ ................................ ................................ ................................ 36 3.2 Research Goa l and Objectives ................................ ................................ ................................ ..... 36 3.3 Nature of Research and Approach ................................ ................................ .............................. 37 3.4 Data Collection ................................ ................................ ................................ ............................ 37 3.5 Study Variables ................................ ................................ ................................ ........................... 39 3.6 Data Analy sis ................................ ................................ ................................ ............................... 42 3.6.1 Data Logging and Cleaning ................................ ................................ ................................ .. 42 3.6.2 Content Analysis ................................ ................................ ................................ ................. 42 3.6.3 Data Coding ................................ ................................ ................................ ......................... 50 3.6.4 Data Cleaning ................................ ................................ ................................ ...................... 63 3.6.5 Validation of Revised Risk Intensi ty and Partnering Level Assessment Models ................. 63 3.6.6 Statistical Tests for Hypothesis Testing ................................ ................................ ............... 64 3.7 Quality Measures ................................ ................................ ................................ ........................ 66 vii CHAPTER 4 RESULTS AND FINDINGS ................................ ................................ ................................ ...... 67 4.1 Descriptive Statistics ................................ ................................ ................................ ................... 67 4.2 Exploratory Data Analysis ................................ ................................ ................................ ........... 70 4.3 Model Validation ................................ ................................ ................................ ......................... 71 4.4 Revised Models of Risk and Partnering ................................ ................................ ...................... 72 4.4.1 Risk intensity assessment model ................................ ................................ ........................ 73 4.4.2 Partnering level assessment model ................................ ................................ .................... 79 4.5 Characteristics and Normality Tests for Dependent Variables ................................ ................... 84 4.6 Hypothesis Testing ................................ ................................ ................................ ...................... 90 4.6.1 Risk - Partnering Fit versus Schedule Growth ................................ ................................ ....... 90 4.6.2 Risk - Partnering Fit versus Cost Growth ................................ ................................ .............. 93 4.6.3 Risk - Partnering Fit versus Increase in Participant Satisfaction ................................ ........... 94 4.6.4 Risk - Partnering Fit versus Number of change orders ................................ ......................... 96 4.7 Summary ................................ ................................ ................................ ................................ ..... 98 CHAPTER 5 CONCLUSIONS ................................ ................................ ................................ ................... 100 5.1 Conclusions from Results and Findings ................................ ................................ ..................... 100 5.2 Deliverables and Implications ................................ ................................ ................................ ... 100 5.3 Limitations and Discussion ................................ ................................ ................................ ........ 101 5.4 Recommendations for Future Research ................................ ................................ ................... 103 APPENDICES ................................ ................................ ................................ ................................ .............. 105 APPENDIX A Sample project award application ................................ ................................ ................ 106 APPENDIX B Risk scaling grades and measures in literat ure ................................ ............................. 119 APPENDIX C Survey consent form ................................ ................................ ................................ ..... 125 APPENDIX D Survey e - mail and design ................................ ................................ .............................. 126 REFERENCES ................................ ................................ ................................ ................................ .............. 129 viii LIST OF TABLES Table 1 List of Risk Elements and Risk Factor Categories in the Literature ................................ ................ 12 Table 2 Number of PPY Applications per Year ................................ ................................ ............................ 39 Table 3 Post - EDA list of Risk Factors and constituent Risk Elements ................................ ......................... 45 Table 4 Post EDA grading scale across Partnering factors ................................ ................................ .......... 49 Table 5 Coding Form for Risk Intensity Assessment (Category 1 Projects) ................................ ................ 51 Table 6 Coding Form for Risk Intensity Assessment (Category 2 Projects) ................................ ................ 53 Table 7 Snapshot of Coding for Risk Intensity Assessment ................................ ................................ ........ 56 Table 8 Coding Form for Partnering Level Assessment (Category 1 Projects) ................................ ............ 58 Table 9 Coding Form for Partnering Lev el Assessment (Category 2 Projects) ................................ ............ 59 Table 10 Snapshot of Coding for Partnering Level Assessment ................................ ................................ .. 60 Table 11 Snapshot of Coding for Performance Outcomes ................................ ................................ ......... 62 Table 12 Classification of Projects by Year of Completion ................................ ................................ .......... 67 Table 13 Classification of Project Locations by States ................................ ................................ ................ 68 Table 14 Classification of Projects per proj ect type ................................ ................................ ................... 69 Table 15 Classification of Projects per project delivery method ................................ ................................ 70 Table 16 Classification of Projects per budget category ................................ ................................ ............. 70 Table 17 Pearson's Chi - Square Test Results ................................ ................................ ............................... 72 Table 18 Risk register for the revised Risk Intensity assessment model ................................ .................... 74 Table 19 Revised Risk Intensity Assessment Model (Category 1 Projects) ................................ ................. 75 Table 20 Revised Risk Intensity Assessment Model (Category 2 Projects) ................................ ................. 77 Table 21 Partnering register for the revised Partnering Level assessment model ................................ ..... 81 Table 22 Rev ised Partnering Level Assessment Model (Category 1 Projects) ................................ ............ 82 ix Table 23 Revised Partnering Level Assessment Model (Category 2 Proje cts) ................................ ............ 83 Table 24 Normality Test for Schedule Growth ................................ ................................ ............................ 85 Table 25 Normality Test for Cost Growth ................................ ................................ ................................ ... 86 Table 26 Normality Test for Increase in Participant Satisfaction ................................ ................................ 87 Table 27 Normality Test for Number of change orders ................................ ................................ .............. 89 Table 28 Kruskal - Wallis Test for Fit versus Schedule Growth ................................ ................................ ..... 91 Table 29 Dunn Test for Fit versus Schedule Growth ................................ ................................ ................... 91 Table 30 Kruskal - Wallis Test for Fit versus Cost Growth ................................ ................................ ............ 93 Table 31 Kruskal - Wallis Test for Fit versus Increase in Participant Satisfaction ................................ ......... 95 Table 32 Kruskal - Wallis Te st for Fit versus Number of change orders ................................ ....................... 96 Table 33 Exhibit - A of Risk Analysis Scale (Source: Hannah, Thomas & Swanson, 2013) .......................... 119 Table 34 Exhibit - B of Risk Analysis Scale (Adopted from: Baccarini & Archer, 2001) ............................... 119 Table 35 Exhibit - C of Risk Analysis Scale (Source: Kindinger and Darby, 2000) ................................ ....... 119 Table 36 Post - EDA Risk Element Grading (Horizontal Projects) ................................ ............................... 121 Table 37 Post - EDA Risk Element Grading (Non - Horizontal Projects) ................................ ....................... 123 x LIST OF FIGURES Figure 1 Depiction of an AEC project (Syal, 2017) ................................ ................................ ........................ 1 Figure 2 Risk Scalability Matrix I (adopted from International Partnering Institute) .............................. 15 Figure 3 Risk Scalability Matrix II (adopted from International Partnering Institute) ............................. 16 Figure 4 Risk Scalability Matrix III (adopted from International Partnering Institute) ............................ 17 Figure 5 Partnering Level Assessment I (adopted from International Partnering Institute) ................... 32 Figure 6 Partnering Level Assessment II (adopted from International Partnering Institute) .................. 33 Figure 7 Partnering Level Assessment III (adopted from International Partnering Institute) ................. 34 Figure 8 Collected (sample) data set with respect to population ................................ ............................... 38 Figure 9 Number of projects by year of completion ................................ ................................ ................... 67 Figure 10 Number of projects per state ................................ ................................ ................................ ...... 68 Figure 11 Classification per Project Type ................................ ................................ ................................ .... 69 Figu re 12 Classification per Project Delivery Method ................................ ................................ ................. 69 Figure 13 Classification of Projects per original contract amount ................................ .............................. 70 Figure 14 Average Schedule Growth across Fit Categories ................................ ................................ ........ 85 Figure 15 Average Cost Growth across Fit Categories ................................ ................................ ................ 86 Figure 16 Average Increase in Participant Satisfaction across Fit Categories ................................ ............ 87 Figure 17 Average Number of change orders across Fit Categories ................................ ........................... 89 Figure 18 Fit versus S chedule Growth ................................ ................................ ................................ ........ 92 Figure 19 Fit versus Cost Growth ................................ ................................ ................................ ................ 94 Figure 20 Fit versus Increase in Participant Satisfaction ................................ ................................ ............ 96 Figure 21 Fit versus Number of change orders ................................ ................................ ........................... 97 1 CHAPTER 1 INTRODUCTION 1.1 Background An Architecture, Engineering and Construction (AEC) project can be considered to be a series of activities and tasks undertaken over a specific period of time (Syal, 2017) in which its stakeholders define goals , specific objectives, design s and specifications for the project; and then strive to achieve those goals with available and limited resources (e.g., budget, time, manpower ). Not only is there a high number of project elements (e.g., tasks, sp ecialists, and subsyste ms) in AEC projects, but there also exists substantial variation and interdependency among them (Baccarini, 1996). This makes AEC projects complex. Moreover , there exist probable events, whose occurrences may affect the performance a nd successful completion of an AEC project. Poor performance of project elements or failure to achieve project goals can prove to be costly for its stakeholders (e.g., accidents, economic losses, damage to organizational image) (And erson & Merna, 2003) . Therefore, AEC projects are invariably characterized as being risky. Figure 1 Depiction of an AEC project (Syal, 2017) R isk cannot be eliminated from AEC projects, but it can be managed (Smith, Merna, & Jobling, 2014) . The process of aiding project management decision - making by utilizing the practices of risk identification, analysis, response planning, and monitoring and control is called r isk m anagement (Project Management 2 Institute, 2009) . Risk m anagement is an integral component of p roject m anagement, the effectiveness of which is associated with the performance of projects. One vital part of AEC projects , which also constitute s a major portion of their complexity, is its multidisciplinary and fragmented project teams. They are required to work and coordinate with each other to achieve project goals and objectives. These teams usually have little to no prior experience of working together nor adequate time to develop relationships. Over the last few decades, as construction projects have become riskier , there is a rise in the need and level of collaboration. T he industry is observing a rise in the adoption of several collaboratio n - based project delivery methods (e.g., Design - Build, Integrated Project Delivery) , technologies (e.g., BIM), and practices (e.g., Lean Construction, Project Partnering). Research shows that inter - organizational teamwork and level of collaboration among AE C project teams affects project performance (e.g., cost, schedule, quality, safety) (Chan, Ho, & Tam, 2001; Azmy, 2012). 1.2 Need Statement In project and risk management literature, c ollaboration among project participants is often mentioned as a requiremen t for effective risk management (Al - Bahar & Crandall, 1990; Azmy, 2012; Hanna, Thomas, & Swanson, 2013). In fact, some researchers theorize that non - cooperative behavior is a threat to the effectiveness of risk management (Hanna, Th omas, & Swanson, 2013) . However, a lthough both researchers and practitioners commonly discern that collaboration is an effective risk management strategy, there exists a theoretical gap in providing empirical reinforcement supporting this assertion. A part of this gap is due to the lack of a structured framework for investigating collaboration. The practice of Project Partnering provides such a structured framework to study collaboration analytically . Project Partnering is a project delivery practice; adopting which, two or more organizatio ns commit to harboring an environment of collaboration (e.g., effective communication, shared vision, goal alignment, trust) in a structured approach, with the intention of achieving optimum shared project performance goals (e.g., reduced costs, delays, li tigation). Additionally, e xisting Partnering literature is largely qualitative and presents conceptual models, potential benefits to adopting Partnering, barriers to its adoption, critical success factors, and performance measurement and evaluation methods . Quantitative research in this domain is limited and has studied projects either coming largely from a single source of ownership (e.g., DOTs) or a project type (e.g., 3 horizontal infrastructure projects) at a time. There is a lack in Partnering literature of evidence - based quantitative research that identifies specific factors linked to partnered - project success using data from a large and diversified sample of projects. Thus, there was a need for an empirical assessment of the association between risk (sp ecifically its intensity) and collaboration (or partnering), and its impact on project performance. This need prompted the undertaking of this study. The outcomes of this study are twofold. Firstly, it add s to the body of knowledge of project and risk mana gement via collaborat ion . The pragmatic significance is assistance to owne rs, stakeholder s and project managers alike in setting and managing expectations of partaking in collaborative behavior; thus, helping them to ma ke informed decisions when entering into collaborative arrangements for risk management on their projects . Secondly, this study adds to the theory of best practices in Partnering with respect to partnering for effective risk management. 1.3 Research Scope As dis cussed in the previous section, i t is rather difficult to capture the concept of collaboration across various types of projects in a structured and consistent manner for the purpose of such a study. Project Partnering offers a potential solution. Partneri ng is a project delivery practice constituting a structured approach by which two or more organizations commit to harboring an environment of effective communication, shared vision and trust with the intention of achieving common project objectives (e.g., reduced costs, change orders, litigation). resolution strategies, etc., usually with the help of a third - party neutral facilitator. As partnered pro jects possess a structured framework within which efforts for collaboration are systematized and assessable, this study focuses on studying projects that have adopted Project Partnering. With respect to studying risk for this research , Tah and Carr (2000) support the assertion there are two types of construction project risks internal and external. Internal risks (e.g., cost pressure) manifest from causes whose origins fall within the general management scope of the project and in general can be controll ed. External risks (e.g., government shutdown) are usually not controllable because the factors causing them are outside the general management scope of the project. This research will primarily focus on risks internal to project s . 4 D ue to lack of existen ce of predefined or commonly accepted terminology to capture the concept of this Fit denotes the relation between risk intensity and level of partne ring on a project: The higher the level of collaboration (i.e., partnering) with respect to intensity of risk in a project better is the fit. A lthough due diligence was done during revision and subsequent verification of models to capture the constructs of risk intensity and collaboration (or partnering) level during this research , the study largely relie d on existing literature about what contributes to risk and collaboration i n a project. The primary focus of this research was to capture the interplay between these constructs and investigate its correlation with project performance. Hence, sophisticated methodology (e.g., f actor analysis ) for verifying the authenticity of the models was outside the scope of this research. Lastl y , quantitative analysis was used to test the study hypothesis designed in adherence to research goals and objectives stated in subsequent section s . In summary, the framework of Partnering is adopted in this research to facilitate the study of collaborati on in a structured manner. Hence, the units of analysis for the study are partnered projects in the US. In the 1.4 Research Goals and Objectives Responding to the need statement stated in Section 1.2 , the goal of the research was to investigate partnered projects for the impact of the fit between their intensity of risk and adopted partnering level on their performance outcomes (e.g., cost, schedule). The need statement and goal of this research can be effectively assimilated into the research question : partnered AEC projects, does the fit between risk intensity and level of partnering correlate with performance of the project To answer this question, the following hypothesis was developed : (e.g., cost growth, schedule growth) To achieve the study go al, the following objectives were delineated : 5 1. V ia a literature review, identify and develop (if required) models to ascertain following constructs of interest : a. Risk i ntensity of a project; b. Partnering level of a project; c. Fit between risk intensity and part nering level of projects; and d. Project performance outcomes. 2. U sing the outcomes of objective one , systematically code quantitative measures of these constructs from project details in the data set. 3. Conduct statistical hypothesis testing to ascertain if the re exists a correlation between the fit between risk intensity and partnering level in partnered projects and its performance outcomes. 1.5 Overview of Research Methodology Data collected and used for this research is archival in nature containing details of 1 27 partnered AEC projects from the United States completed between 2010 and 2018. These details comprise of answers to a questionnaire enquiring performance, etc. A detailed description of the data set is presented in Section 3.4 . For objective one , the researcher commenced a literature review to present the state - of - art theory about risk, collaborative practices (especially partnering) and effect of their interplay on project outcomes. Models used in the industry to assign risk intensity and pa rtnering level to a partnered project were found as a result. Following the above finding , preliminary content analysis was attempted for a few projects in the data set, to code constructs of interest (e.g., risk intensity, partnering level) using t he se mo dels within the available project data . However, it was realized that the models had limitations with respect to available project data in the data - set . Hence, exploratory data analysis was undertaken to address these limitations with the objective to revi se the models per objective one . A ddition al literature review was conducted and revised risk intensity and partnering level assignment models were developed. As the models found via literature study were revised, it was imperative to test the correctness of these models. For this, a survey was presented to experts (i.e., neutral third - party part nering facilitators ) of the projects in the data set requesting the m to assign values for risk intensity and partnering level to their project. Statistical analysis was used to test the hypothesis that the values assigned by the experts for both risk intensity and partnering level are not significantly different from thos e assigned via the revised 6 models. It was proved that the hypothesis was true and h ence, values for risk intensity and partnering level assigned using the theoretically developed models were used for further analysis. Further, d ue to lack of predefined or commonly accepted terminology to capture the concept of this Literature supports the introduction of such variables. For example , in economics, the change in demand versus the change in price of goods. Fit denotes the relation between risk intensity and level of partnering on a project. F it level for a project was defined by the researcher as follows : Partnering level higher than intensity of risk indicated a P artnering level equal to intensity of risk indicated a Partnering level lesser than intensity of risk indicated a Thus, th e product of completing objective one was development of models and procedures to be used to identify constructs of interest of this study viz., risk intensity, partnering level, fit, and performance outcomes. For objective two, content analysis and codin g were employed to systematically identify and assign values for the constructs of interest (risk intensity, partnering level, fit, and performance outcomes) using models and procedures developed via objective one. The product of completing objective two w as that each project in the data set was assigned quantitative values for performance outcome metrics (e.g., schedule growth, cost growth); and was categorized into one of the three fit categories (positive, neutral and negative). For objective three , the research hypothesis that intensity of risk and adopted partnering level, better is its performance tested using statistical tests for comparison of samples across the three f it categories. For this purpose, a performance outcome metric of projects categorized into the three fit categories was considered as three different samples . For example, it was statistically tested whether values for the schedule growth metric was differ ent for projects in at least two of the three fit categories (positive, neutral and negative). If a statistically significant difference was found, further statistical tests comparing two samples (or fit categories) at a time were conducted to identify whi ch fit category performed better in terms of schedule growth compared to the other two fit categories. 7 Throughout the research, appropriate measures to maintain research quality were undertaken at every step. For example, construct validity was establishe d via comprehensive literature review. Further, revised risk intensity and partnering level models were verified via a survey. During the coding exercise, inter - coder reliability was maintained by utilizing two coders followed by random cross - checks . Detai led description of quality efforts is outlined in Section 3.7 . 1.6 Deliverables and Outcomes The deliverables of this study are tools or models for risk intensity assessment and simultaneous determination of recommended level of partnering. The study was able to contribute to the body of knowledge of risk management via collaboration by providing empirical reinforcement to the association or lack of thereof between risk, collaboration and performance. Lastly, this study provides guidelines for best practices in Partnering contributing to effective risk management on AEC projects. Decision - makers in the construction industry (e.g., owner organization, stakeholders) will be able to support to their request or demand to implement an intensity of collaborative efforts on projects with respect to assessed risk intensity . Owners will be able to manage t heir expectations with regards to the impact of collaboration on the risk of their project. In addition, empirical evidence about how risk and collaboration affect performance could assist in convincing other project team members like contractors and trade s about the expectations and importance of partaking in collaborative behavior. 1.7 CHAPTER 1 of this research presented the background of the domain of ri sk management via collaboration. It was followed by establishing the need for such a study. Then, the scope of this study was delineated, and its goals and objectives mentioned. Planned research methodology to achieve these goals and objectives was explain ed and finally expected deliverables were laid out. CHAPTER 2 presents the literature review conducted for this study. It presents existing theories about risk and ap proach to its management, an overview of various collaborative practices in the industry and details about the particular practice of Partnering. CHAPTER 3 presents the r esearch m ethodology, which expands on the goals and objectives of this research in detail as well as describes the data collection strategy, data coding efforts, approach and 8 techniques used for data analysis. It concludes with explanation f or the quality measures adopted at various stages in this study. CHAPTE R 4 pre sents the findings of the study . Beginning with descriptive statistics outlining releva nt characteristics of the data set, it proceeds to explaining the process and results of exploratory data analysis . It presents details and results of hypothesis tests . CHAPTER 1 concludes this thesis by presenting a summary of the findings, outlining lessons learned, reviewing expected and actual deliverables, presenting limitations of this study and recommending directions for future research. 9 CHAPTER 2 LITERATUR E REVIEW 2.1 Introduction This chapter presents the literature review for this research, based on which the researcher will provide perspective for selection of research methods and subsequent quantitative analysis. The literature review is divided into three parts (i) theory and state of practice of risk & its management; (ii) overview of collaborative practices in the AEC industry; and (iii) description of the collaborative project delivery practice of Project Partnering. The first section of the literatur e review covers aspects of risk and its management (e.g., risk definition, identification, risk analysis and approaches to risk management) in project management literature with a focus on AEC projects. At the outset, risk is defined and then the character istics of AEC projects that contribute to risk are listed and classified. Then, risk analysis metrics, processes and ranking scales from existing studies are presented. Further, state - of - art approaches to risk management recognized in literature are presen ted. This part concludes with underlining the trend of rise in need for collaborative risk management in the AEC industry. Keeping this trend of the rise in need for collaboration in mind, the second section presents an overview of the practices in the AEC information and communications technologies (e.g., BIM, cloud - based integration); (ii) project delivery practices (e.g., Design - Build, IPD); and (iii) project delivery practic es (e.g., Lean Construction, Project Partnering). An effort is made to identify risk management aspects of these practices. The third section presents the collaborative project delivery practice of Project Partnering in depth, which is the focus of this re search. A brief history of Project Partnering is presented along with its definition, description of its tools and its critical success factors. A special effort is made to study the state - of - art risk management aspects of this practice. A risk identificat ion, analysis and ranking matrix found in literature is presented and critiqued. 2.2 Risk in AEC Projects 2.2.1 Defining Risk Risk is a critical characteristic of AEC projects, which determines the selection of suitable project managerial actions required to comple te the project successfully (Baccarini, 1996). Project Risk 10 Management is a vital part of Project Management (Project Management Institute, 2009) . There is significant debate and lack of consensus on the definition and composi management literature (Whitty & Maylor, 2009; Gratt, 1989). For the purpose of this research, the following definition of risk will be considered: nt, or series of events of various (Gratt, 1989) . Complexity is another characteristic of an AEC project and it is important to understand its role in ; and can be As far as the relationship between complexity and risk is concerned, researchers suggest that complexity is a factor of project risk (Weidong & Lee, 2005; Vidal & Marle, 2008). Fang and Marle (2012) advocate that complexity of a project leads to the existence of a netwo rk of interdependent risks. In fact, construction companies are exposed to risk at a high level because of increasing complexity (Hanna, Thomas, & Swanson, 2013) . This research considers complexity to be a component of risk. Fu rther, Tah & Carr (2000) support that there are two types of construction project risks internal and external. Internal risks (e.g., cost pressure) manifest from causes whose origins fall within the general management scope of the project and in general government shutdown) are usually uncontrollable because the factors causing them are outside the general management scope of the project. This research primarily deals with risks internal to the pro ject. 2.2.2 Risk Identification and Classification Risk identification is considered the first step in project risk management as it provides the basis for the next steps of risk management (e.g., risk analysis , risk response planning, and risk control) (PMI, 2 007). Chapman (2001) opines that the process of risk identification has direct influence on the contribution that risk analysis and management makes to the overall project management of construction. Effective risk identification leads to effective risk ma nagement (Tchankova, 2002). Generally, contractors rely on experience and rules of thumb when dealing with risk because there is a lack of a systematic way to prioritize risk and its elements (Al - Bahar & Crandall, 1990) . Hence, it is important to not only identify risks but also classify them. Risk classification aids in expanding awareness 11 about the risks involved for its stakeholders and deciding risk management strategies per the nature of risks (Al - Bahar & Crandall, 1990) . Several efforts have been made to identify and classify risks in the AEC industry. These efforts are usually part of a larger risk management model and the categories reflect the risk management tools or practices presented in those models. This res earch studied various risk factors and their risk classification categories in literature. A summarized list of the risk elements and their classification categories are presented in Table 1 below: 12 Table 1 List of Risk Elements and Risk Factor Categories in the Literature Source Risk Element Risk Factors Luo, He, Xie, Yang, & Wu, 2017 Organizational Vertical and horizontal differentiation across or ganizations; degree of operational interdependencies and interaction; number of members, departments, organizations, regions, nations, languages, time zones; power structure, number and diversity of actors, diversity of the cultural human mindset, size, r esources, project team, trust, and risk; contractual conditions, number of contract/work packages, coordination of stakeholders, and project planning and scheduling. Technological/ infrastructural Variety or diversity of some aspect of a task; interdependencies among tasks and teams; technology, innovation system, uncertainty of the process or demand; density of activities in a spatial and temporal frame; building type, overlapping of design and construction works, and dependency on project oper ation; variety of technologies employed and technological newness of the project; site compensation and clearance, transportation systems, and qualifications required for contractors. Resource Project scale; budget size. Directional/objective Degree of independence when defining operations to achieve given goals; ambiguity related to multiple potential interpretations of goals and objectives; ambiguity of project scope, and project size in terms of capital; various project in managing and keeping track of the large number of different interconnected tasks and activities. Sociopolitical Administrative policies/procedures, number of appli cable laws and regulations, local experience expected from parties, and influence of politics. Environmental Local climatic conditions, geographic conditions, and environmental risks. Al - Bahar & Crandall, 1990 Acts of God Flood, earthquake, landslide, fire, wind, lightning. Physical Damage to structure, damage to equipment, labor injuries, material and equipment fire or theft. Financial and economic Inflation, availability of funds from client, exchange rate fluctuation, financial default of subcontractor, non - convertibility. Political and environmental Changes in laws and regulations, war and civil disorder, requirements for permits and their approval, pollution and safety rules, expropriation, embargoes. Design Incomplete design scope, defective design, errors and omissions, inadequate specifications, different site conditions. 13 Table 1 (cont d) Zavadskas, Turskis, & External risk Political - changes in government laws of legislative system, regulations and policy, improper administration system; economic - inconstancy of economy in the country, repayment situation in manufacture sphere, inflation, funding, and contractor could not properly assess either their probability o r their cost impact; social, weather. Project risk Time, cost, work quality, construction, technological, resource. Internal risk Resource; project member - team member turnover, staffing build up, insufficient knowledge among team members, cooperation, motivation, and team communication issues; construction site, documents and information. Baccarini & Archer, 2001 Method of establishing targets The way cost targets were established; the way time targets were established; the way quality targets were established. Consequence of failure to meet targets The effect if cost targets are not met; the effect if time targets are not met; the effect if quality targets are not met. Project features Uniqueness of the product; complexity of deliverables; financing; adequacy of funds; project location; project surroundings; hazardous materials; definition of project; site availability; project justification; project approvals; lity and competency of contractors; procurement method; stakeholder interest. International Partnering Institute, 2018 Project value Scale of the project (e.g., mega, large, small); project budget. Complexity Technical complexity, design complexity, construction complexity, schedule constraint, uncommon materials. Political significance Visibility, oversight, strategic significance, organizational image at stake, size of client, importance of location of project. Relationships New project relationships between owner, contractor, subcontractors, etc., turnover rate of subcontractors, potential for conflict, strained relationships, previous litigation. Desired Level of Engagement Seeking risk mitigation, seeking high project efficiencies. 14 2.2.3 Risk I ntensity and its Assessment Some risks are more significant to the stakeholders than others are ; and hence the risk analysis process is important step between risk identification and determination of r isk management strategy (Al - Bahar & Crandall, 1990) . There is a need to rank and prioritize risks in a project in order to focus the risk management e ff ort on the greater risks (Baccarini & Archer, 2001). If the definition tha t risk is a function of the probability of occurrence of an uncertain event and the severity of its impact is considered, it is then logical to say that higher the probability of occurrence of a risky event and more severe the impact of its occurrence, hig her is the intensity of risk. Hanna, Thomas, & Swanson (2013) suggested the use of the following two parameters to grade risk: (i) probability of risk realization and (ii) extent of impact on project objectives if the risk realizes. Both the parameters were measured in percentage like risk intensity, was then defined as the product of the two parameters. However, in the AEC industr y, there is no exact science to calculate probability of risk occurrence , nor estimate the severity of its impact. Moreover, different stakeholders may have different perception about the probability and severity of realization of a risk. Nonetheless, ris k intensity assessment is a vital step towards risk management and must be undertaken as an effective project management strategy. A preliminary literature search for existing risk intensity assessment models led to the finding of matrices (International Partnering Institute, 2018) developed by the International Partnering Institute ( IPI ) who administered the questionnaire for the collection of data used in this research. A discussion with a member of IPI (who was the Director a t the time when the matrices were developed ) affirmed that t committees comprised of industry experts from the AEC industry (personal communication, March 04, 2019). These models have been presented bel ow: 15 Figure 2 Risk Scalability Matrix I (adopted from International Partnering Institute) 16 Figure 3 Risk Scalability Matrix II (adopted from International Partnering Institute) 17 Figure 4 Risk Scalability Matrix III (adopted from International Partnering Institute) 18 Before utilizing the matrices for further analysis, they were e valuate d based on peer - reviewed risk assessment literature for validity. To do so, first , various characteristics of the models were observed and then their basis in literature, if any, was examined. Observations about the models are as follows: i. Separate matrices exist for horizontal (e.g., roads, bridges), vertical (e.g., commercial buildings) and avia tion (e.g., runways, terminals). It is important to note that aviation projects include both vertical (e.g. terminal buildings, watch towers) and horizontal (e.g., runway) projects. The justification for a separate aviation category is that the number, structure and power to influence of stakeholders (e.g., security, customs) in aviation projects add an extra layer of risk, which is acknowledged via a separate catego ry. However, it is observed that the models for vertical and aviation are practically similar. This suggests that risk intensity perception varies per project type (e.g., vertical, horizontal, aviation) . ii. In each matrix, various risk elements (e.g., projec t size, visibility, complexity) are identified which are then classified into risk factors (e.g., project value, complexity, political significance, relationships a nd desired level of engagement). Although the levels of risk are different per the project t ype, risk factors, constituent risk elements and their descriptions are common across all three matrices. iii. The intensity of risk of a particular ri sk factor is graded on a n ordinal scale of 1 - 4 for horizontal projects and 1 - 5 for vertical and aviation project s. Although no rule for identifying the overall intensity of risk is seen, overall risk intensity presumably is again scaled on a scale of 1 - 4 for horizontal projects and 1 - 5 fo r vertical p rojects based on consensus of project stakeholders involved in risk assessment exercise. With regards to observation one, it is found that although the risk factors on a project are common across the AEC industry, their intensity varies dependi ng on the type of project. For example, horizontal AEC projects experience a lower level of risk because of the high percentage of self - performed work. On the contrary, vertical and aviation projects usually consist of a prime contractor coordinating sever al trades or subcontractors who perform the work, thus adding an extra layer of risk to the project. Thus, there is basis in literature for the development of separate models for horizontal (e.g., roads, bridges), vertical (e.g., buildings) and aviation (e .g., runways, terminals). Of these, the models for vertical and aviation are practically similar and hence will be analyzed as one henceforth. 19 For observation two, Sections 2.2.1 and 2.2.2 have already present ed the basis for risk identification and the need for classification . With respect to common risk factors and elements acro ss all project types, there is evidence in literature that risk assessment models or processes (Hanna, Thomas, & Swanson, 2013; Kindinger & Darby, 2000; Baccarini & Archer, 2001) commonly consider a generic set of risk elements and classifications for risk assessment across project types. For observations three, Kindinger & Darby (2000) recommend that risk analysis must be conducted using a graded approach with the intention of fitting the risk management approach to the needs of the project (e.g., project size, data availability, requirements of project team). Hanna, Thomas, & Swanson (2013) have used a 5 - point Likert scale in their risk rating model to grade risk elements. Baccarini & Archer (2001) in their risk model have collected the ratings given by t he different stakeholders and averaged the numbers to present risk factors and their corresponding grading scale. In summary, it was concluded that there is support in peer - reviewed literature for the risk models developed by IPI . Hence, these models were analysis phase. 2.2.4 Approaches to Risk Management Risk Management is a systematic process of identification and assessment of risks and determination of strategies for risk mitigation and responses to occurrences of risk events over the life of a project with the objective of achieving project goals (e.g., cost, schedul e, safety) and maximizing project value (e.g., participant satisfaction, quality) (Al - Bahar & Crandall, 1990) . In the previous sections, the researcher elaborated on the methods of risk identification and risk analysis. In this section, a general overview of the approaches to risk management is presented. 2.2.4.1 Traditional Approach to Risk Management Appropriate risk allocation which refers to the contractual distribution of risk among project stakeholders (e.g., owner, designer, an d contractor) often, seems to be the objective behind most of the risk management models. Risk allocation is often seen as a contentious process in AEC industry because parties attempt to transfer as much risk as possible to other parties (Hanna, Thomas, & Swanson, 2013) . Bargaining power or power imbalances influence risk allocation (Zhang, Zhang, Gao, & Ding, 2016) and in practice, most of the risk 20 gets transferred to parties least capable of hand ling them because of their limited bargaining power (Chen & Hubbard, 2012) . Risk misallocation negatively affects the cooperative behavior of stakeholders in AEC projects (Zhang, Zhang, Gao, & Ding, 2016 ) . However, the goal of optimal risk management should be to minimize the total cost of risks to a project, not necessarily the costs to each party separately (CII 1993) and risk - irrespective of whose risk it may be (ASCE 1979). 2.2.4.2 Collaborative Approach to Risk Management The possibility of non - collaborative behavior seems to be inherent in the Risk Allocation approach towards risk modelling. Rahman & Kumaraswamy (2004) commented on this approach to risk modeling and set forth the following reasoning for adopting a collaborative approach towards risk modeling. According to them, first, a complete, definitive and exhaustive allocation of risks cannot be achieved because not all construction risks foreseeable at the outset. Second, quantification of foresee able risks may be neither always possible nor correct because as a project progresses, the nature and extent of foreseeable risks may change, new risks may emerge, and existing risks may change in importance. Third, some of the risks may require the combin ed efforts of all contracting parties for their effective management. complexity of AEC projects. Her model is based on a new classification of risk suggeste d by Das & Teng, (2001). They suggested that all risk can be classified as either relational risk relates to achieving the goals of collaboration or performance risk achieving the goals of the technical undertaking provided that the collaboration funct making a case for a collaborative approach to risk management was that technical solutions are well developed in the AEC industry and thus substantial potential for performa nce improvement is embedded in human interaction as well as rise in recognition for need for collaboration in the AEC industry. The rise in the need for collaboration in the AEC industry has led to the development of various collaborative practices in the form of (i) technologies (e.g., BIM, Cloud - based integration); (ii) project delivery methods (e.g., Design - Build, Integrated Project Delivery); and (iii) project delivery practices (e.g., Lean Construction, Project Partnering). These collaborative practic es are discussed in the next section. 21 2.3 Collaborative Practices in the AEC Industry 2.3.1 Collaborative Information and Communication Technologies (ICTs) This section deals with Information and communications technologies (ICT) in the AEC industry. ICT refers to the extension of the use of information technology to include communication systems (e.g., wireless systems) and enterprise software that enable organizations to access, store, transmit and manipulate information. All these technologies help in facilitatin g efficient and error - free information exchange among project stakeholders and team members. Hence, they are characterized as being collaborative in nature. ICT has witnessed applications in the AEC industry as response to a spectrum of organizational and managerial issues in project management. Some prime examples of ICT applications seen in the AEC industry are (Lu, et al., 2015) : (i) Web Based Systems; (ii) Virtual/Augmented Reality Technologies; (iii) Wireless Technologies; (iv) Electronic Data Interchange/Electronic Data Management System; and (v) Building Information Modelling (BIM). A brief overview of these technologies, their collaborative implications and risk management aspects is presented below. 2.3.1.1 Web - Based Systems Mo st ICT applications depend heavily on the web technology development in both the design and management process. Examples of web - based ICT applications include decision support systems, information management systems, and collaborative contract change manag ement systems. The benefits of using web technology are generally focused on effective collaboration, communication and coordination, and the decision - making process (Lu, et al., 2015) 2.3.1.2 Virtual/Augmented Reality technologies - end user interface which is interactive, spatial, in real time (Burdea and Coiffet, 2017) . Augmented reality (AR) is a variation of VR that creates an environment in which digital information is inserted in a predominantly real - world view (Wang et al. 2013). Many studies have focused on the application of VR and AR technologies in the design a nd construction process. These studies provide evidence that VR/AR technology is effective (i) in promoting collaboration among participants such as during design review process (Hammad, Wang, & Mudur, 2009) ; (ii) monitoring 22 co nstruction progress (Golparvar - Fard, Peña - Mora, & Savarese, 2009) ; and (iii) improving organizational performance and decision making capacity for architecture design companies (Lu, et al., 2015) . 2.3.1.3 W ireless Technologies Wireless technology allows the transfer of information between two or more points that are not connected by an electrical conductor. Common wireless technologies in the construction industry include radio frequency identification techn ology, personal digital assistants (PDAs), and global positioning systems (GPS). All of these applications offer considerable benefits in terms of information collection, exchange and storing, thus improving collaborative work (Lu, et al., 2015) ; the decision - making process; and project performance (Wang L. , 2008) . In addition, wireless technology can be integrated with other ICT applications. For example, the integration of wireless technology and a gent - based systems can support collaborative work by providing real - time field data capturing (Lu, et al., 2015) . 2.3.1.4 Electronic Data Interchange/Electronic Data Management System Electronic data interchange (EDI) or electronic data management system (EDMS) is a seamless data interchange tool that enables communication among different computer systems or computer networks. As a result, various EDI systems in the construction industr y are designed to remove the barriers for collaboration in a geographically fragmented industry. 2.3.1.5 Building Information Modelling (BIM) Not all construction risks are foreseeable at the outset (Rahman & Kumaraswamy, 2004) . Uncer tainty in AEC projects arises because architects, engineers and constructors cannot completely visualize and therefore predict potential design, construction, or operational issues that could occur on the project. Building information modeling (BIM) has em erged as an innovative way to virtually design and manage AEC projects. With BIM technology, an accurate virtual model of a building is digitally constructed which helps architects, engineers, and constructors visualize what is to be built in a simulated e nvironment. BIM modelling has various applications or functionalities (Azhar, 2011) like: (i) Visualization (e.g., 3D renderings); (ii) Construction sequencing (e.g., 4D scheduling); (iii) Cost estimating (e.g., 5D cost estimat ion); (iv) Fabrication/Shop Drawings; (v) Code review (e.g., fire code review); (vi) Conflict detection; (vii) Forensic analysis; and (viii) Facilities management. Additionally, BIM encourages integration of the roles of all stakeholders on a project. It facilitates inter - organizational and intra - organizational collaboration, communication, and cooperation (Dossick & Neff, 2011) . 23 Adoption of BIM greatly improves the predictability of building performance and operation thereby e liminating some uncertainty in AEC projects. The process of applying BIM is a systematic way for managing risks and is expected to play a significant role in facilitating risk management in AEC projects. (Zou, Kiviniemi, & Jones, 2015). Primarily, BIM has the potential to manage risks in the planning and design phase, where one significant risk is alignment of the design with project feasibility, cost estimates and constructability (Miller & Lessard, 2001) . In addition , collabor ation within project teams leads to reduction in financial risk (e.g., improved profitability, reduced costs), schedule risk (e.g., better time management) and relational risk (e.g., improved customer client relationships). 2.3.2 Collaborative Project Delivery M ethods For AEC projects, which usually have numerous project players (e.g., owner, designer, contractor, supplier, stakeholders, financers, end - users), an operational structure is needed for smooth operations. This structure must account for appropriate s election of project players, assignment of tasks and roles for them highlight financing, processes and procedures and define hierarchies. Such a structure with legal agreements and lines of privity is called a project delivery method. Design - Bid - Build (DB contracts separately with the contractor and the designer and there exists no line of privity between the contractor and the designer. The contractor further contracts subcontract ors. This method has several weaknesses (Sweet & Schneier, 2009) like: (i) lack of contractor input during the design process; (ii) longer project duration owing to linear sequencing of design and construction activities; and ( iii) risk of adversarial and non - collaborative behavior by project players, among others. This led to the evolution of variations to this delivery method like Design - Build (DB), Construction Manager as General Contractor (CM/GC) or at Risk (CMAR), Integra ted Project Delivery (IPD), etc. Though these variations differ among themselves, one common undertone is an intention to facilitate collaboration. e ffort is made to define them, highlight their collaborative nature and their advantages in terms of risk management. 24 2.3.2.1 Design - Build - bid - project delivery where the owner contracts separately with the designer or engineer to design the project and with a contractor to execute the design. One principal variation of this system involves combining - hod of project delivery. DB provides the means to overcome some of the fragmentation in the AEC industry as it inherently requires more collaboration. It fosters this collaboration among parties that normally harbor adversarial relationships (e.g., design er and contractor), and this facilitates the creation of shared project objectives and mutual goals (Yates, 1995) . Primary advantages of DB include (Sweet & Schneier, 2009; Yates, 1995; Koch, Gransberg, & Molenaar, 2010): (i) m inimization of contractual lines of privity from three (owner, designer, constructor) to two (owner and design - construction entity); (ii) ability to fast - track the project; (ii) minimization of administrative task of owner; (iii) reduction of design risk; (iv) reduction in change orders; and (iv) facilitation of collaborative decision making leading to innovation. There are some disadvantages to DB as well (Sweet & Schneier, 2009) like: (i) loss of the designer as a paid designer - constructor combined entity; (iii) contention between the owner and contractor on the whether the project should be fixed price or an open - ended cost contract; and (iv) potential problems with progress payments. 2.3.2.2 Construction Management Construction Management developed as an alternative to the traditional project delivery because of the c onstruction details and its cost; and contractors sometimes lacked construction technique skills (Sweet & Schneier, 2009) . This project delivery method is essentially characterized by the involvement of a entity, which provides professional services to deliver a project within a Guaranteed Maximum Price (GMP) while acting as a consultant to the owner. Sometimes, the CM may also contract with subcontractors to provide part or whole of the construction services. These variations have been classified in two classifications or approaches to Construction Management (i) Construction 25 Manager as Agent/Risk (CMAR); and (ii) Construction Manager as Constructor/General Contractor (CMGC). Advantages of CM proje ct delivery method include (Asmar, Hanna, & Loh, 2016; Gransberg, Shane, & Schierholz, 2013): (i) early involvement and input of CM before design is complete enabling stakeholder collaboration and minimizing design risk; (ii) improvement in project perform ance metrics (e.g., cost, schedule) in comparison to other delivery methods; and (iii) risk transfer to the CM. One major disadvantage of the CMGC/CMAR project delivery method is that, like DBB, there is no contractual privity between the CM and the design er. This places the owner between these entities to resolve any issues that may arise. Other disadvantages include (Molenaar, Harper, & Yugar - Arias, 2014) : (i) projects are not competitively bid but negotiated with the CM entit y; and (ii) GMP may include large contingency to cover risk and incomplete design. 2.3.2.3 Integrated Project Delivery Integrated Project Delivery (IPD) is a form of Relational Contracting mechanism that seeks to distribute equitably the responsibilities and bene fits of the contract transparently between the parties, based on the underlying principle of collaboration. Matthews & Howell (2005) defined IPD as an organiza tion able to apply IPD seeks to mitigate some issues with traditional project delivery methods (Forbes & Ahmed, 2011) like: (i) lack of field input in designing the project; (ii) inhibition of cooperation and innovation; and (iii) lack of coordination among planning systems. Mutual respect, mutual benefit, early goal definition , enhanced communication, clearly defined open standards, appropriate technology, high performance and leadership are some essential principles of IPD (AIA California Council, 2007) . 2.3.3 Collaborative Project Delivery Practices Operational structures for AEC projects with no legal obligations for the parties are called project delivery practices . Just like legal variations to the DBB project delivery method highlighted above, operational structures like Lean Construction and Project Partnering have been developed with the intent of mitigating the risk of non - collaborative behavior in DBB. Due to no associated legal structures to these structures, they can be implemented with any project delivery method. 26 The project delivery practice of Lean Construction has been highlighted in this sub - section. A separate section is dedicated to Project Partnerin g, as it is the focus of this research where it will be reviewed in detail. - based production ximum possible (Koskela, Howell, Ballard, & Tommelein, 2002) . The Construction Industry Institute identified five principles of lean construction (Ballard, Kim, Jang, & Liu, 2007) : (i) Customer focus; (ii) Culture and people; (iii) Workplace organization and standardization; (iv) Elimination of waste; and (v) Continuous improvement and built - in quality. The report stated that lean project delivery seeks Various tools and techniques have been developed to implement the Lean Construction project delivery system (Ballard, Tommelein, Koskela, & Howell, 2002) during various phases of project delivery like: (i) Lean work structuring; (ii) Last Planner S ystem; (iii) Lean design; (iv) L ean supply; (v) Lean assembly; and (vi) Lean installation. Lean constructio n applications are noted to be successful with forms of contract that reward collaboration. Collaboration among project team members is often listed as a requirement or an outcome of implementing Lean project delivery practices. 2.4 Partnering in AEC Projects 2.4.1 Background and Definition of Partnering cooperate in (Cook & Hancher, 1990) . In 1987, the Construction Industry Institute (CII) established a task force to evaluate the feasibility of using Partnerin g in the construction industry. The task force concluded that Partnering offered many opportunities to improve the total quality and cost effectiveness of construction projects while developing an atmosphere conducive to innovation, teamwork, trust, and co mmitment (CII, 1989) . achieving specific business objectives by maximizing the effectiveness of each participant's resources. The 27 relationship is based on trust, dedication to common goals, and an understanding of each other's Several other definitions of Partnering can be found in literature: should not be confused with other good project management practice, or with longstanding relationships, negotiated contracts, or preferred supplier arrangement, all of which lack the structure and objective measu (Construction Industry Board, 1997) . together regularly throughout the life of a proje ct. Partnering provides a space for communication, improved strategy, and issue resolution. Over time, partnered teams build trust, a reliable predictor of high performing teams. Through Partnering, fragmented teams coalesce and unify around a shared objec (International Partnering Institute, 2018) . In summary, Project Partnering is a collaborative project delivery practice that encourages the development of communication, trust and mutual comm itment among participating stakeholders with the objective of successfully achieving the goals of a project. 2.4.2 Partnering Elements and Tools V arious models for project P artnering in construction have been devel oped in literature. For example, Cheng & Li (200 1), Crowley & Karim (1995), CII (1996), etc. These models present partnering processes, identify key factors affecting partnering processes and outline challenges to partnering implementation. A summary of Partnering elements and tools used in the AEC indu stry is given below (California Department of Transportation, 2013): 2.4.2.1 Executive Sponsorship Executive Sponsorship refers to the support of top management in implementing Partnering on the project. Usually financial in nature, executive sponsorship represent s executive level commitment to the process of Partnering on the project which is identified as a key element in partnering success. Executive Sponsorship is utilized for Partnering activities such as training or team building activities. 28 2.4.2.2 Facilitat ion A fa cilitator is an entity, which possesses knowledge about both P artnering and con struction. The Facilitator is hired to facilitate the partnering process on a project. The facilitator is selected before the start of the Partnering process, which can be at an y phase in the project. However, the ideal time to hire the facilitator is upon selection of the project team (e.g., general contractor, construction manager, designer. A facilitator can either be (i) In - House; or (ii) Third Party Neutral Facilitator . The selection is usually made based on the scale and desired level of collaboration deemed necessary for the project. The facilitator is selected by the project stakeholders either via a request for qualification process or prior working experience as well. Fa cilitator certifications are awarded by organizations such as IPI , etc. which assist project members make an informed selection. Once selected, the role of the Partnering Facilitator is to : (i) arrange the Kick - off Par tnering Workshop and facilitate subsequent Partnering Workshops; (ii) train and guide the participants in the Partnering process; (iii) assist the project stakeholders and team members to develop the Partnering Charter and update it as the project progresses; (iv) conduct team building activities; (v) administer partnering performance surveys and interpret responses to improve the partnering process; (vi) develop issue resolution process for the project. 2.4.2.3 Partnering Meetings/Workshops/Sessions Partnering Meetings (also called Workshops o r Sessions) are where project stakeholders and team members get together to discuss the progress of the project, forecast upcoming challenges and resolve existing challenges and disputes. There are three types of Workshops viz. (i) Kick - off; (ii) Interim; and (iii) Close - out A Kick - off Workshop is the very first workshop, which signals the commencement of Partnering on the project. The workshop is attended by the project stakeholders, team members and supply chain parties. This workshop is formally facilita ted, and its agenda includes deciding mutual goals and objectives for the project, outlining these in the Partnering Charter, formulating a dispute resolution plan and identifying key risks for project success. Interim Workshops are Partnering Workshops he ld during the course of the project. The frequency of these workshops is decided during the Kick - off Workshop. It can be either monthly, quarterly, half - yearly or yearly depending upon various factors (e.g., risk level, desired level of collaboration on th e project). 29 These workshops may or may not be facilitated. The agenda of these workshops is to follow - up on project and collaborating performance. Activities include updating the Partnering Charter, resolving disputes, holding team building events, holding Partnering performance evaluations, etc. The Close - out Workshop signals the conclusion of Partnering on the project and sometimes even the end of the project itself. The objective of the close - out Workshop is to analyze, and record lessons learnt on the p roject and recognize efforts on the participants. Partnering Awards, which is the formal rewarding of exceptional Partnering performances on the project, is another practice that may or may not be held during the Close - out Workshops 2.4.2.4 Partnering Charter The Partnering Charter is one of the key elements of the Partnering process. It is a document usually prepared at the Kick - Off Workshop . It highlights the goals and objectives set by the project stakeholders, team and supply chain parties for the p roject in terms of cost, quality, safety, as well as communication, trust, etc. Participants sign the Charter signifying their commitment to Partnering on the project. The Charter guides the Partnering process as the project progresses. The Charter may be updated at subsequent Workshops to reflect new goals. Partnering Charter can be used as a good measure of goal alignment on the project. 2.4.2.5 Alternative Dispute Resolution Alternative Dispute Resolution practices are adopted in Partnering to resolve disputes a nd avoid claims or litigation. A Dispute Resolution Association/Board is set up consisting of member representative of all project stakeholders and teams. An Issue Resolution Ladder is also set up with ranks ranging from the field level, project manager le vel, executive level, upper management to owner level. Such a set up encourages dispute resolution at the field level where they arise and only for the serious issues to be escalated upwards. As the time an issue can stay at a level is fixed, speedy and ef fective resolution of disputes is seen on projects that implement this practice. 2.4.2.6 Evaluation Surveys Partnering Surveys are used to evaluate the performance of Partnering efforts on the project. These surveys are usually administered during the Interim Work shops and the responses are collected by the Facilitator confidentially. He/ She then decides the future course of action by analyzing these responses. 30 A Facilitator Evaluation survey may also be conducted, usually at the Close - Out Workshop, to evaluate th e performance of the Facilitator. This survey is collected by the owner agency. 2.4.2.7 Multi - tiered Partnering Multi - tiered Partnering is an advanced Partnering tool. Under this tool, separate Partnering Workshops are held for the Executive, Core and Project Team s to discuss issues specially pertaining to those management levels. 2.4.2.8 On - Boarding/Off - Boarding Another advanced Partnering tool, it can be applied to both subcontractors and stakeholders. This tool is most effective when many stakeholders and subcontractor s are involved in the project and their roles in the project are expected to be intermittent. It refers to stakeholders/subcontractors involved in the Partnering Workshops when their roles are current and letting them go when their roles are over. This hel ps the Partnering Workshops be more focused towards on - going tasks and issues on the project. 2.4.2.9 Special Task Force Special Task Force refers to the creation of a team involving stakeholders, team members, etc. to resolve a challenge or issue. The team is cre ated to resolve that one issue, focused towards solving it and is dissolved once the solution is obtained. This tool helps in effective resolution of challenging issues that require specialized knowledge, time and focus to solve. In summary, Project Partne ring possesses a structured framework to implement collaborative practices. Hence, the researcher has chosen partnered projects as the focus of this research. 2.4.3 Partnering Level and its Assessment As Partnering is a structured process for collaboration invo lving the implementation of its tools, the level of such collaboration can be regulated to a certain degree via the variations in the implementation of the tools. For example, partnered projects facilitated by a neutral third - party facilitator show a highe r level of collaboration compared to a project that is facilitated by in - house project members . Another example is that partnered project participants are more likely to show increased cooperation when partnering sessions or workshops are held more frequen tly (e.g., monthly compared to quarterly or yearly) . Thus, number and/or frequency of the implementation of the Partnering tools determines the of 31 collaboration on the pr oject; sometimes also ing . The structured framework Project Partnering provides helps assigning a quantitative measure to the level of collaboration on an AEC project. The level of Partnering on an AEC project is a function of the whether various Partnering tools were u sed or not and the frequency of some tools. The International Partnering Institute ( IPI ) is a non - profit organization at the forefront of providing tools for Partnering implementation. A preliminary literature search led to Partnering Level assessment matr ices developed by IPI (International Partnering Institute, 2018) . These matrices have been presented below: 32 Figure 5 Partnering Level Assessment I (adopted from International Partnering Institute) 33 Figure 6 Partnering Level Assessment II ( adopted from International Partnering Institute) 34 Figure 7 Partnering Level Assessment III (adopted from International Partnering Institute) 35 Like risk intensity matrices, before utilizing the partnering level matrices for further analysis, they were evaluated based on peer - reviewed partnering level assessment literature for validity. To do so, first, various characteristics of the models were observed and then their basis in literature, if any, was examined. Observations about the models have been noted below: i. Separate matrices exist for horizontal (e.g., roads, bridges), vertical (e.g., buildings) and aviation (e.g., runways, terminals) out of which, the models for vertical and aviation are practically similar. This suggests that partnering level implementation varies per project type (e.g., vertical, horizontal, aviation). ii. In each matrix, various partnering tools (e.g., facili tation, workshops) are identified which are then differentiated via their frequency ( e.g. , 2 project surveys, monthly surveys). Although the partnering levels are different per the project type, partnering tools and their frequency variations are common ac ross all three matrices. iii. The overall partnering level is graded on an ordinal scale of 1 - 4 for horizontal projects and 1 - 5 for vertical and aviation projects. With regards to observation one, it is found that although the partnering tools on a project are common across the AEC industry, resultant level of collaboration varies depending on the type of project. For example, horizontal AEC projects, due to the rigid nature of contract types, long project durations and less number and variety of project stakeh older members, lesser level of collaboration is achieved with the use of same partnering tools and frequency as would be achieved on a horizontal or aviation project. In addition, if partnering level is to be aligned and selected based on the risk intensit y then it makes sense to have the same separate matrices for partnering level as for risk level. For observation two, Section 2.4.2 explains how partnering tools and practices have variations dependent on their use or lack of thereof as well as frequency of use. For observation three, if partnering level is to be aligned and selected based on the risk intensity then it is natural that the partnering level scales must be on the same scale as those determined for the risk levels. In summary, it was concluded that there is support in peer - reviewed literature for the partnering level models developed by IPI . Hence, these models were used for preliminary content analysis. 36 CHAPTER 3 RESEARCH METHODOLOGY 3.1 Introduction CHAPTER 1 provided the background of the research topic followed by CHAPTER 2 , which presented a literature review about its state - of - art. Combined, the chapters led up to providing peer - reviewed affirmation about the existence of an epistemological gap in the investigation into the collective impact of the interplay between risk and collaboration via partnering on project performance. It furthers this study by outlining the resea rch goals, objectives and hypothesis, followed by describing the nature o f this study and the research approach adopted. It also provides a detailed description of collected data , which is archival in nature . Further, it repor ts on the results of preliminary content analysis, subsequent exploratory data analysis, development of revised risk intensity and partnering level assessment models, and their validation via a survey . Lastly, it presents the analysis methods used and finally co ncludes with outlining research quality measures implemented in this study . 3.2 Research Goal and Objectives Literature review showed that Partnering literature lacks evidence - based quantitative research that identifies specific factors linked to partnered - pro ject success using data from a large and diversified sample of projects. In addition, in project management literature, there was a need for an empirical assessment of the association between risk (specifically its intensity) and collaboration (or partneri ng), and its impact on project performance. This twofold need prompted the undertaking of this study. Responding to this need, the goal of the research was to investigate partnered projects for the impact of the fit between their intensity of risk and ado pted partnering level on their performance outcomes (e.g., cost, schedule). The need statement and goal of this research can be effectively assimilated into the research question: f partnering correlate To answer this question, the following hypothesis was developed: better is its performance (e.g., 37 To achieve the study goal, the following objectives were delineated: 1. Via a literature review, identify and develop (if required) models to ascertain following constructs of interest: a. Risk intensity of a project; b. Partnering level of a project; c. Fit between risk intensity and partnering level of projects; and d. Project performance outcomes. 2. Using the outcomes of objective one, systematically code quantitative measures of these constructs from project details in the d ata set. 3. Conduct statistical hypothesis testing to ascertain if there exists a correlation between the fit between risk intensity and partnering level in partnered projects and its performance outcomes. 3.3 Nature of Research and Approach The nature of this re search is explanatory, and the approach is quantitative. Constructs in the data set are primarily quantitative in nature. Literature review was used to identify an array of variables from the data to produce a hypothesis concurring with the research question. Content analysis was used to identify and encode constructs of interest in the data set. Furt her, quantitative analysis will be used to test the study hypothesis mentioned in the previous section, designed in adherence to research goals and objectives. 3.4 Data Collection The population of interest for this study is AEC projects that implemented Proj Partnered projects are a subset of larger pool of AEC projects that intentionally implement collaborative practices for project delivery. It is not possible to collect data from all these projects. Hence, sampling is needed to collect a representative sample of the population. The researcher collected archival data for this research as the sample . This d ata was obtained from a single source International Partnering Institute ( IPI ) , which is a non - profit organizatio n providing guidance, education, recognition and networking support to its member organizations comprising of owners, CM firms, designers , contractors, and facilitators, etc. in the domain of achieving better project performance in the construction industr y via a culture of collaboration. 38 The data set includes 127 partnered projects in the US between 2010 and 2018. Each project, or data point, is an application for IPI Project of the Year (PPY) awards. These applications were collected via the web and application was voluntary. A sample of the award application is presented APPENDIX A Documen t collected for each project is in the form of responses to a questionnaire consisting of inquiries (e.g., project type, location , delivery method, etc.) project performance metrics (e.g., original contract amount, final contract amount, planned duration, actual duration, etc.) and particulars of collaborative practices (e.g., frequency of partnering workshops, use of partnering charters, etc.) . The documents per project varied between 20 - 40 pages in length. The part nered projects represented in these documents are the unit of analysis of this study. Application to the PPY award were open to any partnered project completed within the previous year of the application and thus, the probability that a partnered project would apply for this award is assumed equal for all partnered projects). Thus, the sampling is considered random and hen ce, the use of statistical analysis tools is justified. As IPI is a US - based organization, geographical bias might exist in the data in the sense that partnered projects from the US are more likely to apply than from outside US. To eliminate this bias, th e researcher will present the results of the research as applicable to partnered projects in the US only. Hence, in conclusion, the sample of 127 projects is a random sample representative of all partnered projects in the US. The following table shows the number of applications received sorted by year. Partnered Project of the Year Applications (Data Set) Partnered Projects Architecture Engineering Construction Projects Figure 8 C ollected (sample) data set with respect to population 39 Table 2 Number of PPY Applications per Year Year Number of PPY Applications 2010 12 2011 1 2012 4 2013 13 2014 10 2015 27 2016 21 2017 14 2018 25 Total 127 This study also collected data in the form of risk intensity and partnering level assessment of the projects in the data set. This information was collected from partnering facilitators of the project, via an online survey. The survey was designed to assess perceived risk and partnering levels of the projects and distributed to all project partnering facilitators in the data - set (i.e., 50 facilitators for 127 projects). Survey participants received an email including consent form for participation, list of projects to fill out the survey for, and a survey. The survey requested assessment of overall risk intensity and partnering level of a given project using a Likert scale of 1 - 4 (for horizontal projects) and 1 - 5 (for non - hori zontal projects) (i.e., 1=lowest level 4 and 5=highest level). Out of all 50 facilitators: 10 could not be reached (e.g. , failure of email delivery, retirement) 16 responded (40% response rate) accounting for: o 53 out of the 127 projects (41.7%). 3.5 Study Variables (between risk intensity and partnering level), whereas measures of performance outcomes (e.g., schedule growth, cost growth) are the dependent or respons e variables. The description of these study variables is presented: 40 3.5.1.1 Independent Variable Due to lack of predefined or commonly accepted terminology to capture the concept of the interplay between risk intensity and level of collaboration, the study propose Fit denotes the relation between risk intensity and level of partnering on a project. Fit level for a project was defined by the researcher as follows: Partnering level higher tha n intensity of risk indicates ; Partnering level equal to intensity of ris k indicates Partnering level lesser than intensity of risk indicates 3.5.1.2 Dependent Variables 3.5.1.2.1 Cost Growth Cost Growth (CO) is a common measure of project cost performance and is defined as the increase in the contract amount of a project with respect to the original contract amount. Mathematically, the calculation can be described as: Cost growth is typically expressed in percentage and is positive if the project is over - budget and negative if it is under - budget. Along with details about the original and final contract amou nt, data for this research contained details about the cost (in dollars) associated with owner change orders and how much of it was due to owner scope changes. While comparing performance metrics across two decades of literature, Sullivan, Asmar, Chalhoub, & Obeid, (2017) observe that there are discrepancies in how studies account for impacts of owner scope additions on measuring performance outcomes. To capture a realistic idea of the actual cost performance of projects, the researcher subtracted the cost associated with owner change orders pertaining to scope changes from the final contract amount, when calculating the cost growth of projects in the data set. 3.5.1.2.2 Schedule Growth Schedule Growth ( SG ) is a common measure of project schedule performance and is defined as the increase in the duration of a project with respect to the original planned duration. Mathematically, the calculation can be described as: 41 Schedule growth is typically expressed in percentage and is positive if the project was delayed and negative if it was ahead of scheduled. Along with details about the original and final durations, data for this research contained details about the duration (in days) associated with owner change orders and how much of it was due to owner scope changes. As expressed before, while comparing performance metrics across two decades of literature, Sulliva n, Asmar, Chalhoub, & Obeid, (2017) observe that there are discrepancies in how studies account for impacts of owner scope additions on measuring performance outcomes. To capture a realistic idea of the actual schedule performance of projects, the research er subtracted the days associated with owner change orders pertaining to scope changes from the final duration, when calculating the schedule growth of projects in the data set . 3.5.1.2.3 Increase in Participant Satisfaction Leung, Ng, & Cheung, (2004) have presente d the case for the use of metrics to emasure participant satisfaction in AEC proejcts, since construction is a service industry. The questionnaire used to collect data used in this research inquired about the overall satisfaction of participants of the pro ject on a scale of 1 10 before and after the project. Thus, based on available data, the researcher construed the variable 3.5.1.2.4 Number of Change Orders A seminal study (Gransberg, Dillon, Reynolds, & Boyd, 1999) compare performances across partnered refers to the summation of both owner - initiated and field - initiated change orders. 3.5.1.2.5 Additional Dependent Variables In addition to the above performance metrics, the researcher looked at other standard metrics to measure project performance (e.g., safety performance ) . With respect to these metrics , not enough data was available to warrant their inclusion into the depend ent variables considered in this study. 42 3.6 Data Analysis T he nature of this study is quantitative; and is associated with finding statistical evidence to either reject or support the hypothesis that, sity of risk and O'Leary, (2018) recommends that the first step in effective data analysis is systematic logging and cleaning of data, followed by content analysis, data coding, summari zing descriptive statistics, selecting the appropriate statistical test and finally, assessing statistical significance of the analysis. This section describes quantitative data analysis procedures adopted in this study in accord with the abovementioned recommendation. 3.6.1 Data Logging and Cleaning As described in Section 3.4 , the population of interest for this study is AEC projects that implemented Partnering; and accordingly, collected data contained details about 127 such projects from the United States, completed between 2010 and 2018. Data was received from the International Partnering Institute in the form of PDF and Word documents files sorted by application year. The researcher conducted a quality control check confirming that each project file belonged to the right year folder. Word files were converted to PDF for consistency of file type. Files within a folder were cross - checked for repetition and sorted accordingly. In the end, each file was assigned a serial identification numbe r, which represented a unique partnered project. 3.6.2 Content Analysis 3.6.2.1 Preliminary Content Analysis Content analysis is a process of systematic examination of collected data to identify and assign values to constructs of interest for further data analysis (Flick, Kardorff, & Steinke, 2010) . Based on the background, scope, need and goals established, the interest of thi s study is quantitative measurement of the constructs risk intensity , partnering level and performance outcome indicators of AEC projects. Objective 2 of this study is to code quantitative measures of these constructs from project details in the data set systematically . The researcher undertook the task of preliminary content analysis searchin g for and coding the sub - constructs from the models presented because of Objective 1 . 43 After commencing the preliminary content analysis for risk intensity assessment , the researcher realized that , although a few sub - constructs in the model were straightfo rward to identify and code the data (e.g., project value in $ amount) , some others were highly ambiguous (e.g., high desired level of engagement) . Subsequently, other limitations of the models were discovered : The models seemed to lack a comprehensive list of risk elements and risk factors. Although, the researcher acknowledges that no model can comprehensively cover all risk elements and factors, a few more common and vital risk elements and factors could be included in the model (e.g. project approvals, s afety risk) . Some risk elements are not described in adequate detail for the purpose of the content analysis. A part of this limitation exists due to lack of definition s across the grading scale for each risk factor. For example, it is unclear and highly s ubject to open and varied interpretation as to what is meant In addition, there are no descriptive indicators for differentiating between levels of metrics such as complexity. Each risk fact or has a scale from 1 - 4 or 1 - 5 but there is no rule regarding calculat ing the overall risk intensity of the project. For example, if the project budget is below $5M (level 1) but the complexity is high technical (level 5), it is unclear what final risk int ensity level should be assigned to a project . Similarly, limitations were discovered when conducting preliminary content analysis and coding for assessment of partnering level using the preliminary models: The level of collaboration via partnering is depe ndent on the use or lack of use of a group of partnering tools on a project . Such a single criteria rule for deciding partnering level of a project lacks robustness . Each group of partnering tools in the models is representative of a particular partnering level. However, it is unclear how to assign the overall partnering level if tools across two or three levels only two partnering surveys (partnering level 1) but conducts quarterly partnering meetings (level 4). 3.6.2.2 Exploratory Data Analy sis (EDA) Preliminary content analysis and data coding efforts revealed limitations in the risk intensity and partnering level as sessment models adopted from the literature . Hence, per objective one of the study 44 design, there was a need to revise or develop revised processes or models to assess the risk intensity and partnering le vel of projects in the data set for the purpose of co mprehensive content analysis. To do so, t he researcher resorted to the literature to revise the existing IPI level assessment by addressing their limitations . 3.6.2.2.1 EDA for Revising Risk Intensity Assessment Model T o address the limitations of the model for risk intensity assessment identified in previous section, following solutions based on exploratory data analysis were proposed: 1. Limitation one of the risk intensity model was a seeming lack of an enough list of r isk elements and risk factors. To address this, the research referred to the comprehensive list of risk elements and factors identified in Section 2.2.2 Risk Identification and Classification of this study and particularly in Table 1 . The list was scrutinized for repetition of risk factors and constituent risk elements and they were combined and represented as one. For example, the risk factors from Al - Bahar & Crandall, (1990), effectively contained the same risk elements and thus were combined . Later, these risk factors and risk e lements were compared with those covered by the model for risk intensity assessment by IPI to identify further overlaps and maintain standard nomenclature . The result was a post - E DA list of risk factors and risk elements that fell within the scope of this study (internal risks controllable via project management ). The list is presented in Table 3 . 2. Limitation two of the model was lack of definitive descriptions across the scaling grade for each risk factor. To address this limitation, the researcher referred to the literature to identify scaling grades and their descriptions across each risk element represented in Table 3 , preferably across a 4 - and 5 - point grading scale. The results of the exercise are presented in APPENDIX B It shows the various risk grading models and scales developed by various researchers and concludes with two comprehensive table s for horizontal and non - horizontal (vertical and aviation) projects separatel y depicting scaling for each risk element from the consolidated list represented in Table 3 . 45 Table 3 Post - E DA list of Risk Factors and c onstitue nt Risk Elements Risk factors Risk element(s) Description Project value Budget Planned project budget (in $) (adjusted for inflation to CPI * 2018) Duration Planned project duration (in calendar days) Work per day Planned project budget (as above) /planned project duration (as above) Project - based risk Project approvals Number of project approvals required and difficulty of obtaining them Site & environmental conditions Probability, severity and controllability of occurrence of unfavorable site and environmental conditions Safety, a ccess ibility & o n - going operations Probability, severity and controllability of occurrence of accidents; existence of on - going operations or access issues and severity of impact on construction activities and vice - versa Construction complexity Probability, severity and controllability of occurrence of constructability challenges Design complexity Probability, severity and controllability of incompleteness, omission, error, underd evelopment of design; uniqueness of project in terms of design Sociopolitical risks Third - party s takeholder & Public Interest Number of third - party stakeholders & public, their level of interest in project and required level of interaction and interdependency for smooth operations Project relationships Inter - stakeholder relationships Previous relationship between the owner, stakehold ers, contractor designer, etc. because of working together; history of strained working relationships, litigation, etc. Desired Level of Engagement Cost pressure The pressure on a project team to del iver the project within budget; b ased on: feasibility of budget, surety and adequacy of funding per the budget & contingency Schedule pressure The pressure on a project team to d eliver the project on schedule; b ased on: contingency in schedule; risk of missing deadlines Quality pressure The pressure on a project team to deliver the project within strict q uality norms; b ased on incentive s for quality, quality plan detail, ext ernal reviews esp. federal, number o f specs *CPI = Consumer Price Index 46 Two categories of projects were classified category 1 included horizontal projects (excluding aviation horizontal projects) and category 2 included vertical and aviation (including horizontal) projects. Separate grading scales for category 1 and category 2 p rojects were deemed necessary because in horizontal - performed, whereas for buildings (vertical projects), the work is performed by sub - , communication, March 04, 2019). To accommodate for the increased risk intensity based on the need for an increased level of collaboration in vertical projects , the researcher added an extra level of risk intensity to this model . The process resulted in two variations of the model : Risk intensity assessment model for category 1 horizontal projects (excluding aviation horizontal projects) Risk intensit y assessment model for category 2 vertical and aviation (including horizontal) projects . 3. Limitation three of the risk intensity model s by IPI was unclear rules on calculating the overall risk intensity of the project. It was arbitrarily decided to use the average score of each risk element , round it up to the higher level and designate it as the overall risk intensity of the project. Thi s does not indicate that the researcher assumes each risk element is of equal importance i.e. has equal relative weightage when contributing to overall risk of a project. Conducting a detailed analysis to determine the relative weightages of the risk eleme nts was outside the scope of this study. Hence, an arbitrary rule to decide the overall risk intensity was used. This assumption was later validated using a survey, the details of which are presented in a later section 3.6.3 47 3.6.2.2.2 EDA for Revising Partnering Level Assessment Model Based on exploratory data analysis to address the limitations of the partnering level assessment model identified in previous section, fo llowing solutions were proposed: 1. Limitation one of the partnering level assessment model by IPI was a single criteria rule for decidin g partnering level of a project. T he researcher observed that it was inadequate to determine the overall partnering level merely based upon the implementation or lack of thereof of partnering tools. During exploratory data analysis, it was observed that projects adopt partnering tools to i mprove areas of collaboration (e.g., dispute resolution, facilitation) per requirement of the project. Thus, it was more resolution, facilitation) analogical T o address this limitation, the researcher sort ed the partnering tools into Dispute/Issue Resolution Facilitation Partnering Workshop Frequency Partnering Survey Frequency Goal Alignment & Team - Building Stakeholder Involvement These partnering factors are akin to the risk factors of the model for risk intensity assessment. The researcher believes that dividing partnering tools per the partnering factors and then developing a grading scale across them adds robustness to the model of partnering level assessment. It allows the choice to implement partnering tools of a higher level across a partnering factor (e.g., dispute resolution) depending upon the need of the project. 2. Limitation two of the partnering level assessm ent model by IPI was unclear rules on calculating the overall partnering level of the project. Before such a rule was established, the researcher developed a grading scale across each partnering factor above. The grading scale across each partne ring factor can be seen in Table 4 . Later, rules were established to determine the overall partnering level. For example, the use of a third - party neutral partnering facili tator is one of the most important tools of the Partnering framework. Hence, one of the rules of deciding the overall Partnering level is that irrespective of the scores of other Partnering factors, the overall partnering level of a project shall not excee 48 rules include bonus point for holding a close - out workshop and lessons learned workshop within 49 Table 4 Post EDA grading scale across Partnering factors Partnering factor Level 1 Level 2 Level 3 Level 4 Dispute/ issue resolution Field - level decision making Issue resolution ladder developed Dispute resolution board formed Facilitated dispute resolution Facilitation Self - directed In - house or internal NA Third - party facilitation Partnering workshop frequency Kick - off only More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly / weekly or more Partnering survey frequency At least once More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly / weekly or more Goal alignment & team - building Charter developed Goals revisited at least once Partnering training Partnering recognition/awards/ special task forces Stakeholder involvement Subcontractor on - boarding/off - boarding Stakeholder on - boarding/off - boarding Some form of stakeholder involvement Executive sponsorship / multi - tier partnering 50 3.6.3 Data Coding 3.6.3.1 Coding for Risk Intensity Assessment The outcome of the exploratory data analysis exercise was development of revised models for risk intensity assessment, which als o served as the coding forms for final content analysis and coding. The revised coding forms for risk intensity assessment are presented below: 51 Table 5 Coding Form for Risk Intensity Assessment ( Category 1 Projects) Risk factor analysis and rating form for Category 1 projects Risk factors Code Risk element(s) Level 4 Level 3 Level 2 Level 1 Project value RF1 Budget $250M - $500M+ $10M - $250M $5M - $10M $0 - $5M RF2 Duration 18 24+ months 12 18 months 6 12 months <6 months RF3 Work per day $100,000 - $200,000+ $50,000 - $100,000 $25,000 - $50,000 $0 - $25,000 Project - based risk RF4 Project approvals Large number of approvals required; high level of difficulty/stringency expected; may impact project severely Some approvals of possible difficulty/stringency required; budget and schedule impact possible Regular approvals required; no impact on project Approvals pre - obtained; no impact on project RF5 Site & environmental conditions History of differi ng site conditions that may affect schedule, cost, quality, or safety; moderate to extreme weather No history of differing site conditions; controllable site conditions; will not affect schedule, cost, quality or safety; moderate weather Favorable site con ditions; minimal risk to schedule, cost, quality, or safety; precautions taken; minor weather delays expected Favorable site conditions with no risk to the schedule, cost, quality or safety; no weather conditions expected RF6 Safety, accessibility & on - going operations Risk of catastrophe/fatality; staging within occupied areas/on - going construction; challenging accessibility issues Moderate risk; risk of disability; additions to occupied areas/staging adjacent to on - going operations; no accessibility issues Minor risk of damage; well clear of occupied areas; no accessibility issues Minor to no risk & greenfield site; no accessibility issues RF7 Construction complexity Very high; new/innovative methods involved, constructability affected by external factors like location Moderate; complex operations required Low; minor constructability challenges expected Very low; little to no constructability challenges expected RF8 Design complexity Design & specs based on in complete information; risk of design omissions Designer is inexperienced or design team is improper; probability of design errors Improper/incomplete design scheme communicated by client; experienced and competent designer Design scheme well communicated b y client; experienced and competent designer; proper design reviews completed Sociopolitical risks RF9 Third - party stakeholder & Public Interest High level of political, community or media sensitivity; high profile of client Third - party stakeholder groups may be involved Project may attract stakeholder or media interest Project unlikely to attract stakeholder or media interest Project relationships RF10 Inter - stakeholder relationships New relationships; history of litigation; big joint ventures; less to no time for relationship development No prior working relationship but sufficient time for relationship development before project start Some previous working experience with neutral to good working relationships Sufficient previous working experience with prior positive relationships 52 Desired Level of Engagement RF11 Cost pressure Unclear/insufficient budget, budget feasibility not established, lack of confidence regarding financing, strictly no scope for additional funds, little to no contingency Budget feasibility not established, adequate funds and sources secured but request for additional funds would be lengthy and embarrassing, enough contingency Budget feasibility established, adequate funds and sour ces secured and some scope for additional funds, enough contingency Budget feasibility established using benchmarks, adequate scope for additional recurrent funds and sources secured, generous contingency RF12 Schedule pressure The basis for the current schedule is unclear or the duration is likely to be inadequate The basis for the schedule is clear, but there are indications that overruns are possible Benchmarks were used to establish schedule; tight contingencies Benchmarks used to establish th e schedule and adequate contingencies exist RF13 Quality pressure High contractual quality requirements; if unmet could affect cost & schedule significantly Moderate contractual quality requirements; if unmet could affect cost & schedule moderately Minor contractual quality requirements; if unmet, some probability of affecting cost & schedule No special contractual quality requirements; if unmet, will not affect cost & schedule 53 Table 6 Coding Form for Risk Intensity Assessment ( Category 2 Projects) Risk factor analysis and rating form for Category 2 projects Risk factors Code Risk element(s) Level 5 Level 4 Level 3 Level 2 Level 1 Project value RF1 Budget $250M - $500M+ $25M - $250M $10M - $25M $5M - $10M $0 - $5M RF2 Duration >24 months 18 24 months 12 18 months 6 12 months <6 months RF3 Work per day >$200,000 $100,000 - $200,000 $50,000 - $100,000 $25,000 - $50,000 $0 - $25,000 Project - based risk RF4 Project approvals Large number of approvals required; High level of difficulty/stringency expected; may impact project severely Significant number of approvals required; Medium level of difficulty/stringency expected; may impact project significantly Some approvals required; Possible difficulty/stringency expected; provisions in budget and schedule for delay Minimal number of approvals required; Regular approvals (zero difficulty/stringency expected); no impact on project No approval required or all have been obtained; no impact on project RF5 Site & environmental conditions history of differing site conditions; Site conditions uncontrollable; may affect schedule, cost, quality or safety; Site location prone to acts of God, Extreme weather history of differing site conditions; Controllable (planned for) site conditions but may affect schedule, cost, quality, or safety; Moderate to extreme weather No history of differing site conditions; Controllable site conditions; will not affect schedule, cost, quality or safety; basic precautions taken; Moderate weather Favorable site conditions; minimal risk to schedule, cost, quality, or safety;; precautions taken; Slight weather delays expected Favorable; no risk to the schedule, c ost, quality or safety; Established and known; no history of differing site conditions; no weather conditions expected RF6 Safety, accessibility & on - going operations Risk of Catastrophe numerous fatalities & Activities in occupied areas/On - going operati ons; Challenging access issues Risk of fatality; Staging within occupied areas/On - going construction; Challenging access issues Moderate Risk; Risk of disability; Additions to occupied areas/Staging adjacent to on - going operations or construction; No access issues Minor risk of damage; Well clear of occupied areas; No access issues Minor to no risk & Greenfield site; No access issues RF7 Construction complexity Very High; Never done before Innovative and risky operations required Hig h; Never done before but safe operations required Moderate; Complex operations required Low; Minor challenges expected Very Low; Little to no challenge expected RF8 Design complexity incomplete design and high probability of design change and review; underdeveloped specs design & specs based on incomplete information; risk of omissions designer is inexperienced or design team is improper; probability of design errors improper/incomplete design scheme communicated by client; experienced and comp etent designer design scheme well communicated by client; experienced and competent designer; proper design reviews completed 54 Sociopolitical risks RF9 Third - party stakeholder & Public Interest High level of political, community or media sensitivity High profile client or project; Third - party stakeholder groups may be involved Third - party stakeholder groups may be involved Project may attract stakeholder or media interest Project unlikely to attract stakeholder or media interest Project relationships RF10 Inter - stakeholder relationships Client with no experience. New Relationships; History of Litigation; Joint Ventures; Less to No time for relationship development Mixed experience amongst clients or Rel ationships, Less to no scope for developing relationships Some experience amongst clients or time for relationship development before project start Some previous project experience and neutral to good working relationships & exp eriences Sufficient previous working experience with prior positive relationships Desired Level of Engagement RF11 Cost pressure No clear budget, budget seems insufficient, budget feasibility not established, inadequate funds or sources not secured, less to no contingency Budget feasibility not established, adequate funds and sources identified but financing not secured, strictly no scope for additional funds, little to no contingency Budget feasibility not established, adequate funds and sou rces secured but request for additional funds would be lengthy and embarrassing, sufficient contingency Budget feasibility established, adequate funds and sources secured and some scope for additional funds, sufficient contingency Budget feasibility establ ished using benchmarks, adequate & sure scope for additional recurrent funds and sources secured, generous contingency RF12 Schedule pressure There is no clear schedule, or the schedule is clearly insufficient The basis for the current schedule is unclear or the schedule is likely to be inadequate The basis for the schedule is clear, but indications are that overruns are possible Benchmarks were used to establish schedule Benchmarks were used to establish the schedule and adequate contingencies exis t RF13 Quality pressure High Quality requirements; if unmet could affect cost, schedule, project significantly Moderate Quality requirements; if unmet could affect cost, schedule, project slightly Minor contractual quality requirements; if unmet, some probability of affecting cost & schedule Quality requirements not specifically mentioned; will not affect project No mention about importance of quality requirements; will not affect project 55 Following rules were adopted for coding: i. Firstly, the type of each project in the data - set was identified ( category 1 or 2 ) ii. Then, project details were examined by two different coders, to identify risk elements from the coding forms above iii. Once identified, based on available data and the coding forms above, a grade (1 - 4 for category 1 and 1 - 5 for category 2 projects) was assigned for each of the identified risk element by the two coders iv. Lastly, overall risk level was computed as the average of all scores of each risk element rounded off to the higher grade. A snapshot of the coding sheet is provided in Table 7 . The outcome of the secondary coding exe rcise was that each project in the dataset was assigned an overall risk intensity level on a scale of 1 - 4 for category 1 and 1 - 5 for category 2 projects. 56 Table 7 Snapshot of Coding for Risk Intensity Assessment Sr. Number Project ID Type per Matrix Budget Duration Work Per Day Cost Pressure Schedule Pressure Quality Pressure Project Approvals Site & Environmental Conditions Safety Risk Construction Complexity Design Complexity Third - party Stakeholder & Public Interest Relationships Overall Risk Level: Average (Continuous) Overall Risk Level: Average (Roundup) SN ID TYP. RF1 RF2 RF3 RF4 RF5 RF6 RF7 RF8 RF9 RF10 RF11 RF12 RF13 AVG. AVG. 1 2018_01 1 3 4 4 2 4 3 2 4 4 4 4 2 3 3.31 4 2 2018_02 3 4 1 5 4 5 4 4 3 5 4 2 3 2 3.54 4 3 2018_03 3 4 5 4 2 5 4 2 4 4 3 2 2 5 3.54 4 4 2018_04 3 3 1 4 2 5 3 2 4 3 3 3 3 2 2.92 3 5 2018_05 1 1 2 1 2 2 2 2 3 4 3 2 3 2 2.23 3 57 3.6.3.2 Coding for Partnering Level Assessment The outcome of the exploratory data analysis exercise was development of revised models for partnering level assessment, which also served as the coding forms for final content analysis and coding. The revised coding forms for partnering level assessment a re presented in Table 8 an d Table 9 . Following rules were adopte d for coding: i. Firstly, the type of each project in the data - set was identified ( category 1 or 2 ) ii. Then, project details were examined by two different coders, to identify use of partnering tools and their characteristics (e.g., frequency) from the coding fo rms above iii. Once identified, based on available data and the coding forms above, a grade (1 - 4 for category 1 and 1 - 5 for category 2 projects) was assigned for each of the identified partnering factor by the two coders. Some additional rules for assigning the scores were as follows: a. Irrespective of the scores of other Partnering factors, the overall partnering level of a b. ed if the - category 1 and category 2 projects respectively c. for category 1 and category 2 projects respectively iv. Lastly, overall partnering level was computed as the average of all scores of each partnering factor rounded off to the higher grade. A snapshot of the coding sheet is provided in Table 10 . The outcome of the secondary coding exercise was that each project in the dataset was assigned an overall partnerin g level on a scale of 1 - 4 for category 1 and 1 - 5 for category 2 projects . 58 Table 8 Coding Form for Partnering Level Assessment ( Category 1 Projects) Partnering level for category 1 projects Partnering factor Level 1 Level 2 Level 3 Level 4 Bonus points & other notes Dispute/issue resolution Field - level decision making Issue resolution ladder developed Dispute resolution board formed Facilitated dispute resolution NA Facilitation ** Self - directed In - house or internal NA Third - party facilitation NA Partnering workshop frequency Kick - off only More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly / weekly or more Bonus: close - out workshop Partnering survey frequency At least once More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly / weekly or more Bonus: lessons learned analyzed Goal alignment & team - building Charter developed Goals revisited at least once Partnering training Partnering recognition/awards/ special task forces NA Stakeholder involvement Subcontractor on - boarding/off - boarding Stakeholder on - boarding/off - boarding Some form of stakeholder involvement Executive sponsorship / multi - tier partnering NA 59 Table 9 Coding Form for Partnering Level Assessment ( Category 2 Projects) Partnering level for category 2 projects Partnering factor Level 1 Level 2 Level 3 Level 4 Level 5 Bonus points & other notes Dispute/issue resolution Field - level decision making Issue resolution ladder developed Dispute resolution board formed Facilitated dispute resolution Na NA Facilitation ** Self - directed In - house or internal NA NA Third - party facilitation NA Partnering workshop frequency Kick - off only More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly Weekly or more Bonus: close - out workshop Partnering survey frequency At least once More than once but less th an quarterly Quarterly or more but less than monthly Monthly or more but less than weekly Weekly or more Bonus: lessons learned analyzed Goal alignment & team - building Charter developed Goals revisited at least once Partnering training Partnering recognition/awards Special task forces NA Stakeholder involvement Subcontractor on - boarding/off - boarding Stakeholder on - boarding/off - boarding Some form of stakeholder involvement Executive sponsorship Multi - tier partnering NA ** Irrespective of the scores of other Partnering factors, the overall partnering level of a project shall not exceed the sco 60 Table 10 Snapshot of Co ding for Partnering Level Assessment Sr. Num ber Projec t ID Type per Matrix Field - Level Decision Making Issue Resolution Ladder (IRL) Dispute Resolution Advisor (DRA)/ Board (DRB) Facilitated Dispute Resolution (FDR) Facilitation Frequency of Workshops Bonus Point - Close - Out Workshop Held? Frequency of Surveys Bonus Point - Lessons Learned Analyzed Project Charter Goals Revised/Revisited Partnering Recognition/Awards Partnering Training Special Task Forces for Issue Resolution Subcontractor On - Boarding/Off - Boarding Stakeholder On - Boarding/Off - Boarding Stakeholder Involvement Executive Sponsorship Multi - Tiered Partnering - Executive, Core & Stakeholder Factor 1 (Dispute Resolution) Score Level 2 (Facilitation) Score Factor 3 (Workshop Frequency) Score Factor 4 (Survey Frequency) Score Factor 5 (Goal Alignment & Team Building) Score Factor 6 (Stakeholder Involvement) Score Overall Partnering Level: Factor Average Score Overall Partnering Level: Average Roundup 1 2018_ 01 1 1 1 1 0 4 2 0 4 1 1 1 1 0 0 1 1 1 0 0 3 4 2 4 3 3 3.1 7 4 2 2018_ 02 3 1 1 0 0 5 4 1 4 1 1 1 0 0 1 1 1 1 0 0 3 5 5 5 3 3 3.9 2 4 3 2018_ 03 3 1 1 0 1 5 3 1 4 1 1 1 1 0 1 1 1 0 0 0 4 5 4 5 4 2 3.9 6 4 4 2018_ 04 3 1 0 0 0 5 5 1 1 1 1 1 1 0 0 1 1 1 1 0 1 5 5 2 3 4 3.3 8 4 5 2018_ 05 1 1 1 0 0 4 4 1 4 1 1 1 1 0 1 1 1 0 0 0 2 4 4 4 4 2 3.3 3 4 61 3.6.3.3 Coding for Project Performance Evaluation Metrics Lastly, values for the various performance outcome metrics were coded for each project in the data - set. Several studies have undertaken the task of measuring or comparing performance of AEC projects. One such seminal study (Gransber g, Dillon, Reynolds, & Boyd, 1999) quantitatively analyzed performance of partnered projects via several performance indicators (e.g., cost growth, average cost per change order , time growth). Examining that list, and available data, following perform ance indicators were identified as relevant for this study: Cost Growth Schedule Growth Increase in Participant Satisfaction As noted, projects in the data set were completed over a span of years (2010 to 2018). Thus, to achieve a fair comparison of their cost performance, it was necessary to adjust coded cost data for inflation. The researcher compared values for Consumer Price Index (C PI) published by the Bureau of Labor Statistics for each year with the CPI index value for 2018 and adjusted the cost data accordingly . To do so, the percent increase of CPI index was computed for each year (compared to 2018) and applied to the coded cost data of every year to compute cost adjusted for inflation. Thus, a uniform measure of cost with respect to project completion year was achieved. A snapshot of the performance outcome coding sheet is provided in Table 11 . The outcome of the coding exercise was that each project in the dataset was assigned values for performance outcome metrics (e.g., cost growth, schedule growth) which were used during data analysis and hypothesis testing. 62 Table 11 Snapshot of Coding for Performance Outcomes Sr. Number Project ID Schedule Growth Schedule Growth minus CCO Time Extension Cost Growth Cost Growth Minus Owner's CO Number of CO p rocessed Number of Owner Initiated CO Number of Field Initiated CO Number of Claims Accepted Number of Unresolved Claims at Close - Out Participation Satisfaction 1 2018_01 20% 0% 1.12% - 10.25% 140 77 63 2 0 - 14% 2 2018_02 0% 0% 1.60% 0.00% 24 21 3 0 0 NA 3 2018_03 5% 5% 17.97% 17.96% 3 3 1 0 0 20% 4 2018_04 - 31% - 31% - 15.42% - 20.11% 11 7 4 0 0 31% 5 2018_05 - 1% - 1% 28.21% 5.25% 4 4 3 0 0 3% 63 3.6.4 Data Cleaning After the data coding process, the researcher examined coded data sheets for irregularities. Outliers were identified and examined for clarifications for reasons the data point differed significantly from other observations. Following cleaning actions were performed: Projects that had missing data points for a dependent variable were removed during analysis concerning that variable. For example, out of the 127 projects in the data set, 3 projects did not contain measures to determine schedule growth and thu s were removed during analysis of risk - partnering fit versus schedule growth. Schedule growth data was examined for reasons of delay. Projects that were delayed for reasons beyond human control were adjusted accordingly. For example, a project reported a 7 - month the project was recalculated after discounting for that delay. 3.6.5 Validation of Revised Risk Intensity and Partnering Level Assessment Models The outcome of the coding exercise was that every project in the data set was assigned a quantitative measure of risk intensity, partnering level and performance outcome indicators. The coding forms used for assessment of risk intensity and partnering level were derived from models based on the ones deve loped by IPI, which were revised via exploratory data analysis. Hence, before proceeding with data analysis, it was necessary to validate the se revised models. The revised models because they use measures of variables to predict ris k intensity and partnering level. Validation is the process of assessing whether prediction of the construct of interest (risk intensity and partnering level in our case) is within the confidence interval deemed acceptable for the intended use of the model physical measurements for the quantity of interest by carrying out a hypothesis test of whether or not (Na tional Research Council, 2012) as deemed by the intended use of the model. For this study, these physical measurement s for the quantities of interest (risk intensity and partnering level) were obtained via a survey . The survey was designed to assess perceived risk and partnering levels of the projects and distributed to a ll project partnering facilitators in the data - set (i.e., 50 facilitators for 127 projects). Survey participants 64 received an email including consent form for participation ( APPENDIX C ), list of projects to fill out the survey for, and a survey ( APPENDIX D ). The survey requested assessment of overall risk intensity and partnering level of a given project using a Likert scale of 1 - 4 (for category 1 projects) and 1 - 5 (for category 2 projects) (i.e ., 1=lowest level 4 and 5=highest level) . Out of all 50 facilitators : 10 could not be reached (e.g. , failure of email delivery, retirement ) 16 responded (40 % response rate) accounting for: o 53 out of the 127 projects (41.7%). The response rate was deeme d adequate for proceeding with model validation. As prescribed by National Research Council (2012), revised risk intensity and partnering models were verified via statistical hypothesis testing Chi Square Test of Homogeneity . The test i s employed to a single categorical variable from two populations to determine whether distribution of frequency counts is identical across different populations. In this case, the single categorical variable is the measure of risk intensity measure and pa rtnering level taken one at a time; and the two populations are the model output and survey responses. Because category 1 and category 2 projects are scaled differently, the test was conducted separately for survey data from category 2 and category 1 projects for both risk intensity and partnering level surveys. 3.6.6 Statistical Tests for Hypothesis Testing T he nature of this study is quantitative; and is associated with finding statistical evidence to either reject or support the following overall study hypothesis : I n a partnered project, better the fit between the intensity of risk and adopted partnering level, better is its performance (e.g., cost growth, schedule growth). The hypothesis was modified for each performance outcome (i.e., dependent variabl e) and tested separately. In this study, Performance outcome metrics (e.g., cost growth, schedule growth) are the response variables, which are continuous in nature. 65 So far, the out come of content analysis and data coding exercise was that each project in the data set was assigned a measure for the construct fit (based on overall risk intensity and partnering level), and performance outcome metrics (e.g., schedule growth, cost growth ). Thus, projects in the data set , and therefore their performance outcome measures were divided into 3 categories viz. positive, neutral and negative. Thus, the problem of testing the hypothesis of this study reduces to the problem of comparing the freque ncy distribution of the groups of data representing performance outcome measures across the three categories of fit (positive, neutral, and negative) . Note that the hypothesis will be tested separately for each performance outcome metric. If no difference in the frequency distributions were found, it would mean that for that performance outcome metric, there is no variability introduced because of fit i.e. there is no association between fit (between risk intensity and partnering level) and the performance outcome metric. Alternatively, if the re were difference in the frequency distributions were found, it would mean that there is an association between fit and that performance outcome metric. Several statistical p arametric and non - parametric tests are avail able for comparing groups. The decision to choose a particular test depends on various factors like number of groups to be compared, existence of pairing between them, variable type, and normality (Motulsky, 1995) . In this study, there are three groups ( dependent va riables data across three fit categories) to be compared, they are independent samples. Shapiro - Wilk test for normality was conducted on these samples and the Kruskal - Wallis test was determined to be the most appropriate for testing the hypothesis of this study. (Kruskal & Wallis, 1952) is a non - parametric statistical test that assesses continuous variable. T he null hypothesis of the Kruskal - Wallis specifies that the groups are subsets from the same population . T o test this null hypothesis, the groups are combined into a single group and variable of interest are ranked based on their order . The new rank scores are summed by group and, along with group sample sizes, are used to calculate the H statistic , which reflects the variance in ranks between is compared by refer ring to a ch exceeds a critical value, we may conclude that the groups do not come from the same population . For this study, the Kruskal - Wallis test was conducted using RStudio statist ical software. 66 Once it is established that the groups do not come from the same population, it was further necessary to determine the order among the groups. For example, if it was found that schedule performance varies across fit categories, it is import ant to determine, which fit category shows better schedule performance compared to others. For this purpose, the researcher used the Dunn Test. It reports the results among multiple pairwise comparisons after a significant Kruskal - Wallis test for the numbe r of groups (3 in our case). 3.7 Quality Measures For the results of a study to be valid, a researcher must examine and address potential sources of error and reliability (Fellows & Liu, 2008) . It is important to present the measures taken to ensure research quality for the findings to be used in other research and in application in practice. As data collected for this research is archival in nature, it is important to check for potential bias in the data arising due to the data collection agency as well as the data collection process. Descriptive statistics were examined to observe bias arising due to the data collection agency (e.g., bias due to geography, project ty pe, project delivery metho d). In addition , because the data set contained projects over a span of time (2010 to 2018), the researcher normalize d values for constructs such as project budget using Consumer Price Index (CPI) adjustments . Quality during content analysis was maintaine d via random quality checks and reliability checks. Two researchers conducted the data coding exercise. Researcher 1 is the investigator in this study, who is a graduate student of construction management, with prior experience in partnering and AEC indust ry research, while Researcher 2 is undergraduate student in civil engineering familiar with construction management terminology. Firstly, Researcher 1 and 2 reviewed the coding forms together and clarified any discrepancy in understanding the codes. Prelim inary coding was then conducted by Researcher 2. Coded data was randomly checked by Researcher 1 for quality control. Most of the constructs in this study (e.g., performance measures and partnering tools used or not) are factual and hence had no researcher bias component to them. Outliers in the data were examined separately for potential errors and fixed accordingly. For this quantitative study, the researcher state d the confidence (or significance) levels to help determine the applicability of results. 67 CHAPTER 4 RESULTS AND FINDINGS 4.1 Descriptive Statistics This section describe s the characteristics of the archival data collected for this study. Overall, 127 AEC projects were studied. All these projects were completed in the United States between 2010 and 20 18. The following table shows the number of applications received sorted by their year of completion: Table 12 Classification of Projects by Year of Completion Year Number of Projects Completed in the Year 2010 12 2011 1 2012 4 2013 13 2014 10 2015 27 2016 21 2017 14 2018 25 Total 127 Figure 9 N umber of projects by y ear of c ompletion 68 Table 13 shows the state - wise distribution of the projects in the data - set. Table 13 Classification of Project Locations by Stat es Project Location (State) Number of Projects CA 79 AZ 12 UT 11 NV 5 OH 5 CO 3 CT 3 MD 2 MI 2 NC 2 PA 1 TN 1 VA 1 Total Number of Projects 127 Figure 10 Number of projects per state 69 It is interesting to note that a majority of the projects (approx. 62%) were located in the state of California. One possible explanation for such a skew could be that the award agency IPI ( from whom the data is collected ) is based out of California. Thus, it is possible that applicants located close to the agency were more aware of the awards and hence applied in larger numbers. Alternatively, the skew can also be explained from obse rvations from previous studies (National Academies of Sciences, Engineering, and Medicine, 2019) that a majority of partnered projects are located in the West Coast of the US. Out of the 127 projects, 86 projects (68%) were horizontal type, 22 projects (17%) were vertical and the remaining 19 (15%) w ere aviation. Table 14 tabulates the distribution of projects in the data set per their project type. Table 14 Classification of Projects per pro ject type Project Type Number of Projects % of Total number of Projects Horizontal 86 68% Vertical 22 17% Aviation 19 15% Total Number of Projects 127 100% Figure 12 Classification per Project Delivery Method Of the 127 projects in the data - set, 61 (48 %) are Design - Bid - Build (DBB), 26 (21%) are Design - Build (DB), 22 (17%) are Construction Manager as Agency (CMA) and 18 (14%) are Construction Manager at Risk or as General Contractor (CMR/GC) . Table 15 below tabulates the distribution of projects in the data set per their project type. GC 26% DBB 30% DBB 24% CMA 20% Classification based on Project Delivery Methods GC DBB DBB CMA Horizontal 68% Vertical 17% Aviation 15% Classification based on Project Type Horizontal Vertical Aviation Figure 11 Classification per Project Type 70 Table 15 Classificat ion of Projects per project delivery method Project Delivery Method Number % of Total number of Projects DBB 61 48% DB 26 21% CMA 22 17% CMR/GC 18 14% Total Number of Projects 127 100% Based on original contract amount ( not adjusted for inflation ) , t he breakdown of the 127 projects is as presented in Table 16 . Table 16 Classification of Projects per budget category Budget Category Number % of Total number of Projects <$25M 52 41% $25M - $250M 67 53% $250M+ 8 6% Total Number of Projects 127 100% Figure 13 Classification of Projects per original contract amount 4.2 Exploratory Data Analysis O bjective one and two of this study were to identify and revise (if necessary) models or processes to measure the constructs of interest (risk intensity and partnering level). During the literature review, <$25M , 52 $25M - $250M , 67 $250M+ , 8 <$25M $25M-$250M $250M+ 0 10 20 30 40 50 60 70 80 Classification based on Award Budget Amount <$25M $25M-$250M $250M+ 71 content analysis, exploratory data analysis and data coding exercises undertaken to achieve these objec tives, following lessons were learned: 1. Ideally, risk intensity of a project should be determined prospectively i.e. before project start via identifying relevant risks and then determining their probability of occurrence and severity of impact upon realiza tion. However, for the purpose of a study like this one, risk intensity has to be assessed retrospectively from project details via models. 2. Risk intensity assessment processes and models from peer - reviewed literature endorse the use of a risk register of c ommon project risks to prospectively measure risk intensity of a project. 3. There exists literature that questions the appropriate ness of using an ordinal scale of measurement for expressing risk intensity. However, when the outcome of a risk intensity mode l is to be used as a decision - model (e.g., to further determine a risk management strategy like selecting a partnering level), it is acceptable to use ordinal scale of measurement (e.g., 1 - 5 scale) 4. Although risk factors and their constituent elements are c ommon across different project types, non - horizontal (e.g., vertical projects like commercial buildings and aviation projects like terminals and runways) projects experience higher risk intensity compared to horizontal (e.g., roads, utility) projects. This is because the number, interdependency and influence of stakeholders in non - horizontal projects is higher than horizontal projects thereby adding an extra layer of risk over the same risk factors. 5. It is inadequate to determine the overall partnering level merely based upon the implementation or lack of thereof of partnering tools. The researcher observed that projects adopt partnering tools to improve areas of collaboration (e.g., dispute resolution, facilitation) per requirement of the project. Thus, it w as more suitable to determine partnering level 4.3 Model Validation The Chi Square Test of Homogeneity was conducted separately for horizontal a nd non - horizontal projects for both risk intensity and partnering level measures. In all four cases, the null hypothesis was : H 0 : P measure via model output = P measure via survey result 72 That is, the distribution of frequency counts of the measures of risk i ntensity and partnering level are distributed identically across the two populations (mode l output and survey responses), and the alternative being that the null hypothesis is false. The results of the test are below . Table 17 Pearson's Chi - Square Test Results Model and Case 2 DF p - value Critical p - value Observation Result Risk Intensity Model Case 1: Category 1 Projects 6.593 3 0.086 0.05 p > p critical H 0 is not rejected Case 2: Category 2 Projects 4.646 3 0.199 0.05 p > p critical H 0 is not rejected Partnering Level Model Case 1: Category 1 Projects 7.769 3 0.051 0.05 p > p critical H 0 is not rejected Case 2: Category 2 Projects 5.431 3 0.142 0.05 p > p critical H 0 is not rejected In both cases, since the p - value is greater than the decided significance level of 0.05, the null hypothesis cannot be rejected i.e., there is no statistically significant difference in the distribution of frequency counts of measures of risk intensity as well as partnering level between predicted values using the revised models and physical measurements received via expert survey responses. Hence, it was decided that the measures of constructs of interest (risk intensity and partnering le vel) used for further data analysis. 4.4 Revised M odels of Risk and Partnering The outcome of the exploratory data analysis was the revision of models t o determine measures of risk intensity and partnering level of AEC projects. Note that the revised models were validated via a survey followed by statistical model validation . The models are presented below: 73 4.4.1 Risk intensity assessment model Table 18 below, contains a list of the risk factors and risk elements, with their description, that constitute the risk register for this model. Following the risk regis ter table are Table 19 and Table 20 , representing the revised models for risk inte nsity assessment for horizontal and non - horizontal projects respectively. The steps for determining overall risk intensity of an AEC project are: STEP 1. Identify the type of project a. Category 1 Project A project is a horizontal project if most of its scope in volves heavy civil construction whose length is longer than its height. Examples of such projects include bridges, roads, utility projects, etc. b. Category 2 Project Includes: i. V ertical project s A project is a vertical project if a majority of its scope stretches vertically. Example of vertical construction projects include commercial buildings, hospitals, etc. ii. A viation project s A project is an aviation project if a majority of its scope involves construction on or close to airports and requires s ignificant interaction with airport authorities. Examples of aviation projects include runways, control towers, terminals, etc. Note: It is important to note that aviation projects may be both vertical and horizontal in scope. For the purpose of this resea rch, if a project can be classified as aviation then it cannot be classified as Category 1 . STEP 2. Based on the project type determined above, c hoose the appropriate risk intensity assessment model Category 1 or Category 2 project risk assessment model STEP 3. Based on project details and characteristics , assign the risk level for each risk element identified in the model. For example, if your project is horizontal and its original contract amount is $300M, then model. STEP 4. Compute overall risk level as the average of all scores of each risk element s rounded up to the higher level . 74 Table 18 Risk register for the revised Risk Intensity assessment model Risk factors Risk element(s) Description Project value Budget Planned project budget (in $) (adjusted for inflation to CPI* 2018) Duration Planned project duration (in calendar days) Work per day Planned project budget (as above) /planned project duration (as above) Project - based risk Project approvals Number of project approvals required and difficulty of obtaining them Site & environmental conditions Probability, severity and controllability of occurrence of unfavorable site and environmental conditions Safety, accessibility & on - going operations Probability, severity and controllability of occurrence of accidents; existence of on - going operations or access issues and severity of impact on construction activities and vice - versa Construction complexity Probability, severity and controllability of occurrence of constructability challenges Design complexity Probability, severity and controllability of incompleteness, o mission, error, underdevelopment of design; uniqueness of project in terms of design Sociopolitical risks Third - party stakeholder & Public Interest Number of third - party stakeholders & public, their level of interest in project and required level of interaction and interdependency for smooth operations Project relationships Inter - stakeholder relationships Previous relationship between the owner, stakeholders, contractor designer, etc. because of working together; history of strained working relations hips, litigation, etc. Desired Level of Engagement Cost pressure The pressure on a project team to deliver the project within budget; based on: feasibility of budget, surety and adequacy of funding per the budget & contingency Schedule pressure The pressure on a project team to deliver the project on schedule; based on: contingency in schedule; risk of missing deadlines Quality pressure The pressure on a project team to deliver the project within strict quality norms; based on incentives for quality, quality plan detail, external reviews esp. federal, number of specs *CPI = Consumer Price Index 75 Table 19 Revised Risk Intensity Assessment Model ( Category 1 Projects) Risk Intensity Assessment Model ( Category 1 Projects) Risk factors Risk element(s) Level 4 Level 3 Level 2 Level 1 Project value Budget $250M - $500M+ $10M - $250M $5M - $10M $0 - $5M Duration 18 24+ months 12 18 months 6 12 months <6 months Work per day $100,000 - $200,000+ $50,000 - $100,000 $25,000 - $50,000 $0 - $25,000 Project - based risk Project approvals Large number of approvals required; high level of difficulty/stringency expected; may impact project severely Some approvals of possible difficulty/stringency required; budget and schedule impact possible Regular approvals required; no impact on project Approvals pre - ob tained; no impact on project Site & environmental conditions History of differing site conditions that may affect schedule, cost, quality, or safety; moderate to extreme weather No history of differing site conditions; controllable site conditions; will not affect schedule, cost, quality or safety; moderate weather Favorable site conditions; minimal risk to schedule, cost, quality, or safety; precautions taken; minor weather delays expected Favorable site conditions with no risk to the schedule, cost, quality or safety; no weather conditions expected Safety, accessibility & on - going operations Risk of catastrophe/fatality; staging within occupied areas/on - going construction; challenging accessibility issues Moderate risk; risk of disability; add itions to occupied areas/staging adjacent to on - going operations; no accessibility issues Minor risk of damage; well clear of occupied areas; no accessibility issues Minor to no risk & greenfield site; no accessibility issues Construction complexity Very high; new/innovative methods involved, constructability affected by external factors like location Moderate; complex operations required Low; minor constructability challenges expected Very low; little to no constructability challenges expe cted Design complexity Design & specs based on incomplete information; risk of design omissions Designer is inexperienced or design team is improper; probability of design errors Improper/incomplete design scheme communicated by client; experienced and competent designer Design scheme well communicated by client; experienced and competent designer; proper design reviews completed Sociopolitical risks Third - party stakeholder & Public Interest High level of political, community or media se nsitivity; high profile of client Third - party stakeholder groups may be involved Project may attract stakeholder or media interest Project unlikely to attract stakeholder or media interest Project relationships Inter - stakeholder relationships New relationships; history of litigation; big joint ventures; less to no time for relationship development No prior working relationship but sufficient time for relationship development before project start Some previous working experience with neutral to good working relationships Sufficient previous working experience with prior positive relationships 76 Table 19 Desired Level of Engagement Cost pressure Unclear/insufficient budget, budget feasibility not established, lack of confidence regarding financing, strictly no scope for additional funds, little to no contingency Budget feasibility not established, adequate funds and sources secured but request for additional funds would be lengthy and embarrassing, sufficient contingency Budget f easibility established, adequate funds and sources secured and some scope for additional funds, sufficient contingency Budget feasibility established using benchmarks, adequate scope for additional recurrent funds and sources secured, generous contingency Schedule pressure The basis for the current schedule is unclear or the duration is likely to be inadequate The basis for the schedule is clear, but there are indications that overruns are possible Benchmarks were used to establish schedule; tight conting encies Benchmarks used to establish the schedule and adequate contingencies exist Quality pressure High contractual quality requirements; if unmet could affect cost & schedule significantly Moderate contractual quality requirements; if unmet could affect cost & schedule moderately Minor contractual quality requirements; if unmet, some probability of affecting cost & schedule No special contractual quality requirements; if unmet, will not affect cost & schedule 77 Table 20 Revised Risk Intensity Assessment Model ( Category 2 Projects) Risk Intensity Assessment Model ( Category 2 Projects) Risk factors Risk element(s) Level 5 Level 4 Level 3 Level 2 Level 1 Project value Budget $250M - $500M+ $25M - $250M $10M - $25M $5M - $10M $0 - $5M Duration >24 months 18 24 months 12 18 months 6 12 months <6 months Work per day >$200,000 $100,000 - $200,000 $50,000 - $100,000 $25,000 - $50,000 $0 - $25,000 Project - based risk Project approvals Large number of approvals required; High level of difficulty/stringency expected; may impact project severely Significant number of approvals required; Medium level of difficulty/stringency expected; may impact project significantly Some approvals required ; Possible difficulty/stringency expected; provisions in budget and schedule for delay Minimal number of approvals required; Regular approvals (zero difficulty/stringency expected); no impact on project No approval required or all have been obtained; no i mpact on project Site & environmental conditions history of differing site conditions; Site conditions uncontrollable; may affect schedule, cost, quality or safety; Site location prone to acts of God, Extreme weather history of differing site conditions; Controllable (planned for) site conditions but may affect schedule, cost, quality, or safety; Moderate to extreme weather No history of differing site conditions; Controllable site conditions; will not affect schedule, cost, quality or safety; basic precautions taken; Moderate weather Favorable site conditions; minimal risk to schedule, cost, quality, or safety;; precautions taken; Slight weather delays expected Favorable; no risk to the schedule, cost, quality or safety; Established and known; no history of differing site conditions; no weather conditions expected Safety, accessibility & on - going operations Risk of Catastrophe numerous fatalities & Activities in occupied areas/On - going operations; Challenging access issues Risk of fatality; Staging within occupied areas/On - going construction; Challenging access issues Moderate Risk; Risk of disability; Additions to occupied areas/Staging adjacent to on - going operations or construction; No access issues Minor risk of damage; Well cle ar of occupied areas; No access issues Minor to no risk & Greenfield site; No access issues Construction complexity Very High; Never done before Innovative and risky operations required High; Never done before but safe operations required Moderate; Complex operations required Low; Minor challenges expected Very Low; Little to no challenge expected Design complexity incomplete design and high probability of design change and review; underdeveloped specs design & specs based on incomplete i nformation; risk of omissions designer is inexperienced or design team is improper; probability of design errors improper/incomplete design scheme communicated by client; experienced and competent designer design scheme well communicated by client; experienced and competent designer; proper design reviews completed 78 Table 2 0 Sociopolitical risks Third - party stakeholder & Public Interest High level of political, community or media sensitivity High profile client or project; Third - party stakeholder groups may be involved Third - party stakeholder groups may be involved Project may attract stakeholder or media interest Project unlikely to attract stakeholder or media interest Project relationships Inter - stakeholder relationships Client with no experience. New Relationships; History of Litigation; Joint Ventures; Less to No time for relationship development Mixed experience amongst Relationships, Less to no scope for developing relationships Some experience amongst sufficient time for relationship development before project start Some previous project experience and neutral to good working relationships & experiences Sufficient previous working experie nce with prior positive relationships Desired Level of Engagement Cost pressure No clear budget, budget seems insufficient, budget feasibility not established, inadequate funds or sources not secured, less to no contingency Budget feasibility not establis hed, adequate funds and sources identified but financing not secured, strictly no scope for additional funds, little to no contingency Budget feasibility not established, adequate funds and sources secured but request for additional funds would be lengthy and embarrassing, sufficient contingency Budget feasibility established, adequate funds and sources secured and some scope for additional funds, sufficient contingency Budget feasibility established using benchmarks, adequate & sure scope for additional recurrent funds and sources secured, generous contingency Schedule pressure There is no clear schedule, or the schedule is clearly insufficient The basis for the current schedule is unclear or the schedule is likely to be inadequate The basis for the schedule is clear, but indications are that overruns are possible Benchmarks were used to establish schedule Benchmarks were used to establish the schedule and adequate contingencies exist Quality pressure High Quality requirements; if unmet could affect cost, schedule, project significantly Moderate Quality requirements; if unmet could affect cost, schedule, project slightly Minor contractual quality requirements; if unmet, some probability of affecting cost & schedule Quality requirements no t specifically mentioned; will not affect project No mention about importance of quality requirements; will not affect project 79 4.4.2 Partnering level assessment m odel Table 21 contains a list of the partnering factors and constituent partnering tools, the implementation of which contributes to the partnering score of that factor . This table is akin to a risk registe r and the . Following the list are Table 22 and Table 23 , representing the revised models for partnering level assessment for horizontal and non - horizontal projects respectively. The steps for determining overall partnering level of an AEC project are: STEP 1. Identify the type of project a. Category 1 Project A project is a horizontal project if a majority of its scope involves heavy civil construction whose length is longer than its height. Examples of such projects include bridges, roads, utility projects, etc. b. Category 2 Project Includes: i. Vertical projects A project is a vertical project if a majority of its scope stretches vertically. Example of vertical construction projects include commercial buildings, hospitals, etc. ii. Aviation project A project is an aviation project if a majority of its scope involves construction on or close to airports and requires significant interaction with airport authorities. Examples of aviation projects include runways, control towers, terminals, etc. Note: It i s important to note that aviation projects may be both vertical and horizontal in scope. For the purpose of this research, if a project can be classified as aviation then it cannot be classified as Category 1 . STEP 2. Based on the project type determined above, ch oose the appropriate partnering level assessment model Category 1 or Category 2 project partnering level assessment model STEP 3. Based on the partnering tools implemented on the project, assign a score (1 - 4 for Category 1 and 1 - 5 for Category 2 ) for each partne ring factor per the model STEP 4. Note that, i rrespective of the scores of other Partnering factors, the overall partnering level of a 80 STEP 5. - out Partnering workshop frequency Partnering survey frequency maximum possible points available for that factor STEP 6. Compute overall partnering level as the average of all scores of each partn ering rounded off to the higher level 81 Table 21 Partnering register for the revised Partnering Level assessment model Partnering f actor Partnering Tools or frequency Dispute/issue resolution Field - level decision making Issue resolution ladder developed Dispute resolution board formed Facilitated dispute resolution Facilitation Self - directed In - house or internal Third - party facilitation Partnering workshop frequency Kick - off only More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly / weekly or more Close - out Workshop Partnering survey frequency At least once More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly / weekly or more Lessons learned analyzed Goal alignment & team - building Charter developed Goals revisited at least once Partnering training Partnering recognition/awards/ special task forces Stakeholder involvement Subcontractor on - boarding/off - boarding Stakeholder on - boarding/off - boarding Some form of stakeholder involvement Executive sponsorship / multi - tier partnering 82 Table 22 Revised Partnering Level Assessment Model ( Category 1 Projects) Partnering Level Assessment Model ( Category 1 Projects) Partnering factor Level 1 Level 2 Level 3 Level 4 Bonus points Dispute/ issue resolution Field - level decision making Issue resolution ladder developed Dispute resolution board formed Facilitated dispute resolution NA Facilitation** Self - directed In - house or internal NA Third - party facilitation NA Partnering workshop frequency Kick - off only More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly / weekly or more Bonus: close - out workshop Partnering survey frequency At least once More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly / weekly or more Bonus: formal lessons learned analyzed Goal alignment & team - building Charter developed Goals revisited at least once Partnering training Partnering recognition/awards/ special task forces NA Stakeholder involvement Subcontractor on - boarding/off - boarding Stakeholder on - boarding/off - boarding Some form of stakeholder involvement Executive sponsorship / multi - tier partnering NA 83 Table 23 Revised Partnering Level Assessment Model ( Category 2 Projects) Partnering Level Assessment Model ( Category 2 Projects) Partnering factor Level 1 Level 2 Level 3 Level 4 Level 5 Bonus points Dispute/issue resolution Field - level decision making Issue resolution ladder developed Dispute resolution board formed Facilitated dispute resolution Na NA Facilitation** Self - directed In - house or internal NA NA Third - party facilitation NA Partnering workshop frequency Kick - off only More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly Weekly or more Bonus: close - out workshop Partnering survey frequency At least once More than once but less than quarterly Quarterly or more but less than monthly Monthly or more but less than weekly Weekly or more Bonus: formal lessons learned analyzed Goal alignment & team - building Charter developed Goals revisited at least once Partnering training Partnering recognition/awards Special task forces NA Stakeholder involvement Subcontractor on - boarding/off - boarding Stakeholder on - boarding/off - boarding Some form of stakeholder involvement Executive sponsorship Multi - tier partnering NA ** Irrespective of the scores of other Partnering factors, the overall partnering level of a project shall not exceed the score 84 4.5 Characteristics and Normality Tests for Dependent Variables Based on the hypothesis of this study, measures of the performance outcomes of cost growth, schedule growth, and increase in participant satisfaction, were identified as the dependent variables. This section presents descriptive statistics of those indepen dent variables, including information about their statistical distribution. Schedule Growth Post data cleaning for non - responses and removal of outliers, schedule performance of 124 of the 127 projects in the data set was obtained. Out of the 124 projects, implemented partnering level), the risk intensity of the p roject); and risk intensity on the project) The highest average schedule growth of projects was observed in fit category 1 (negative) at 30.28%, followed by fit category 2 (neutral), where the average schedule growth across projects was 4.02%. Least average schedule growth per category was observed in fit category 3 (positive) at - 3.52%. Negative schedule growth indicates that the project was completed ahead of its original planned schedule. The trend is graphically represented in Figure 14 . 85 Figure 14 Average Schedule Growth across Fit Categories The results of the Shapiro - Wilk test of normality for the schedule growth are presented in Table 24 . Table 24 Normality Test for Schedule Growth Normality Test (Shapiro - Wilk) Characteristic Fit Category 1 (Negative) Fit Category 2 (Neutral) Fit Category 3 (Positive) p - value 1.01E - 07 1.11E - 08 7.94E - 02 skewness 2.43 2.68 0.10 kurtosis 5.62 12.66 1.83 z 6.00 9.00 0.00 Result Not Normal Not Normal Normal As observed , p < 0.05 for categories 1 and 2, thus indicating that data in categories 1 and 2 are not normally distributed. Cost Growth Post data cleaning for non - responses and removal of outliers, cost performance of 118 of the 127 projects in the data set was obtained. Out of the 118 projects, 35 (29.6 n implemented partnering level), 66 ( 55.9 the risk intensity of the project); and 17 (14.4 vel higher than risk intensity on the project) 30% 4% - 4% -10% 0% 10% 20% 30% 40% SCHEDULE GROWTH (%) FIT CATEGORY AVERAGE SCHEDULE GROWTH ACROSS FIT CATEGORIES 1 (Negative) 2 (Neutral) 3 (Positive) 86 negative i.e. the final contract amount was lesser than the original contract amount, thus indicating savings. The highest average cost growth of projects was observed in fit category 1 (negative) at - 1.24% , followed by fit category 2 (neutral), where the average cost growth across projects was - 1.68% . Least average cost growth per category was observed in fit category 3 (positive) at - 5.03% . The trend is graphically represented in Figure 15 . Figure 15 Average Cost Growth across Fit Categories The results of the Shapiro - Wilk test of normality for cost growth are presented in Table 25 Table 25 Normality Tes t for Cost Growth Normality Test (Shapiro - Wilk) Characteristic Fit Category 1 (Negative) Fit Category 2 (Neutral) Fit Category 3 (Positive) p - value 1.71E - 08 4.96E - 06 3.58E - 03 skewness 3.87 - 0.05 - 1.80 kurtosis 19.89 1.59 3.97 z 10.00 0.00 - 3.00 Result Not Normal Not Normal Not Normal As observed , p < 0.05 for categories 1, 2 and 3, thus indicating that data in all categories are not normally distributed. Increase in Participant Satisfaction Post data cleaning for non - responses and removal of ou tliers, increase in participant satisfaction performance of 80 of the 127 projects in the data set was obtained. Out of the 80 projects, - 1.2% - 1.7% - 5.0% COST GROWTH (%) FIT CATEGORY AVERAGE COST GROWTH ACROSS FIT CATEGORIES 1 (Negative) 2 (Neutral) 3 (Positive) 87 implemented partnering level), risk intensity of the project); and risk inten sity on the project) Figure 16 Average Increase in Participant Satisfaction across Fit Categories It is notable that across all fit categories increase in participant satisfaction is positive i.e. the final participant satisfaction score was greater than the participant satisfaction score at the beginning of the project. The lowest average increase in participant satisfaction of projects was obse rved in fit category 1 (negative) at 6.6% , followed by fit category 2 (neutral), where the average increase in participant satisfaction across projects was 8.2% . The highest average increase in participant satisfaction per category was observed in fit category 3 (positive) at 12.5% . The results of the Shapiro - Wilk test of normality for increase in participant satisfaction are presented in Table 26 Table 26 Normality Test for Increase in Participant Satisfaction Normality Test (Shapiro - Wilk) Characteristic Fit Category 1 (Negative) Fit Category 2 (Neutra l) Fit Category 3 (Positive) p - value 0.14 7.16E - 05 0.01 skewness 0.95 1.60 1.07 kurtosis 1.71 4.54 - 0.48 z 2.00 5.00 2.00 Result Normal Not Normal Not Normal 6.6% 8.2% 12.5% INCREASE IN PARTICIPANT SATISFACTION (%) FIT CATEGORY AVERAGE INCREASE IN PARTICIPANT SATISFACTION ACROSS FIT CATEGORIES 1 (Negative) 2 (Neutral) 3 (Positive) 88 As observed , p < 0.05 for categories 2 and 3, thus indicating that data in categories 2 and 3 are not normally distributed. As observed, w ith respect to distribution of the population of performance outcomes, there is no basis for assuming their normality . Moreover, the Shapiro - Wilk test for normality for performance outcome mea sures across the three categories failed for most analys e s. T his eliminated the possibility of using parametric statistical tests (e.g., ANOVA) as they rely on the basis that the samples are normally distributed. Hence, the Kruskal - Wallis non - parametric st atistical test was used, which does not require that data from the samples be normally distributed. Number of Change Orders Post data cleaning for non - responses and removal of outliers, number of change orders performance of 1 2 3 of the 127 projects in the data set was obtained. Out of the 1 2 3 projects, 36 (2 9 implemented partnering level), 71 ( 58 l equivalent to the risk intensity of the project); and 1 6 intensity on the project) The trend of number of change orders across the fit categories is graphically represented below 89 Figure 17 Average Number of change orders across Fit Categories The results of the Shapiro - Wilk test of normality for number of change orders are presented in Table 27 . Table 27 Normality Test for Number of change orders Normality Test (Shapiro - Wilk) Characteristic Fit Category 1 (Negative) Fit Category 2 (Neutral) Fit Category 3 (Positive) p - value 4.27E - 06 1.11E - 15 1.62E - 01 skewness 1.93 6.56 3.21 kurtosis 3.63 49.51 11.09 z 5.00 23.00 6.00 Result Not Normal Not Normal Not Normal As observed, p < 0.05 for all categories, thus indicating that data in all categories are not normally distributed. As observed, with respect to distribution of the population of performance outcomes, there is no basis for assuming their normality. Moreover, the Shapiro - Wilk test for normality for performance outcome measures across the three categories failed for most analyses. This eliminated the possibility of using parametric statistical tests (e.g., ANOVA) as they rely on the basis that the samples are normally distributed. Hence, the Kruskal - Wallis non - parametric statistical test was used, which does not require that data from the samples be normally distributed. 41 65 25 0 10 20 30 40 50 60 70 AVG. NO. OF CHANGE ORDERS FIT CATEGORY AVERAGE NO. OF CHANGE ORDERS ACROSS FIT CATEGORIES 1 (Negative) 2 (Neutral) 3 (Positive) 90 4.6 Hypothesis Testing The overall study hypothesis In a partnered project, better the fit between t he intensity of risk and adopted partnering level, better is its performance (e.g., cost growth, schedule growth). For the purpose of statistical analysis, the study hypothesis can be paraphrased as of measures for performance outcome metrics (schedule growth, cost growth and increase in participant satisfaction) differ across the risk - The unit of anal ysis is partnered projects . where: Fit category 1 project shows risk intensity higher than implemented partnering level, F it category 2 a partnering level equivalent to the risk intensity; and Fit ca tegory 3 represents fit the project shows a partnering level higher than risk intensity . The dependent variables are cost growth, schedule growth and increase in participant satisfaction. Because the impact of the risk - partnering fit on pro ject performance will be tested for each performance metric separately, three sub - hypotheses were generated from the overall study hypothesis. Hypothesis 1: risk - part Hypothesis 2: - Hypothesis 3: ribution of measures for increase in participant satisfaction performance differ across the risk - Hypothesis 4: number of change orders differ across the risk - 4.6.1 Risk - Partnering Fit versus S chedule G rowth This involved examining if the risk - partnering fit is correlated to the schedule growth performance of partnered projects. For statistical testing, the assumed correlation is represented by Hypothesis 1 that, 91 - partnering fit categories (negative, neutral and positive). The above hypothesis was considered as an alternative hypothesis (H A ) when conducting the Kruskal - Wallis test, where the null hypothesis (H 0 ) indicated that there is no difference in distribution of schedule growth measures (estimated via the median) across t he risk - represented as: Table 28 shows t he results of the Kruskal - Wallis tes t: Table 28 Kruskal - Wallis Test for Fit versus Schedule Growth Kruskal - Wallis Test (Fit versu s Schedule Growth) H 0 The samples come from populations with equal medians H 1 T he samples come from populations with medians that are not all equal Observation CHISQ = 5.12 > 4.605 (rejection region) Hypothesis Testing H 0 Rejected p - value 0.08 < 0.1 (90% significance level ) Thus, as observed, the n ull hypothesis was rejected , thereby implying that the alternative hypothesis, which indicated that there is a variability introduced in schedule growth performance because of the risk - partnering fit of a project , is true. T he result offers empirical evidence to assert that there exists a statistically significant (CHISQ = 5.12, p < 0.1) correlation betw een risk - partnering fit and schedule performance of partnered projects. The researcher then sought to identify which of the categories differed from each other and the order between them. To achieve this objective, the Dunn - test was conducted . The results of the Dunn Test are tabulated in Table 29 . Table 29 Dunn Test for Fit versus Schedule Growth Difference p - value Result Mean fit category 1 - Mean fit category 3 2.138072 0.0163 The difference is significant at 95% confidence level (CI) Mean fit category 1 - Mean fit category 2 1. 685832 0.0459 The difference is not significant at 95% CI, but is significant at 90% CI Mean fit category 2 - Mean fit category 3 1.055765 0.1455 The difference is not significant at 95% CI The results of the Dunn test can be interpreted as follows: 92 Schedule growth performance of projects in Fit Category 3 (positive fit) is statistically significantly less than that of projects Fit Category 1 (negative fit) at 95% confidence (p=0.01 < 0.05). That is, when it comes to the performance outcome metric of schedule growth, one can say that , 95% of the time, projects with partnering level higher than risk intensity (positive fit) perform better than projects with partnering level lower than risk intensity (negative fit). Schedule growth performance of project s in Fit Category 3 (positive fit) is not statistically significantly different than that of projects Fit Category 2 ( neutral fit) ( p=0.14 ) Schedule growth performance of projects in Fit Category 2 (neutral fit) is not statistically significantly different from that of projects Fit Category 1 (negative fit) at 95% confidence (p=0.04). However, the difference is significant at 90% confidence interval. That is, one can say that, 90% of the time, projects with partnering level equal to risk intensity (neutral fit) perform better than projects with partnering level lower than risk intensity (negative fit). Figure 18 below provides a visual representation of the values of sche dule growth across the three fit categories . Figure 18 Fit versus Schedule Growth 93 4.6.2 Risk - Partnering Fit versus Cost Growth This involved examining if the risk - partnered projects. For statistical testing, the assumed correlation is represented by Hypothesis 2 that, t growth performance differ across the risk - partnering fit categories (negative, neutral and positive). The above hypothesis was considered as an alternative hypothesis (H A ) when conducting the Kruskal - Wallis test, where the null hypothesis (H 0 ) indicate d that there is no difference in distribution of cost growth measures (estimated via the median) across the risk - represented as: Table 30 shows t he results of the Kruskal - Wallis tes t. Table 30 Kruskal - Wallis Test for Fit versus Cost Growth Kruskal - Wallis Test (Fit versus Cost Growth) H 0 The samples come from populations with equal medians H 1 The samples come from populations with medians that are not all equal Rejection Region CHISQ > 5.991 Observation CHISQ = 0.046 < 5.991 Hypothesis Testing H0 Not Rejected p - value 0.9772 > 0.1 (90% significance level ) Thus, as observed, the null hypothesis was not rejected, thereby implying that the alternative hypothesis, which indicated that there is a variability introduced in cost growth performance because of the risk - partnering fit of a proje ct, is not true. Thus, the result provides no empirical evidence to assert that there exists a statistically significant correlation between risk - partnering fit and cost performance of partnered projects. Figure 19 below provides a visual representation of the values of cost growth across the three fit categories . 94 Figure 19 Fit versus Cost Growth 4.6.3 Risk - Partnering Fit versus Increase in Participant Satisfaction This involved examining if the risk - i ncrease in participant satisfaction performance of partnered projects. For statistical testing, the assumed correlation is represented by Hyp othesis 3 that, increase in participant satisfaction performance differ across the risk - partnering fit categories (negative, neutral and positive). The above hypothesis was considered as an alternative hypothes is (H A ) when conducting the Kruskal - Wallis test, where the null hypothesis (H 0 ) indicated that there is no difference in distribution of i ncrease in participant satisfaction measures (estimated via the median) across the risk - and was represented as: Table 31 shows t he results of the Kruskal - Wallis tes t. 95 Table 31 Kruskal - Wallis Test for Fit versus Increase in Participant Satisfaction Kruskal - Wallis Test (Fit versus Increase in Participant Satisfaction ) H 0 The samples come from populations with equal medians H 1 T he samples come from populations with medians that are not all equal Rejection Region CHISQ > 5.991 Observation CHISQ = 0.046 < 5.991 Hypothesis Testing H0 Not Rejected p - value 0.9772 > 0.1 (90% significance level ) Thus, as observed, the null hypothesis was not rejected, thereby implying that the alternative hypothesis, which indicated that there is a variability introduced in i ncrease in participant satisfaction performance because of the risk - partnering fit of a project, is not true. Thus, the result provides no empirical eviden ce to assert that there exists a statistically significant correlation between risk - partnering fit and i ncrease in participant satisfaction performance of partnered projects. Figure 20 below provides a visual representation of the values of i ncrease in participant satisfaction across the three fit categories. 96 Figure 20 Fit versus Increase in Participant Satisfaction 4.6.4 Risk - Partnering Fit versus Number of change orders This involved examining if the risk - number of change orders n partnered projects. For statistical testing, the assumed correlation is represented by Hypothesis 4 th at, number of change orders performance differ across the risk - partnering fit categories (negative, neutral and positive). The above hypothesis was considered as an alternative hypothesis (H A ) when conducting t he Kruskal - Wallis test, where the null hypothesis (H 0 ) indicated that there is no difference in distribution of number of change orders measures (estimated via the median) across the risk - represented as: Table 32 shows t he results of the Kruskal - Wallis tes t. Table 32 Kruskal - Wallis Test for Fit versus Number of change orders Kruskal - Wallis Test (Fit versus Number of change orders ) 97 H 0 The samples come from populations with equal medians H 1 T he samples come from populations with medians that are not all equal Rejection Region CHISQ > 5.991 Observation CHISQ = 2. 98 < 5.991 Hypothesis Testing H0 Not Rejected p - value 0. 2253 > 0.1 (90% significance level ) Thus, as observed, the null hypothesis was not rejected, thereby implying that the alternative hypothesis, which indicated that there is a variability introduced in number of change orders performance because of the risk - partnering fit of a project, is not true. Thus, the result provides no empirical evidence to assert that there exists a statistically significant correlation between risk - partnering fit and number of change orders performance of partnered projects. Figure 21 below provides a visual representation of the values of number of change orders ac ross the three fit categories. Figure 21 Fit versus Number of change orders 98 4.7 Summary Descrip tive characteristics of samples of project performance measures distributed across the three risk - partnering fit categories show that: 1. With respect to averages of schedule growth performance (after discounting for contractual time extensions granted on acc ount of owner scope additions) across the fit categories, a. Mean s chedule growth is least in projects in fit category 3 (positive), followed by fit category 2 (neutral) and then by fit category 3 (negative). This implies that on an average, schedule perform ance improves as the risk - partnering fit improves. b. Mean schedule growth is negative in projects in fit category 3 (positive) , thus implying that on an average, projects that adopted a partnering level higher than risk intensity (fit category 3) completed t he original scope of the projects ahead of the original planned duration. 2. With respect to averages of cost growth performance (after discounting for contractual change order costs accepted on account of owner scope additions) across the fit categories, a. Me an cost growth across the three fit categories was negative, thus implying that on an average, projects that adopt partnering complete the original scope of the projects under - budget compared to the original contract amount. b. Further, the mean cost growth i s least in projects in fit category 3 (positive), followed by fit category 2 (neutral) and then by fit category 3 (negative). This implies that on an average, that as project as the risk - partnering fit improves, more savings are realized in partnered proje cts. 3. With respect to averages of increase in participant satisfaction performance across the fit categories, a. Mean increase in participant satisfaction across the three fit categories was positive, thus implying that on an average, on projects that adopt partnering, after completion, project participants leave the project with higher sense of satisfaction compared to the beginning of the project. b. Furthe r, the mean increase in participant satisfaction is most in projects in fit category 3 (positive), followed by fit category 2 (neutral) and then by fit category 3 (negative). This implies that on an average, that as project as the risk - partnering fit impro ves, 99 project participants experience higher increase in project satisfaction in partnered projects. Statistical testing of the study hypothesis yielded a significant result when tested for the impact of risk - partnering fit on schedule growth performance . I t was discovered that s chedule growth performance of projects in Fit Category 3 (positive fit) were found to be statistically significantly less than that of projects Fit Category 1 (negative fit) at 95% confidence (p=0.01 < 0.05). That is, 95% of the time , projects with partnering level higher than risk intensity (positive fit) perform better than projects with partnering level lower than risk intensity (negative fit). Thus, there exists empirical evidence to support the assertion better the fit Although statistical testing of hypothesis for other performance measures (cost growth and increase in participant satisfaction ) did not yield s ignificant results, the researcher believes that if additional data to establish normality of the samples was collected, parametric statistical tests (e.g., ANOVA) would show significant results of trends like those discovered for schedule growth performan ce. 100 CHAPTER 5 CONCLUSIONS 5.1 Conclusions from R esults and F indings Following conclusions can be made from the results and findings of this study: 1. There exists statistical evidence to the existence of a correlation between risk - partnering fit and schedule performance (measured via schedule growth) of partnered AEC projects. 2. P artnered AEC projects that adopted a higher level of collaboration (via partnering) compared to the intensity o f risk, demonstrate significantly improved schedule performance (measured via schedule growth) than projects that adopted a lower level of collaboration (via partnering) compared to the intensity of risk. 3. Statistical evidence was not found to assert a cor relation between risk - partnering fit and other performance outcomes considered in this study, i.e. , cost performance (measured via cost growth) and increase in participant satisfaction . 4. D escriptive characteristics of the variables (cost growth and increase in participant satisfaction ) clearly exhibit a trend showing that as the fit category improved, average values for both variables demonstrated improved cost and participant satisfaction performance. 5.2 Deliverables and Implications This study offers the following deliverables to the theory of project (specifically risk) management via collaboration: 1. Revised models for risk intensity and partnering level assessment, which are statistically verified via surveys, presented to industry experts. 2. The study f il led the gap by conducting the empirical investigation into the impact of the interplay between risk and collaboration (via partnering) on project performance . a. The results of this study support the claim that partnering is an effective project delivery prac tice for improved collaboration and subsequent improvement in project performance. b. The researcher observed that projects encountering significant risk intensity could reduce its probability of occurrence and/or the severity of its impact in case it materia lizes to prevent it from affecting project performance. 101 c. It was also observed that a significant portion what could be classified as technical risk (e.g., constructability, design) comprised of risk of mistrust or non - collaborative behavior when determining its intensity. This paves a way for understanding the causality between collaboration and risk management. d. By empirically demonstrating how risk is managed and project performance improved by adopting the appropriate level of partnering, this study adds to the theory of best practices in Partnering . Although, previous researchers recommended this best practice, there did not exist empirical reinforcement to the same. In addit ion, this research supports the use of Partnering as an effective structured approach to achieve collaboration on AEC projects. Currently, partnering seems to be the only structure via which collaboration can be stratified, adjusted and adopted into variou In addition to the above, t he outcomes of this study offer the following pragmatic deliverables to AEC industry practice as well: 1. This study presents m odels to assess risk intensity and partnering level of projects , which can be utilized by pro ject stakeholders (owner organization, construction manager, etc.) to assess the intensity of risk on their project and decide a level of collaboration (via partnering) to adopt, with the goal of achieving improved project performance outcomes. 2. The result s of this study demonstrate tangible benefits (e.g., improvement in cost and schedule performance) of implementing the appropriate level of collaboration (via partnering). These results can help project managers or stakeholders c onvincing upper management about the benefits of implementi ng a s tructured collaboration practice like Partnering. 5.3 Limitations and Discussion I t is important that to recognize the limitation of this study and discuss their origin and possible solutions . Such a discussion would assis t future researchers in this field to be mindful about the limitations; and therefore, account for them in their research design . F ollowing are potential limitations encountered in this study and discussions about them: One might argue the choice of partnered projects as the unit of analysis to study collaboration in this research. The researcher acknowledges this argument but counters that there is a l ack of 102 availability of other generally acceptable and structured models to implement a nd study collaboration. Moreover, a vailable literature strongly suggests that Partnering is possibly the only framework to study collaboration in an analytical manner for the purpose of quantitative research in the domain of collaboration. There are some i nherent limitations to using archival data for research. The researcher had no control over the design or standard of the questionnaire used to collect data. For example, t he questionnaire did not directly request data about risk or its intensity to its re spondents. Although the researcher developed a validated model to assess risk intensity from available project details in the data - set, it is possible that some risk elements were not reported in those details. Descriptive statistics show that a large number of projects (62%) were located in California. Although this a cause for concern regarding location bias, the researcher finds no connection as to how that would affect the results of this research as none of the study constructs (risk intensity, partnering level, project performance) are known to be location - dependent. Nevertheless, the researcher has provided explanations justifying the bias in Section 4.1 . R inherent drawbacks. Risk assessment should ideally be conducted at or before the beginning of a project. As the data set c ontained projects that had already been completed, it is possible that probability of occurrence of the risk elements or severity of their impact. Although re liability checks were conducted to maintain the quality of coding, t he researcher recommends that the risk intensity model presented in this study be used for assessing risk intensity level before commencement of the project. Alternatively , risk assessment methods suggested by Hanna at al. (2013) could be used. There is a debate regarding ordinal measurement or quantification of risk. It is argued that risk cannot be graded on scales of say 1 5 beca use, for example, it is difficult to perceive with certainty how a risk of level of 4 is exactly doubly as risky as a level 1 risk . However, this study presented several peer - reviewed journals support research using ordinal scales for risk assessment and measurement. In addition, helpful statistical analys is is difficult to conduct on continuous data. Further, it is tricky to assign one number representing the overall risk intensity or partnering level of a project. However, it might be necessary to do so when making a decision like which partnering level to adopt based on the risks of the project. As recommended in the later section, f urther 103 research in this domain might suggest a one to one correspondence between the intensity of individual risk element and level of individual partnering tool to be select ed. However, based on available theory , this researcher considered it apt to use ordinal measurement to represent overall risk intensity level and partnering level of a project. 5.4 Recommendations for F uture R esearch This research initiated an empirical inves tigation into the impact of the interplay between risk and colla boration on project performance. During the study, there were several lessons learned, limitations experienced as well as avenues identified for further research. Based on them, t he researcher suggests following strategic research directions: 1. Refinement of Risk Intensity and Partnering Level assignment models Survey - based r esearch efforts followed by factor analysis can be taken to refine the accuracy of risk intensity and partnering level as signment models used in this study. By reaching out to a variety of project participants like owners, stakeholders, designers, contractors and trades, the objective of such a study could be to understand if there is a difference in how risk intensity and p artnering level assignments are perceived by project participants with different roles. The study could also attempt to assign weightages to the risk and partnering factors with the aim to develop a stand - alone decision - making tool for prospective risk int ensity and partnering level assessment. 2. Best Practices for Risk Management via Collaboration Using the models presented in thi s study, efforts can be taken to map usage of specific partnering tools to alleviate specific risk element s or factor s . This effort could comprise of interview - based or case study research. The outcome would be a prescriptive model for decision - makers to select a particular partnering tool and its level to combat a certain identified risk and its intensity. 3. Comparison of Pa rtnered versus Non - Partnered Projects By conducting a data collection effort to collect data of similar nature from a similar variety of AEC projects that did not implement Partnering, one could compare the performance of partnered and non - partnered proj ects. Such a study would check if and how the adoption of partnering efforts improve specific performance outcomes of a project compared to those of non - partnered projects. 4. O bserving impacts on relational risk instead of standard project risks During con tent analysis, the researcher observed that the impact of collaboration on project performance via risk reduction appears to be indirect . That is, p roject details in the data - set often revealed that collaboration reduced the risk of non - collaborative behav ior (e.g., mistrust, contentious 104 communication) , which in turn led to reduction of the probability of occurrence and/or severity of impact of the s tandard project risk (e.g., construction and design complexit y, unfavorable site conditions). Thus, a study s imilar to this one could be undertaken to firstly identify risk elements of non - Lehtiranta, 2011); then develop models to determine relational risk intensity and correlate it to the level collaboratio n (via partnering) and proejct performance. Such a study might establish causal relations linking collaboration, risk reduation and proejct performance. Continuing research in the domain of the interplay between risk, collaboration and performance wi ll help decision makers adopt collaboration in a more informed and structured manner. It will allow for the development of metrics by which the outcomes of adopting collaboration could be anticipated, predicted and measured. The researcher hopes that such developments will ultimately help the AEC industry to be recognized as a collaborative industry and consequently produce innovative and sustainable built - environment solutions as a result. 105 APPENDICES 106 APPENDIX A Sample project award application 107 108 109 110 111 112 113 114 115 116 117 118 119 APPENDIX B Risk scaling grades and measures in literature Table 33 Exhibit - A of Risk Analysis Scale (Source: Hannah, Thomas & Swanson, 2013) Scale Probability Of Risk Realization Risk Impact (extent of impact on project objectives if the risk realizes) Product Score Risk Rating Scale 1 Very low (< 10% chance) Negligible and routine procedures sufficient to deal with the consequence (<5% impact) 0 5 1 2 Low chance (10% 35% chance) Minor and would threaten an element of the function (5 10% impact) 6 10 2 3 Medium (35% 65% chance) Moderate and would necessitate significant adjustment to the overall function (10 20% impact) 11 15 3 4 High (65% - 90% chance) Significant and would threaten goals and objectives (20 50% impact) 16 20 4 5 Very High (> 90% chance) Extreme and would stop achievement of functional goals and objectives (>50% impact) 20 25 5 Table 34 Exhibit - B of Risk Analysis Scale (Adopted from: Baccarini & Archer, 2001) Risk Factor Risk Rating Scale 5 4 3 2 1 The way the cost targets were established There is no clear budget or the budget is clearly insufficient The basis for the current budget is unclear or the budget is likely to be inadequate The basis for the budget is clear, but indications are that overruns are possible Benchmarks wer e used to establish budgets Benchmarks were used to establish the budget and adequate contingencies exist The effect if the cost targets are not met No additional funds available and project will not proceed No additional funds available and scope reduced Request for additional funds would be lengthy and embarrassing Some scope for additional funds Additional funds available Uniqueness of the product Prototype incorporating new techniques Unusual project (out of the ordinary) Conventional project Modifications to an existing design One of a series of repetitions Table 35 Exhibit - C of Risk Analysis Scale (Source: Kindinger and Darby, 2000) 120 Risk Analysis Scale Non/Low Risk Medium Risk High Risk Risk factor can be tackled via Known resources and knowledge of the organization Resources and knowledge need to the adapted to tackle the risk factor New resources need to be procured or new knowledge needs to be developed to tackle the risk factor 121 Table 36 Post - EDA Risk Element Grading (Horizontal Projects) Risk factor scaling grade for horizontal projects Risk factors Risk element(s) Level 4 Level 3 Level 2 Level 1 Source Project value Budget $250m - $500m+ $10m - $250m $5m - $10m $0 - $5m IPI, 2018 Duration 18 24+ months 12 18 months 6 12 months <6 months IPI, 2018 Work per day $100,000 - $200,000+ $50,000 - $100,000 $25,000 - $50,000 $0 - $25,000 EDA observations Project - based risk Project approvals Large number of approvals required; high level of difficulty/stringency expected; may impact project severely Some approvals required; possible difficulty/stringency expected; provisions in budget and schedule for delay Minimal number and regular approvals (no difficulty/ stringency expected); no impact on project No approval required or all have been obtained; n o impact on project Baccarini & Archer, 2001 Site & environmental conditions History of differing site conditions; conditions may affect schedule, cost, quality, or safety; moderate to extreme weather conditions No history of differing site conditions; c ontrollable site conditions; will not affect schedule, cost, quality or safety; basic precautions taken; moderate weather Favorable site conditions; minimal risk to schedule, cost, quality, or safety;; precautions taken; slight weather delays expected Favo rable; no risk to the schedule, cost, quality or safety; established and known; no history of differing site conditions; no weather conditions expected Chan D. W., Chan, Lam, Yeung, & Chan, 2011 Safety, accessibility & on - going operations Risk of catastrophe/fatality; staging within occupied areas/on - going construction; challenging access issues Moderate risk; risk of disability; additions to occupied areas/staging adjacent to on - going operations or construction; no access issues Minor risk of dama ge; well clear of occupied areas; no access issues Minor to no risk & greenfield site; no access issues Baccarini & Archer, 2001 Construction complexity Very high; new/innovative methods involved, constructability affected by environment Moderate; complex operations required Low; minor challenges expected Very low; little to no challenge expected Based on EDA observations 122 Design complexity Design & specs based on incomplete information; risk of omissions Designer is inexperienced or design team is improper; probability of design errors Improper/incomplete design scheme communicated by client; experienced and competent designer Design scheme well communicated by client; experienced and competent designer; p roper design reviews completed Baccarini & Archer, 2001 Sociopolitical risks Third - party stakeholder & Public Interest High level of political, community or media sensitivity; high profile client External stakeholder groups involved; less to no level of sociopolitical sensitivity Project may attract stakeholder or media interest Project unlikely to attract stakeholder or media interest Baccarini & Archer, 2001 Project relationships Inter - stakeholder relationships New relationships; history of litigation; joint ventures; less to no time for relationship development New relationships, some scope for developing relationships Some previous project experience and neutral to good working relationships & experiences Lots of previous working experience and relati onships developed. Baccarini & Archer, 2001 Desired Level of Engagement Cost pressure Unclear/insufficient budget, budget feasibility not established, lack of clarity regarding financing, strictly no scope for additional funds, little to no contingency Budget feasibility not established, adequate funds and sources secured but request for additional funds would be lengthy and embarrassing, sufficient contingency Budget feasibility established, adequate funds and sources secured and some scope for additional funds, sufficient contingency Budget feasibility established using benchmarks, adequate & sure scope for additional recurrent funds and sources secured, generous contingency Baccarini & Archer, 2001 Schedule pressure The basis for the current schedule is unclear or the schedule is likely to be inadequate The basis for the schedule is clear, but indications are that overruns are possible Benchmarks were used to establish schedule Benchmarks were used to establish the schedule and adequat e contingencies exist Baccarini & Archer, 2001 Quality pressure High quality requirements; if unmet could affect cost, schedule, project significantly Moderate quality requirements; if unmet could affect cost, schedule, project slightly Quality requirements not specifically mentioned; less to no probability of affecting the project No mention about importance of quality requirements; will not affect project Baccarini & Archer, 2001 123 Table 37 Post - EDA Risk Element Grading (Non - Horizontal Projects) RISK FACTOR CODING FORM FOR NON - HORIZONTAL ( VERTICAL & AVIATION ) PROJECTS Risk factors Risk element(s) Level 5 Level 4 Level 3 Level 2 Level 1 Source Project value Budget $250M - $500M+ $25M - $250M $10M - $25M $5M - $10M $0 - $5M IPI, 2018 Duration >24 months 18 24 months 12 18 months 6 12 months <6 months IPI, 2018 Work per day >$200,000 $100,000 - $200,000 $50,000 - $100,000 $25,000 - $50,000 $0 - $25,000 based on EDA observations Project - based risk Project approvals Large number of approvals required; High level of difficulty/stringency expected; may impact project severely Significant number of approvals required; Medium level of difficulty/stringency expected; may impact project significantly Some approvals required; Possible difficulty/stringency expected; provisions in budget and schedule for delay Minimal number of approvals required; Regular approvals (zero difficulty/stringency expected); no impact on project No approval req uired or all have been obtained; no impact on project Baccarini & Archer, 2001 Site & environmental conditions history of differing site conditions; Site conditions uncontrollable; may affect schedule, cost, quality or safety; Site location prone to acts of God, Extreme weather history of differing site conditions; Controllable (planned for) site conditions but may affect schedule, cost, quality, or safety; Moderate to extreme weather No history of differing site conditions; Controllable site conditio ns; will not affect schedule, cost, quality or safety; basic precautions taken; Moderate weather Favorable site conditions; minimal risk to schedule, cost, quality, or safety;; precautions taken; Slight weather delays expected Favorable; no risk to the sch edule, cost, quality or safety; Established and known; no history of differing site conditions; no weather conditions expected Chan D. W., Chan, Lam, Yeung, & Chan, 2011 Safety, accessibility & on - going operations Risk of Catastrophe numerous fatalities & Activities in occupied areas/On - going operations; Challenging access issues Risk of fatality; Staging within occupied areas/On - going construction; Challenging access issues Moderate Risk; Risk of disability; Additions to occupied areas/Staging adjacent to on - going operations or construction; No access issues Minor risk of damage; Well clear of occupied areas; No access issues Minor to no risk & Greenfield site; No access issues Baccarini & Archer, 2001 Construction complexity Very High; Never done before Innovative and risky operations required High; Never done before but safe operations required Moderate; Complex operations required Low; Minor challenges expected Very Low; Little to no challenge expected based on EDA observations 124 Design complexity incomplete design and high probability of design change and review; underdeveloped specs design & specs based on incomplete information; risk of omissions designer is inexperienced or design team is improper; probabilit y of design errors improper/incomplete design scheme communicated by client; experienced and competent designer design scheme well communicated by client; experienced and competent designer; proper design reviews completed Baccarini & Archer, 2001 Sociopolitical risks Third - party stakeholder & Public Interest High level of political, community or media sensitivity High profile client or project Stakeholder groups involved Project may attract stakeholder or media interest Project unlikely to attract stakeholder or media interest Baccarini & Archer, 2001 Project relationships Inter - stakeholder relationships Client with no experience. New Relationships; History of Litigation; Joint Ventures; Less to No time for relationship development Mixed experience amongst clients or Relationships, Less to no scope for developing relationships New Relationships, Some scope for developing relationships Some previous project experience and neutral to good working relationships & experiences Lots of previous working experience and relationships developed. Baccarini & Archer, 2001 Desired Level of Engagement Cost pressure No clear budget, budget seems insufficient, budget feasibility not established, inadequate funds or sources not secured, less to no contingency Budget feasibility not established, adequate funds and sources identified but financing not secured, strictly no scope for additional funds, little to no contingency Budget feasibility not established, adequate funds and sou rces secured but request for additional funds would be lengthy and embarrassing, sufficient contingency Budget feasibility established, adequate funds and sources secured and some scope for additional funds, sufficient contingency Budget feasibility establ ished using benchmarks, adequate & sure scope for additional recurrent funds and sources secured, generous contingency Baccarini & Archer, 2001 Schedule pressure There is no clear schedule or the schedule is clearly insufficient The basis for the current schedule is unclear or the schedule is likely to be inadequate The basis for the schedule is clear, but indications are that overruns are possible Benchmarks were used to establish schedule Benchmarks were used to establish the schedule and adequat e contingencies exist Baccarini & Archer, 2001 Quality pressure High Quality requirements; if unmet could affect cost, schedule, project significantly Moderate Quality requirements; if unmet could affect cost, schedule, project slightly Quality requirements not specifically mentioned; less to no probability of affecting the project Quality requirements not specifically mentioned; will not affect project No mention about importance of quality requirements; will not affect project Baccarini & Arc her, 2001 125 APPENDIX C Survey consent form 126 APPENDIX D Survey e - mail and design Survey E - Mail: Dear (survey respondent), My name is H. (Harsh) Kalbhor, and I am a researcher with Dr. Sinem Mollaoglu at Michigan State University on an IPI - funded research project. Our data shows that you were the Partnering Facilitator on one or more projects submitted as applicants for the Partnered Project of the Year Awards between 2010 and 2018, held annually by IPI. For our current research examining the relation betwe performance outcomes, we request your participation in a short survey. Please find attached the following to this e - mail: Research Participant Information and Consent Form for your review Survey File Pl ease fill out the 10 - minute survey and attach it back in a reply to this e - mail (to kalbhorh@msu.edu ) Your response is highly valued and would greatly With best regards and thanks, H. (Harsh) Kalbhor 127 128 129 REFERENCES 130 REFERENCES Abedi, M., Fathi, M., & Rawai, N. M. (2012). Cloud Computing Technology for Collaborative Information System in Construction Industry. 18th International Business Information Management Association (IBIMA). Istanbul, Turkey. AIA California Council. (2007). A Working Definition - Integrated Project Delivery. Sacramento: AIA California Council. Al - Bahar, J. F., & Crandall, K. C. (1990). Systematic Risk Management Approach for C onstruction Projects. Journal of Construction Engineering and Management, 116(3), 533 - 546. Retrieved October 22, 2018, from https://ascelibrary.org/doi/abs/10.1061/%28ASCE%290733 - 364%281990%29116%3A3%28533%29 Anderson, D. K., & Merna, T. (2003). Project M anagement Strategy project management represented as a process based set of management domains and the consequences for project management strategy. International Journal of Project Management, 21(5), 387 - 393. Retrieved 11 11, 2018, from https://ac.els - cdn .com/S026378630200087X/1 - s2.0 - S026378630200087X - main.pdf?_tid=1b260025 - 2bbb - 42c1 - ad9f - 07de4dd4163e&acdnat=1541953168_5593fb20ca9d7f854caec86d9d19d6b6 Asmar, M., Hanna, A., & Loh, W. - Y. (2016). Evaluating Integrated Project Delivery Using the Project Quart erback Rating. Journal of Construction Engineering and Management, 142(1). Retrieved November 30, 2018, from https://ascelibrary.org/doi/pdf/10.1061/%28ASCE%29CO.1943 - 7862.0001015 Azhar, S. (2011). Building Information Modeling (BIM): Trends, Benefits, Ri sks, and Challenges for the AEC Industry. Leadership and Management in Engineering, 11(3), 241 - 252. Retrieved 11 18, 2018, from https://ascelibrary.org/doi/10.1061/%28ASCE%29LM.1943 - 5630.0000127 Azmy, N. (2012). The Role of Team Effectiveness in Construct ion Project Teams and Project Performance. Ames, Iowa: Iowa State University. Retrieved November 29, 2018, from https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=3272&context=etd Baccarini , D. (1996). The concept of project complexity - a review. In ternational Journal of Project Managemen, 14(4), 201 - 204. Retrieved 11 11, 2018, from http://ieg.ifs.tuwien.ac.at/~aigner/projects/planninglines/evaluation/Project_Management/papers/bac carini96complexity.pdf Baccarini, D., & Archer, R. (2001). The risk ra nking of projects: a methodology. International Journal of Project Management, 19(3), 139 - 145. Retrieved 11 12, 2018, from https://www.sciencedirect.com/science/article/pii/S0263786399000745#TBL1 Ballard, G., Kim, Y. - W., Jang, J. - W., & Liu, M. (2007). R oadmap for Lean Implementation at the Project Level. Austin: Construction Industry Institute. Retrieved December 2, 2018, from https://www.researchgate.net/profile/Glenn_Ballard/publication/232322523_Roadmap_for_Lean_Impl ementation_at_the_Project_Level/lin ks/00b7d52011d137218e000000/Roadmap - for - Lean - Implementation - at - the - Project - Level.pdf 131 Ballard, G., Tommelein, I., Koskela, L., & Howell, G. (2002). Lean construction tools and techniques. In R. Best, & G. Valence, Design and Construction: Building in Value (pp. 227 - 255). Oxford: Butterworth - Heinemann. California Department of Transportation. (2019, January 05). Partnering with Caltrans. Retrieved from Caltrans Division of Construction: http://www.dot.ca.gov/hq/construc/partnering/documents/Field_Guide_to_P artnering_on_Caltrans_C onstruction_Projects_final.pdf Chan, A. P., Chan, D. W., Chiang, Y. H., Tang, B. S., Chan, E. H., & Ho, K. S. (2004). Exploring Critical Success Factors for Partnering in Construction Projects. Journal of Construction Engineering an d Management, 130(2), 188 - 198. Retrieved December 4, 2018, from https://ascelibrary.org/doi/10.1061/%28ASCE%290733 - 9364%282004%29130%3A2%28188%29 Chan, A. P., Ho, D. C., & Tam, C. M. (2001). Effect of Interorganizational Teamwork on Project Outcome. Journal of Management in Engineering, 17(1), 34 - 40. Retrieved November 11, 2018, from https://ascelibrary.org/doi/pdf/10.1061/%28ASCE%290742 - 597X%282001%2917%3A1%2834%29 Chan, D. W., Chan, A. P., Lam, P. T., Yeung, J. F., & Chan, J. H. (2011). Ri sk ranking and analysis in target cost contracts: Empirical evidence from the construction industry. International Journal of Project Management, 29(6), 751 - 763. Retrieved 11 12, 2018, from https://www.sciencedirect.com/science/article/pii/S0263786310001 201?openDownloadIssueModal=tr ue Chapman, R. J. (2001). The controlling in¯uences on e • ective risk identi ® cation and assessment for construction design management. International Journal of Project Management, 19(3), 147 - 160. Retrieved 10 29, 2018, from https://ac.els - cdn.com/S0263786399000708/1 - s2.0 - S0263786399000708 - main.pdf?_tid=6d32c5df - dd53 - 40ab - 9474 - 0f909f5117c7&acdnat=1540819386_84c28a6fd5ebce8a85a852d76c2683d2 Chapman, R. J. (2001). The controlling influences on effective risk identification and assessment for construction design management. International Journal of Project Management, 19(3), 147 - 160. Retrieved November 25, 2018, from https://www.sciencedirect.com/science/article/pii/S0263786399000708 Chen, C., & Hubbard, M. (2012). Power relatio ns and risk allocation in the governance of public private partnerships: A case study from China. Policy and Society, 31(1), 39 - 49. Retrieved 10 28, 2018, from https://www.tandfonline.com/doi/full/10.1016/j.polsoc.2012.01.003?scroll=top&needAccess=true Ch eng, E. W., & Li, H. (2001). Development of a conceptual model of construction partnering. Engineering, Construction and Architectural Management, 8(4), 292 - 303. Retrieved December 3, 2018, from https://doi.org/10.1108/eb021190 Cheng, E. W., & Li, H. (200 2). Construction Partnering Process and Associated Critical Success Factors: Quantitative Investigation. Journal of Management in Engineering, 18(4), 194 - 202. Retrieved Decemeber 4, 2018, from https://ascelibrary.org/doi/abs/10.1061/(ASCE)0742 - 597X(2002)18 :4(194) 132 CII. (1989). Partnering RT - 018 (Archive). Retrieved from Construction Industry Institute: Knowledge Base: https://www.construction - institute.org/resources/knowledgebase/best - practices/partnering/topics/rt - 018 CII. (1996). Model for Partnering Ex cellence. Construction Industry Institute. Construction Industry Board. (1997). Partnering in the Team: A Report by Working Group 12 of the Construction Industry Board (CIB Reports). London: Thomas Telford Ltd. Cook, E., & Hancher, D. (1990). Partnering: Contracting for the future. Journal of Management in Engineering, 6(4), 431 - 446. Retrieved November 25, 2018, from https://ascelibrary.org/doi/pdf/10.1061/(ASCE)9742 - 597X(1990)6%3A4(431) Crowley, L. G., & Karim, M. A. (1995). Conceptual Model of Partnerin g. Journal of Management in Engineering, 11(5), 33 - 39. Retrieved December 3, 2018, from https://ascelibrary.org/doi/pdf/10.1061/%28ASCE%290742 - 597X%281995%2911%3A5%2833%29 Das, T. K., & Teng, B. - S. (2001). A risk perception model of alliance structuring. Journal of International Management, 7(1), 1 - 29. Retrieved October 29, 2018, from https://pdfs.semanticscholar.org/5033/dbc8f302739444dccee0bdb73a7107231d77.pdf Dossick, C., & Neff, G. (2011). Messy talk and clean technology: communication, problem - solvin g and collaboration using Building Information Modelling. The Engineering Project Organization Journal, 83 - 93. Retrieved November 19, 2018, from http://www.com.washington.edu/faculty/assets/neff_messy_talk.pd Fang, C., & Marle, F. (2012). A simulation - based risk network model for decision support in project risk management. Decision Support Systems, 52(3), 635 - 644. Retrieved 11 11, 2018, from https://ac.els - cdn.com/S0167923611002016/1 - s2.0 - S0167923611002016 - main.pdf?_tid=c40649e1 - 11a6 - 4496 - ad 44 - 935cc19ae7b1&acdnat=1541956447_049fb7873815644b4c7a7eaf8c1cb543 Fellows, R., & Liu, A. (2008). Research Methods for Construction (Third Edition ed.). Oxford, United Kingdom: Wiley - Blackwell. Retrieved January 8, 2019 Flick, U., Kardorff, E. v., & Stei nke, I. (2010). A Companion to Qualitative Research. London: SAGE. Forbes, L., & Ahmed, S. (2011). Lean Based Project Delivery Methods. In Modern Construction: Lean Project Delivery and Integrated Practices (pp. 167 - 201). Boca Raton, Florida: CRC Press. G olparvar - Fard, M., Peña - Mora, F., & Savarese, S. (2009). Application of D4AR - A 4 - Dimensional augmented reality model for automating construction progress monitoring data collection, processing and communication. Journal of Information Technology in Const ruction, 14(Special issue Next Generation Construction IT: Technology Foresight, Future Studies, Roadmapping, and Scenario Planning), 129 - 153. Retrieved December 2, 2018, from http://www.itcon.org/2009/13 Gransberg, D. D., Dillon, W. D., Reynolds, L., & B oyd, J. (1999). Quantitative Analysis of Partnered Project Performance. J. Constr. Eng. Manage., 125(3), 161 - 166. 133 Gransberg, D., Shane, J., & Schierholz, J. (2013). NCHRP Proejct 10 - 85: A Guidebook for Construction Manager - at - Risk Contracting for Higwhay Projects. Washington: Transportation Research Board. Gratt. (1989). The Definition of Risk and Associated Terminology for Risk Analysis. In J. J. Bonin, & D. E. Stevenson, Risk Assessment in Setting National Priorities. Advances in Risk Analysis (Vol. 7) . Boston, MA: Springer. Retrieved October 22, 2018, from https://link.springer.com/chapter/10.1007/978 - 1 - 4684 - 5682 - 0_73 Gratt, L. B. (1987). Risk Analysis or Risk Assessment; A Proposal for Consistent Definitions. In L. L. Covello V.T., Uncertainty in Ris k Assessment, Risk Management, and Decision Making. Advances in Risk Analysis (Vol. 4). Boston, MA: Springer. Retrieved October 22, 2018, from https://link.springer.com/chapter/10.1007/978 - 1 - 4684 - 5317 - 1_20 Hammad, A., Wang, H., & Mudur, S. (2009). Distrib uted Augmented Reality for Visualizing Collaborative Construction Tasks. Journal of Computing in Civil Engineering, 23(6), 418 - 427. Hanna, A. S., Thomas, G., & Swanson, J. R. (2013). Construction Risk Identification and Allocation: Cooperative Approach. J ournal of Construction Engineering and Management, 139(9), 1098 - 1107. Retrieved October 22, 2018, from https://ascelibrary.org/doi/abs/10.1061/%28ASCE%29CO.1943 - 7862.0000703 International Partnering Institute. (2018, November). Owner's Toolbox. Retrieved 11 11, 2018, from International Partnering Institute: https://partneringinstitute.org/owners - toolbox/ International Partnering Institute. (2018). What is Construction Partnering? Retrieved fr om International Partnering Institute: https://partneringinstitute.org/about/what - is - construction - partnering/ International Partnering Institute. (n.d.). IPI Owner's Toolbox. Retrieved from https://partneringinstitute.org/owners - toolbox/#toggle - id - 4 - closed Kindinger, J., & Darby, J. L. (2000). Risk factor analysis a new qualitative risk management tool. Project Management Institute Annual Seminars & Symposium. Houston, TX. Newtown Square, PA: Project Management Institute. Koch, J. E., Gransberg, D. D., & Molenaar, K. R. (2010). Chapter 7: Design - Build Change Orders. In Project Administration for Design - Build Contracts: A Primer for Owners, Engineers, and Contractors (pp. 191 - 217). Reston, Virginia: American Society of Civil Engineers. Retrieved November 3 0, 2018 Koskela, L., Howell, G., Ballard, G., & Tommelein, I. (2002). The foundations of lean construction. In R. Best, & G. Valence, Design and Construction: Building in Value (pp. 211 - 225). Oxford, UK: Butterworth Heinemann. Retrieved December 2, 2018, from file:///C:/Users/SPDC/Desktop/OneDrive%20 - %20Michigan%20State%20University/IPI%20Research%20Project/Literature/Koskela_Ch.14_Lean%20Co nstruction.pdf Kruskal, W. H., & Wallis, W. A. (1952). Use of Ranks in One - Criterion Variance Analysis. Journal of t he American Statistical Association, 47(260), 583 - 621. 134 Lehtiranta, L. (2011). Relational Risk Management in Construction Projects: Modeling the Complexity. Leadership and Management in Engineering, 11(2), 141 - 154. Retrieved October 29, 2018, from https:/ /ascelibrary.org/doi/10.1061/%28ASCE%29LM.1943 - 5630.0000114 Leung, M. - Y., Ng, T., & Cheung, S. - O. (2004). Measuring construction project participant satisfaction. Construction Management and Economics, 22(3), 319 331. Lu, Y., Li, Y., Skibniewski, M., Wu, Z., Wang, R., & Le, Y. (2015). Information and Communication Technology Applications in Architecture, Engineering, and Construction Organizations: A 15 - Year Review. Journal of Management in Engineering, 31(1). Retrieved December 2, 2018, from https://asce library.org/doi/pdf/10.1061/%28ASCE%29ME.1943 - 5479.0000319 Luo, L., He, Q., Xie, J., Yang, D., & Wu, G. (2017). Investigating the Relationship between Project Complexity and Success in Complex Construction Projects. Journal of Management in Engineering, 3 3(2). Retrieved October 27, 2018, from https://ascelibrary.org/doi/abs/10.1061/%28ASCE%29ME.1943 - 5479.0000471 Matthews, O., & Howell, G. (2005, April). Integrated Proejct Delivery An Example of Relational Contracting. Lean Construction Journal, 2(1), pp. 46 - 61. Retrieved November 30, 2018, from http://www.leanconstruction.org/media/docs/lcj/LCJ_05_003.pdf Miller, R., & Lessard, D. (2001). The Strategic Management of Large Engineering Projects. Cambridge, Massachusetts.: The MIT Press. Retrieved 11 19, 201 8, from http://faculty.london.edu/icooper/assets/documents/StrategicManagement.pdf Molenaar, K., Harper, C., & Yugar - Arias, I. (2014). Guidebook for Selecting Alternative Contracting Methodsfor Roadway Projects: Project Delivery Methods, Procurement Proce dures, and Payment Provisions. Boulder: University of Colorado. Motulsky, H. (1995). Ch. 45 Statistical Traps to Avoid. In Intuitive Biostatistics: A Nonmathematical Guide to Statistical Thinking (pp. 424 - 441). London: Oxford University Press Inc. Natio nal Academies of Sciences, Engineering, and Medicine. (2019). Guidebook for Integrating Collaborative Partnering into Traditional Airport Practices. Washington, DC: The National Academies Press. National Research Council. (2012). Ch. 5 Model Validation an d Prediction. In Assessing the Reliability of Complex Models: Mathematical and Statistical Foundations of Verification, Validation, and Uncertainty Quantification (pp. 52 - 83). Washington, D.C.: The National Academies Press. Retrieved from https://doi.org /10.17226/13395 O'Leary, Z. (2018). Steps in Quantitative Analysis. Retrieved June 8, 2019, from Sage Publishing: https://study.sagepub.com/oleary3e/student - resources/analysing - data/steps - in - quantitative - analysis Project Management Institute (PMI). (2007). Construction Extension to the PMBOK® Guide. Newton Square, PA: Project Management Institute, Inc. Project Manage ment Institute. (2009). Practice Standard for Project Risk Management. Newtown Square: Project Management Institute, Inc. 135 Rahman, M. M., & Kumaraswamy, M. M. (2004). Potential for Implementing Relational Contracting and Joint Risk Management. Journal of Management in Engineering, 20(4). Retrieved 10 29, 2018, from https://ascelibrary.org/doi/pdf/10.1061/%28ASCE%290742 - 597X%282004%2920%3A4%28178%29 Rahman, S. A., Endut, I. R., Faisol, N., & Paydar, S. (2014). The Importance of Collaboration in Constructio Management and Technology Research. 125, pp. 414 - 421. Malaysia: Procedia - Social and Behavioral Sciences. Retrieved from https://ac.els - cdn.com/S1877042814028778/1 - s2.0 - S18 77042814028778 - main.pdf?_tid=828abb04 - 270e - 46a2 - 8b9b - 90b448f22b8a&acdnat=1543778862_5901db1fa77b6d82ce261dbe7d4cf441 Smith, N. J., Merna, T., & Jobling, P. (2014). Managing Risk in Construction Projects (3rd ed.). Hoboken, NJ: John Wiley & Sons, Ltd. Retr ieved 11 17, 2018, from https://media.wiley.com/product_data/excerpt/34/11183472/1118347234 - 4.pdf Sparkling, A. (2014). A Research Synthesis Of Key Partnering Drivers And Performance Outcomes In Architecture, Engineering, And Construction Research. Michig an State University, Construction Management. East Lansing: Michigan State University. Sullivan, J., Asmar, M. E., Chalhoub, J., & Obeid, H. (2017). Two Decades of Performance Comparisons for Design - Build, Construction Manager at Risk, and Design - Bid - Buil d: Quantitative Analysis of the State of Knowledge on Project Cost, Schedule, and Quality. J. Constr. Eng. Manage., 143(6). Sweet, J., & Schneier, M. (2009). Combining Designing and Building. In Legal Aspects of Architecture, Engineering and Construction Process (pp. 365 - 369). Stamford, CT: Cengage Learning. Syal, M. (2017). Introduction and Overview. In M. Syal, Construction Project Management (p. 5). Tah, J. H., & Carr, V. (2000). A proposal for construction project risk assessment using fuzzy logic. C onstruction Management and Economics, 18(4), 491 - 500. doi:10.1080/01446190050024905 Tchankova, L. (2002). Risk identification basic stage in risk management. Environmental Management and Health, 13(3), 290 - 297. Retrieved October 22, 2018, from https://w ww.emeraldinsight.com/doi/pdfplus/10.1108/09566160210431088 Tchankova, L. (2002). Risk identification basic stage in risk management. Environmental Management and Health, 13(3), 290 - 297. Tchankova, L. (2002). Risk identification - basic stage in risk m angement. Environmental Management and Health, 13(3), 290 - 297. Retrieved November 25, 2018, from https://www.emeraldinsight.com/doi/pdfplus/10.1108/09566160210431088 Vidal, L. - A., & Marle, F. (2008). Understanding project complexity: implications on project management. Kybernetes, 37(8), 1094 - 1110. Retrieved 11 11, 2018, from https://www.emeraldinsight.com/doi/pdfplus/10.1108/03684920810884928 136 Wang, L. (2008). Enhancing construction quality inspection and management using RFID techno logy. Automation in Construction, 17(4), 467 - 479. Wang, S., Dulaimi, M. F., & Aguria, M. Y. (2004). Risk management framework for construction projects in developing countries. Construction Management and Economics, 22(3), 237 - 352. Retrieved 10 29, 2018 , from https://www.tandfonline.com/doi/pdf/10.1080/0144619032000124689?needAccess=true Weidong, X., & Lee, G. (2005). Complexity of Information Systems Development Projects: Conceptualization and Measurement Development. Journal of Management Information Systems, 22(1), 45 - 83. Retrieved 11 11, 2018, from https://www.jstor.org/stable/pdf/40398758.pdf?refreqid=excelsior%3A8f1aa6afc63608432bf1d4563e2 292c4 Whitty, S. J., & Maylor, H. (2009). And then came Complex Project Management (revised). International Jo urnal of Project Management, 27(3), 304 - 310. Retrieved 11 11, 2018, from https://ac.els - cdn.com/S0263786308000410/1 - s2.0 - S0263786308000410 - main.pdf?_tid=52e8e450 - 531b - 4002 - a770 - 10624b2fc401&acdnat=1541954256_b750c8e648af43cafe7bc092d704a309 Yates, J. K. ( 1995). Use of Design/Build in E/C Industry. Journal of Management in Engineering, 33 - 38. Retrieved November 30, 2018, from https://ascelibrary.org/doi/pdf/10.1061/%28ASCE%290742 - 597X%281995%2911%3A6%2833%29 . (2010). Risk assessment of construction projects. Journal of Civil Engineering and Management, 16(1), 33 - 46. Retrieved 10 28, 2018, from https://www.tandfonline.com/doi/pdf/10.3846/jcem.2010.03?needAccess=true Zhang, S., Zhang, S., Gao, Y., & Ding, X. ( 2016). Contractual Governance: Effects of Risk Allocation on Management, 165(8), 854 - 863. Retrieved 10 28, 2018, from https://ascelibrary.org/doi/pdf/10.106 1/%28ASCE%29CO.1943 - 7862.0000175 Zhang, X., & Liu, Y. (2007). The Impact of ICT on Supplier - Buyer Relationship in Different Types of Supply Chain. International Conference on Wireless Communications, Networking and Mobile Computing, (pp. 4694 - 4697). Shang hai. Zou, Y., Kiviniemi, A., & Jones, S. (2017). A review of risk management through BIM and BIM - related technologies. Safety Science, 97, 88 - 98. Retrieved 11 18, 2018, from https://www.sciencedirect.com/science/article/pii/S0925753516000072?via%3Dihub Z ou, Y., Kiviniemi, A., & Jones, S. W. (2015). BIM - based Risk Management: Challenges and Opportunities. Proc. of the 32nd CIB W78 Conference 2015, (pp. 847 - 855). Eindhoven. Retrieved 11 19, 2018, from https://livrepository.liverpool.ac.uk/3001636/