MASS TIMBER CURRICULUM DEVELOPMENT FOR ENGINEERING STUDENTS USING DACUM AND DELPHI METHODOLOGIES By Christiana Kiesling A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Civil Engineering – Master of Science 2024 ABSTRACT Mass timber is an emerging construction technology growing in popularity in the United States. One obstacle in the gradual adoption of mass timber construction is the limited availability of qualified engineers and designers. While successful efforts have been made to address research topics related to mass timber design and to identify common barriers and challenges to the adoption of mass timber as a construction material, little research has been conducted to identify desired student outcomes for undergraduate civil or structural engineering students working with mass timber after graduation. This thesis describes the development of an industry-guided educational resource for curriculum development in order to improve alignment between undergraduate student outcomes and employer requirements related to mass timber design. A “Developing A Curriculum” (DACUM) chart was produced by a panel of qualified industry professionals to identify the tasks, duties, general knowledge, skills, and worker characteristics required of successful entry-level structural engineers working with mass timber. A second panel of qualified subject matter experts was assembled for implementation of the Delphi Survey Methodology, in which experts were instructed to draw upon their opinions and perspectives honed through years of industry experience to rank their level of agreement with each item and propose changes to the chart. Findings reflect a prioritization of competencies related to design of mass timber elements and structures, understanding of material characteristics, navigation of available design resources, contributions to project deliverables, and support of sustainability goals. The finalized DACUM chart provides clarity on curriculum needs to ensure graduating engineers are well-prepared to work with mass timber elements as the industry continues to grow in popularity. The chart can be used as a resource to guide the development of mass timber engineering educational materials, including learning modules, virtual tours, and student assignments to be presented independently or implemented with existing engineering coursework. ACKNOWLEDGEMENTS First and foremost, I’d like to thank Dr. Kristen Cetin for her constant support and guidance over the past two years. She has gone above and beyond her responsibilities as an academic advisor and has given me countless opportunities to grow and experience new things. From research trips to Alaska to an internship that has paved the way for my future career, she has unquestionably changed my life in a positive and permanent way. Her ability to balance serving as an expert in her field, supporting her family, and advocating for the individual needs and interests of her students is inspiring. I would also like to thank Dr. George Berghorn for his constant support and willingness to help me reach my academic goals. I would not have developed a passion for mass timber without his guidance and I am grateful that he introduced me to this project. I’d like to thank both him and MassTimber@MSU for their successful efforts to cultivate student interest in this area and continued advocacy for the growth of the mass timber industry in Michigan. I am extremely thankful to Dr. Berghorn and Dr. Peter Savolainen for serving as a part of my thesis committee. I’d also like to thank my peers within Dr. Cetin’s research group for their collaboration and valuable contributions to the projects I’ve been a part of. In particular, I’d like to thank Olivia Pauls and Cade Person for their assistance in the work related to this thesis. Thank you to each of the DACUM panelists and Delphi experts for their participation in this study and willingness to improve the educational futures of undergraduate students working with mass timber. This research is supported by the National Science Foundation (NSF) under award numbers 2220236 and 2203123. The opinions, findings, conclusions, and views of the author expressed in this thesis do not necessarily represent those of the NSF. I’d also like to thank the Michigan State University College of Engineering for their financial support. I moved to East Lansing with the primary goal of gaining an enriching educational experience, but emerged with an unexpected gift - lifelong friendships. I am profoundly grateful for the camaraderie and support that I’ve received from each of my Michigan friends. They made my graduate school experience truly memorable and have changed my life for the better. Lastly, I’d like to thank my family for their unwavering support throughout my (somewhat lengthy) academic journey. I would not have found what I’m passionate about without their constant encouragement and support. iii TABLE OF CONTENTS INTRODUCTION AND BACKGROUND ................................................................................... 5 Significance ................................................................................................................................. 6 Need for MT Educational Materials ............................................................................................ 8 DACUM Background ............................................................................................................... 12 Delphi Background ................................................................................................................... 13 METHODS ................................................................................................................................... 15 Overview ................................................................................................................................... 15 DACUM Panel .......................................................................................................................... 16 DACUM Meeting ...................................................................................................................... 17 DACUM Panelist Review ......................................................................................................... 18 Delphi Panel .............................................................................................................................. 19 Delphi Survey Method .............................................................................................................. 21 RESULTS AND DISCUSSION ................................................................................................... 25 A Priori Results: Preliminary DACUM Chart .......................................................................... 25 Results by Delphi Round........................................................................................................... 27 Final Results: Final DACUM Chart .......................................................................................... 31 Limitations of DACUM and Delphi Panels .............................................................................. 42 CONCLUSION ............................................................................................................................. 45 REFERENCES ............................................................................................................................. 47 APPENDIX ................................................................................................................................... 50 iv INTRODUCTION AND BACKGROUND Mass timber, a class of engineered wood products including cross-laminated timber (CLT), glued-laminated timber (glulam), mass plywood panels (MPP), and laminated veneer lumber (LVL), has gained substantial traction in the construction industry within the United States. This surge in popularity can be attributed to several factors, including the environmental benefits, design versatility, and efficient assembly offered by mass timber (MT) construction. MT sequesters carbon and may produce lower greenhouse gas (GHG) emissions during production than conventional building materials like steel and concrete (Riddle, 2023; Oliver et al., 2014). Wood and engineered wood products also have an excellent strength-to-weight ratio and possess strong insulation and acoustical properties (Asdrubali et al., 2017). Additionally, MT is significantly lighter than concrete by volume and can be prefabricated off-site, facilitating faster construction times and reduced labor costs that make MT a popular option for developers seeking efficient and cost-effective building solutions (Hassan et al., 2022; Smith et al., 2016). The continued development and improvement of MT products has allowed for applications beyond traditional light-frame construction buildings (Kuzmanovska et al., 2018). The 2021 International Building Code (IBC) included provisions for tall wood structures and has been adopted by numerous states and jurisdictions in the United States, increasing the feasibility and appeal of using this construction approach for large scale domestic projects (Woodworks, 2024; Riddle, 2023). Studies analyzing the performance of MT elements under seismic loading and fire conditions have led to the development of members and connections resistant to earthquakes and fire, addressing issues and misconceptions that once served as barriers to the adoption of MT construction materials (Izzi et al., 2018; Timmers & Jacobs, 2018; Ceccotti et al., 2013; Shahnewaz et al., 2017; Muszyński et al., 2019; Frangi et al., 2018). Addressing these challenges has allowed the industry to meet the rapidly increasing interest in MT construction in the United States, which is reflected by the substantial increase in the number of completed MT projects across the nation as shown in Figure 1 (Woodworks, 2023). The United States’ established dependence on the production and consumption of wood products (particularly in the construction industry) coupled with the environmental sustainability, design versatility, and construction efficiency offered by MT construction continue to propel this material to the forefront of the construction industry in the United States (Alderman, 2022; Brandeis et al., 2021). 5 Figure 1: Mass Timber Projects Constructed in the United States as of December 2023 (Woodworks, 2023) Significance The rapid emergence and continued evolution of MT construction in the United States underscores the importance of ensuring that university curriculum adequately prepares students for professional success working with new technologies. Recent advancements must be integrated into existing curricula or the program risks becoming outdated and irrelevant to current industry needs. Including emerging technologies and practices in undergraduate programming equips students with the skills needed to adapt to changing professional landscapes. It is crucial that university curriculum aligns with current industry needs and trends not only to meet the demands of the modern workforce, but also to ensure the professional success of graduating students. A well-structured college program allows students to explore a diverse range of topics that enables them to develop a holistic understanding of their chosen area of study. Additionally, a well-crafted curriculum creates opportunities for interdisciplinary work and the development of practical skills beyond topics included on course syllabi. Effective programs provide students with both the essential knowledge and technical skills relevant to their field and critical thinking abilities, problem-solving capabilities, and interpersonal competencies necessary for success in a dynamic work environment. Well-structured college programs are particularly important in the areas of civil and structural engineering, where the need to gain a deep understanding of a wide 6 breadth of knowledge challenges the ability of students to adequately develop the skills necessary for professional success after graduation. The extent to which engineering graduates are adequately prepared for entry-level positions after graduation from their university program is a topic that requires frequent and ongoing investigation (Balogh and Criswell, 2013). Concerns regarding student’s academic preparation for the workforce also exist among instructors and faculty members. A recent poll of 176 structural engineering professors at ABET- accredited universities found that only 12% of respondents answered “most definitely; they have the tools and skills, and are ready to perform structural engineering” when asked if their undergraduate students were adequately prepared to enter the workforce after graduation (Francis, 2019). This result emphasizes two things: 1: if students are to be well-prepared for the modern workforce, then current needs and relevant modern technologies (like MT) must be addressed, and 2: included coursework must be carefully and intentionally cultivated to produce well-equipped graduates without adding to existing course load. Particular care must then go into developing MT curriculum, the inclusion of which has already been validated because of its unquestionable growth in popularity in the construction industry. Because of the challenges associated with adding a new course to undergraduate engineering programs, including high student course load, fulfillment of ABET accreditation, institution-specific requirements, lack of funding, and availability of qualified faculty, it is perhaps a more feasible option to integrate elements of MT education into existing engineering coursework. This also presents an opportunity to take an interdisciplinary approach to MT design. However, as mentioned above, coursework must be carefully and intentionally designed to mitigate existing challenges associated with the completion of a civil or structural engineering degree. If curriculum is going to be integrated by individual institutions and instructors, then resources must be available to assist in the development of these materials. MT’s status as an emerging construction technology means that availability of comprehensive educational materials is limited compared to those for more established engineering courses such as steel or concrete design. This scarcity of existing resources to assist in teaching MT design at the undergraduate level serves as a further barrier to developing and implementing high-quality curriculum. To address this challenge, a systematic approach must be taken to identify the essential knowledge and skills required for MT engineering and develop educational resources that effectively convey these concepts to undergraduate students. 7 Bridging the gap between industry needs and student outcomes ensures that students graduate from their university program having learned concepts and materials relevant to current industry needs and are well-prepared to succeed in their first career. This can be achieved through collaboration with practicing engineers and other industry professionals with adequate MT design experience. The implementation of the Developing a Curriculum (DACUM) process allows for the creation of a framework for MT engineering education using the diverse opinions and perspectives of industry professionals. The validation of the DACUM chart through the application of Delphi methodology serves to further enhance the credibility of the results and maximize relevance to industry needs. By soliciting input from a panel of MT subject matter experts with relevant industry experience, the validity of the identified competencies and learning outcomes can be corroborated to ensure alignment with industry needs and educational best practices. Need for MT Educational Materials MTs rise in popularity in the United States has not been accompanied by the development of related undergraduate level educational materials. While a lack of existing MT educational resources for collegiate structural engineering programs underlines the importance of developing such materials and highlights the relevance of this study, it also prevents the analysis of existing literature regarding this topic. As a result, several related topics and themes are discussed to highlight the need for MT educational materials and provide context for the current condition of the MT industry. The identification of barriers hindering widespread adoption of MT construction in the United States allows for the analysis of knowledge gaps to address these challenges. Various studies and workshops have been conducted to identify barriers limiting the growth rate of MT construction that may have previously prevented its acceptance as a conventional construction material. A survey distributed to the architecture community in 2015 identifying barriers to adoption of CLT identified several challenges, including high material and construction costs, code compatibility issues, and lack of manufacturing ability within the United States. Additionally, the results highlighted deficiencies in the knowledge distribution needed to address misconceptions regarding high maintenance costs and fire performance (Mallo & Espinoza, 2015). Similarly, a questionnaire distributed to respondents in the construction and engineering 8 communities identified several challenges limiting the growth of MT, including cost, poor coordination between project team members, design-related difficulties, and lack of experience in timber construction (Ahmed & Arocho, 2020). Issues related to high cost, manufacturing ability, and code barriers have largely been addressed since the initial survey in 2015. Remaining issues include the inexperience and resulting knowledge gaps of designers and constructors working with MT. A semi-structured questionnaire survey sent to 1200 construction companies, 300 architecture firms, and 55 MT manufacturers similarly found that construction practitioners in the United States have a low involvement level and work experience in timber building design and construction. More specifically, 45% of respondents reported ever being involved in a MT construction project while only 12% had more than five years of experience working on a MT building project (Ahmed & Arocho, 2021). Survey participants also stated that both owners and developers lack formal education regarding MT construction and identified lack of experience in timber construction as the most substantial challenge during the construction period. Similarly, a survey sent out to MT subject matter experts in industry and academia in Canada found that 96.4% of survey participants ranked “insufficient design experts in the market” as a potential barrier in the widespread use of MT (Syed, 2020). Additional studies have further indicated that the lack of inclusion of MT-related curriculum in AEC educational programs serve as a barrier to producing new graduates with the established knowledge necessary to work successfully on MT projects (Laguarda Mallo & Espinoza, 2014; Sheine et al., 2019). Identification of barriers to the widespread adoption of MT also allows for the prioritization of research needs to address these challenges. Several workshops and seminars have been conducted to compile the opinions of AEC professionals regarding current MT research needs, including ongoing research needs assessments through the USDA Forest Products Laboratory, which utilize the expertise of more than 100 professionals with MT experience through the effort-impact method of prioritization (Zelinka et al., 2019). Organized findings from these meetings result in extensive lists of current research needs necessary to further the growth of MT as a building material. Many of these research topics, including seismic resistance, fire studies, and updates to building code have been addressed since the commencement of these needs assessments in 2015. Despite industry misconceptions, most experts in the MT sector agree that research areas such as fire resistance, seismic performance, 9 and sound transmission have been explored extensively and are largely resolved (Lehmann & Kremer, 2023). Addressing these research needs was undeniably crucial in overcoming identified barriers to the adoption of MT construction. However, these research items are largely technical in nature and relate primarily to the material performance, construction costs and logistics, building code, manufacturing ability, and industry perceptions related to MT construction. Far less progress has been made regarding workforce preparation or educational programming, which has consistently remained an identified barrier to the adoption of MT in past decades. This gap demonstrates a need to further resources in this area to keep pace with the rapid progress in other areas of MT research. There is a relative lack of available MT engineering courses and curriculum at the undergraduate collegiate level in the United States (Person, 2024). Additionally, even traditional timber engineering courses are offered with less frequency than traditional design courses for materials like steel or concrete (Perkins, 2016; Francis, 2019). Several inventories of timber engineering coursework in the United States and Canada have been conducted in recent years (Gupta & Gopu, 2005; Perkins, 2016; Francis, 2019; Daneshvar et al., 2021; Person, 2024). A curriculum survey through the National Council of Structural Engineers Associations (NCSEA) in 2016 analyzed course offerings from 251 ABET-accredited engineering schools in the United States, including courses offered in timber design. Survey results revealed that 55% of the included universities offered courses teaching fundamentals of timber engineering to undergraduate or graduate students (Perkins, 2016). The repeated survey implemented by NCSEA in 2019 found that 52% of 258 surveyed universities offered a wood design course to undergraduates (Francis, 2019). This relative lack of course availability, particularly when compared to offerings related to materials like concrete or steel, was investigated as part of the survey. Primary reasons cited included lack of student demand, lack of school support, lack of timber design professors, and imposed unit restrictions (Francis, 2019). This implied lack of importance placed on timber coursework does not bode well for inclusion of MT curriculum in university programs. However, the continued growth of MT construction is undeniable and will only strengthen the need for related design curriculum in future years. As a result, it is necessary to consider options for implementing MT design 10 curriculum in existing coursework, even at universities where timber engineering coursework is not offered. Approaching timber and MT design in the context of composite structures allows for integration of timber-related design concepts into existing courses focused on other construction materials. In 2019, two timber learning modules were successfully integrated into an existing structural steel design course at an accredited university in Canada. A survey distributed to the students revealed that the learning modules were successful in providing students with some general knowledge of timber and that 76% of students indicated they had some level of confidence in contributing to the design of a timber building after completing the modules. Additionally, 52% of participating students responded to an open-ended survey question stating an interest in further participation in a timber course or module (Chorlton et al., 2019). Efforts have also been made to incorporate MT design into existing coursework at the university level using an interdisciplinary approach. The implementation of an interdisciplinary integrated design studio at the University of Oregon allowed structural engineering and architecture students to work in tandem to design timber structures, combining traditional lecture, lab, and studio formats to provide students with necessary information and encourage collaboration throughout the design process (Sheine et al., 2019). The project was successfully repeated and further developed in following years to incorporate additional MT structures and design elements. The intentional design of this program allowed the coursework to extend beyond the basics of MT design and encouraged students to develop their communication and problem-solving skills through collaboration with members of their design team. The ability to integrate MT educational modules or units into existing coursework makes this opportunity more accessible to multiple universities and further emphasizes the need for available educational materials. Several professional organizations and resources exist to further the MT design knowledge of current industry professionals through on-the-job training (OJT), certifications, and seminars. Far fewer of these resources exist for undergraduate students. Some resources, such as templates for the creation of syllabi, have been developed to guide instructors unfamiliar with timber engineering with creating an undergraduate course (Lawson et al., 2020). Additionally, NCSEA has published a proposed course outline for a timber education module created to fulfill their list of core timber knowledge requirements (Dong, 2015). The Wood Education Institute (WEI) was created in 2008 to assist in offering wood education for 11 undergraduate, graduate, and continuing education programs, but has since been dissolved due to lack of funding (Okoye et al., 2017). However, recent interviews with 11 structural engineering instructors teaching timber-related courses reflected a belief that there is a significant gap in educational resources related to timber design, including lecture materials, notes, practice problems, assessment materials, and student design projects (Person, 2024). Increasing public awareness of existing materials while also developing additional educational resources that can be customized to an instructor’s specific needs will increase the feasibility of including timber and MT related material at the collegiate level. This paper aims to identify desired competencies and characteristics of a structural engineer-in-training (EIT) working with MT to serve as a resource for the alignment of student outcomes at the undergraduate level. Background information regarding the use of the DACUM and Delphi methodologies will be presented. The selection method for DACUM and Delphi panelists will be described, and the implementation of each study presented. Results from each process will be described and compared to analyze items considered most crucial for inclusion in MT design curriculum. DACUM Background The DACUM process, widely used in vocational education and workforce development, offers a systematic approach to identify the knowledge, skills, and tasks required for effective performance in a specific occupation (CETE, 2024). A DACUM chart can then be developed to outline the foundational competencies and learning outcomes necessary for MT engineering education. The resulting chart serves as an educational resource to guide the development of curriculum that incorporates the insights of subject matter experts. This method places importance on collaboration between educators and industry professionals to bridge the gap between academic student outcomes and desired worker competencies. The completed DACUM chart ensures that educational content is in close alignment with industry needs and serves as a blueprint for the development of academic and training programs. A specific job title, along with an implied level of expertise, must be selected for the creation of a DACUM chart. After identifying this role, a group of industry professionals with experience regarding this job title are invited to serve as DACUM panelists. A facilitator conducts a meeting with these panelists to systematically determine what tasks and competencies are typically required for the selected profession. Panelists may also intentionally select verbs 12 from Bloom’s Taxonomy for each task with the intention of assigning a minimum cognitive level to each item, a process also used in the development of the Civil Engineering Body of Knowledge (CEBOK) (ASCE, 2004). The panel identifies the desired skills, traits, characteristics, and knowledge that an individual should possess to be successful in the selected position. The subject matter experts are expected to share their opinions based on their own experience and engage in collaborative discussion, allowing for the iterative development of tasks and competencies incorporating the perspectives of all panelists. The resulting tasks and competencies are typically documented and shared on a large screen during the meeting. These results are later organized and reformatted into the finalized DACUM chart. The job title of “Mass Timber Engineer-in-Training (EIT)” was selected. It was determined that the resulting DACUM chart would be created to guide the development of undergraduate curriculum and that the identified tasks and competencies should be relevant to a hypothetical new employee who had recently graduated from their undergraduate program and was within one year of working in the selected position. The final chart is intended to be used in the creation, design, or analysis of curriculum to ensure that recent graduates are well-equipped to address the demands of the modern market. Delphi Background The Delphi method, a structured technique used to achieve consensus among a panel of experts on a particular topic, allows for iterative rounds of feedback and refinement of ideas. Originally developed in the 1950s, the method is named after the Oracle of Delphi in ancient Greece to symbolize the forecasting ability of a group of knowledgeable individuals sharing their collective wisdom (Keeney et al., 2011). The Delphi method has a variety of applications and can be used to facilitate group problem solving, aid in decision making, forecast future scenarios, assist in program planning, augment areas of incomplete knowledge, or investigate and predict what does not yet exist (Skulmoski et al., 2007; Balogh et al., 2013). The Delphi method has been successfully used for a variety of topics and remains a common research method for doctoral dissertations (Skulmoski et al., 2007). It has been used in several engineering areas, including competencies for graduate structural engineering students, service systems engineering, and engineering pedagogy (Balogh et al., 2013; Sorby et al., 2005; Rüütmann, 2022). The Delphi method’s potential use as a forecasting technique and reliance upon the 13 unified opinion of an expert panel lends its credibility for application to an emerging construction technology like MT. The method involves the use of multi-round anonymous surveys intended to generate informed opinions without the bias that may arise from face-to-face interactions. A selected panel completes an initial survey within their area of expertise, the results of which are compiled by a facilitator. The facilitator removes any items that have reached consensus and redistributes a new survey with only the items that have not yet reached agreement. The facilitator also provides a summary of responses to the panelists to encourage them to revise their opinions in subsequent rounds, particularly regarding items that may be a source of disagreement in the group. The results of the subsequent survey are compiled, and the process is continued until consensus is reached on all items. In the context of a Delphi survey, consensus refers to the measure used to ascertain whether the panel of qualified experts has reached a certain level of agreement on a specific item. The consensus method is often selected based on the size or characteristics of the selected panel. Commonly used consensus methods include the statistical measures of mode, median, mean, interquartile range, coefficient of relative variation, and average percent of majority opinion (APMO). The selected consensus method for this study was median (measure of central tendency). It was determined that items would reach consensus when at least 70% of the answer values fell within a half bin of the median answer of the group. It is worth noting that consensus can be reached in favor of or against a task, meaning panelists have the ability to either include or eliminate items during the Delphi process. For this study, the Delphi methodology was applied to the preliminary DACUM chart because of MT’s emergence as a recent technology. Additionally, the opinions of the qualified expert panelists completing the Delphi process serve to validate and supplement the perspectives of the original DACUM subject matter experts. The final version of the chart after the Delphi process is thus subjected to more scrutiny than the traditional DACUM chart and is therefore better suited to guide the development of curriculum to prepare new graduates for success in the MT industry. 14 METHODS Overview Review of existing literature and recent coursework inventories revealed a lack of available timber design courses in the United States, highlighting the extensive gaps in MT education and emphasizing the need of materials to assist in curriculum development. A job analysis was performed to identify and characterize key occupational roles in the field of MT engineering and requirements for successful employees. This was done by conducting a DACUM workshop of selected subject matter experts to identify the tasks and duties required of a new engineer working in the MT industry. This engagement with industry professionals yielded a DACUM chart aimed to assist in the success of university students by outlining the requirements of the industry. The results of this meeting also served as a bridge between the broad, overarching concepts identified in needs analysis and the specific tasks a MT EIT may be required to perform. Task verification was conducted through an initial review of the DACUM chart to ensure the accuracy and relevance of tasks and duties determined during the panel meeting. This validation process involved redistributing the compiled results of the DACUM meeting to the original six panelists one week after the meeting concluded. Panelists were asked to indicate their general level of agreement with each task and duty by indicating “agree”, “needs improvement”, or “disagree” to verify the importance of each item for a MT EIT. Panelists were also asked to provide feedback regarding any items that they felt were missing or should be excluded from the finalized chart. The passage of time since the convening of the panel paired with the change in format of the review process (face-to-face verbal interaction in a group setting versus individual written review) strengthened the validatory power of the task verification. It was determined that all reviewed tasks included on the preliminary DACUM chart would be subjected to the Delphi survey method. The scrutiny placed on each task by a new group of panelists would serve as the task selection to further validate the results of the chart and identify high-priority tasks to be included in training and curriculum development. Additionally, items which reached consensus on “disagree” or “strongly disagree” during the Delphi process were eliminated from the DACUM chart, meaning all tasks and duties remaining after the task selection were curated by the panelists and deemed particularly significant to the role of a MT 15 EIT. The process used for the creation of the DACUM chart and validation through the Delphi Method is shown in Figure 2. DACUM Panel Figure 2: Overview of DACUM and Delphi Progression DACUM panelists were intentionally selected because of their relevant work experience, contributions to multiple MT projects, and proven expertise in the area of MT engineering. Potential participants were drawn from the list of engineering professionals attending the International Mass Timber Conference in 2023, which concluded the day prior to the panel’s convening. This conference was the primary professional development event for MT professionals and it was assumed that the majority of individuals qualified to serve on a MT DACUM panel would also be attending the event. Seven individuals were contacted regarding their participation in the DACUM panel and six were able to attend the meeting. Four of the selected panelists were licensed engineers working at structural engineering or AEC firms who had recently worked on MT projects. The remaining panelists worked for a CLT manufacturer and construction technology firm. The panelists included three men and three women. The six selected panelists’ job titles, state of residence, and years of professional experience at the time of the panel are shown in Table 1. 16 Table 1: DACUM Panelist Information Job Title Licensure Location Professional Experience Panelist #1 Partner at structural engineering firm P.E., Dipl. Ing. British Columbia, CA 30+ years Panelist #2 Associate at engineering firm P.E., S.E. Oregon, USA 16 years Panelist #3 Vice President of Products and Technology at construction technology firm Panelist #4 Vice President - Structures at structural engineering firm Panelist #5 Panelist #6 Senior Director of Mass Timber at mass timber manufacturer Principal and Co-Founder at AEC consulting firm - Washington, USA 19 years P.E., S.E. New York, USA 18 years - Wisconsin, USA 8 years S.E. Illinois, USA 10 years DACUM Meeting The selected panelists met in person to collaborate in the creation of a DACUM chart outlining the desired MT-related competencies of a hypothetical new EIT immediately after their completion of an undergraduate engineering degree at an accredited university. The panelists were instructed to identify the duties, tasks, traits, and skills required by the hypothetical EIT’s employer to successfully complete a MT project. These findings would be documented in the DACUM chart to serve as a resource in determining desired student outcomes at the university undergraduate level. A prepared list of possible skills, work areas, concepts, and objectives required of a new engineer was displayed at the start of the meeting to help panelists eliminate any items not specific to MT projects and to brainstorm items that were not included on the prepared list. Panelists exchanged ideas and worked together to identify items the group considered to be most important. Panelists were then instructed to add verbs from Bloom’s Taxonomy to each item to create a task to be fulfilled by the hypothetical engineer. These tasks were grouped together to identify an overarching concept that encompassed each set of items. Tasks were combined, elaborated upon, and clarified as needed. Concept groupings were re-worded to become “duties” under which each series of tasks fell. A simplified example of this process is shown in Figure 3. After the preliminary tasks and duties were completed, panelists were asked to rate each task as Level 0 (basic familiarity), Level 1 (identify/describe), Level 2 (assess/analyze), or Level 3 (create) depending on the anticipated level of knowledge for the hypothetical EIT. Panelists also viewed a list of general skills and worker characteristics to identify traits that were most relevant 17 to a MT engineer. Panelists were told that any verb additions, concept groupings, or skills identification that were not completed during the allotted meeting time would be included for their review in the meeting results distributed the following week. Figure 3: Example Creation Process for Tasks and Duties DACUM Panelist Review Results of the panel meeting were compiled to match the traditional DACUM format, then redistributed to the original panelists for a secondary analysis one week after the panel convened. Panelists were instructed to review the preliminary DACUM chart and designate their level of agreement (“agree”, “needs improvement", or “disagree”) with each task. They were also asked to re-identify crucial worker characteristics and leave general comments regarding the results of the chart. Four of the six panelists completed the review process and returned comments on the chart. Coincidentally, Panelist #3 and Panelist #5 did not complete the review process, meaning the four reviewed charts came from the engineers participating in the panel. Several tasks were eliminated from the DACUM chart in light of the panelist’s review comments by considering the average of the four respondent’s answers and removing any tasks with an average level of agreement falling below “needs improvement”. Comments from each panelist were also used to re-word 10 tasks, primarily to alter the Bloom’s Taxonomy verb or to clarify or elaborate upon the meaning or scope of the task. The DACUM chart was updated to reflect these changes and prepared for distribution to a new panel of qualified experts. 18 Delphi Panel Because the effectiveness of the Delphi process is dependent upon the opinion of a small group of qualified experts, it was particularly important to select panelists whose experience and expertise in the field of MT engineering yielded them qualified to review the results of the DACUM panel. A qualifications survey was developed in a digital format and distributed to known contacts working in the MT industry. Several requirements related to work experience, licensure, and educational background were assessed through the survey with the purpose of identifying individuals with enough experience and industry knowledge to serve as Delphi subject matter experts. Table 2 displays the questions and results of the qualifications survey. Respondents were able to leave additional comments at the end of the survey clarifying or elaborating upon their survey answers. Fourteen respondents completed the qualifications survey and nine were selected for participation in the Delphi Process. Table 2: Results of Delphi Qualifications Survey 4 Respondent Number: 2 1 3 5 6 7 8 9 10 11 12 13 14 Did you receive an engineering degree from an accredited university? Have you completed at least one university-level structural design course (i.e. concrete, steel, timber, masonry, etc.)? Do you have a Professional Engineer (PE) license? How many years have you worked as a licensed structural engineer? Have you signed/sealed plans for a structure utilizing mass timber? How many mass timber-focused projects have you worked on? Point total: Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes 21 23 36 20 0 45 15 0 0 25 10 0 15 26 No No Yes Yes No No Yes No Yes Yes No No No No 25+ 15 50 8 8 8 5 8 2 4 0 100 0 50 80 8 8 4 4 8 4 7 10 15 4 8 4 8 Experience requirement met? Yes Yes Yes Yes Yes No Yes No Yes Yes Yes Yes Yes Yes The first four questions in the qualifications survey contributed to the accumulation of points based on respondent’s answers. Respondents were given one point for completing at least one engineering design course at the university level and three points if they received an engineering degree from an accredited university, both of which were true for all respondents. The third and fourth questions assessed the length of time the respondent had worked in the structural engineering industry and if they held a professional license. It should be noted that the 19 fourth question asked how long the respondent had worked as a licensed structural engineer when it could have been more accurately worded as “How many years have you worked as a licensed engineer in structural design?”. This question required independent verification of the respondent’s answers after completion of the survey. Respondents were given four points if they had worked for 15 or more years as a licensed engineer, three points for ten to 14 years, two points for five to nine years, or one point for less than five years. The fifth and sixth questions of the survey referred to the respondent’s experience working with MT. The respondent was required to have signed or sealed plans for a structure utilizing MT or worked on at least one MT-focused project to be included in the group of qualified experts. Importance was placed on this requirement because of the focus on finding practicing engineers to provide an industry perspective on MT education as a part of the DACUM process. If the respondent did not meet the criteria outlined in either of these questions, they were automatically eliminated from the panel. The results of the survey showed that two participants (Respondents 6 and 8) did not meet the requirements regarding MT design experience. While Respondents 5, 9, and 12 had adequate design experience, they had point totals of four while the remaining respondents had point totals of at least seven. As a result, these five respondents were eliminated from the pool of qualified candidates. The remaining nine respondents were notified of their inclusion in the panel of qualified experts and invited to participate in the Delphi survey process. Respondent 14 did not respond to their invitation and was removed from the panel before the survey was distributed. All eight of the selected experts had an engineering license and had worked in the engineering industry for at least 10 years. Three experts hold doctoral degrees, and two experts have experience teaching timber related courses at the university level. The experts were asked to provide their job title as a part of the qualifications survey. The job title, credentials, and state of residence of each respondent at the time of survey completion are shown in Table 3. . 20 Table 3: Delphi Expert Information Respondent Number Expert Number Job Title Expert #1 Respondent #1 Principal Lecturer at a public university Credential Location Ph.D., P.E., S.E. Maine, USA Expert #2 Respondent #2 President of construction company P.E. Michigan, USA Expert #3 Respondent #3 President and CEO of engineering firm Ph.D., P.E. Colorado, USA Expert #4 Respondent #4 Vice President and COO of engineering firm, Adjunct Professor in structural engineering P.E. Arkansas, USA Expert #5 Respondent #7 Expert #6 Respondent #10 Principal and Director of Mass Timber at engineering firm Principal in fire protection engineering at engineering firm P.E. Oregon, USA P.Eng. Washington D.C., USA Expert #7 Respondent #11 Associate Principal at engineering firm P.E., S.E. California, USA Expert #8 Respondent #13 Research General Engineer at national laboratory Ph.D., P.E. Wisconsin, USA The Delphi process relies upon the use of an expert panel in which participants are carefully selected because their work experience and qualifications separate them from individuals conducting similar work. Because of the focus on qualified experts, there is no need to select a large number of individuals to ensure that panelists are representative of groups with varying degrees of knowledge and experience working with MT. Rather, panelists are intentionally selected to serve as a representative sample of the relatively small pool of experts, validating the use of a small data set. Delphi Survey Method Round 1 Survey The Delphi procedure was used to validate the tasks, general knowledge and skills, and worker characteristics obtained from the DACUM panel. The results of the DACUM chart reviewed by the panelists in Portland were reformatted to a digital survey format in which respondents were asked to rate each task using the Likert scale (strongly disagree, disagree, neutral, agree, and strongly agree) (see Figure 4). Because the Delphi process includes opinions regarding word choice and clarity, all tasks were copied verbatim from the preliminary DACUM chart to the Round 1 survey and tasks remained grouped by duty. 21 Figure 4: Tasks in DACUM format (left) and Delphi Survey Format (right) The “General Knowledge and Skills” section of the survey required respondents to indicate whether they considered each skill or requirement to be particularly important for EITs working in MT design. Rather than using the Likert scale, respondents were asked to indicate whether they agreed or disagreed that a student should be able to demonstrate each skill after graduating from their university. Similarly, respondents were given a list of worker characteristics and asked whether they agreed or disagreed that each trait was particularly important for MT EITs. It was determined that the Software and Future Trends and Concerns sections of the DACUM chart would not be subjected to the Delphi process to optimize the length of the survey to promote expert participation and retention. The survey began with a brief description of the background and use of the DACUM chart and instructions for completing the survey. Experts were told that their primary role was to rate each task based on whether they believed a newly graduated engineer should be able to perform the task within their first year of work as a structural EIT working on MT projects. Experts were also reminded that they were selected to complete the survey because of their experience working with MT and encouraged to draw upon their professional experiences and knowledge during the completion of the survey. 22 The Round 1 survey was distributed in September 2023. Experts were asked to complete the survey within two weeks of the distribution date. The survey did not have a time limit and allowed experts to complete the questions in multiple sittings. Round 2 Survey Any tasks, worker characteristics, and general knowledge and skills items that reached consensus in the Round 1 survey were removed from the Round 2 survey. Items that had reached consensus with a median value of 1 (strongly disagree) or 2 (disagree) were removed from both the Round 2 survey and the preliminary DACUM chart. Additionally, one task and one worker characteristic were re-worded based on comments from the Round 1 survey. A report of Round 1 survey results was created showing the median value and consensus percentage for each remaining task, skill, and worker characteristic. The report also included relevant comments from experts to allow respondents to anonymously share their opinions regarding certain items. The experts were asked to read through the report (particularly the comments) and encouraged to use the report while completing the Round 2 survey. The inclusion of comments in the report was intended to assist experts in considering different perspectives and answers contrary to their own opinion. An email containing the Round 1 survey results report, instructions, and a link for the Round 2 survey (identical in format to the Round 1 survey) was sent to each expert at the end of October 2023. All eight experts completed the Round 2 survey in less than two weeks. Round 3 Survey Any tasks, worker characteristics, and general knowledge and skills items that reached consensus in the Round 2 survey were removed from the Round 3 survey. Again, items that had reached a consensus on “disagree” or “strongly disagree” were also removed from the preliminary DACUM chart. Unlike the summary report for the results of the Round 1 survey, the summary report for the Round 2 survey was customized specifically for each expert. In addition to showing the group’s median answer and consensus percentage for each task, the Round 2 summary report also included a graphic displaying the respondent’s answer for each task compared to answers from the rest of the group, as shown in Figure 5. The report included relevant comments for each duty from the Round 1 and Round 2 surveys. The experts were asked to read through the report and consider how their answers compared to those of the rest of the group. This comparison was meant to encourage respondents who provided answers far from the median value to reconsider their opinion during the Round 3 survey. The final survey, in the 23 same format as the first two surveys, was distributed to experts in late November 2023 alongside restated instructions and the individualized report of results summarizing Round 2. All eight experts completed the Round 3 survey within three weeks of the initial distribution date. Figure 5: Excerpt from Round 1 Survey Summary Report (left) and Round 2 Survey Summary Report (right) 24 RESULTS AND DISCUSSION A Priori Results: Preliminary DACUM Chart 46 tasks were recorded at the conclusion of the DACUM meeting in Portland. 44 of these tasks were ranked and sorted into one of seven duties while two tasks remained unsorted and unranked at the conclusion of the meeting. The results returned from the DACUM panelists’ review in the weeks following the meeting in Portland eliminated three of the 46 tasks on the initial DACUM chart. Comments voiced by multiple panelists during the DACUM review process included concern that a task was not highly relevant to a structural engineering position or disagreement with the decided level of importance for each task (Level 0, 1, 2, or 3). One panelist also indicated that there were several tasks that a new engineer would not be expected to know how to complete upon graduation from their degree but should be able to learn to perform within their first year of work. Seven duty categories remained after the review of the chart by DACUM panelists. Of the seven duties, the largest were “Duty D: Design safe, serviceable, constructable, and durable mass timber structures” with 12 tasks, “Duty E: Contribute to the development of project deliverables” with nine tasks, and “Duty A: Demonstrate knowledge of engineering design” with six tasks. 27 of the 43 tasks (63.8%) were classified into one of these three duty categories, showing an importance on tasks related to the design of MT structures and resulting development of project deliverables. Because the knowledge level designations assigned to each task correlate with the depth at which the EIT would be expected to understand the task, it is implied that tasks with a high level of understanding are more valued by panelists than those requiring only basic familiarity. This also implies that it would be prudent to include tasks ranked as Level 3 in university curricula if student outcomes are to align well with industry expectations. The level designations assigned to each of the 43 tasks sorted into seven duties are shown in Table 4. Please note that “Duty X: Communicate with stakeholders” is named as such because of this duty’s later elimination during the Delphi survey process. 25 Table 4: Distribution of Task Level Designations by Duty Duty Total number of tasks Level 3 Level 2 Level 1 Level 0 “Create” “Assess/ Analyze” “Identify/ Describe” “Basic Familiarity” A: Demonstrate knowledge of engineering design B: Understand mass timber characteristics C: Navigate network of mass timber resources D: Design safe, serviceable, constructable, and durable mass timber structures X: Communicate with stakeholders E: Contribute to development of project deliverables F: Support project sustainability goals 6 4 3 12 4 9* 5 3 3 1 3 0 2 0 2 1 1 5 1 1 1 1 0 0 4 1 2 3 Totals: 43* 12 12 11 0 0 1 0 2 3 1 7 *One task from Duty E did not receive a level rating during the DACUM panel meeting or review process Nine of the 12 tasks designated as Level 3 were classified within Duties A, B, and D, highlighting a prioritization of tasks related to engineering design, material properties, and the design of MT structures. Inversely, Duties X and F contained no tasks prioritized as Level 3 and only two tasks designated as Level 2, reflecting that expertise in stakeholder communication or understanding of project sustainability goals may not be necessary immediately upon completion of an undergraduate degree. The distribution of knowledge level designations by duty are shown in Figure 6. Level 3 Level 2 Level 1 Level 0 s k s a T f o r e b m u N 12 10 8 6 4 2 0 Duty A Duty B Duty C Duty D Duty X Duty E Duty F Figure 6: Distribution of Task Level Designations by Duty Closer analysis of tasks with low level designations provides insight into industry expectations for new engineers. For example, tasks designated as Level 0 or 1 within Duties E 26 and F were largely industry-related skills dependent upon the preferences and protocols of the hypothetical EIT’s company, including knowledge of project delivery methods and approval processes, engineering and construction workflows, project sustainability goals, and limitations with manufacturing and installation. Panelist comments reflected that these types of tasks could be developed during early years of employment, implying that these topics need not be primary focuses of MT design curriculum at the undergraduate level despite their importance in the successful completion of MT construction projects. It follows that tasks within Duties A, B, and D are then better suited for integration into undergraduate design curriculum as students are expected to have a higher level of expertise in these areas after graduation. The results of the preliminary DACUM chart can be found in Table 14 and Table 15 in the Appendix. Results by Delphi Round The results of the Delphi method serve as a measure of validation of the preliminary DACUM chart. Because each survey round could not be concluded until all experts had the survey, results from each round of the Delphi process can be analyzed individually. A summary of the number of tasks reaching consensus by survey round are shown in Table 5. Please note that the term “Consensus - Included” refers to items included on the final DACUM chart which reached consensus with a median value of “neutral”, “agree”, or “strongly agree.” Tasks referred to as “Consensus - Excluded” were eliminated from the final DACUM chart when consensus was reached with a median value of “disagree” or “strongly disagree” for these items. Information regarding the methodology and rationale for the inclusion or elimination of tasks during “Post-Delphi Analysis” can be found in section “Round 3 Survey Results”. Table 5: Summary of Consensus Tasks by Survey Round Number of tasks in survey Tasks reaching consensus Consensus - Included Consensus - Excluded Round 1 Round 2 Round 3 Post-Delphi Analysis 43 30 19 14 12 5 5 10 1 6 0 4 13 11 5 - 27 Round 1 Survey Results The Round 1 survey distributed as part of the Delphi process included 43 tasks grouped into seven duties. Additionally, 32 worker characteristics and 11 general knowledge and skills items were included in the survey. Results showed that 13 of the 43 tasks, ten of the 11 general knowledge and skills items, and 27 of the 32 worker characteristics reached consensus during the first Delphi round. Most of these tasks reached consensus in favor of including the item on the DACUM chart. One task and one general knowledge and skills item were removed from the chart as the consensus reached was in favor of excluding these items. The one task that was eliminated was classified as Level 0 by the DACUM experts, implying that both groups placed a relatively low level of importance on this task. Of the 12 tasks that reached consensus for inclusion in the chart, five had no contrary answers among the experts (that is, all eight experts answered either “strongly agree”, “agree”, and “neutral” or “strongly disagree”, “disagree”, and “neutral”). For the remaining seven tasks, only one expert answered contrary to the rest of the group (for example, answered “disagree” while the other seven experts answered “neutral”, “agree”, and “strongly agree”). The contrarian answers did not consistently come from the same expert, although Expert #7 answered contrary to the rest of the group for three of these tasks. Comments from experts were helpful in ascertaining the reasoning behind answers distant from the median value and gaining insight into the approach they took to complete the survey. Several comments voiced the opinion that new graduates typically require on-the-job training and direct supervision to develop the skills necessary to complete a task. Others debated whether included tasks should be expected of an EIT or if they should be the responsibility of an experienced engineer or project manager. Additionally, several experts included comments indicating that their past experience with EITs had influenced their perspective regarding expectations for a newly hired EIT, for example, Expert #3 stated “It would be wonderful if new grads could produce complete member design and lateral design, but I have stopped expecting it for some time”. Comments from experts were also used to consider items that could be re-worded for clarity or specificity. One task in Duty D related to the use of current MT design standards was re-worded to clarify that the engineer would only be expected to follow current standards relevant to their location and industry sector. The worker characteristic “has hands-on work 28 experience” was rewritten for clarification as “has prior work experience (internship)” for future rounds. Round 2 Survey Results The Round 2 survey contained 30 tasks, five worker characteristics, and one general knowledge and skills item. 11 of the 30 remaining tasks reached consensus, of which five were included and six excluded from the final DACUM chart. All five remaining worker characteristics reached consensus while the item in the general knowledge and skills section remained unresolved. Of the items that reached consensus during the Round 2 survey, more reached consensus in favor of elimination than inclusion. The six eliminated tasks came from “Duty D: Design safe, serviceable, constructable, and durable mass timber structures”, “Duty X: Communicate with stakeholders”, and “Duty E: Contribute to development of project deliverables”. “Duty E: Contribute to development of project deliverables” contained nine tasks at the start of the Delphi process. Eight of these tasks reached consensus after the two survey rounds, four in favor of inclusion and four in favor of exclusion. Comments from experts conveyed the opinion that most tasks under this duty could be learned during the hypothetical EIT’s early years of employment. Other comments related to the extent to which a new engineer should be able to perform the tasks, for example: “For [this set of tasks], I think that they should be able to do this, with some time, effort, and possibly support. In other words, they should be able to call a supplier to determine available MT, they should understand the likely failure modes and material properties listed in specifications, and they should be able to explain but not necessarily calculate long term changes, but not necessarily have a full understanding of their ramifications...” The experts left more comments for the Round 2 survey than they had for the Round 1 survey, often addressing the statements from other experts on the results report. This allowed experts to consider the opinions and perspectives of the rest of the group to a greater extent while completing the Round 3 survey. Round 3 Survey Results The Round 3 survey contained 19 tasks and one general knowledge and skills item. Five of the 19 remaining tasks reached consensus, all of which were included in the DACUM chart. The remaining general knowledge and skill item did not reach consensus and was eliminated from the chart as most experts consistently voted to exclude this item. It was previously 29 determined that the Delphi process would only include three survey rounds to keep all eight experts engaged in participating throughout the duration of the study. As a result, additional analysis of survey results was needed to determine whether to include or exclude the remaining items that had not yet reached consensus. Of the 14 remaining tasks, six had results containing no contrary answers among the experts (that is, all eight experts answered “strongly agree”, “agree”, or “neutral” for each of these six tasks). While these items did not technically meet the selected measure for consensus (limited deviation from median value), the rationale that all experts agreed with or were neutral regarding the inclusion of these items allowed for the decision to include each task on the DACUM chart. Conversely, one task that had not reached consensus was eliminated from the final DACUM chart as all eight experts answered “strongly disagree”, “disagree”, or “neutral” for this item. Of the seven remaining tasks, five had results containing only one contrary answer among the experts (for example, seven experts answered neutrally or in favor of inclusion while only one expert answered in favor of elimination). For two tasks (both grouped under “Duty F: Support project sustainability goals”), all experts responded with “strongly agree”, “agree”, or “neutral” except for Expert #4. It was determined that these two tasks would be included in the DACUM chart. Similarly, for three tasks (all related to communication with stakeholders and the development of project deliverables), all experts responded with “strongly disagree”, “disagree”, or “neutral” except for Expert #2 and were eliminated from the DACUM chart. The consistent positions of Expert #4 as a contrarian to the inclusion of sustainability related items and Expert #2 as a proponent for items other experts considered better developed through OJT further validated the rationale for including or eliminating these five tasks. The two remaining tasks had a neutral median and average value as the eight experts’ answers were distributed fairly evenly between “agree”, “neutral”, and “disagree”. Both tasks were included in the final DACUM chart as it was more prudent to include the tasks than to eliminate them. After this final curation of tasks, it was discovered that all four tasks within “Duty X: Communicate with stakeholders” in the preliminary DACUM chart had been eliminated from the final chart. To reflect these changes, Duty X was removed from the DACUM chart. 30 Final Results: Final DACUM Chart The results of the final DACUM chart are shown in Table 6. This chart can be compared to Table 14 and Table 15 in the Appendix, which present the results of preliminary DACUM Chart prior to exposure to the Delphi Methodology to identify items that were eliminated or re- worded from the preliminary DACUM chart during the application of the Delphi Methodology. A parenthetical note after the title of each duty signifies whether any tasks were eliminated from that category during the Delphi process and are no longer shown on the DACUM chart. 31 Table 6: Results of Final DACUM Chart Duty A: Demonstrate knowledge of engineering design (0 tasks eliminated) A1 Produce complete member design and sizing (including connection design and design for fire resistance) A2 Design connections A3 Evaluate fastener types and behavior with various materials A4 Analyze design needs for composite structural elements A5 Examine vibration analysis results A6 Apply lateral design (seismic, wind) to MT system Duty B: Understand MT characteristics (0 tasks eliminated) B1 Assess MT material properties and failure modes B2 Evaluate material grades and species B3 Identify types of MT available from specific suppliers/manufacturers B4 Explain long-term/post occupancy changes, including creep, shrinkage, and fatigue Duty C: Navigate network of MT resources (0 tasks eliminated) C1 Summarize available MT design resources (Woodworks, ThinkWood, design manuals, etc) C2 Acquire and analyze research data to apply to design C3 Participate in ongoing education (presentations, code committees, etc.) Duty D: Design safe, serviceable, constructable, and durable MT structures (2 tasks eliminated) D1 D2 Assess what is allowable under design code (IBC, ICC) including construction types, gross floor areas, fire -rating requirements, building heights, etc. Follow current standards relevant to location and industry sector, for example: NDS, O86 Eurocode (EC5), -PRG320, LVL standard, Glulam standard (ANSI 117/190.1), ASTM standards (D) D3 Evaluate paths for MT fire design D4 Understand necessary fabrication tolerances D5 Design for standardization/replicability D6 Design for durability/adaptability D7 Understand testing methods for development of standards and NDE techniques D8 Understand general MT testing, certifications, and requirements D9 Understand basic principles for DFMA including panelization, sequencing, and site and design constraints D10 Design for bulk water/moisture management for structural elements Duty E: Contribute to development of project deliverables (5 tasks eliminated) E1 Evaluate project drawing set and shop drawings E2 Sequence mass timber design tasks into workflow E3 Develop design aids and programs for internal use E4 Participate in site visits and observe installation Duty F: Support project sustainability goals (0 tasks eliminated) F1 Explain LCA, carbon, and sustainability principles/counterpoints in the context of MT F2 Describe sustainable forestry and material procurement F3 Assess EPDs and comparative LCAs F4 Explain design for deconstruction principles (circular economy) F5 Identify sustainability/LCA tools *Acronym definitions can be found in Figure 11: Published Finalized DACUM Chart: Page 3 in the Appendix 32 No tasks included on the preliminary DACUM chart in Duty A, B, C, or F were eliminated during the Delphi survey process, illustrating a mutual agreement between DACUM panelists and Delphi experts in favor of including these tasks and duties. Conversely, more than half of the tasks in Duty E and all tasks from Duty X were eliminated during the Delphi survey process. Because eliminated tasks had to reach consensus to be removed, the Delphi panelists had a unified opinion regarding these items. However, these tasks also highlight the difference in opinion between the Delphi experts (in agreement in favor of elimination) and the DACUM panelists (in agreement in favor of inclusion). The number of tasks in each duty before and after the Delphi method are shown in Figure 7. Figure 7: Comparison of Preliminary and Final DACUM Chart Of the duties that remained unaltered during the Delphi process, “Duty A: Demonstrate knowledge of engineering design” and “Duty B: Understand mass timber characteristics” are largely related to comprehension of member design and material properties, two concepts common in core competencies seen in traditional design courses like concrete or steel. Both duties resulted in a relatively high level of agreement throughout the Delphi survey rounds, with all tasks earning median answers of at least 4 (“agree”) in every round. The inclusion of all tasks 33 from Duties C and F demonstrates a similar prioritization on a student’s ability to navigate available MT resources and their knowledge of MT sustainability principles. There are other metrics that can be used to consider the implied importance of a task or duty besides consistency before and after the Delphi method. Table 7 shows the average and median of all answers for all tasks within each duty. Duties B and C had the highest averages when considering all answers, indicating that the Delphi experts were most strongly in favor of including tasks from this category in the DACUM chart. While this is not necessarily indicative of the duty that had the greatest area of consensus, these are the sections in which the opinions of the Delphi experts most closely aligned with those of the DACUM panelists. Table 7: Total Answers by Duty Average of all answers Median of all answers Total answers Duty A: Demonstrate knowledge of engineering design Duty B: Understand mass timber characteristics Duty C: Navigate network of mass timber resources Duty D: Design safe, serviceable, constructable, and durable mass timber resources Duty X: Communicate with stakeholders Duty E: Contribute to development of project deliverables Duty F: Support project sustainability goals 3.88 4.30 4.17 3.33 2.48 3.05 3.43 4 4 4 3 2 3 4 128 64 32 216 88 120 88 Because tasks went through multiple rounds of survey exposure until they reached consensus, some tasks and duties underwent more scrutiny than others. The first tasks to reach consensus during the Delphi survey process are indicative of tasks of which the experts had the most aligned opinions without being influenced by each other’s answers. Among these items, those that had a median answer greater than or equal to 4 (“agree”) reflected a desire to include tasks and were indicative of items of high priority to the Delphi experts. These seven tasks that reached consensus on agreement in the first survey round are shown in Table 8. Six tasks that reached consensus in the first round had median values ranging from 2.5 to 3.5 and were not considered as tasks of high importance to the experts because of the neutrality of the answers. No tasks that reached consensus in the first round of the survey had median values less than 2.5. 34 Table 8: Consensus Agreement Items from Delphi Round 1 Survey Median Consensus Percentage Task ID Task 5 4.5 4.5 4.5 4 4 4 75% C1 Summarize available mass timber design resources (WoodWorks, ThinkWood, design manuals, etc.) 100% B2 Evaluate material grades and species 88% 75% 75% 75% 75% C3 Participate in ongoing education (presentations, code committees, etc.) E4 Participate in site visits and observe installation A3 Evaluate fastener types and behavior with various materials D8 Understand general mass timber testing, certifications, and requirements F5 Identify sustainability/LCA tools Conversely, tasks that were slowest to reach consensus were indicative of items that were a source of disagreement among the Delphi experts. Two tasks that had not reached consensus by the end of the Delphi process required additional consideration to determine whether they should be included in the final DACUM chart (a process described in Delphi Round 3 Results). Comments from experts were helpful in identifying why each task was unable to easily reach consensus. Common concerns regarding “Task F2: Describe sustainable forestry and material procurement” were related to the scope of sustainable forestry and the substantial amount of coursework likely required to cover such a topic, especially as it does not relate to engineering. The expected level of comprehension for this task was also a source of disagreement. While no comments specifically addressed “Task D5: Design for standardization/replicability”, it is possible that the lack of consensus on this item was caused by the differing personal opinions of the experts as to whether MT should move towards mass production or if the ability to customize MT elements is a part of its appeal as a material. Comprehension level designations assigned by the original DACUM panelists were not included or evaluated as part of the Delphi surveys. The preliminary DACUM chart contained seven Level 0 tasks, 11 Level 1 tasks, 12 Level 2 tasks, and 12 Level 3 tasks. It stands to reason that if the Delphi experts’ opinions aligned with those of the DACUM panelists, the items eliminated through the Delphi surveys would primarily have lower comprehension level designations. Table 9 confirms that this was the case as the majority of eliminated tasks were classified as Level 0 (basic familiarity) or Level 1 (identify/describe) while only two Level 2 (assess/analyze) tasks were eliminated. All Level 3 (create) tasks remained on the finalized 35 DACUM chart, signifying an alignment between the opinions of the DACUM and Delphi panelists regarding tasks of particular importance. Table 9: Final Decisions for Each Task Level Designation Included Eliminated Level 0 2 tasks 5 tasks Level 1 7 tasks 4 tasks Level 2 10 tasks 2 tasks Level 3 12 tasks 0 tasks A more detailed examination of the level distribution within similar types of tasks (for example, tasks related to engineering design) provides insight as to which specific types of tasks are expected of a new engineer upon their first day of employment. Design related tasks such as member and connection design, knowledge of material grades and properties, fastener design, interpretation of design code, awareness of design resources, and development of personal design aids were designated as Level 3. These concepts could be classified as basic knowledge of MT material properties and member design, similar in scope to concepts included in typical undergraduate courses such as concrete or steel design. More advanced design topics, including understanding of vibration analysis, post occupancy (long-term) changes, hybrid composite behavior, lateral design, retrofits, and sustainable design were designated as Level 1 or 2. This is indicative of the type of content more commonly covered in graduate level courses or early years of employment that may not be required of an undergraduate student upon completion of their degree. Eliminated tasks are indicative of items considered less crucial for a hypothetical EIT to master during their first year of employment (shown in Table 10). Nine of the 11 eliminated tasks were grouped in duties related to professional communication, stakeholder collaboration, and project deliverables. While these abilities are undeniably crucial for a successful engineer working in the AEC industry, Delphi results reflect that it may be more prudent to develop these abilities through on the job training or during early years of employment. Furthermore, these content areas are often not considered in traditional design courses like concrete or steel and are typically relegated to final capstone or senior-level design project courses. Additionally, many items in “Duty E: Contribute to development of project deliverables”, are largely specific to the firm at which the EIT is employed and cannot be adequately addressed using a standardized approach at the undergraduate level. 36 Table 10: Tasks from Preliminary DACUM Chart Eliminated During Delphi Method Duty D: Design safe, serviceable, constructable, and durable MT structures X1 Understand constraints of manufacturing process, tooling, C&C X2 Design, strengthen, retrofit existing timber structures Duty X: Communicate with stakeholders X3 Coordinate/communicate design task process/order with non-structural teams, architect, manufacturer X4 Knowledge of approval process/alternative means and methods X5 Educate owners X6 Train and include AHJ in design process Duty E: Contribute to development of project deliverables X7 Differentiate among various project delivery methods X8 Describe BIM execution plan X9 Describe principles of R&D, product development X10 Differentiate among different services (performance vs prescriptive) when submitting proposals X11 Describe CNC cutter paths Level 2 1 Level 2 1 0 0 Level 1 1 0 0 0 Tasks from “Duty X: Communicate with stakeholders” contained several comments questioning whether the tasks were within the scope of an EIT's job description. Additionally, common themes emerged that the role of a university should be to address fundamental engineering design skills and that tasks related to communicating with stakeholders or managing deliverables are better left to OJT during the EIT’s early years of employment. Comments from several Delphi experts are shown in Table 11 and reflect their rationale for answering in favor of eliminating these items. 37 Table 11: Expert Comments Regarding Duty X Referenced Task(s) X3 - X6 “This particular list is more the responsibility of the project manager. I'd rather the structural designer focus on the basics of design with the material. These items can be learned on the job.” X3 - X5 “I feel like coordination is a PM's job, and that fresh graduates should not be expected to manage projects. In fact all [three] of these feel like senior engineering responsibilities…” X3 - X5 “These tasks are those learnt on the job and would be the responsibility of more senior engineers…” X6 “It is too much to expect an engineer in training to potentially "train" an AHJ who may have 30 years experience. they just need to communicate with an AHJ” X3 - X5 “All of the issues in this group are the job of professional mentors, not universities. Universities should remain focused on those fundamental skills that cannot reasonably be taught on the job. To make an absurd example, I expect university graduates to be absolutely fluent in statics, shear and moment diagrams, and basic structural analysis. I cannot afford the time or effort to teach these basic skills. However, I can teach them easily how to engage with an AHJ by bringing them along with me to meetings and having them observe, and engaging in discussion on the drive back to the office. Likewise with all these examples.” X4 “You should have a project liaison that understands the process of means and methods for the project.” X3 - X5 “In my opinion, these items generally fall under the responsibility of senior engineers/project managers. I would never let one of our designers right out of school directly coordinate with architects, other trades, project owners, or AHJ independently. As talented as a student can be right out of school, it takes a fair bit of experience to successfully take on these kinds of responsibilities.” In addition to the Delphi expert’s comments from each survey round, the distribution of their survey answers and variations in consistency between survey rounds can provide insight into the extent to which they were influenced by the answers of the other experts. The expert’s perceived prioritization of various content areas can also be identified by examining duties in which they consistently answered in favor of inclusion. A summary of the answer choices made by each expert in the three survey rounds is shown in Table 12. Considering the response distribution, average, and median of the expert’s answers for each round provides insight into their consistency throughout the survey process. 38 Table 12: Expert Response Distribution in Delphi Survey Delphi Round # of 1s # of 2s # of 3s # of 4s # of 5s Average Median Round 1 Expert #1 Round 2 Round 3 Round 1 Expert #2 Round 2 Round 3 Round 1 Expert #3 Round 2 Round 3 Round 1 Expert #4 Round 2 Round 3 Round 1 Expert #5 Round 2 Round 3 Round 1 Expert #6 Round 2 Round 3 Round 1 Expert #7 Round 2 Round 3 Round 1 Expert #8 Round 2 Round 3 5 2 0 0 0 0 1 2 0 0 0 0 1 4 0 0 0 0 5 3 1 0 0 0 11 12 4 6 1 2 12 8 6 9 3 6 15 13 5 0 5 4 8 5 1 1 6 0 10 5 11 9 13 2 9 8 2 5 6 2 21 3 9 4 0 2 10 8 7 15 10 7 12 8 4 24 16 15 15 12 11 21 13 8 5 10 5 20 11 11 16 10 9 21 10 4 5 3 0 4 0 0 6 0 0 8 8 3 1 0 0 19 14 2 4 4 1 6 4 8 3.02 2.93 3.00 3.60 3.50 3.68 3.30 3.00 3.26 3.65 3.87 3.42 2.77 2.63 3.00 4.35 4.13 3.58 3.14 3.23 3.42 3.74 3.40 4.05 3 3 3 4 4 4 3 3 4 4 4 4 3 2 3 4 4 4 3 3 4 4 3 4 Each expert’s personal interpretation of the Likert scale also plays a role in the analysis of Delphi results. Some experts tended to select responses on either end of the scale (1 = strongly disagree, 5 = strongly agree). Different interpretations of what it means to answer something “strongly” leads to issues with the alignment of expert opinions, which can be a barrier to reaching consensus on a particular task. For example, Experts #2, #4, #6, and #8 did not give any ratings of 1 (strongly disagree) throughout the three survey rounds. The selection of answer rating 5 (strongly agree) was also inconsistent among the group. Experts #6, #4, and #8 gave 35, 19, and 18 ratings of “strongly agree”, respectively, while Expert #5 only selected this option for one task throughout all three survey rounds. The prepared survey result reports distributed with the Round 2 and Round 3 surveys were intended to help experts consider the opinions of others and lead the group towards 39 consensus. The extent to which each expert was swayed by the responses of the rest of the panel differed among the group. Expert 7 had relatively consistent answer distributions in all three survey rounds, with most of his answers falling between 3 (neutral) and 4 (agree). Expert 3 had nearly identical answers between the Round 2 and Round 3 surveys (only two answers varied), indicating that he either maintained a consistent opinion between rounds or may have had the results report open while responding and intentionally selected the same answers as the previous round. Conversely, the distribution of Expert 6’s responses varied throughout rounds, particularly regarding tasks in Duties X and E, which were predominantly industry-focused tasks that other experts were in favor of eliminating. While Expert #6 answered with 4s and 5s for these tasks in the Round 1 survey, he later selected 2s for these items, which was more in alignment with the median value and responses of other experts. However, the change in expert’s answers throughout survey rounds cannot automatically be attributed to their conceding towards the opinions of others. Expert #1’s answer distribution moved towards neutrality throughout the three survey rounds, but it is not possible to know whether change was due to consideration of other expert’s feedback or was a result of fatigue through the survey rounds. Extenuating factors affecting the completion of the survey were not considered in the interpretation of results. The use of the prepared survey result reports may have also had unanticipated consequences, such as influencing how the experts interpreted the answer choices. For example, in the Round 1 survey, Expert 6 did not choose to eliminate any tasks and indicated that they strongly agreed (answer choice 5) with 44% of tasks, but later shifted towards more neutral answer selections. It is not possible to definitively determine whether Expert 6’s answer distribution was altered because their opinion regarding survey content was swayed by the perspectives of the rest of the panel or if they reevaluated their interpretation of the answer choices between survey rounds based on the answer distributions of the other experts. There were 15 general knowledge and skills items and 32 worker characteristics in the preliminary DACUM chart. Two general knowledge and skills items (“conduct independent research” and “complete cost analysis when using mass timber in building design”) and one worker characteristic (“has prior work experience”) were eliminated during the Delphi process. The curated list of general knowledge and skills items and worker characteristics shown on the final DACUM chart are presented in Table 13. 40 Table 13: Final General Knowledge and Skills Items and Worker Characteristics General Knowledge and Skills Collaborate with design team Collaborate with MT manufacturer Conduct finite element analysis Adaptable Analytical thinker Collaborative Comfortable with failure Committed to safety Committed to work Conscientious Consistent Creative Decisive Detail-oriented Creative problem solving Explain anisotropic material behavior Holistic thinking (general) Worker Characteristics Ethical Flexible Handles stress well Honest Innovative Interdisciplinary interests Methodical Motivated Organized Passionate about work Holistic thinking (related to sustainability) Oral communication Written communication Persistent Personal integrity Self-motivated Self-starter Solution-oriented Strong work ethic Team player Thorough Thoughtful Works well on a team Worker characteristics and general knowledge and skills desired for a new MT EIT include an emphasis on both knowledge of MT design and the ability to effectively collaborate with coworkers and clients. These areas are particularly important because of the need for qualified engineers to understand and advocate for the use of MT, especially as public awareness regarding MT construction continues to grow. University curriculum can be designed to provide students with adequate knowledge and design experience while also developing the strong communication skills necessary to collaborate effectively with stakeholders. There is also an emphasis on the ability to adapt and problem-solve as a new EIT. The importance placed on these items may be due to the independent study and extra work MT EITs likely conduct during the early years of their career to bridge the gap between academic concepts and real-world applications, which can be exacerbated for students who did not have exposure to timber design during their undergraduate experience. Additionally, while new materials, design aids, and resources continue to emerge for MT designers, difficult concepts or non-standard projects may require independent verification or the insight of an experienced expert. These traits were included specifically due to their relevance for a MT EIT when compared to those of an EIT working with traditional materials like concrete or steel, which are studied more extensively in university programs and have numerous design resources available. 41 Limitations of DACUM and Delphi Panels The ability to customize the DACUM and Delphi methodologies to specific needs is one of the things that makes it attractive to researchers and allows for a variety of applications. However, the lack of strictly imposed standards for things like panel size, round requirements, and expert qualifications can lead to ambiguity. Procedural changes, such as continuing the Delphi surveys longer than three rounds or loosening selection criteria for qualified experts, could alter findings. The knowledge level assignments or Bloom’s Taxonomy verbs placed on each task also could have been integrated into the Delphi process or used to organize tasks on the final DACUM chart. Additionally, the size and composition of the pool of selected respondents had an undeniable effect on the creation and validation of the DACUM chart. While the sizes of both the DACUM panel (six panelists) and Delphi panel (eight experts) were relatively small, they were intentionally selected to serve as a representative sample of current MT experts working in the engineering industry. DACUM panelists and Delphi experts lived in 11 different states (including one Delphi expert in Washington D.C.) and one Canadian province at the time the study was conducted. Comparing the geographical location of each panelist to the states in which MT construction is the most popular provides insight into the experience of study participants. Figure 8 shows a map of the United States with each state shaded according to the number of documented MT projects (in design and completed) as of December 2023 (Woodworks, 2023). Colored outlines represent states from which one (red) or two (blue) participants originated. 42 Figure 8: Geographical Distribution of DACUM Panelists and Delphi Experts from Each State Four of the 14 participants worked in the Pacific Northwest at the time of the study, where MT construction is the most popular and well-established. Four additional participants worked in the Midwest. These groupings were adequate for the study since the intention was to produce a sample representative of current MT experts rather than experts from various geographical areas. Future studies could prioritize finding participants from each state or correlate the number of representatives based on the number of active MT projects in each state. Additionally, repeating the Delphi process with a larger pool of respondents could strengthen findings, especially as more qualified subject matter experts emerge in the MT community. The panel of qualified experts assembled for the Delphi methodology were largely selected because of their experience working with MT. Experts were asked to estimate how many MT projects they had worked on and to report the number of years that they had worked as a licensed engineer. Because the most experienced experts were selected, the panel participants had between ten to 45 years of experience. While an overall higher level of experience lends to the group’s credibility, it also limits the inclusion of experts who have worked as EITs in recent years. It is likely that MT engineers who have entered the AEC industry in the past ten years 43 have different perspectives, priorities, and experiences than the selected experts. Additionally, selected experts were not asked about their work experience immediately after graduating from their university program. It is then not known if experts had any experience working with timber or MT as an EIT or if they entered the MT industry later in their career as the material grew in popularity. While detailed written instructions for completing the surveys were provided in multiple locations for each stage of the DACUM and Delphi process, it is not possible to ensure that each respondent had the same assumptions while participating in the study. For example, instructions indicated that the experts should consider tasks that a hypothetical EIT should be able to perform within their first year of employment. However, for brevity, the survey restated before each task: “After graduating from their university, students should be able to…”. The survey may have had different results if the prompt before each question instead said: “After graduating from their university and within their first year of work as an EIT, students should be able to…”. There is also ambiguity as to whether tasks completed within the first year of work should be developed through OJT rather than university programming, yielding items that may not be suited for inclusion on a DACUM chart. There may have also been inconsistencies in the rationale participants used to answer questions. While the experts knew that they were selected for participation because of their work experience and were asked to provide answers in line with current industry needs, they were also aware that the DACUM chart would be used to guide the development of educational resources. As a result, experts may have answered questions based on their perceived feasibility of incorporating each task into a university program rather than providing their unbiased opinion as to what a new EIT should ideally be able to do within their first year of work. This approach may have caused experts to answer questions drawing upon their own educational experience rather than relying upon their insight as a current industry expert, introducing unintentional biases to the way they completed the survey. 44 CONCLUSION This study addressed a critical gap in the education of undergraduate civil and structural engineering students in the United States by focusing on the emerging field of MT construction. The growing popularity of MT as a sustainable and innovative construction material necessitates the preparation of qualified engineers and designers who possess the necessary knowledge and skills to meet industry demands. Review of existing literature and coursework inventories revealed a lack of MT design resources and courses oriented towards undergraduate students. A DACUM workshop was held to analyze the requirements of a MT EIT position based on the experience of six panelists currently working in the MT industry. The panelists verified the compiled findings of the meeting, resulting in a list of tasks, duties, worker characteristics, and general knowledge and skills items. Eight qualified industry professionals were selected to serve as subject matter experts to validate the preliminary findings through the Delphi methodology. The survey results yielded a finalized DACUM chart reflecting the current and future needs of the MT construction industry. A closer analysis of the methodology and findings associated with the DACUM and Delphi processes provided valuable insight into the priorities of industry professionals, including importance placed on knowledge of engineering design, understanding of MT material characteristics, the ability to navigate available MT resources, design of MT structures, contributions to project deliverables, and support of project sustainability goals. Additionally, worker traits and characteristics related to collaboration, adaptability, and problem-solving skills were considered to be especially relevant and should be facilitated in undergraduate coursework. The core competencies on the finalized DACUM chart can be used to aid in the creation of relevant and timely undergraduate curriculum. Moving forward, the finalized DACUM chart serves as a valuable resource that offers guidance to educational institutions seeking to integrate MT engineering education into existing coursework. The chart will be made publicly available to assist in the development of learning modules, virtual tours, and student assignments that align with industry requirements and enhance student readiness for careers in MT construction. Additionally, similar Delphi-validated DACUM charts will be created for entry-level positions in the architecture and construction industries. As the popularity of MT continues to grow, the implementation of the DACUM chart's recommendations will play a crucial role in ensuring that graduating engineers are well- 45 prepared to contribute to the advancement of sustainable building practices and address the multifaceted challenges posed by an evolving construction industry. 46 REFERENCES Ahmed, S., Arocho, I. (2020). Mass timber building material in the U.S. construction industry: Determining the existing awareness level, construction-related challenges, and recommendations to increase its current acceptance level. Cleaner Engineering and Technology. https://doi.org/10.1016/ j.clet.2020.100007 Ahmed, S., Arocho, I. (2021). Feasibility Assessment of Mass Timber as a Mainstream Building Material in the US Construction Industry: Level of Involvement, Existing Challenges, and Recommendations. ASCE Practice Periodical on Structural Design and Construction. https://doi.org/10.1061/(ASCE)SC.1943-5576.0000574 Alderman, D. (2022). U.S. Forest Products Annual Market Review and Prospects, 2015-2021. General Technical Report FPL-GTR-289. U.S. Department of Agriculture, Forest Service, Forest Products Laboratory. American Society of Civil Engineers (ASCE) (2004). Civil Engineering Body of Knowledge for the 21st Century: Preparing the Civil Engineer for the Future. https://doi.org/10.1061/9780784409657 Asdrubali, F., Ferracuti, B., Lombardi, L., Guattari, C., Evangelisti, L., Grazieschi, G.(2017). A review of structural, thermo-physical, acoustical, and environmental properties of wooden materials for building applications. Building and Environment. https://doi.org/10.1016/j.buildenv.2016.12.033 Balogh, Z. E., Criswell, M. E. (2013). Framework of Knowledge for Master’s-Level Structural Engineering Education. Journal of Professional Issues in Engineering Education and Practice. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000176 Balogh, Z. E., Criswell, M. E., De Miranda, M. A. (2013). Expectations for the Masters-Level Structural Engineering Graduate Preparedness using the Delphi Method. 120th ASEE Annual Conference & Exposition. Brandeis, C., Taylor, M., Abt, K. L., Alderman, D., Buehlmann, U. (2021). Status and Trends for the U.S. Forest Products Sector: A Technical Document Supporting the Forest Service 2020 RPA Assessment. U.S. Department of Agriculture. https://doi.org/10.2737/SRS-GTR-258 Ceccotti, A., Sandhaas, C., Okabe, M., Yasumura, M., Minowa, C., Kawai, N. (2013). SOFIE project – 3D shaking table test on a seven-storey full-scale cross-laminated timber building. Earthquake Engineering and Structural Dynamics. https://doi.org/10.1002/eqe.2309 Center on Education and Training for Employment (CETE). (2024). DACUM International Training Center. Ohio State University. Chorlton, B., Mazur, N., Gales, J. (2019). Incorporating timber education into existing accredited engineering programs. Proceedings of the Canadian Engineering Education Association (CEEA). https://doi.org/10.24908/pceea.vi0.13871 47 Daneshvar, H., Goni, T., Zhang, S., Kelterborn, R., Chui, Y.H. (2021). Structural Timber Design in Curricula of Canadian Universities: Current Status and Future Needs. Education Sciences. https://doi.org/10.3390/educsci11120765 Dong, K. (2015). Structural Education Deficiencies: Timber and Masonry. STRUCTURE Magazine. Francis, S. M. (2019). 2019 NCSEA Structural Engineering Curriculum Survey Results. STRUCTURE Magazine. Frangi, A., Fontana, M., Knobloch, M. (2018). Fire Design Concepts for Tall Timber Buildings. Structural Engineering International. Gupta, R., Gopu, V. (2005). Wood Engineering Education—Trends and Challenges. In Proceedings of the Structures Congress 2005, New York, NY, USA. Hassan, O. A. B., Emad A.A., N., Gabriel Abdulahad, G. (2022). A comparative study between glulam and concrete columns in view of design, economy and environment. Case Studies in Construction Materials. https://doi.org/10.1016/j.cscm.2022.e00966 Izzi, M., Casagrande, D., Bezzi, S., Pasca, D., Follesa, M., Tomasi, R. (2018). Seismic behaviour of Cross-Laminated Timber structures: A state-of-the-art-review. Engineering Structures. https://doi.org/10.1016/j.engstruct.2018.05.060 Keeney, S., Hasson, F., McKenna, H. (2011). The Delphi Technique in Nursing and Health Research. Wiley-Blackwell. ISBN 978-1-4051-8754-1 Kuzmanovska, I., Gasparri, E., Tapias Monne, D., Aitchison, M. (2018). Tall Timber Buildings: Emerging Trends and Typologies. 2018 World Conference on Timber Engineering. Laguarda Mallo, M.F., Espinoza, O. (2014). Outlook for cross-laminated timber in the United States. Bioresources. Laguarda Mallo, M. F., Espinoza, O. (2015). Awareness, perceptions and willingness to adopt Cross-Laminated Timber by the architecture community in the United States. Journal of Cleaner Production. https://doi.org/10.1016/j.jclepro.2015.01.090. Lawson, J. W., Kam-Biron, M., Perkins, B. (2020). Used the Most, Taught the Least. STRUCTURE Magazine. Lehmann, S., Kremer, P. D. (2023). Filling the Knowledge Gaps in Mass Timber Construction: Where are the Missing Pieces, What are the Research Needs? Mass Timber Construction Journal. Muszyński, L., Gupta, R., Hong, S. H., Osborn, N., Pickett, B. (2019). Fire resistance of unprotected cross-laminated timber (CLT) floor assemblies produced in the USA. Fire Safety Journal. https://doi.org/10.1016/j.firesaf.2018.12.008. Okoye, U., Kam-Biron, M., Perkins, B., Barnes, C. (2017). Higher Education That Includes Timber Engineering. STRUCTURE Magazine. Oliver, C. D., Nassar, N. T., Lippke, B. R., McCarter, J. B. (2014). Carbon, Fossil Fuel, and Biodiversity Mitigation With Wood and Forests. Journal of Sustainable Forestry. 48 Perkins, B. (2016). 2016 NCSEA Structural Engineering Curriculum Survey. STRUCTURE Magazine. Riddle, A. A. (2023). Mass Timer: Overview and Issues for Congress. Congressional Research Service. Rüütmann, T. (2022). Reinforcement Of Engineering Pedagogical Competencies Of Engineering Faculty Members And Doctoral Students. Land Economy and Rural Studies Essentials. European Proceedings of Social and Behavioural Sciences. https://doi.org/10.15405/epsbs.2022.02.90 Shahnewaz, Md., Tannert, T., Shahria Alam, M., Popovski, M. (2017). Seismic Performance of Cross Laminated Timber (CLT) Platform Building by Incremental Dynamic Analysis. 39th IABSE Symposium – Engineering the Future. Vancouver, Canada. Sheine, J., Donofrio, M., Gershfeld, M. (2019). Promoting Interdisciplinary Integrated Design Education Through Mass Timber. In 2019 Reynolds Symposium: Education by Design. Portland, Oregon, October 18-19, 2019. Skulmoski, G. J., Hartman, F. T., Krahn, J. (2007). The Delphi Method for Graduate Research. Journal of Information Technology Education. Smith, R. E., Griffin, G., Rice, T., Hagehofer-Daniell, B. (2016). Mass timber: evaluating construction performance. Architectural Engineering and Design Management. Sorby, S. A., Bohmann, L. J., Drummer, T. D., Frendewey, J. O., Mattila, K. G., Sutherland, J. W. (2005). Development of a Curriculum for Service Systems Engineering Using a Delphi Technique. Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition. Syed, M.T. (2020). Identifying Mass Timber Research Priorities, Barriers to Adoption and Engineering, Procurement and Construction Challenges in Canada. MS Thesis, Department of Forestry, University of Toronto. Timmers, M., Jacobs, A. T. (2018). Concrete apartment tower in Los Angeles reimagined in mass timber. Engineering Structures. https://doi.org/10.1016/j.engstruct.2017.11.047 Woodworks (2023). Market Trends Map: Snapshot of Mass Timber Projects in the U.S. https://www.woodworks.org/resources/mapping-mass-timber/ Woodworks (2024). Status of Building Code Allowances for Tall Mass Timber in the IBC. https://www.woodworks.org/resources/status-of-building-code-allowances-for-tall-mass-timber- in-the-ibc/ Zelinka, S.L., Williamson, T., Martinson, K.L., Ritter, M. (2019). Research needs assessment for the mass timber industry: Proceedings of the 2nd North American mass timber research needs workshop. General Technical Report FPL-GTR-273. Madison, WI: U.S. Department of Agriculture, Forest Service, Forest Products Laboratory. 49 Table 14: Results of Preliminary DACUM Chart (Prior to Delphi Methodology): Duties A to D Level A. Demonstrate knowledge of engineering design APPENDIX A1 Produce complete member design and sizing (includes impact of connection design and fire) A2 Develop connection design A3 Evaluate fastener types and behavior with various materials A4 Analyze design needs for hybrid composite materials A5 Examine vibration analysis results A6 Review lateral design, seismic/wind design (MT) B. Understand MT characteristics B1 Assess MT material properties and failure modes B2 Evaluate material grades (glulam, CLT), species B3 Appraise knowledge of MT types available from specific suppliers/manufacturing capacity B4 Explain long term (including post occupancy) changes- creep, shrinkage, fatigue, etc. C. Navigate network of MT resources C1 Summarize available MT design resources (Woodworks, ThinkWood, design manuals, etc.) C2 Acquire and analyze research data and design applications C3 Interact with ongoing education (code committee, presenting) D. Design safe, serviceable, constructable, and durable MT structures D1 Assess what is allowable under design code (construction types, gross floor areas, fire rating - requirements, building heights) (IBC, ICC) D2 Explain NDS (materials), O86 Eurocode (EC5), PRG320, ANSI 190.1, LVL standard, Glulam standard, -ASTM standards (D) D3 Evaluate paths for MT fire design D4 Understanding of necessary tolerances D5 Design for standardization, replicability D6 Design for durability, adaptability D7 Understand constraints of manufacturing process, tooling, C&C D8 Understand testing methods for development of standards and non-destructive evaluation techniques D9 Understand general MT testing, certifications, requirements D10 Knowledge of basic principles for DFMA (panelization, sequencing, site constraints, design constraints) D11 Design for bulk water, moisture management for structural elements D12 Design, strengthen, retrofit existing timber structures 3 3 3 2 2 1 Level 3 3 3 2 Level 3 2 0 Level 3 3 3 2 2 2 2 1 2 1 1 1 *Task identification numbers have been stricken through to reflect tasks that were eliminated during the Delphi survey process. 50 Table 15: Results of Preliminary DACUM Chart (Prior to Delphi Methodology): Duties X to F Level X. Communicate with stakeholders X1 Coordinate/communicate design task process/order with non-structural teams, architect, manufacturer X2 Knowledge of approval process/alternative means and methods X3 Educate owners X4 Train and include AHJ in design process E. Contribute to development of project deliverables E1 Evaluate project drawing set and shop drawings E2 Sequence design tasks - structural/non- structural/services (MT workflow) E3 Differentiate among various project delivery methods E4 Describe BIM execution plan E5 Develop programs/job aids/internal programs/design aids E6 Describe principles of R&D, product development E7 Differentiate among different services (performance vs prescriptive) when submitting proposals E8 Describe CNC cutter paths E9 Participate in site visits, observations, installations, transportation F. Support project sustainability goals F1 Explain LCA, carbon, sustainability principles/ counterpoints F2 Describe sustainable forestry, procurement F3 Assess EPDs and comparative LCAs F4 Explain design for deconstruction principles (circular economy) F5 Identify sustainability/LCA tools 2 1 0 0 Level 3 2 1 1 3 0 0 0 NA Level 1 1 2 1 0 *Task identification numbers have been stricken through to reflect tasks that were eliminated during the Delphi survey process. 51 Table 16: Results of Preliminary DACUM Chart (Prior to Delphi Methodology): General Knowledge and Skills and Worker Characteristics Collaborate with design team Collaborate with MT manufacturer Conduct finite element analysis Conduct independent research Adaptable Analytical thinker Collaborative Comfortable with failure Committed to safety Committed to work Conscientious Consistent Creative Decisive Detail-oriented General Knowledge and Skills Cost analysis when using MT in building design Holistic thinking (related to sustainability) Creative problem solving Oral communication Explain anisotropic material behavior Holistic thinking (general) Worker Characteristics Ethical Flexible Handles stress well Has hands-work experience Honest Innovative Interdisciplinary interests Methodical Motivated Organized Passionate about work Written communication Persistent Personal integrity Self-motivated Self-starter Solution-oriented Strong work ethic Team player Thorough Thoughtful Works well on a team 52 Table 17: Summary of Delphi Survey Results: Duties A to C Round 1 Round 2 Round 3 After graduating from their university, students should be able to... Consensus Median Consensus Median Consensus Median Verdict Produce complete member design and sizing (including connection design and design for fire resistance) Design connections Evaluate fastener types and behavior with various materials Analyze design needs for composite structural elements Examine vibration analysis results Apply lateral design (seismic, wind) to mass timber system Assess mass timber material properties and failure modes Evaluate material grades and species Identify types of mass timber available from specific suppliers/manufacturers Explain long-term/post occupancy changes, including creep, shrinkage, and fatigue Summarize available mass timber design resources (WoodWorks, ThinkWood, design manuals, etc.) Acquire and analyze research data to apply to design Participate in ongoing education (presentations, code committees, etc.) Duty A: Demonstrate knowledge of engineering design Duty B: Understand MT characteristics Duty C: Navigate network of MT resources 25% 50% 75% 63% 50% 38% 63% 3 4 4 4 4 4 4 50% 38% - 38% 63% 63% 4 4 - 4 4 4 100% 4.5 100% 4.5 - 50% 4 50% - 4 25% 4 88% 4.5 75% 25% 5 4 - - 75% 3.5 88% 4.5 - - 75% 4 Include 75% 4 Include - - Include 63% 4 Include 75% 4 Include 63% 4 Include - - - - Include Include 63% 4 Include - - - - - Include - Include - Include - Include 53 Table 18: Summary of Delphi Survey Results: Duty D After graduating from their university, students should be able to... Consensus Median Consensus Median Consensus Median Verdict Round 1 Round 2 Round 3 Duty D: Design safe, serviceable, constructable, and durable MT structures Assess what is allowable under design code (IBC, ICC) including construction types, gross floor areas, fire rating requirements, building heights, etc. Follow current standards relevant to location and industry sector, for example: NDS, O86 Eurocode (EC5), PRG320, LVL standard, Glulam standard (ANSI 117/190.1), ASTM standards (D) * Evaluate paths for mass timber fire design Understand necessary fabrication tolerances Design for standardization/replicability Design for durability/adaptability Understand constraints of manufacturing process and tooling Understand testing methods for development of standards and non- destructive evaluation techniques Understand general mass timber testing, certifications, and requirements Understand basic principles for design for manufacturing and assembly (DFMA) including panelization, sequencing, and site and design constraints Design for bulk water/moisture management for structural elements Design, strengthen, and retrofit existing timber structures 25% 4 63% 4 50% 4 Include 63% 4 50% 4 63% 4 Include - 4 3 3 - - Include 75% 3.5 Include 25% 75% 3 3 Include Include 38% 0% 25% 88% 3.5 - 25% 38% 38% 50% 3 3 3 3 75% 2.5 75% 3 75% 4 - - - - 50% 3 75% 3.5 - - - - - Exclude - Include - Include - Include 50% 3.5 50% 4 50% 4 Include 38% 3 88% 2.5 - - Exclude 54 Table 19: Summary of Delphi Survey Results: Duties X to E Round 1 Round 2 Round 3 After graduating from their university, students should be able to... Consensus Median Consensus Median Consensus Median Verdict Coordinate design task process with non- structural teams, architects, and manufacturers 63% 3.5 38% 3 63% 3 Exclude Duty X: Communicate with stakeholders Understand approval process/alternate means and methods 0% Educate project owners 0% 63% 2 63% 50% 2.5 63% Train and include authority having jurisdiction (AHJ) in design process Evaluate project drawing set and/or shop drawings Sequence mass timber design tasks into workflow Differentiate among various project delivery methods Describe BIM Execution Plan Develop design aids and programs for internal use Describe principles of research and development (R&D) for product development Differentiate among different services (performative vs. perspective) when submitting proposals Describe computer numerical control (CNC) cutter paths Participate in site visits and observe installation Duty E: Contribute to development of project deliverables 2 2 Exclude Exclude - Exclude - - Include Include - - - 50% 3 Exclude - - - - - - - - Exclude Include - Exclude - Exclude - - Exclude Include 3 3 2 4 25% 63% 63% 50% 4 3 75% 3.5 75% 2.5 88% 3.5 - 88% 1.5 75% 38% 4 - 3 100% 2.5 - - - - 25% 3 75% 2 38% 2 88% 1.5 75% 4.5 - - 55 Table 20: Summary of Delphi Survey Results: Duty F After graduating from their university, students should be able to... Consensus Median Consensus Median Consensus Median Verdict Round 1 Round 2 Round 3 Duty F: Support project sustainability goals Explain life cycle assessment (LCA), carbon, and sustainability principles/counterpoints in the context of mass timber Describe sustainable forestry and material procurement Assess environmental product declarations (EPDs) and comparative LCAs Explain design for deconstruction principles (circular economy) Identify sustainability/LCA tools 50% 50% 4 4 63% 50% 63% 3.5 63% 75% 3.5 75% 4 - - 4 3 4 - - 50% 4 Include 38% 3 Include 50% 4 Include - - - - Include Include 56 Table 21: Summary of Delphi Survey Results: General Knowledge and Skills After graduating from their university, students should demonstrate the following skills... % Agree % Agree % Agree Verdict General knowledge and skills Creative problem solving Oral communication Written communication Holistic thinking (general) Holistic thinking (related to sustainability) 75% 100% 100% 100% 75% - - - - - - - - - - Include Include Include Include Include After graduating from their university, students should have the ability to complete the following tasks... % Agree % Agree % Agree Verdict General knowledge and skills Conduct finite element analysis Conduct independent research Collaborate with design team Collaborate with mass timber manufacturer Complete cost analysis when using mass timber in building design Explain anisotropic material behavior 75% 50% 100% 88% 25% 100% - - Include 38% 38% Exclude - - - - - - - - Include Include Exclude Include 57 Table 22: Summary of Delphi Survey Results: Worker Characteristics Please indicate if you consider the following worker characteristics to be particularly important for new engineers (EITs) working in MT design. % Agree % Agree % Agree Verdict Adaptable Analytical thinker Collaborative Comfortable with failure Committed to work Committed to safety Conscientious Consistent Creative Decisive Detail-oriented Ethical Flexible Handles stress well Has prior work experience Worker Characteristics Honest Innovative Interdisciplinary interests Methodical Motivated Organized Passionate about work Persistent Personal Integrity Self-motivated Self-starter Solution-oriented Strong work ethic Team player Thorough Thoughtful Works well on a team 75% 88% 100% 88% 100% 100% 100% 100% 63% 50% 88% 100% 88% 88% 38% - - - - - - - - 75% 75% - - - - 25% 100% - 50% 75% 100% 100% 100% 88% 88% 100% 88% 63% 100% 100% 100% 100% 88% 100% 75% - - - - - - - - 75% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Include Include Include Include Include Include Include Include Include Include Include Include Include Include Exclude Include Include Include Include Include Include Include Include Include Include Include Include Include Include Include Include Include 58 Figure 9: Published Finalized DACUM Chart: Page 1 59 Figure 10: Published Finalized DACUM Chart: Page 2 60 Figure 11: Published Finalized DACUM Chart: Page 3 61