-_-...-..-_._ J fHESSS GAN STA I, IIIIIIIIIIIII II IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII 301 706 9984 ’3 This is to certify that the thesis entitled EVALUATION OF FIVE AND TEN-YEAR LAKE STATES FVS AND TWIGS GROWTH PROJECTIONS FOR UPLAND HARDWOODS IN THE NORTHERN LOWER PENINSULA OF MICHIGAN presented by Sean Joseph Canavan has been accepted towards fulfillment of the requirements for Masters degree in Forestry War/Z” Major professor Date ‘i/m/W 0—7639 MS U is an Affirmative Action/Equal Opportunity Institution LIBRARY MIchlgan State Unlverslty PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. DATE DUE DATE DUE DATE DUE »APR" :3: .‘ EM 3 MAX] I #22002“ L— t 1/98 c]ClRC/Da!eDue.p6&p.1-S EVALUATION OF FIVE AND TEN-YEAR LAKE STATES FVS AND TWIGS GROWTH PROJECTIONS FOR UPLAND HARDWOODS IN THE NORTHERN LOWER PENINSULA OF MICHIGAN. By Sean Joseph Canavan A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Department of Forestry 1997 ABSTRACT EVALUATION OF FIVE AND TEN-YEAR LAKE STATES FVS AND TWIGS GROWTH PROJECTIONS FOR UPLAND HARDWOODS IN THE NORTHERN LOWER PENINSULA OF MICHIGAN. By Sean Joseph Canavan Five and ten-year diameter growth, basal area growth, and mortality for seven upland hardwood species in northern Lower Michigan were compared to growth projections from F VS Lake States and Lake States TWIGS, two individual-tree grth models designed for use in this area. The data consisted of individual tree measurements from 44 stands across 10 ecological land type phases on the Manistee National Forest. The stands were measured in 1986, 1991, and 1996. Four sets of projections were done: two for the period from 1986-1996 differing in cycle length, and one each from 1986- 1991 and from 1991-1996. Sets of projections were composed of three different FVS growth simulations, differing in the amount of initial information included in each simulation, and one TWIGS simulation with all initial information included. FVS projections using all initial information produced the most accurate estimates of ten-year diameter and basal area growth. Neither F VS nor TWIGS consistently produced more accurate estimates of ten-year mortality. Drought and insect defoliation, which occurred between 1986 and 1991, may have played a large role in the inaccuracy of five-year growth predictions produced by both projection systems for five of the seven species examined. ACKNOWLEDGMENTS While my name is on this thesis, a number of people deserve credit for the help they gave me throughout this project’s two year duration. First I would like to thank my graduate committee chairman, Dr. Carl Ramm for really making this my project. I can’t say how much I appreciate you getting me started, and then letting me go; leaving it up to me to decide how to get from start to finish and which questions to ask. It’s been an invaluable experience. I would also like to thank the other members of my graduate committee, Dr. Karen Potter-Witter and Dr. Connie Page for helping me, through their ideas and edits, to produce a quality thesis. It’s finally almost done. I would like to thank a number of people with the USDA Forest Service for their help during my field season and with the FVS program. Thanks to Larry Throop, Rose Ingram and Matt Sands at the Cadillac-Manistee Ranger District office for providing information on the stands where my plots were located, maps to help me get out to the plots, and for being willing to sit down and talk about forestry in general. Thanks also to Renate Bush for teaching me how to use FVS and being calm when I called during periods of F VS-stress, and to Gary Dixon and Richard Teck for helping me solve my mysterious problem of bizarre output. In addition, I’d like to thank Dr. Don Dickmann for helping me decide how to handle plots located near managed stands and trees with impossible grth patterns. It’s iii very disconcerting when your data indicates that a few of the trees whose growth you’re trying to model are shrinking over time. There are a number of graduate students who’ve helped me throughout this project and whom I really must thank, most especially Lisa, Chuck, Sarah, Lars and Thomas for listening to my ideas, offering advice, lending an car when I needed to vent my fi'ustrations, and making me leave the office every now and then and have a good time. In addition, there are a number of family and friends who aren’t here at MSU that have helped me throughout these last two years. Thanks Seamus, Maggie, Elizabeth, Partha, Abby, Maya, Julian and Karthik for being there when I needed to get my mind out of Michigan for a while. Finally, I’d like to thank my parents. Thank you both for never doubting me and making me always believe I could do whatever I put my mind to. Your support has been one of the constants in my life on which I have always relied. Thanks Mom for making sure I had whatever I needed and always telling me that no matter what, everything would be okay in the end. Thanks Dad for being the example by which I measure myself. A better role model I could never find. This project, with all its problems included, has been an incredible learning experience; one which I wouldn’t trade for anything. Thanks everyone for helping me get past this step and preparing me for the next one. This project was supported by grants from the McIntire-Stennis Cooperative Forestry Research Program and the Department of Forestry at Michigan State University. iv TABLE OF CONTENTS LIST OF TABLES ....................................................................................................... vii LIST OF FIGURES ...................................................................................................... xi INTRODUCTION ......................................................................................................... 1 PROBLEM DEFINITION ............................................................................................. 4 OBJECTIVES ............................................................................................................... 7 LITERATURE REVIEW .............................................................................................. 8 METHODS ................................................................................................................. 28 Field Methods and Data Collection ....................................................................... 28 Data .............................................................................................................. 28 1996 Field Measurements .............................................................................. 33 Data Compilation .......................................................................................... 35 FVS & IWIGS Simulations .................................................................................. 37 Data Analysis ........................................................................................................ 43 Validation Statistics ....................................................................................... 43 Error Statistics .............................................................................................. 44 SASATEST Validation .................................................................................. 45 RESULTS ................................................................................................................... 48 Influence of Additional Information on FVS Prediction Accuracy .......................... 49 Diameter Growth Error Analysis ................................................................... 49 Trees/Acre Error Analysis ............................................................................. 58 Basal Area Error Analysis .............................................................................. 66 Comparison of F VS and TWIGS Prediction Accuracy .......................................... 74 Diameter Growth Error Analysis ................................................................... 74 Trees/Acre Error Analysis ............................................................................. 80 Basal Area Error Analysis .............................................................................. 86 v DISCUSSION ............................................................................................................. 93 FVS Validation ..................................................................................................... 94 lO-Year Projection ........................................................................................ 94 F ive-Year Projection .................................................................................... 105 FVS and 'I'WIGS Comparison ............................................................................. 107 Conclusion .......................................................................................................... 1 16 Recommendations ............................................................................................... 1 17 Future Considerations ......................................................................................... 1 18 APPENDIX ............................................................................................................... 121 BIBLIOGRAPHY ...................................................................................................... 124 vi Table 1. Table 2. Table 3. Table 4. Table 5. Table 6. Table 7. Table 8. LIST OF TABLES Descriptions of the Ecological Land Type Phases (ELTPs) included in this analysis. ......................................................................................................... 30 Sample sizes, stand density, average size, age, and site quality by species and ecological land type phase (ELTP) in 1986. ................................................... 32 Mean diameter (Dbh) growth and standard deviation (s) of mean diameter growth (in.) for each of the two five-year periods of the study (1986-1991, 1991-1996). ................................................................................................... 38 Mean change in trees/acre (TPA) and standard deviation (5) of mean change in trees/acre for each the two five-year periods of the study (1986-1991, 1991- 1996). ............................................................................................................ 39 Mean basal area (BA) growth and standard deviation (8) of mean basal area growth (it2 / acre) for each the two five-year periods of the study (1986-1991, 1991-1996). ................................................................................................... 39 Mean error (bias), and standard deviation (3) of mean error for estimated dbh (in) by species and size class for lO-year FVS projections run over a single lO-year cycle. Bias is expressed as the predicted value minus the observed value. ............................................................................................................. 50 Mean error (bias), and standard deviation (5) of mean error for estimated dbh (in) by species and size class for lO-year FVS projections run over two five- year cycles. Bias is expressed as the predicted value minus the observed value. ............................................................................................................ 53 Mean error (bias), and standard deviation (3) of mean error for estimated dbh (in) by species and size class for 5-year FVS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. .............................................................................................. 55 vii Table 9. Mean error (bias), and standard deviation (s) of mean error for estimated dbh Table 10. Table 11. Table 12. Table 13. Table 14. Table 15. Table 16. Table 17. (in.) by species and size class for 5-year F VS projections (1991-1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. .............................................................................................. 57 Mean error (bias), and standard deviation (3) of mean error for estimated trees/acre (TPA) by species for lO-year FVS projections run over a single lO-year cycle. Bias is expressed as the predicted value minus the observed value. ........................................................................................................... 60 Mean error (bias), and standard deviation (5) of mean error for estimated trees/acre (TPA) by species for lO-year FVS projections run over two five- year cycles. Bias is expressed as the predicted value minus the observed value ............................................................................................................. 61 Mean error (bias), and standard deviation (5) of mean error for estimated trees/acre (TPA) by species for five-year FVS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. ...................................................................................... 62 Mean error (bias), and standard deviation (3) of mean error for estimated trees/acre (TPA) by species for five-year FVS projections (1991-1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. ...................................................................................... 64 Mean error (bias), and standard deviation (8) of mean error for estimated stand-level trees/acre (TPA) by FVS simulation. Bias is expressed as the predicted value minus the observed value. .................................................... 65 Mean error (bias), and standard deviation (5) of mean error for estimated basal area (BA) (fi2 /acre) by species for lO-year FVS projections run over a single lO-year cycle. Bias is expressed as the predicted value minus the observed value. ........................................................................................................... 67 Mean error (bias), and standard deviation (s) of mean error for estimated basal area (BA) (it2 /acre) by species for lO-year FVS projections run over two five-year cycles. Bias is expressed as the predicted value minus the observed value. ........................................................................................................... 68 Mean error (bias), and standard deviation (3) of mean error for estimated basal area (BA) (ft2 /acre) by species for five-year FVS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. ............................................................................. 7O viii Table 18. Table 19. Table 20. Table 21. Table 22. Table 23. Table 24. Table 25. Table 26. Mean error (bias), and stande deviation (8) of mean error for estimated basal area (BA) (t’t2 lacre) by species for five-year FVS projections (1991-1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. ............................................................................ 71 Mean error (bias), and standard deviation (5) of mean error for estimated stand-level basal area (BA) (Pr2 lacre) by FVS simulation. Bias is expressed as the predicted value minus the observed value. .......................................... 73 Mean error (bias), and standard deviation (s) of mean error for estimated dbh (in) by species and size class for lO-year FVS and 10-year TWIGS projections run over a single lO-year cycle, and the 10-year TWIGS projection run over two S-year cycles. Bias is expressed as the predicted value minus the observed value. ........................................................................................................... 75 Mean error (bias), and standard deviation (5) of mean error for estimated dbh (in.) by species and size class for S-year FVS (Full) and TWIGS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. .................................................... 77 Mean error (bias), and standard deviation (3) of mean error for estimated dbh (in.) by species and size class for S-year FVS (Full) and TWIGS projections (1991—1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. .................................................... 79 Mean error (bias), and standard deviation (5) of mean error for estimated trees/acre (TPA) by species for lO-year F VS and TWIGS projections run over two five-year cycles, and a 10-year TWIGS projection run over a single lO-year cycle. Bias is expressed as the predicted value minus the observed value. ........................................................................................................... 8 1 Mean error (bias), and standard deviation (3) of mean error for estimated trees/acre (TPA) by species for five-year FVS and TWIGS projections (1986- 1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. ................................................................... 83 Mean error (bias), and standard deviation (s) of mean error for estimated trees/acre (TPA) by species for five-year FVS and TWIGS projections (1991- 1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. ................................................................... 84 Mean error (bias), and standard deviation (s) of mean error for stand-level trees/acre (TPA) by FVS and TWIGS simulation. Bias is expressed as the predicted value minus the observed value. .................................................... 86 ix Table 27. Table 28. Table 29. Table 30. Mean error (bias), and standard deviation (5) of mean error for estimated basal area (BA) (it2 lacre) by species for lO-year FVS and 'I'WIGS projections run over a single lO-year TWIGS and a 10-year TWIGS projection run over two five-year cycles. Bias is expressed as the predicted value minus the observed value. ........................................................................................................... 87 Mean error (bias), and standard deviation (5) of mean error for estimated basal area (BA) (ft2 lacre) by species for five-year FVS and 'I'WIGS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. .................................................... 89 Mean error (bias), and standard deviation (s) of mean error for estimated basal area (BA) (t’t2 lacre) by species for five-year FVS and 'I'WIGS projections (1991-1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. .................................................... 90 Mean error (bias), and standard deviation (3) of mean error for stand-level basal area (BA) (ft2 / acre) by FVS and TWIGS simulation. Bias is expressed as the predicted value minus the observed value. .......................................... 91 Appendix Table 1. Crown class codes used in remeasurements of ECS plots. ............ 121 Appendix Table 2. Stands excluded from projection by projection period, and the reason for exclusion (M - management, V - vandalism). ......................................... 121 Appendix Table 3. Sample plots included in the diameter growth analysis for the three different projection periods. ........................................................................ 122 Appendix Table 4. Sample plots included in the plot-level basal area and trees/acre analyses for the three different projection periods. ...................................... 123 Figure 1. Figure 2. Figure 3. Figure 4. Figure 5. Figure 6. Figure 7. Figure 8. LIST OF FIGURES Location of TWIGS Validation Plots in the Manistee National Forest. .......... 31 Mean and standard deviation for mean error of predicted DBH (inches) and BA (ft2 /acre) by simulation level for projection over a single 10-year cycle (positive values are overpredictions). ........................................................... 95 Mean and standard deviation for mean error of predicted TPA over a single lO-year cycle for projection over a single lO-year cycle (positive values are overpredictions). .......................................................................................... 96 Mean and standard deviation for mean error of predicted DBH (inches) for 10- year projections by cycle length and simulation level (positive values are overpredictions). .......................................................................................... 97 Mean and standard deviation for mean error of predicted DBH (inches) by DBH size-class and species for lO-year projections over a single lO-year cycle. This set of figures is indicative of the pattern of prediction observed for all 10- year projections. Only DBH Classes with more than 10 samples are shown. (Positive values are overpredictions). ...................................................... 98-99 Mean and standard deviation for mean error of predicted basal area (it2 /acre) for lO-year projections by cycle length and simulation level (positive values are overpredictions). ........................................................................................ 102 Mean and standard deviation for mean error of predicted TPA for lO-year projections by cycle length and simulation level (positive values are overpredictions). ........................................................................................ 102 Mean and standard deviation for mean error (inches) of predicted TPA for five- year projections by projection period and simulation level (positive values are overpredictions). ........................................................................................ 106 xi Figure 9. Mean and standard deviation for mean error (inches) of predicted DBH for 10- Figure 10. Figure 11. Figure 12. Figure 13. Figure 14. Figure 15. year projections by growth simulator and species. FVS projections were done over a single lO-year cycle. TWIGS projections were done over two five-year cycles. (positive values are overpredictions). ............................................. 108 Mean and standard deviation for mean error (inches) of predicted DBH for 5- year projections (1986-1991) by growth simulator and species. FVS and TWIGS projections were done over a single five-year cycle. (positive values are overpredictions). ................................................................................. 110 Mean and standard deviation for mean error (inches) of predicted DBH for 5- year projections (1991-1996) by growth simulator and species. FVS and TWIGS projections were done over a single five-year cycle. (positive values are overpredictions). ................................................................................. 111 Mean and standard deviation for mean error of predicted TPA for S-year projections (1986-1991) by growth simulator and species. FVS and TWIGS projections were done over a single five-year cycle. (positive values are overpredictions). ....................................................................................... 1 12 Mean and standard deviation for mean error of predicted TPA for 5-year projections (1991-1996) by growth simulator and species. FVS and TWIGS projections were done over a single five-year cycle. (positive values are overpredictions). ....................................................................................... 1 13 Mean and standard deviation for mean error of predicted BA (f’t2 /acre) for 5-year projections (1991-1996) by growth simulator and species. FVS and TWIGS projections were done over a single five-year cycle. (positive values are overpredictions). ................................................................................. 114 Mean and standard deviation for mean error of predicted BA (f’t2 /acre) for 5-year projections (1986-1991) by growth simulator and species. FVS and TWIGS projections were done over a single five-year cycle. (positive values are overpredictions). ................................................................................. 1 15 xii INTRODUCTION “All models are wrong, but some are useful,” claimed noted statistician George Box (Chatfield, 1990). The question becomes, which are more useful and which are more wrong? As many forest managers will confirm, even poor estimates provided by poor models can be better than no estimates at all. This is because effective timber resource management is a complex problem and impossible without accurate information on firture growth and yield. How can managers tell if a model is good or poor and how usefirl it can be? Information of this type can be difficult to find. This study tries to answer these questions for forest managers in Michigan’s Lower Peninsula using a growth and yield projection system known as FVS by validating the model for this area using measured growth data. The state of Michigan covers approximately 36 million acres, of which 19.3 million acres are timberland (Leatherberry and Spencer 1996). This timberland acreage is the fifth largest in the United States. Ownership is primarily in the hands of non- industrial private landowners (53%), with state and federal ownership next largest (21% and 14% respectively), followed by forest industry (8%) and farmers (4%). Hardwoods comprise 75% of the total forested acreage with softwoods contributing the remaining 25%. The primary forest type is maple/birch, covering 7.2 million acres, followed by aspen forest types covering 3.2 million acres. Michigan’s forests support 150,000 jobs 2 and add $9 billion to the state’s economy (Michigan Department of Natural Resources, 1993). Forest-based tourism and recreation support an additional 50,000 jobs and add and additional $3 billion. In order to continue to play such a vital role in Michigan’s economy, the forest industry must rely on productive management of the 1.5 million forested acres it controls. Management of this large forest resource often involves stands of mixed species composition, including several highly valuable hardwood species. Mixed species forests present a very complex problem to forest managers. With no standard approach to management of such areas, managers often need accurate estimates of growth and yield resulting from the implementation of different management techniques in order to decide the best management approach. The problem faced by many forest managers is how to produce accurate estimates of future growth and yield. Forest inventories, while providing information on present forest conditions, are only a snapshot in time. They provide little indication of what may result from the implementation of different management alternatives. Forest managers need a way to predict the forest structure and composition which will result from their management decisions. A number of methods exist to provide this information. Stand table projection is one method which produces direct estimates of stand composition and structure for relatively simple stands. Systems of equations known as growth and yield models have also been developed for a number of species in almost every region of the country. These systems use current forest stand and tree characteristics taken from sample plots to produce estimates of individual tree and stand- 3 level characteristics for a future time. Several of these models have the ability to simulate both managed and unmanaged stand grth in complex and simple stands alike. Such models provide a dynamic dimension to the forest management process. Forest managers can use growth and yield models to assess the effects of different management techniques and to help select a management alternative which optimizes timber production, future forest health and sustainability, wildlife and public acceptance. However, this is true only if the selected grth simulator provides accurate predictions of forest growth. PROBLEM DEFINITION Forest simulation concentrates primarily on timber projection. There are many projection systems available but, as Belcher et al. (1982) point out, the majority of these are species-specific, being restricted in their application to a single species or species- group. Such models often have only limited usefiilness for forest managers who manage multi-species forests under a wide range of forest conditions. One multi-species model receiving wide use today is TWIGS, The Woodsman’s Ideal Growth Projection System (Miner er al. 1988). TWIGS is an individual tree, distance-independent growth and yield simulator developed by the USDA Forest Service North Central Forest Experiment Station. It was derived from a main-frame based growth and yield simulator called STEMS. A number of modifications to this simulator resulted in the version known as TWIGS, one of the first personal computer based growth projection systems. The Lake States variant of TWIGS was developed for use in Michigan, Minnesota and Wisconsin (Miner et al. 1988). A survey of forestry organizations in nine North Central states by Ram and Miner (1986) found that Lake States TWIGS was the most common growth and yield model used in Michigan. A new version of TWIGS, the Lake States variant of the Forest Vegetation Simulator (F VS) has also been developed by the USDA Forest Service. FVS Lake States uses the same 5 equations as Lake States TWIGS, but in an input-output format now standard for all growth and yield projection systems across the United States. The problem which many forest managers in Michigan face is whether or not Lake States TWIGS provides accurate estimates of growth and yield. The managers’ objectives are to use the estimates of growth and yield produced by a grth simulator in order to optimize forest use under constraints due to time, personnel and market changes. This can only be done with any usefirlness when the estimates produced are accurate. The best and most common alternative in Michigan is Lake States TWIGS. However, TWIGS is not without its flaws. When it was developed, the only Lower Peninsula data available was for conifer plantations. As a result, applications of this projection system to hardwoods in Michigan’s Lower Peninsula are, in effect, extensions of the model beyond the region for which it was calibrated (Smith 1983). In addition, early validation tests at the regional level demonstrated that the predictive power of the model decreased as application moved south and east within the lake states, the model’s range of calibration (Leary et al. 1979). The combination of these two factors led to speculation concerning the accuracy of Lake States TWIGS growth and yield predictions for hardwood species in the Lower Peninsula of Michigan. The problem which must now be resolved is to determine how accurate Lake States TWIGS projections for hardwood growth in Michigan are, where does the model fall down, and how may it be improved? This study is a continuation of the initial sub-regional validation of the Lake States TWIGS model for upland hardwoods in Michigan begun by Guertin and Ram (1996). The goals of this study are not to recalibrate the model, but to evaluate the 6 predictive power of the existing Lake States TWIGS and the Lake States variant of F VS for 10-year growth of upland hardwood species in Michigan’s Lower Peninsula, and to examine the possible effects of pest outbreaks and drought on S-year model predictions. OBJECTIVES Specific objectives of this project were to: l) 2) 3) 4) Compare the effect of varying the amount of initial information included in a simulation on the ability of FVS Lake States to accurately predict diameter growth, change in basal area per acre, and mortality over 5 and 10 year intervals for upland hardwoods in Michigan’s Northern Lower Peninsula. Analyze the effects of different projection cycle lengths on model prediction accuracy over a total projection length of 10 years. Compare the abilities of Lake States TWIGS and the FVS Lake States variant to accurately predict diameter growth, change in basal area per acre, and mortality over 5 and 10 year intervals for a variety of upland hardwood ecosystems. Examine the possible effects of drought and defoliation due to pest outbreaks on 5 year TWIGS prediction accuracy through the comparison of two different 5 year projections. LITERATURE REVIEW For the past 30 years, researchers have been developing computer models of fiiture stands, using past forest changes and mathematics as building blocks (Miner & Walters 1984). By 1981, research had produced over 100 growth simulation models for North American forest and tree species (T rimble & Shriner 1981). In recent years, more emphasis has been placed on constructing comprehensive projection systems that can better simulate the dynamics of a diverse resource base (Belcher et a1. 1982). Such simulators may be divided into two general categories: (1) stand-level models, and (2) individual tree-level models (Munro 1974). Stand-level models operate on entire stands, basically consisting of a series of equations which update stand level parameters over time. These models have primarily been developed for even-aged, single-species stands. However, modern planning efforts require increasingly detailed tree and stand information, which stand-level models cannot provide. Individual tree models, which use a series of equations to update individual tree parameters over time, can produce the detailed information needed. These models simulate forest stand growth by aggregating the projected growth of individual trees (Ek & Dudek 1980). The highly detailed stand descriptions provided by this type of model give it certain advantages over less detailed models for analysis of responses to alternative management treatments (Alig et al. 1984, Randall et al. 1988). Still, even with the advent of this type of model, economically 9 unimportant species, mixed stands, and forest resources other than timber received little attention for a long time (Belcher et al. 1982). In 1975, research was begun at the North Central Forest Experiment Station to develop a comprehensive system of tree growth projection for the Lake states region in order to fill this gap (Holdaway & Brand 1983). The individual-tree growth projection system developed represented just one component of a broader Forest Resources Evaluation Program (FREP). The goal of the F REP project was to provide the capability to simulate changes in all major forest resources (Belcher et al. 1982). This would serve two purposes. First, it would provide a mechanism for detailed updates of previously inventoried resources in order to estimate current resource levels. Second, it would allow the detailed projection of future resource levels based on known or estimated current levels. The ability to make projections with and without silvicultural treatments was incorporated in order to provide a basis for resource assessment. The tree growth projection system included in FREP was later modified and renamed STEMS, the Stand and Tree Evaluation and Modeling System (Belcher et al. 1982). STEMS is a distance-independent, individual tree growth model developed at the USDA’S North Central Forest Experiment Station in St. Paul, Minnesota. It was originally calibrated for two major geographic areas: the Pacific Northwest and the Lake States (Belcher 1981, Shifley 1981, USDA Forest Service 1979, Ek et al. 1980, Hahn et al. 1979, Smith and Raile 1979, Lundgren and Essex 1979, Holdaway and Brand 1983, Miner et al. 1988). STEMS can be used to project diameter growth and mortality for even-aged and uneven-aged, mixed species forest stands (Holdaway and Brand 1983). It 10 consists of two linked programs: one to simulate tree growth, mortality, timber management, and regeneration; and another which produces detailed summaries of the output. The results of a study by Holdaway and Brand (1983) into the predictive performance of STEMS led to the release of an updated version of the model in 1985. STEMS85 (Holdaway and Brand 1986) is identical to ST EMS in all but two ways: (1) a diameter adjustment factor was added to the growth component, and (2) the mortality model was modified. The diameter adjustment (Holdaway 1985) was added to reduce diameter prediction errors. The revised mortality model included an adaptation to more realistically predict survival rates for small trees and large, slow-growing trees (Buchman et al. 1983, Buchman 1983, Holdaway and Brand 1986). In 1988 the next version of the FREP projection system was released. This new version, known as TWIGS (The Woodsman’s Ideal Growth projection System), is the microcomputer version of the STEMS85 grth projection system (Miner et a1 1988). The Lake States version of TWIGS was specifically developed and calibrated for use in the north central states of Minnesota, Wisconsin, and Michigan. As in the case of the STEMS and STEMS85 models, the TWIGS model “grows” a forest stand through time based on the attributes of the individual trees in a stand (Kowalski and Gertner 1989). Updated tree and stand attributes, including stand and volume tables, can be obtained for any year in a projection. TWIGS also simulates silvicultural management prescriptions and performs economic analyses of these prescriptions. 1 1 Originally developed for the Lake States, TWIGS was recalibrated with data from throughout the Central States (Indiana, Illinois, and Missouri). This new version, known as Central States TWIGS, contains grth models similar in form to those in the Lake States variant, with mortality and other relevant equations altered or recalibrated for the Central States. This study focuses on the equations used to predict forest growth in the Lake States version of TWIGS. Growth and mortality equations are the core of TWIGS as a growth simulator. These equations were developed for the Lake States using measurements of 1,500 plots with over 80,000 trees, and for the Central States from measurements of 3,000 plots with over 60,000 trees (Miner et al. 1988). The model recognizes 31 different species groups in the prediction of individual tree characteristics. During the early 1980’s, the National Forest Systems (NF S) Timber Management Staff selected the individual-tree, distance-independent model form as the nationally supported framework for growth and yield modeling (Bush and Brand 1995). The modular structure and capabilities of an existing individual-tree, distance-independent grth and yield model developed for use in the Inland Empire area of Idaho and Montana, known as Prognosis (Stage 1973), were incorporated into the national model framework. This model was called the Forest Vegetation Simulator (F VS). A number of regional growth models including Lake States TWIGS were reformulated and embedded into F VS. Lake States F VS and Lake States TWIGS both use the same grth and mortality equations. However, there are five important differences between the two programs (Bush and Brand 1995): (1) Lake States 'I'WIGS predicts diameter growth and 12 mortality on a yearly cycle, while Lake States FVS predicts diameter growth and mortality on a ten year cycle, (2) Lake States FVS predicts individual tree height using equations developed by Ek et al. (1984)., while Lake States TWIGS does not consider height at all, (3) if diameter growth and/or tree height is included in the stand data for 5 or more trees of the same species, Lake States FVS recalibrates diameter growth and/or height growth for that species in accordance with the growth characteristics of the site, while Lake States TWIGS does not, (4) regeneration in Lake States TWIGS consists only of bare-ground planting using keywords, while natural regeneration can be simulated in Lake States F VS, and (5) in Lake States TWIGS, “ingrowth” trees of any diameter can be added once every cycle, while Lake States FVS does not have this capability. The data used to calibrate the different TWIGS variants came from across large regions. As Holdaway and Brand (1983) point out, a projection system that performs well on calibration data from across a region may still fail to satisfactorily fit data from independent subsets of the region. Miner et al. (1988) used this as an argument for using TWIGS to compare the relative differences in growth, yield, and economic effects between alternative management strategies, rather than as a tool for obtaining exact projections. Both Smith (1983) and Holdaway (1985) have questioned TWIGS’ ability to accurately represent local growth potentials. The coefficients used in the model reflect the average of the natural forces operating in these regions as a whole. The program therefore reflects average species growth and mortality for the region, and not for specific localities (Miner el al. 1988). Thus, the level of accuracy in the predictions may decrease 13 when the model is applied intensively to one or more sub-regions. This regional aspect of TWIGS is the first of many concerns with the model’s predictive ability. It is also important to consider the model’s performance outside the range of the calibration data set. This will provide users with an idea not only of how well the model firnctions, but also the range of conditions over which it functions well (Holdaway and Brand 1983). Smith (1983) points out that when a model is applied, either intensively within a small portion of the range of calibration or beyond the geographic area for which it was calibrated, it should not be unexpected for the model to perform poorly. Application of Lake States TWIGS to Michigan should be done with this in mind. Representation from Michigan’s Lower Peninsula in the data set used to calibrate the growth equations in the STEMS model was limited to red pine (Pinus resinosa) and jack pine (Pinus banksiana) plantations. Although additional plots were added for the TWIGS model, no hardwood data was included from the Lower Peninsula. Most of the data for the northern hardwood and oak-hickory timber types came from Wisconsin (Christensen et al. 1979, Smith 1983). As a result, using TWIGS for projections of northern hardwoods in the Lower Peninsula of Michigan is an extension of the model’s predictive abilities beyond its geographic range of calibration. The major underlying concern in this situation is that areas outside the range of calibration may have significantly different climate and soil factors (Holdaway and Brand 1983). Rauscher (1984) goes one step further, stating that the northern hardwood data used in the STEMS model calibration was taken from areas in Wisconsin with homoclines dissimilar to those of the Lower Peninsula of Michigan. These differences are in average seasonal 14 temperatures, precipitation, and solar radiation. The importance of macroclimatic influences on soil vegetation development has been discussed by Spurr and Barnes (1980), Rauscher (1984), Holdaway (1987), and Reed et al. (1992). Furthermore, early validation work on the STEMS model showed that STEMS predictive power decreased as the model application moved south and east in the Lake States region (Leary et al. 1979, Smith 1983). In addition to regional limitations, the fact that research plots were used to calibrate 'I'WIGS also raises questions about the models predictive ability. Data used in STEMS model development came from a variety of sources, including plot records from cutting experiments, demonstration woodlots, industrial continuous forest inventory, and personal records of forest growth. In all cases, the data came from permanent research plots (Christensen et al. 197 9). Differences between research plots and managed forests result primarily from differences in scale and quality control of management treatments, uniformity of stands, ease of access, and the amount of natural damage (Bruce 1977). Models developed from such data often have a tendency toward over-prediction of individual tree and stand-level characteristics (Holdaway 1985, Bruce 1977). Another potential source of error in model prediction are the assumptions built into the model. Holdaway and Brand (1983) point out that models can be expected to fail on some occasions due to an oversimplification of relationships, situations rarely encountered and often not properly modeled, or extreme conditions which are often overlooked. Some assumptions imbedded into ST EMS, and therefore TWIGS, simplify to a large degree the integration of several forest factors. These include soil, insects, 15 disease, competing vegetation, and land use (Miner and Walters 1984). This alone has led to arguments against using STEMS and TWIGS management prescriptions as anything other than guidelines for forest management. In addition to the regional nature of TWIGS, the lack of hardwood data from Michigan’s Lower Peninsula in the calibration data set, the use of research plot data in model calibration, and the assumptions made by the model, another potential source of prediction error which must be considered is the model’s structure itself. Lake States TWIGS uses a set of three linked equations to form its annual diameter growth model (Miner et al. 1988): Annual Diameter Growth = Potential Growth * Competition Modifier + Diameter Adjustment Factor (1) Potential Growth = b1 + szb3 + b 481 * CR * Db5 (2) Where : Potential growth = potential annual dbh growth (inches/yr.) D = current tree diameter (dbh in inches) SI = site index (feet at age 50) CR = tree crown ratio code (0-10%=1, 11-20%=2, 71-80%=8, 8l%+=9) bl b5 = species specific equation coefficients Source : Hahn and Leary (1979) 16 Competition Modifier = 1 - e'f(R)g(Am“SQRTKBAmax "BM/BA] (3) Where : Competition modifier = an index of competition, bounded by 0 and l BAmax = maximum basal area (f’t2 /acre) expected for the species BA = current basal area (ft2 /acre) R = ratio of an individual tree’s dbh to the average stand diameter AD = average stand diameter (inches) f(R) = a function characterizing the individual tree’s relative diameter effect on the modifier : f(R) = b1[1- e"2R ]"3 + b 4 g(R) = a firnction characterizing the average stand diameter effect on the modifier : g(AD) = cl (AD +1)c2 bl b 4,cl ,c2 = species specific equation coeflicients Source : Holdaway (1984) Diameter Adjustment Factor = a1D + azD2 + a 3 (4) Where : Diameter Adjustment Factor = adjustment in annual dbh growth (inches/yr.) D = current tree diameter (dbh in inches) al ,a2 ,a3 = species specific equation coefficients Source : Holdaway (1985) 17 When 'I'WIGS is used to simulate growth over multiple projection cycles, a mortality function interacts with the diameter growth model to simulate individual tree level mortality. The TWIGS mortality firnction is: Survival = bl - [l / (1 + e" )] (5) Where : Survival = the tree’s annual probability of survival n = b2 +b3(DGR)b4 +b5(D-l)b" *e'b7(D") DGR = predicted annual diameter growth (inches) D = current tree diameter (dbh in inches) bl b7 = species specific equation coefficients Source : Buchman et al. (1983), Buchman (1983), Buchman and Lentz (1984) Sources of error related to this model structure are found in the form of suspect variables in the diameter growth model, and in the way the different equations interrelate. Of the three linked equations which comprise the annual diameter growth model, both the competition modifier and the diameter adjustment factor are primarily firnctions of individual tree diameter and/or average stand diameter. The potential growth model is a fiinction not only of current tree diameter, but also of site index and crown ratio (For information on the goodness of fit of the potential growth model see Table 1. in Hahn and Leary 1979). It is these latter two variables which present potential sources of error. 18 TWIGS uses site index (base age 50 years.) as a measure of site quality. However, site index only has meaning in fully stocked, undisturbed, even-aged stands, in which the dominant and codominant trees measured are free-growing, uninjured, and of a known age. A result of these strict standards is that few stands are actually eligible for site index classification. Coupled with this dilemma are the concerns brought up by Carmean (1975) and Monserud (1984) relating to other factors such as site preparation, brush competition, climatic variation and genetics, which have a large effect on the site index of a stand. All these factors bring into question the application of site index curves to stands other than those from which they were derived. TWIGS requires either stand site index or, in the case of a mixed-species stand, the site index for one of the species present in the stand. In the latter case, Lake States TWIGS uses equations to find the site indices for all species present in the stand from the site index provided for one of the species present (Carmean 1979, Carmean 1982, Carmean and Vasilevsky 1971). The potential for error is evident through the differing site requirements of different species, and the concerns mentioned above. An additional concern is that, if site index is not supplied, TWIGS will use the input data to estimate it, resulting in growth estimates based on other estimates. Any errors induced into TWIGS projections by this estimation will then be compounded over the span of the projection. Crown ratio is defined as the percentage of length of the tree’s stem clothed with living branches (Smith 1962). This variable is often not recorded during forest inventories. When it is recorded, it is generally estimated ocularly, opening the door for error. As in the case of site index, if crown ratio data is not supplied, TWIGS will 19 estimate it, resulting again in growth estimates based on estimates. Validation of the crown ratio model for Michigan’s Northern Lower Peninsula showed that the model overestimated crown ratio values by 0.32 units (Holdaway, 1986). Looking at the role this variable plays in the potential growth equation, errors of this magnitude can have a very noticeable effect on growth predictions. The models used in TWIGS for estimating annual diameter growth are directly or indirectly functions of current diameter, and in the case of the potential growth firnction, site index and crown ratio as well. The potential growth function provides the basis for each projected dbh. This initial growth estimate is then modified into its final form by the competition modifier and diameter adjustment factor. The important roles that site index and crown ratio play in the potential grth function demonstrate their abilities to create large errors in TWIGS predictions. With all of the potential sources of error, the accuracy of TWIGS’ grth predictions can easily be called into question. Before a model can be used effectively, it must be evaluated. The purpose of model evaluation is to increase one’s confidence in the predictive capabilities of the model (Holdaway and Brand 1983, Schaeffer 1980). The need for an evaluation of the equations contained in the TWIGS model is, therefore, clear. Growth models predict growth and mortality of some stands better than others. Forest managers must have information about the accuracy of model predictions for their region, and if possible sub-region, in order to justify using the results of the model’s projections in their management (Brand and Holdaway 1989). 20 Buchman and Shifley (1983) have outlined several key areas to examine when evaluating a forest growth and yield projection system. In addition, validation procedures have been described in articles by Snee (1977), Reynolds (1984), and Burk (1986). Model accuracy is considered to be composed of two parts: bias, the average difference between the true value and the estimate; and precision, estimated by the standard deviation of the differences around the average (Husch et al. 1982, Rauscher 1986, Brand and Holdaway 1989). Therefore, an accurate model is unbiased and precise, with bias close to zero, and with little variability among the individual differences. ATEST, a program developed by Rauscher (1986), and the SASATEST template written by Gribko and Wiant (1992), provide a simple means of comparing predictions with validation data to determine model accuracy. This program computes bias and precision as well as confidence and prediction intervals for sample data. Until very recently, Lake States TWIGS had never been evaluated at a sub- regional level in Michigan. Early testing was done at a regional level. Holdaway and Brand (1983) set the stage for validation of the models contained in STEMS and TWIGS with their evaluation of the STEMS model in the Lake States. They decided that certain key variables covering the most important aspects of stand and tree growth, and which were directly estimated quantities, should be considered in judging the performance of a tree growth model. At the individual tree level, the variable tree diameter at breast height (DBH) was selected to evaluate the growth component of the model. DBH is important in management strategies, tree volumes, and mortality estimation. At the stand level, Holdaway and Brand decided the number of trees per acre (TPA) and stand basal area 21 per acre (BA) should be examined. TPA is used to evaluate the mortality component. BA, an important measure of stand density, is used to evaluate the combination of the diameter grth and mortality components, thereby providing information on overall model performance. The STEMS model validation done by Holdaway and Brand (1983) at the regional level showed that, while the model did a good job of predicting the three evaluation variables at the regional level, prediction errors for diameter grth and BA were highest in the Manistee National Forest, the only source of data fiom Michigan’s Lower Peninsula. Diameter growth errors for this forest (overprediction of 0.28 inches over a 10 year period) were more than twice as large as those of any other forest. Ten- year mean diameter growth errors for sugar maple (Acer saccharum), basswood (T ilia americana), white ash (Fraxinus americana), and several members of the red oak group were all greater than 0.45 inches. Considerable errors were also found in predictions of northern red oak (Quercus rubra) mortality. An evaluation of the STEMS85 model, also conducted by Holdaway and Brand (1986), showed that the addition of the diameter adjustment factor and the modified mortality model noticeably improved model predictions in several areas. The same five forests examined in the STEMS validation: Cloquet Experimental Forest - Minnesota, Chequamegon National Forest - Wisconsin, Nicolet National Forest - Wisconsin, Hiawatha National Forest - Upper Peninusala of Michigan, Manistee National Forest - Lower Peninsula of Michigan, were used to evaluate STEMS85. Holdaway and Brand found that the new version of the model consistently predicted slower grth than the 22 original STEMS model for all five forests. While diameter growth was consistently overpredicted in the STEMS validation study, STEMSSS was found to underpredict 10- year diameter growth by between 0.03 and 0.12 inches across four of the forests. Diameter growth for the Manistee National Forest was still overpredicted by an average of 0. 15 inches. The average TPA error of prediction for the Manistee National Forest (12) was the second largest of the five forests, while its average BA error of prediction (7.1 ft2 lacre) was more than three times as large as that of any of the other four forests. A validation study of the Central States TWIGS model was done by Kowalski and Gertner (1989) to evaluate the model’s predictive ability for Illinois forests. Using error statistics similar to those in ATEST, they looked at errors across quadratic mean diameter measurements as well as TPA and BA. They concluded that, in general, TWIGS did an adequate job of predicting typical Illinois forests, with relative diameter and basal area errors of prediction less than 10% of the true mean. Tree survival predictions were less precise, with errors up to 22% of the true mean. However, in general errors for several economically important hardwood species were less than 12% over 10 years. A study by Gertner, Rink and Budelsky (1989) indicated that the primary source of variability in projections made with STEMS was the individual tree mortality model. They suggested that the most efficient way to improve the predictive ability of STEMS would be to improve the mortality model through additional data collection and model restructuring. The first sub-regional validation of the TWIGS model in Michigan was published in 1996 (Guertin and Ramm). Following the methods of Holdaway and Brand in the 23 STEMS and STEMSSS validation studies, predictions errors for five-year diameter growth, BA, and TPA were examined for five species groups across 44 stands in Michigan’s Manistee National Forest. The results of this study indicated that TWIGS overpredicted diameter growth across several species and size-class combinations. Errors were found, in general, to be larger for many species than those produced in the STEMS85 validation study, despite what were generally larger sample sizes than used in the STEM885 validation. However, diameter growth model performance was judged to be acceptable. Mean five-year BA estimates were found to be within 5.0 fiz / acre for all species examined, with the largest error (-4.4 ft2 lacre) occurring in the “other red oak” group (black oak and n. pin oak). Model performance was judged to be acceptable for all species except sugar maple, which had a small mean bias but a very large standard deviation. Errors for five-year predictions of TPA ranged between -33 TPA for the “other red oak” group, and +16 TPA for sugar maple. Very large standard deviations of prediction errors were also noted for several species. TWIGS mortality model performance was judged to be unacceptable for five-year projections for the five species examined. Several studies have compared the predictive abilities the STEMS and TWIGS models to those of other growth and yield models. Crow (1986) compared growth estimates of STEMS and the Sugar Maple Projection System (SMPS) for stands dominated by sugar maple (Acer saccharum). Using ratios of predicted to actual net growth, both models were found to perform equally well. Neither model, however, gave reliable estimates of mortality. 24 Perkey and Carvell (1988) compared 10, 15 and 25 year ‘I'WIGS model BA projections with those of two other stand level growth models (GROAK and GROPOP) for cove broadleaf and upland oak stands in Morgantown, West Virginia. Results indicated that GROAK performed the best at predicting BA growth in upland oak stands, while TWIGS overpredicted growth by up to 15%. Schuler et al. (1993) compared stand level development of North East TWIGS (NE-TWIGS) with three other growth simulators: SILVAH, FIBER, and OAKSIM. Results indicated that NE-TWIGS performed well for beech-birch-maple (Fagus-Betula- Acer) and cherry-maple (Prunus serotina-Acer) forest types. However, SILVAH and OAKSIM performed better than NE-TWIGS in oak-hickory (Quercus-Catya) forest types. Desanker et al. (1994) compared the predictive abilities of several modeling approaches to evaluating forest site, physical, chemical and climatic stresses. They used intensive measurements of tree growth and weather conditions in northern hardwood stands, primarily red oak (Quercus rubra) and red maple (Acer rubrum) in Michigan’s Upper Peninsula, for their comparisons. One of the empirical models examined was STEMS. The results of their study indicated that all models, including STEMS, performed poorly when compared to observed values. Methods have been developed for improving TWIGS estimates for local conditions with additional diameter growth information. Smith (1983) used data from Michigan’s Upper Peninsula and double sampling with ratio of means estimators (Cochran 1977) to reduce diameter errors of prediction. This method uses the ratio of 25 actual to predicted diameter growth for the individual trees on a study plot to produce an adjustment factor. It is applied by multiplying the adjustment factor with the new dbh predictions. Smith reported a 94% decrease in mean annual diameter growth error, from 0.033 inches to -0.002 inches. Gertner (1984) used a sequential Bayesian procedure to improve local predictions of a regional diameter increment model taken from a version of STEMS used in Oregon. This method adjusts regional model parameters in accordance with the precision of sub-regional estimates of diameter growth. Holdaway (1985) developed an additive diameter adjustment factor for the STEMS model using Wisconsin data, which has been incorporated into the TWIGS model. The result was a reduction in the ten year mean diameter growth error of prediction from 0.17 inches to 0.03 inches. Payendeh and Papadopol (1994) adjusted coefficients for the potential diameter growth and tree survival equations in Lake States TWIGS. The adjusted models improved local predictions to within 5% of observed values for spruce-fir, black spruce and red pine plantations in Ontario. While the previous validation studies provide evidence of STEMS and TWIGS weaknesses at sub-regional prediction in some areas, these methods demonstrate how adjusting the equations used in these models can improve local predictions. In addition to modifying existing regional grth and yield models for local growth potentials, a common alternative is to develop new individual tree distance- independent models for projecting diameter growth. Several methods have been reported, ranging from creating growth tables to building complex models which predict fiiture diameter growth. Smith and Shifley (1984) compiled data from over 15,000 trees 26 by diameter class to produce forecasting tables for lO-year growth and yield in Indiana and Illinois forests. Other researchers have taken a different approach and developed new distance independent individual-tree diameter growth models using a variety of different methods. Hahn and Leary (1979) tested a number of different models when developing the original STEMS model, relying on residual plots and coefficients of determination (R2) to determine which model provided the most accurate predictions. The Central States TWIGS potential diameter growth model (Shifley 1987, Miner et al. 1988) is based on a modified Chapman-Richards fiinction (Pienaar and Trumball 1973), with additional variables added to simulate the effects of crown ratio and site index. This same model has been used as the basis for other models as well. Shifley and Brand (1984) proposed using a version of this model, modified by maximum tree size, to predict change in individual tree basal area and diameter over time. Hilt and Teck (1987) also proposed using a modified version of this growth model to predict diameter grth of hardwoods in New England. Daniels and Burkhart (1988) used a modified Chapman-Richards fiinction to model tree basal area and height growth in loblolly pine (Pinus taeda) stands. Zeide (1989) looked at accuracy of diameter growth predictions in the northwest for a modified form of the Chapman-Richards equation, as well as models developed using logistic regression, a modified Weibull equation, and an exponential fiinction. The growth function in the Prognosis model was used as the basis for the development of a model for conifers in the Inland Empire (eastern Washington, northern Idaho, and western Montana) (Wykoffet al. 1982, Wykoff 1990). In several cases besides Zeide’s 27 study (1989) regression has been used to develop models (West 1981, Mawson 1982, Harou, Mack and Mawson 1985, Harrison et al. 1986, Johnson and Weigel 1990, Avila and Burkhart 1992, Ritchie and Powers 1993, Hynynen 1995, Hasenauer and Monserud 1996). Attempts have also been made to combine regression techniques with cluster analysis (Bolton and Meldahl 1990) and to use multivariate linear models (Guertin 1993). A number of multi-stage models (Hilt 1983, Hilt 1985, West 1981) also exist for predicting either basal area or diameter growth. Despite this proliferation of models, none have been shown to provide more accurate estimates of hardwood growth and yield in Michigan’s Lower Peninsula than TWIGS. For this reason, TWIGS remains the best alternative, and the need for validation of this model for growth estimates over a period greater than five years is even more evident. METHODS Field Methods and Data Collection Data The data used in this study came from a subset of 76 stands sampled and used in the development of an ecological classification system (ECS) for the Huron-Mansitee National Forests (Cleland et al. 1993). Stands in the ECS project were selected based on criteria set up in order to minimize the differences between stands. The criteria used were: 1) Stand ages between 60-80 years old. 2) High stem density (“normal stocking”). 3) Minimal evidence of disturbance since stand establishment. 4) Minimum size of 2.47 acres. 5) Less than 30 ft2 /acre of stand basal area in Aspen (Populus grandidentata and Populus tremuloides). These criteria, along with landforrn maps, aerial photos, and USDA Forest Service database information were used to develop a list of suitable stands (Guertin 1993). Stands used in the ECS development project were chosen from this list using stratified random sampling, with strata defined as the three major landforms of the area: outwash plains, ice contact hills, and moraines (Host et al. 1988). Once a stand was located, 28 29 evaluated, and accepted, three to five satellite points were randomly placed about a central soil pit point within each stand. Soil, ground flora, understory, and overstory information were recorded at each point. The overstory was sampled using a 10 BAF (English) angle gauge at the point center. Overstory information recorded included individual tree species, diameter at breast heightl (Dbh) (inches), total height (feet), merchantable height2 (feet), crown ratio (%), and crown class (Appendix Table 1) for all tally trees with Dbh greater than 1 inch. Individual species site index was estimated for the species present at each point using increment core data from a sub-sample of trees. Site index was determined using equations developed by Hahn and Carmean for the Lake States (1982). Further details on stand selection and sampling procedures are provided by Host et al. (1987). In 1986, stratified random sampling was used to choose 50 northern hardwood stands from the 76 ECS stands for further study. The strata used were 10 ecological land type phases (ELTPS) defined during the ECS development (Table 1.). All stands fell within the boundaries of Wexford, Mansitee, Lake, Mason and Newaygo counties in the northwest portion of Michigan’s Lower Peninsula (Figure 1). Table 2 shows the average stand density, tree size, age, and site quality by ELTP for the 50 stands chosen. Stand density ranged from 53-140 ft2 / acre mean BA and from 102-484 TPA. Mean stand Dbh ranged from 34-162 inches. Stand age ranged from 56-95 years. Site indices (base ‘ Diameter at breast height (Dbh) is measured 4.5 feet above ground level on the uphill side of a tree (Avery & Burkhart 1994). 2 Merchantable height is the height of a tree measured from a one foot stump to a four inch top diameter. 30 Table 1. Descriptions of Ecological Land Type Phases (ELTPS) included in this analysis ELTP Description (Overstory/understory species group) EL T: Outwash Sand Plains 1 Pin Oak/White Oak - Dechampsia 10 Black Oak/White Oak - Vaccinium 12 Black Oak/White Oak - Vaccinium with perched water table EL T: Dry Ice Contact and Sand Hills 20 Mixed Oak/Red Maple - Trientalis 21 Mixed Oak/Red Maple - Trientalis with loamy sand/sandy loam bands EL T: Mesic Ice-Contact and Sand Hills 35 Red Oak/Red Maple - Viburnum with perched water table 37 Red Oak/Red Maple - Desmodium with fine loamy substrata ELT: Herb Poor Moraines 40 Sugar Maple/Beech - Maianthemum 43 Sugar-Maple/Red Oak - Maianthemum with fine textured substrata EL T: Herb Rich Moraines 45 Sugar Maple/White Ash - Osmorhiza with perched water table age 50) ranged from 45-106 ft. for northern red oak, from 27-77 ft. for white oak, and from 49-83 ft. for sugar maple (Guertin and Ram 1996). In comparison, the stands used to develop Lake States TWIGS had stand BA ranging from 25-150 ft2 / acre , mean stand Dbh from 40-100 in., and site indices for all species from 45-75 ft. (base age 50) (Miner et al. 1988). The stands selected were revisited and plot centers were located, mapped, and permanently marked. Marking included installing a 4 fl. section of rebar at the plot center, tagging tally trees with numbered aluminum tags at stump level, and marking 2 to 3 witness trees around each point center with marking paint so that triangulation could be used in the fiiture to find plot centers. Of the 50 stands selected, 49 were relocated in 1986. However, five subplots across four stands were not relocated and were dropped from the study. A 10 BAF angle gauge was again used to sample the 31 e .. C . . . 0:. O. . Q o o . . '. o . O . . e e .e e . O o e e e o :0 '00 . Manistee National , 0 Forest Boundary 0 Validation Plot Figure 1. Location of '1 'WIGS Validation Plots in the Manistee National Forest 32 Table 2. Sample sizes, stand density, average size, age, and site quality by species and ecological land type phase (ELTP) in 1986. Site indexl ELTP n2 BA 3 TPA4 DBH AGE wo NRO BO RM SM 1 16 83 283 7.3 75 42 51 1o 14 106 491 6.3 83 39 54 49 12 12 95 288 7.8 74 49 59 20 20 115 400 7.2 82 52 6O 59 48 21 17 111 376 7.3 82 51 61 64 35 27 139 340 8.7 75 65 80 37 20 113 147 11.9 78 64 87 67 4o 22 109 376 7.3 65 76 61 64 43 11 125 354 8.0 67 89 72 45 24 128 355 8.1 66 88 72 Overalle 112 341 7.8 75 52 74 56 59 69 1 Site index in feet, base age 50, for white oak (W O), northern red oak (NRO), black and pin oaks (BO), red maple (RM), sugar maple (SM). 2 Number of sample points per ELTP. 3 Basal area in ft2 / acre. 4 Trees/acre, calculated for all trees 2 1.0 in. Dbh. overstory (Guertin and Ramm 1996). All “in” (tally) trees with a 1 in. Dbh or greater were measured for the same variables as during ECS development, and were referenced by their aluminum tag number. All data recorded from the 1986 field season was compiled into a database with each record containing information for a single tree. Information compiled included stand number, subplot number, tree number, tree species, Dbh, total height, merchantable height (to a 4 in. top diameter), crown ratio and crown class. Bark thickness and Girard Form Factor were estimated for a few trees at each soil pit point. The total number of tree records in the database at this time was 2,434. Plots were revisited and remeasured in 1991, with individual tree numbers and diameters referenced to 1986 tally sheets while on site. Of the 49 stands located in 1986, 33 48 were relocated in 1991. Vandalized plots with missing plot centers were noted in plot records and estimates of BA were taken from estimated plot center locations. A 10 BAF angle gauge was again used to locate all tally trees. Trees found to be “in” were measured for Dbh, crown ratio and crown class, with total height and merchantable height also estimated for all in trees at soil pit plots. Each tree over 1.0 in. and “out” (not a tally tree) in 1986, but “in” in 1991, henceforth called a “moregrowth” tree, was measured for the same characteristics as tally trees at soil pit plots. The 1986 database was updated to include all 1991 measurements on previously sampled trees, as well as all new moregrowth tree records. With the addition of 65 moregrowth trees in 1991, the total number of individual tree records in the database increased to 2,499. 1996 Field Measurements Plots were revisited and remeasured in 1996, with individual tree numbers and Dbh’s referenced to 1991 tally sheets while on site. A1148 stands located in 1991 were relocated in 1996. All missing plot centers were reestablished using 1986 Dbh measurements and limiting distances. Missing plot centers were re-marked with 3.3 ft. sections of 3/4 in. PVC piping. All plot centers were referenced using a GPS receiver to facilitate filture plot relocation. The receiver model used was a Trimble© GeoExplorer GPS unit in 3-D mode with a maximum positional dilution of precision (PDOP) setting of 6.0 and an elevation mask of 15 degrees. In addition, witness trees at each point were marked with two 4 in. dots of orange fluorescent tree marking paint below the stump line. 34 Variables measured at each plot were identical to those measured in 1991. Dbh measurements were made at 4.5 ft. above the forest floor on the uphill side of each tree using the diameter scale on a Spencer 100 ft. logger’s tape. In the event of cankers or other deformities at Dbh, adjustments were made to the point at which tree diameter was measured, and were recorded on plot tally sheets. Heights were estimated using a Spiegal Relaskop at a horizontal distance from the tree of 66 ft. for trees with total heights less than 60 ft, and at a distance of 100 ft. for trees with total heights greater than 60 ft. Crown ratio and crown class were estimated ocularly for each tree. A new database was constructed at the end of the field season with all 1996 measurements as well as 1991 and 1986 Dbh measurements. Records for trees from the original ECS project which either died or were out before 1986 (78), and tree records from the one stand relocated in 1986 (71), but not in 1991 were not included. With these adjustments and the addition of 1996 moregrowth trees (247), the new database consisted of 2,597 tree records from 48 stands. Several plots experienced disturbances between 1991 and 1996. In each case, whether due to forest users or natural forest pests, the type and level of disturbance was recorded on plot tally sheets. American basswood trees (T ilia americana) across a number of sites showed signs of Lindon looper (Erannis tiliaria) defoliation. Gypsy moth (Lymantria dispar) egg sacs were noted in several oak stands, as was heavy defoliation of scattered white oak trees in a few stands. Defoliation was recorded as an ocular estimate of the percentage of the crown which had been defoliated. Human disturbances included active forest management, vandalism, and littering. 35 Stands with evidence of thinning or other forms of management (small clearcuts, slash piles, etc.) were noted on plot tally sheets. Vandalism occurred on several plots, and consisted of missing aluminum tree tag numbers, and in some cases rebar plot centers. These plots were generally close to roads. In instances where aluminum tag numbers were missing, trees were easily relocated and tags replaced due to the fact that plots were originally point sampled starting from true north and rotating clockwise. Plots with missing plot centers were handled as stated previously, with plot centers reestablished using limiting distances and 1986 Dbh measurements. In addition to these forms of disturbance, two stands showed signs of trash dumping. Data Compilation To match the validation approach used in the STEMS85 analysis and thereby allow for the comparison of results, stands showing evidence of recent disturbance were removed from the study. Stand records and personal communication with personnel at the Forest Service Manistee-Cadillac and Baldwin-White Cloud ranger districts revealed that subplots in some stands had been affected by management during the course of the study. Plots affected by management were ineligible for the Dbh, BA, and TPA analyses due to the effect that management has on the diameter growth of residual trees. After discussing this matter with Dr. Don Dickmann, silviculturalist at Michigan State University, it was decided that any subplots less than two chains (132 ft.) fi'om the edge of a clearcut should be excluded from the study. Previous field season tally sheets were used to determine which subplots were made ineligible for study due to vandalism. These plots were not used in the BA and TPA analyses, due to the inability to locate 36 moregrowth trees in 1986 and 1991 without the established plot center. However, trees on these plots were included in the Dbh analysis. Appendix Tables 2-4 provide lists of which plots were included in the different analyses for the different simulations. Upon examination after the 1996 field season, certain trees appeared to exhibit aberrant diameter growth, with 1991 measurements being either larger than 1996 measurements by one or two tenths of an inch, or smaller than 1986 measurements by one or two tenths of an inch. This is most likely attributable to a slight difl’erence in field procedures for 1991 and the other two field seasons. Diameters were measured in 1991 at 4.5 ft. above ground level on the side of the tree facing the plot center. In accordance with standard procedures (Avery and Burkhart 1994) diameters were measured during the other two field seasons at 4.5 ft. above ground level on the uphill side of each tree, regardless of plot center location. In cases showing aberrant growth, it was decided that the 1991 measurements would be recognized as the source of error due to the non- standard sampling method used. All trees with diameter growth following this pattern (59 trees) were left out of validation tests for TWIGS’ diameter growth fiinction in simulations either starting or ending in 1991. In order to prevent disruption of the BA and TPA analyses, the 1991 diameter measurements for these trees were adjusted. After discussing these situations with Dr. Don Dickmann, it was decided that in cases where 1991 Dbh measurements were less than 1986 measurements, the 1991 Dbh measurements should made equal to the 1986 measurements prior to growth simulation. The intent was to provide the closest possible value to the recorded measurement, and still show realistic growth. In cases where 1991 Dbh measurements were greater than 1996 Dbh 37 measurements, 1991 measurements were made equal to 1996 measurements for the same reason. In four isolated cases, 1991 Dbh measurements were wildly inconsistent with measurements made in 1996. These trees were also removed from the diameter growth firnction validation. Approximate 1991 Dbh values were estimated fi'om growth data for other trees of the same species on the plot, and these values were used in the BA and TPA analysis. Tables 3-5 provide means for observed individual-tree Dbh growth by species and size-class, and plot-level BA growth and change in TPA by species for the periods from 1986-1991 and 1991-1996. Five of the seven species as well as all species combined experienced increases in average five-year diameter growth rate during the second five- year period. Plot-level average five-year BA grth rates also increased for five of seven species and all species combined. Decreases in average BA growth rates for two species were reflected by similar decreases in five-year changes of plot-level TPA for the same species. FVS & TWIGS Simulation The Lake States TWIGS variant of FVS is the new USDA Forest Service endorsed growth and yield projection system for Michigan. Until recently, this honor had gone to the Lake States TWIGS model. While the two growth simulators employ the same equations, they are not identical. As a result, it was decided that a comparative analysis of the predictive abilities of the two would be useful. In addition, as it is new and until now untested in the Lake States, it was also decided that a more in-depth analysis of Table 3. Mean diameter (Dbh) growth and standard deviation (5) of mean diameter growth (in.) 38 for each of the two five-year periods of the study (1986-1991, 1991-1996). 1986-1991 1991-1996 Diameter Mean Dbh Mean Dbh Species Class n I Growth 8 11 Growth 3 N. Red Oak 3.0 - 4.9 16 0.07 0.07 8 0.19 0.06 5.0 - 10.9 226 0.35 0.20 187 0.35 0.18 11.0 - 16.9 330 0.65 0.26 352 0.57 0.26 17.0 + 132 0.94 0.38 173 0.79 0.28 All 706 0.59 0.35 721 0.56 0.29 Other Red Oak 3.0 - 4.9 10 0.14 0.08 5 0.16 0.09 5.0 - 10.9 95 0.34 0.21 79 0.40 0.20 11.0 - 16.9 109 0.53 0.19 110 0.56 0.22 17.0 + 30 0.60 0.23 37 0.66 0.24 All 244 0.45 0.23 231 0.51 0.24 White Oak 3.0 - 4.9 27 0.11 0.11 17 0.12 0.08 5.0 - 10.9 197 0.18 0.15 166 0.22 0.18 11.0 - 16.9 104 0.39 0.20 97 0.35 0.19 17.0 + 8 0.33 0.26 10 0.44 0.17 All 338 0.24 0.20 292 0.27 0.20 Sugar Maple 3.0 - 4.9 27 0.13 0.15 21 0.14 0.14 5.0 - 10.9 166 0.29 0.24 152 0.40 0.23 11.0 - 16.9 42 0.50 0.22 53 0.79 0.37 17.0 + 4 0.70 0.29 5 0.66 0.15 All 244 0.31 0.26 239 0.50 0.74 Red Maple 3.0 - 4.9 48 0.24 0.19 37 0.22 0.13 5.0 - 10.9 68 0.39 0.24 82 0.44 0.24 11.0 - 16.9 23 0.56 0.36 25 0.66 0.28 17.0 + 6 0.73 0.40 10 0.87 0.51 All 151 0.37 0.29 159 0.44 0.30 Basswood 3.0 - 4.9 1 0.10 -- 0 -- -- 5.0-10.9 20 0.13 0.19 16 0.15 0.19 11.0 - 16.9 55 0.31 0.22 56 0.31 0.22 17.0 + 5 0.28 0.20 6 0.48 0.45 All 81 0.26 0.22 78 0.29 0.25 American Beech 3.0 - 4.9 5 0.24 0.05 6 0.30 0.13 5.0 - 10.9 24 0.33 0.17 23 0.33 0.22 11.0 - 16.9 24 0.49 0.25 27 0.39 0.24 17.0 + 3 0.43 0.12 4 0.52 0.25 All 58 0.38 0.22 62 0.36 0.23 All Species 3.0 - 4.9 137 0.16 0.16 95 0.19 0.14 5.0 - 10.9 857 0.29 0.22 752 0.34 0.22 11.0 - 16.9 747 0.55 0.27 778 0.54 0.28 17.0 + 204 0.82 0.40 265 0.74 0.30 All 1962 0.43 0.32 1911 0.47 0.38 1 Number of trees. 39 Table 4. Mean change in trees/acre (TPA) and standard deviation (s) of mean change in trees/acre for the two five-year periods of the study (1986-1991, 1991-1996). 1986-1991 1991-1996 Mean Mean Species nl Change TPA s n Change TPA S N. Red Oak 96 -16.8 45.8 97 -17.2 46.8 Other Red Oak 59 -5.2 23.6 53 -17.3 30.9 White Oak 91 -15.7 54.0 83 -20.1 58.1 Sugar Maple 36 -29.5 103.0 32 8.2 133.9 Red Maple 67 -8.0 22.7 70 41.7 216.4 Basswood 18 -4.0 14.5 15 -2.6 10.7 American Beech 20 57.2 285.5 17 -6.0 12.2 All Species 146 -20.9 145.2 136 4.6 227.1 1 Number of sample points. Table 5. Mean basal area (BA) growth and standard deviation (s) of mean basal area growth (ft2 / acre) for the two five-year periods of the study (1986- 1991, 1991-1996). 1986-1991 1991-1996 Mean Mean Species 111 Change BA S n Change BA 8 N. Red Oak 96 0.5 7.3 97 2.3 7.7 Other Red Oak 59 1.4 5.1 53 -2.5 7.8 White Oak 91 -0.9 5.1 83 -2.2 9.0 Sugar Maple 36 -0.8 7.3 32 5.9 9.8 Red Maple 67 0.9 3.4 70 3.3 8.1 Basswood 18 -1.1 4.7 15 0.0 3.8 American Beech 20 0.5 2.2 17 0.6 2.4 All Species 146 0.3 9.2 136 2.8 13.5 ‘ Number of sample points. 40 the power of Lake States FVS would also be done. This was carried out by varying the amount of input data included in the projections and examining the mean and standard deviation of the bias of prediction which resulted. Four sets of simulations were run, differing by length of the projection period, and projection cycle length. The first set of simulations projected growth from 1986 to 1996 over a single lO-year cycle, implying that growth estimates for 1996 were directly estimated from 1986 field measurements. The second set of simulations also projected growth fiom 1986 to 1996, this time over two consecutive 5-year cycles. In this scenario 1986 field measurements were used to estimate growth for 1991. The 1991 estimates were then used as the starting point for a second projection, estimating growth from 1991 to 1996. The third set of simulations projected grth from 1986 to 1991 over a single 5-year cycle. The fourth set of simulations also used a single 5-year cycle to project growth from 1991 to 1996. These four sets of simulations were done in order to analyze the effect of cycle length on prediction accuracy and to compare grth estimates over two five-year periods experiencing different climatic and natural pest conditions. Miner et al. (1988) claimed that the best results using TWIGS, which predicts diameter growth on a yearly cycle, are achieved with projection intervals of five years or less. A comparison of the first two sets of simulations was used to determine if best results using F VS, which predicts diameter growth on a ten year cycle, are achieved the same way. As stated above, the cycle length for the first set of simulations was 10 years, while the cycle length for the second set was five years. The third set of simulations was run in order to provide 41 an indication of F VS’ prediction accuracy for a period experiencing both severe drought (1988) and defoliation due to tent caterpillar (1987) and gypsy moth (1991) infestations. The fourth set of simulations, for the period from 1991-1996, provided a control for comparison with the results of the 1986 to 1991 simulations. Neither drought, nor significant pest infestations were experienced during this period. Results of FVS prediction accuracy tests for this period were used to determine whether or not these events have an effect on the level of accuracy achieved by the program. Within each set of FVS simulations, three levels of simulation were run, difl’ering in the amount of stand and tree-level information included. In addition to information for the different variables recorded for each tree at each sample point, the first simulation level in each set (“Full”) also included both stand age and stand site index, determined during the development of the ECS project, as well as individual tree diameter growth information, determined from previous measurements. The second simulation level (“noDI”) included the same information as the first level, with the exception of the individual tree diameter growth data which FVS uses to recalibrate its diameter growth function. Of all the information included in the study, it was decided that this information was least likely to be available to most forest managers. This second simulation level was run in order to examine the effect of leaving this information out on prediction accuracy. The third simulation level (“noSI,A,DI”) included the same information as the first level with the exclusion of the individual tree diameter growth data and the site index and age estimates for each stand. It was decided that site index and age information were the next least likely variables after diameter growth to be available to forest managers. This 42 simulation level was run in order to examine the level of prediction accuracy which may be achieved when these data are not available. This made for a total of 12 difl‘erent FVS simulations; four sets of simulations with three simulation levels in each set. A total of four TWIGS simulations were run. The four simulations difi’ered in projection interval and cycle length, with one for each of the four different sets of FVS simulations. Each TWIGS simulation included information for all the variables measured at each sample point, as well as site index information for the species present in highest abundance at each point. Stand age was also included. Diameter growth information was not included as TWIGS cannot recalibrate diameter growth. Unlike FVS, different levels of TWIGS simulations were not examined since this program is not the USDA Forest Service endorsed growth and yield program for the Lake States, and therefore will no longer be revised or updated. Comparisons between F V8 and TWIGS were made with the “Full” FVS simulation level for each set of simulations. This was done in order to compare the results of the two programs under the most ideal conditions possible. All simulations were run at the stand level, with stand files containing individual tree information for all trees across all subplots in a stand not judged invalid due to disturbance. In the case of TWIGS, individual tree measurements were expanded to a per acre basis using TREEGEN (Miner et al. 1988) before simulation. TWIGS was then used to run the simulations. FVS does not require expansion of the data to a per acre basis beforehand. Rather, it will perform the expansion itself in the course of the simulation, provided the number of subplots in a stand is supplied. This results in considerable time saved. Each treelist contained at least 10 trees, nearly all greater than 2 43 in. in diameter, and total projection lengths were less than 30 years, in accordance with recommendations for best results (Miner et al. 1988). The option to group TWIGS results into five diameter classes (0.0-2.9 in., 3.0-4.9 in., 5.0-10.9 in., 11.0-16.9 in., 17.0 in.+) was used as this partition provided the most detailed description of the individual tree output, and because validation results could be directly related to TWIGS’ output without any adjustment (Guertin 1993). Individual tree predictions using FVS were combined into identical groups to allow for comparison of the two models’ results. Data Analysis Validation Statistics As noted previously, Holdaway and Brand (1983) set the stage for validation of the TWIGS model and its ancestors. The variables they chose to evaluate, Dbh, TPA, and BA, cover the most important aspects of stand and tree growth. An analysis of Dbh predictions provides an indication of the predictive ability of the program’s individual tree growth component. The accuracy of predictions of TPA reflect the effectiveness of the mortality fiinction at simulating stand competition. BA predictions, which are effectively a combination of Dbh growth and TPA estimates, provide an indication of overall model performance. The same three variables were used in this study to evaluate the prediction accuracy for FVS and TWIGS, and to compare the two programs’ growth predictions. Validation for each of the projection systems was done with grth data for seven upland hardwood species: northern red oak (Quercus rubra), other red oak (Quercus velutina and Quercus ellipsoidalis), white oak (Quercus alba), sugar maple (A cer saccharum), red maple (A cer rubrum), American basswood (T ilia americana), and 44 American beech (Fagus grandifolia). Sample sizes varied widely across different species, with northern red oak being most abundant. A total of seventeen species were tallied during the study. Of these seventeen species, it was felt that only the seven listed above were present in large enough numbers for meaningful analysis. The height and volume equations embedded in TWIGS and Lake States FVS were not tested for different reasons. Individual tree heights were estimated for over 800 sample trees with a Spiegal Relaskop during the 1996 field season. Measurement error with this tool is i 3 it. It was decided that this error would become confounded with prediction error to produce uninterpretable results. The accuracy of TWIGS and FVS’ estimates of volume are questionable due to the fact that the equations used to generate these estimates were calibrated with data from Michigan’s Upper Peninsula (Raile et al. 1982). However, a validation of these equations for the study area is not reasonable as actual measures of volume cannot be determine without cutting down each tree and sending them through a mill. Therefore, estimates of volume from FVS and TWIGS may only be compared to other estimates based on equations using actual measurements. The results of such a comparison would essentially be useless as an indication of true prediction accuracy. Error Statistics The statistics used in this study to analyze the error in model predictions were previously outlined by Kowalski and Gertner (1989) in their validation of the Central States 'I'WIGS model. They looked at mean error and percent mean error, where error 45 was defined as observed values (yi) minus predicted values (9,). However, while this form is standard in many statistical techniques, validation studies for both the STEMS and STEMS85 projection systems instead focused on errors of the form (yi -yi) (Holdaway and Brand 1983, Holdaway and Brand 1986). For consistency and comparability, this definition of error was also used in this study. Overpredictions therefore appear as positive errors and underpredictions are shown as negative errors. The error statistics used were: (91—1/1) Bias: mean error (E) = ;——, n M: £101 -yi)/y1] percent error = 100* I: , Precision : standard deviation of error = where ei = (yi -yi), n = sample size. SASATEST Validation Once all the projections were run, FVS and TWIGS output treelist files were used to extract predicted diameters for each tree run through the different simulations. These values were then paired up with actual diameter measurements for the same trees. Next, resulting pairs were sorted into different groups to be run through SASATEST. SASATEST is a SAS template for the BASIC program ATEST developed by Rauscher 46 (1986) which analyzes pairs of data using an accuracy test developed by Freese (1960) and modified by Reynolds (1984). ATEST, and therefore SASATEST, incorporate the error statistics of the previous section and provide measures of mean bias, percent error, and standard deviation of error. They also calculate confidence and prediction intervals for firture sample means and for individual sample values respectively. SASATEST was used at the individual tree level to examine prediction error in a number of areas. First, pairs of predicted values and actual values were run for all trees included in each simulation in order to test the models predictive accuracy on a grand scale. Next, pairs of values were separated by species. Pairs of values for all trees in each of the seven species groups selected for study were analyzed by species group to examine how well TWIGS and FVS predict diameter growth for different species. Next, all pairs were aggregated by initial tree diameter into five diameter groups: 0.0-2.9”, 3.0- 4.9”, 5.0-10.9”, 11.0-16.9”, 17.0”+. indicated previously the description of the TWIGS simulations. SASATEST was run with the four largest diameter groups in order to determine how good TWIGS and F VS are at predicting growth in different diameter groups. Not enough samples were present in the smallest group for results to be meaningful. Finally, all pairs were grouped by species and diameter group and run through SASATEST to look for any species and size class combinations where TWIGS or FVS performed poorly. Each of these groups was tested using SASATEST for all sixteen different simulations. SASATEST was used at the plot and stand levels to examine prediction error for BA and TPA. Actual BA per acre at each sample point was estimated using current 47 individual tree basal areas and 1986 expansion factors for each living tally tree at the point. Actual numbers of TPA at each sample point were estimated by summing up the simulations’ initial year expansion factors for each tree still living on a plot at the end of the simulation period. Predicted BA per acre at each point was found by multiplying the predicted diameter for each tree in a plot by the predicted number of TPA represented by that tree, and then summing up these values for each plot. Predicted TPA for each point was found by summing up the predicted TPA represented by each tree at a point. Actual values were then paired with predicted values for all plots and run through SASATEST to determine how well TWIGS and Lake States FVS predict plot-level parameters. Next, predicted and actual values for plot-level BA and TPA for each of the seven species chosen were run through SASATEST to look for patterns in the ways TWIGS and Lake States F VS under or overpredict the individual species components of plot-level BA and TPA. The final step in the SASATEST analysis of the TWIGS and Lake States F VS predictions involved testing model predictions of stand-level BA and TPA. Actual values for these variables were estimated by averaging actual plot level BA and TPA values for each stand. These values were then paired with the stand level estimates given by the models and run through SASATEST to determine how well the models predict stand- level parameters. Results for each simulation have been compiled and are presented in a series of tables in the following section. RESULTS Both the Lake States F VS variant and the Lake States TWIGS overpredict 10- year diameter grth for the seven upland hardwood species examined. No pattern was detected in prediction errors for BA and TPA for all seven species examined. A comparison of the two models shows similar positive mean DBH errors in ten-year predictions, with some disparity in five-year prediction accuracy between the two models. In over two-thirds of the cases examined Lake States FVS produced slightly more accurate diameter growth predictions than TWIGS. Lake States F VS also predicted basal area growth more accurately than TWIGS both at the plot level and at the individual species level for nearly all species examined. Neither model consistently predicted mortality more accurately than the other. It must be noted that, during the periods for which grth was projected, the Manistee National Forest experienced tent caterpillar and gypsy moth outbreaks (1987 and 1991 respectively). The defoliation and associated reduced growth resulting from such outbreaks may have played a significant role in the level of accuracy and precision achieved by the two simulators in this study. In addition to these pest outbreaks, extreme drought in the summer of 1988 may have increased mortality and decreased diameter growth, thereby afi'ecting simulation accuracy. 48 49 Influence of Additional Information on Prediction Accuracy Diameter Growth Error Analysis Before comparing FVS and TWIGS results, it is necessary to first examine the results of the different FVS simulations run using varying amounts of information. Table 3 presents the results of the diameter growth analysis for the three FVS simulation levels run with a projection length of 10 years and a cycle length of 10 years. In each of these simulations, the model began with 1986 tree and stand measurements and then directly estimated the new individual tree and stand-level characteristics for 1996. The model was not used to estimate growth for any year in between 1986 and 1996. Sample point- level estimates were determined from predicted individual tree characteristics. The first simulation level, “Full”, included information in each tree record for all variables measured as outlined in the previous chapter. Additional individual tree record information in the form of diameter growth data was included for many trees, derived from measurements taken during the development of the ECS project. Stand level information in this simulation included stand age as well as site index for one of the species present in highest abundance during sampling. The second simulation level, “noDI”, included the same information as the first level with the exception of the individual tree diameter growth data. The third simulation level, “noSI,A,DI”, included the same information as the first simulation with the exception of the diameter growth data, stand site index, and stand age data. The results presented in Table 6 provide a detailed description of the effects which certain variables have on prediction accuracy (“Mean Error”) and precision 50 Table 6. Mean error (bias), and standard deviation (5) of mean error for estimated dbh (in) by species and size class for lO-year FVS projections run over a single lO-year cycle. Bias is expressed as the predicted value minus the observed value. Full 1 noDI 2 noS1,A,DI 3 Diameter Mean Mean Mean Species Class 11 4 Error 5 EITOI' s Error 5 N. Red Oak 3.0 - 4.9 6 N/ A 5 N/A N/A N/A N/A N/A 5.0 - 10.9 170 0.22 0.30 0.37 0.33 0.32 0.30 11.0 - 16.9 322 0.19 0.42 0.42 0.43 0.27 0.42 17.0 + 193 0.08 0.58 0.27 0.58 0.01 0.57 All 692 0.18 0.45 0.38 0.46 0.23 0.46 Other Red Oak 3.0 - 4.9 1 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 65 0.13 0.48 0.02 0.36 0.06 0.36 11.0 - 16.9 114 0.34 0.50 0.20 0.40 0.24 0.39 17.0 + 40 0.45 0.81 0.18 0.54 0.30 0.56 All 220 0.29 0.57 0.13 0.42 0.19 0.42 White Oak 3.0 - 4.9 17 0.26 0.26 0.35 0.26 0.38 0.27 5.0 - 10.9 169 0.33 0.27 0.44 0.24 0.46 0.24 11.0 - 16.9 105 0.22 0.49 0.41 0.36 0.41 0.35 17.0 + 12 0.29 0.65 0.67 0.52 0.66 0.43 All 303 0.29 0.37 0.43 0.31 0.44 0.30 Sugar Maple 3.0 - 4.9 22 0.21 0.23 0.31 0.25 0.27 0.24 5.0 - 10.9 141 0.41 0.45 0.50 0.46 0.43 0.45 11.0 - 16.9 50 0.14 0.50 0.23 0.53 0.12 0.52 17.0 + 9 N/A N/A N/A N/A N/A N/A All 228 0.32 0.47 0.40 0.48 0.32 0.48 Red Maple 3.0 - 4.9 32 0.14 0.32 0.13 0.33 0.13 0.30 5.0 - 10.9 74 0.19 0.47 0.17 0.48 0.10 0.46 11.0 - 16.9 23 0.42 0.66 0.34 0.55 0.13 0.52 17.0 + 11 1.14 1.57 0.37 0.92 0.01 0.91 All 142 0.23 0.68 0.21 0.51 0.10 0.48 Basswood 3.0 - 4.9 0 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 18 0.36 0.28 0.72 0.41 0.70 0.39 11.0 - 16.9 55 0.42 0.47 0.93 0.40 0.85 0.40 17.0 + 9 N/A N/A N/A N/A N/A N/A All 82 0.45 0.45 0.89 0.41 0.82 0.40 American Beech 3.0 - 4.9 3 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 19 0.23 0.37 0.42 0.42 0.35 0.40 11.0 - 16.9 24 0.45 0.49 0.79 0.55 0.63 0.53 17.0 + 5 N/A N/A N/A N/A N/A N/A All 52 0.32 0.44 0.56 0.52 0.45 0.48 All Species 3.0 - 4.9 81 0.18 0.27 0.23 0.28 0.22 0.27 5.0 - 10.9 697 0.30 0.39 0.37 0.40 0.34 0.39 11.0 - 16.9 754 0.29 0.50 0.42 0.49 0.33 0.47 17.0 + 300 0.23 0.74 0.31 0.63 0.12 0.62 All 1842 0.27 0.50 0.37 0.48 0.29 0.47 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of trees. 5 Sample size was less than 10, so no analysis was run. 51 cc 3, S (standard deviation - ) for 10-year projections run with a cycle length of 10 years. FVS overpredicted diameter growth across all species and size classes in each simulation. Mean bias for each species, aggregated across diameter classes, was less than 0.50 in. over ten years for the “Full” simulation level. Mean bias for basswood, American beech, and large white oak (17in.+) in the “noDI” level, and for basswood and large white oak in the “noSI,A,DI” level were each greater than 0.50 inches. This indicates a large potential for these simulation levels to project trees of these species and size-classes into the wrong diameter classes for a 10-year projection. The “Full” level provided the most accurate diameter growth predictions for northern red oak, white oak, sugar maple, basswood, beech, and all species together. The “noDI” simulation level provided the most accurate predictions for the other red oak group(“ORO”), with grth estimates more than twice as accurate as those of the “Full” level for many diameter classes. The “noSI,A,DI” level produced the most accurate red maple diameter growth predictions. The “Full” simulation level produced more accurate estimates of growth across each diameter class for all species combined with the exception of the largest (17in.+), for which the “noSI,A,DI” level performed the best. No single size class consistently provided the most or least accurate predictions across all species in a simulation for any of the three simulation levels. The degree of precision of the diameter growth estimates reached by each simulation level was nearly equal for several species and size-class combinations, with the “noSI,A,DI” level offering slightly more precise results than the other two levels. Estimates of red maple growth were the most variable for each of the 52 three simulations, while the 17 in.+ diameter class was the least precise for each simulation. A second set of 10-year simulations was run using FVS, with projections being made over two S-year cycles as opposed to one lO-year cycle. This was done in order to test whether the recommendation made by Miner er al. (1988), that TWIGS performs best with short projection intervals, also holds for F VS. The results of the diameter analysis for this set of simulations (Table 7) are very similar to those of the previous analysis, with accuracy being slightly worse (0.01-0.04 in.) in most cases. Diameter growth predictions were more accurate for this set of simulations than those of the previous set at all three simulation levels for the ORO group. The same was true for basswood in the “noDI” and “noSI,A,DI” levels. Measures of precision for this second set of simulations were slightly higher, with magnitudes of difference between the two sets generally ranging from 0.00 - 0.02 inches. As with the previous set of simulations, the “Full” simulation level provided the most accurate overall performance of the three levels examined, with lower mean errors for five of the seven species examined and for three of the four size-classes examined. Once again, however, the “Full” simulation level did a comparatively poor job at predicting diameter grth for red maple and the other red oak group, with both the “noDI” and the “noSI,A,DI” levels producing more accurate diameter predictions across all size-classes for these two species. Along with the two sets of 10-year projections, two sets of 5-year projections were run to examine the effects of gypsy moth outbreak and drought on model prediction accuracy. The first of these two sets of simulations was done using 1986 diameter 53 Table 7. Mean error (bias), and standard deviation (8) of mean error for estimated dbh (in.) by species and size class for lO-year FVS projections run over two five-year cycles. Bias is expressed as the predicted value minus the observed value. Full 1 noDI 2 noSLA,DI 3 Species Diameter Class 11 4 Mean Error 8 Mean Error s Mean Error 3 N. Red Oak 3.0 . 4.9 6 N/A 5 N/A N/A N/A N/A N/A 5.0 - 10.9 170 0.26 0.29 0.38 0.32 0.35 0.30 11.0 - 16.9 322 0.23 0.41 0.44 0.43 0.29 0.42 17.0 + 193 0.11 0.58 0.27 0.58 0.02 0.58 All 692 0.21 0.44 0.39 0.46 0.25 0.47 Other Red Oak 3.0 - 4.9 1 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 65 0.10 0.47 0.01 0.36 0.05 0.37 11.0 - 16.9 1 14 0.33 0.48 0.20 0.40 0.24 0.40 17.0 + 40 0.39 0.76 0.17 0.54 0.29 0.56 All 220 0.27 0.54 0.13 0.42 0.18 0.43 White Oak 3.0 - 4.9 17 0.28 0.25 0.38 0.27 0.39 0.28 5.0 - 10.9 169 0.34 0.28 0.46 0.24 0.47 0.24 11.0 - 16.9 105 0.23 0.49 0.44 0.37 0.45 0.36 17.0 + 12 0.33 0.63 0.70 0.49 0.70 0.41 All 303 0.31 0.38 0.46 0.31 0.47 0.30 Sugar Maple 3.0 - 4.9 22 0.22 0.23 0.29 0.25 0.26 0.24 5.0 - 10.9 141 0.42 0.45 0.52 0.46 0.44 0.45 11.0 - 16.9 50 0.17 0.49 0.24 0.53 0.12 0.52 17.0 + 9 N/A N/A N/A N/A N/A N/A All 228 0.33 0.47 0.41 0.49 0.34 0.48 Red Maple 3.0 - 4.9 32 0.15 0.33 0.15 0.33 0.14 0.31 5.0 - 10.9 74 0.21 0.49 0.18 0.48 0.11 0.46 11.0 - 16.9 23 0.45 0.64 0.37 0.53 0.14 0.54 17.0 + 11 1.15 1.58 0.39 0.94 0.01 0.91 All 142 0.25 0.68 0.22 0.51 0.12 0.48 Basswood 3.0 - 4.9 0 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 18 0.36 0.28 0.71 0.42 0.69 0.40 11.0 - 16.9 55 0.44 0.47 0.92 0.40 0.85 0.40 17.0 + 9 N/A N/A N/A N/A N/A N/A All 82 0.46 0.45 0.88 0.41 0.80 0.41 American Beech 3.0 - 4.9 3 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 19 0.25 0.37 0.42 0.43 0.35 0.41 11.0 - 16.9 24 0.48 0.50 0.80 0.55 0.64 0.53 17.0 + 5 N/A N/A N/A N/A N/A N/A All 52 0.34 0.45 0.56 0.53 0.46 0.49 All Species 3.0 - 4.9 81 0.19 0.27 0.23 0.28 0.21 0.28 5.0 - 10.9 697 0.32 0.39 0.39 0.41 0.36 0.39 11.0 - 16.9 754 0.31 0.50 0.43 0.49 0.35 0.48 17.0 + 300 0.24 0.73 0.31 0.63 0.13 0.63 All 1842 0.29 0.50 0.39 0.48 0.31 0.48 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of trees. 5 Sample size was less than 10, so no analysis was run. 54 measurements, with five-year predictions being compared to 1991 remeasurements. The results of the diameter growth analysis for this set of simulations are found in Table 8. As in the two previous analyses, the same three simulation levels were run, differing in the amount of information included in each simulation. The results for this set of simulations mirror very closely those of the 10-year projections already examined. It is important to note that the 17 .Oin.+ red maple size-class is not present in this set of simulations, as there were fewer than ten red maple trees in this size-class for this period. Prediction accuracy for all sampled red maple trees combined was much better as a result. It should also be noted that the 17.0in.+ diameter class for the red oak group, not included previously, is present in this simulation. The effect of including this size-class was less noticeable on overall red oak accuracy. Once again, the “Full” simulation level provided the most accurate overall results for five of seven species groups, and, in this case, for all four size-class groups. As before, though, the “Full” level produced the poorest estimates of diameter growth for the other red oak and red maple groups. The disparity between estimates from the “Full” level and those from the other two simulation levels for these two species, relative to the length of the simulation, however, was smaller. Unlike the previous two analyses, F VS underpredicted diameter growth for the 17.0 in.+ red oak group in both the “Full” and “noSI,A,DI” simulation levels for this projection. Despite this, each aggregate species group and aggregate size-class group was still overpredicted for both levels. Mean errors for the sugar maple, basswood and beech groups in the “noDI” level and the basswood group in the “noSI,A,DI” level were all greater than 0.25 inches. This again indicates the potential for FVS to project trees of 55 Table 8. Mean error (bias), and standard deviation (s) of mean error for estimated dbh (in.) by species and size class for 5-year F VS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. Full 1 noD12 flask/1,01 3 Species Diameter Class 11 4 Mean Error 8 Mean Error s Mean Error s N. Red Oak 3.0-4.9 15 0.15 0.13 0.19 0.11 0.19 0.11 5.0 - 10.9 200 0.13 0.19 0.21 0.19 0.19 0.18 11.0 - 16.9 336 0.04 0.25 0.20 0.26 0.12 0.26 17.0 + 153 -0.04 0.39 0.12 0.38 -0.02 0.39 All 706 0.07 0.28 0.19 0.28 0.12 0.29 Other Red Oak 3.0 - 4.9 9 N/A 5 N/A N/A N/A N/A N/A 5.0 - 10.9 85 0.14 0.26 0.08 0.21 0.10 0.21 11.0 - 16.9 111 0.22 0.29 0.14 0.21 0.16 0.22 17.0 + 39 0.25 0.40 0.14 0.28 0.20 0.29 All 244 0.18 0.30 0.11 0.23 0.13 0.23 White Oak 3.0 - 4.9 28 0.22 0.21 0.22 0.14 0.23 0.14 5.0 - 10.9 189 0.19 0.17 0.24 0.15 0.24 0.14 11.0 - 16.9 107 0.10 0.23 0.21 0.21 0.22 0.22 17.0 + 12 0.34 0.48 0.44 0.28 0.43 0.23 All 338 0.17 0.22 0.23 0.18 0.24 0.18 Sugar Maple 3.0 - 4.9 23 0.09 0.19 0.11 0.18 0.10 0.18 5.0 - 10.9 159 0.27 0.27 0.32 0.26 0.29 0.26 11.0 - 16.9 52 0.20 0.27 0.24 0.24 0.18 0.24 17.0 + 5 N/A N/A N/A N/A N/A N/A All 244 0.24 0.26 0.28 0.25 0.24 0.25 Red Maple 3.0 - 4.9 39 0.09 0.19 0.09 0.20 0.07 0.19 5.0 - 10.9 73 0.11 0.28 0.10 0.28 0.06 0.26 11.0 - 16.9 23 0.40 0.36 0.31 0.29 0.19 0.31 17.0 + 9 N/A N/A N/A N/A N/A N/A All 151 0.16 0.37 0.14 0.28 0.08 0.26 Basswood 3.0 - 4.9 1 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 18 0.21 0.19 0.39 0.21 0.39 0.21 11.0 - 16.9 56 0.23 0.22 0.47 0.20 0.44 0.22 17.0 + 6 N/A N/A N/A N/A N/A N/A All 81 0.24 0.23 0.45 0.21 0.43 0.22 American Beech 3.0 - 4.9 6 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 21 0.16 0.25 0.25 0.24 0.20 0.23 11.0 - 16.9 23 0.17 0.22 0.34 0.32 0.27 0.30 17.0 + 4 N/A N/A N/A N/A N/A N/A All 58 0.14 0.29 0.27 0.28 0.21 0.26 All Species 3.0 - 4.9 123 0.11 0.19 0.13 0.18 0.13 0.18 5.0 - 10.9 802 0.19 0.23 0.23 0.23 0.21 0.22 11.0 - 16.9 775 0.14 0.29 0.22 0.27 0.17 0.27 17.0 + 246 0.08 0.47 0.18 0.39 0.08 0.39 All 1962 0.15 0.29 0.21 0.27 0.18 0.27 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of trees. 5 Sample size was less than 10, so no analysis was run. 56 these species into the wrong diameter class in projections ten years in length or longer. As before, no size-class consistently presented problems across all species for any of the three levels. Prediction precision was worst for the “Full” simulation level, with other two simulation levels achieving approximately equal levels of precision for most species and size-class combinations. The second set of five-year simulations was done using 1991 diameter measurements, with predictions being compared to 1996 remeasurements. The results of the diameter growth analysis for this set of simulations (Table 9) show a distinctly different pattern than those of the other five-year analysis. Unlike the 1986-1991 simulations, diameter growth for three species groups (other red oak, sugar maple and red maple) and several of the species-size-class combinations for the larger size-classes were underpredicted. The “Full” simulation level produced more accurate estimates than the other two simulation levels for each species, with the exception of the red maple group, which was predicted equally as well by the “noSI,A,DI” level. Growth estimates for each diameter class were more accurate for the “Full” level than for the other two simulation levels when all species were run together, with the exception of the 17.0 in.+ class which was predicted more accurately by the “noSI,A,DI” level. None of the three simulation levels appeared to produce consistently more precise results than the other two levels. The results of the 1991-1996 simulations were more accurate across nearly every species and size-class combination than those of the 1986-1991 simulations. A comparison of the level of precision achieved by each of these sets of simulations, however, did not produce such cut and dried results. Neither set of simulations had 57 Table 9. Mean error (bias), and standard deviation (s) of mean error for estimated dbh (in.) by species and size class for S-year FVS projections (1991-1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. Full 1 noDI 2 noSI,A,DI 3 Species Diameter Class 11 4 Mean Error 3 Mean Error 8 Mean Error 3 N. Red Oak 3.0 - 4.9 6 N/A 5 N/A N/A N/A N/A N/A 5.0 - 10.9 170 0.06 0.19 0.15 0.20 0.13 0.19 11.0 - 16.9 333 0.10 0.27 0.19 0.24 0.13 0.23 17.0 + 211 0.07 0.34 0.10 0.34 0.00 0.33 All 721 0.08 0.28 0.15 0.27 0.10 0.26 Other Red Oak 3.0 - 4.9 1 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 70 -0.04 0.18 -0.06 0.18 -0.05 0.18 11.0 - 16.9 119 0.01 0.22 -0.03 0.21 -0.02 0.21 17.0 + 41 -0.14 0.30 -0.21 0.30 -0.19 0.31 All 231 -0.02 0.23 -0.06 0.23 -0.05 0.23 White Oak 3.0 - 4.9 17 0.06 0.21 0.17 0.21 0.17 0.21 5.0 - 10.9 161 0.06 0.18 0.19 0.17 0.20 0.17 11.0 - 16.9 102 0.03 0.21 0.18 0.20 0.18 0.19 17.0 + 12 -0.03 0.31 0.22 0.27 0.22 0.23 All 292 0.05 0.20 0.19 0.18 0.19 0.18 Sugar Maple 3.0 - 4.9 17 0.05 0.11 0.15 0.12 0.13 0.12 5.0 - 10.9 148 0.02 0.26 0.17 0.25 0.13 0.24 11.0 - 16.9 59 -0.17 0.38 -0.03 0.40 -0.07 0.39 17.0 + 8 N/A N/A N/A N/A N/A N/A All 239 -0.03 0.30 0.12 0.30 0.08 0.29 Red Maple 3.0 - 4.9 30 0.02 0.18 0.02 0.18 0.03 0.17 5.0 - 10.9 85 0.00 0.23 0.07 0.26 0.03 0.25 11.0 - 16.9 31 -0.06 0.29 0.04 0.30 -0.06 0.29 17.0 + 11 -0.14 0.51 0.00 0.51 -0.10 0.51 All 159 -0.01 0.26 0.05 0.27 0.01 0.27 Basswood 3.0 - 4.9 0 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 15 0.12 0.20 0.33 0.25 0.31 0.26 11.0 - 16.9 54 0.09 0.21 0.43 0.20 0.41 0.18 17.0 + 9 N/A N/A N/A N/A N/A N/A All 78 0.10 0.24 0.42 0.23 0.39 0.22 American Beech 3.0 - 4.9 4 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 24 0.07 0.18 0.24 0.23 0.21 0.22 11.0 - 16.9 28 0.17 0.33 0.40 0.32 0.33 0.30 17.0 + 5 N/A N/A N/A N/A N/A N/A All 62 0.1 l 0.27 0.30 0.29 0.25 0.27 All Species 3.0 - 4.9 76 0.03 0.16 0.09 0.18 0.09 0.17 5.0 - 10.9 714 0.04 0.22 0.15 0.23 0.13 0.22 11.0 - 16.9 788 0.06 0.28 0.15 0.29 0.12 0.28 17.0 + 320 0.04 0.35 0.07 0.35 0.00 0.34 All 191 l 0.05 0.27 0.13 0.28 0.10 0.27 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of trees. 5 Sample size was less than 10, so no analysis was run. 58 consistently more precise estimates than the other across the different species and size- class combinations. The mean error for all species and size-classes combined, a quick and dirty means of evaluating overall accuracy, was nearly three times as small for the 91- 96 “Full” simulation level as that of its counterpart in the 89-91 simulations. It appears from these analyses that the “Full” simulation level consistently produces generally more accurate diameter growth estimates than the “noDI” and “noSI,A,DI” levels for both five and ten year projections. In addition, the projections over a single 10-year cycle produced more accurate estimates than did a projection composed of two consecutive 5-year cycles. The results of the two S-year sets of simulations indicate that prediction accuracy is much higher for periods without natural pest outbreaks or drought, with nearly every species and diameter class being predicted more accurately. Trees/Acre Error Analysis Information on the predictive power of the mortality component of F VS is provided by an analysis of the simulator’s predictions of trees/acre (TPA). Poor estimation of TPA indicates that the mortality fiinction is not accurately simulating the effects of existing levels of competition. Before beginning a rigorous analysis of the results of the different F VS simulations run, it is important to note that predicted mortality is sensitive to predicted diameter growth. Small errors in diameter grth estimates can result in noticeable errors in TPA estimates (Holdaway and Brand, 1986). As in the diameter growth analysis, an examination of the predictive ability of FVS’ 59 mortality component is made for the two different sets of ten-year projections, followed by an identical analysis for the two different sets of five-year projections. Each set of simulations is again comprised of the same three simulation levels as in the previous analyses. The sample unit for this analysis is the sample point, with errors expanded to a per acre basis. Results for the set of three FVS simulation levels run with a projection length of 10 years and a cycle length also of 10 are presented in Table 10. Unlike the previous analyses, the “noDI” level produced the most accurate results for all species combined, as well as for three of the seven individual species (northern red oak, red maple, and beech). The “Full” simulation level produced the most accurate results for each of the other four species examined. F VS overpredicted TPA for some species (white oak, sugar maple, basswood), and underpredicted TPA for the others, as well as for all species combined. Only red maple exhibited mean errors of greater than 15 TPA over 10 years for any of the three simulation levels, with errors for the other red oak group being the smallest. Red maple errors were larger than 22 TPA for each of the three simulation levels. None of the three simulation levels consistently provided more precise predictions than the other two. Predictions for sugar maple, followed by all species combined, and then red maple and white oak were the least precise, while basswood predictions were the most precise. As in the diameter growth analysis, differences between TPA estimates for the three simulation levels projected over a ten-year period with two five-year cycles mirrored almost exactly the relationships between results over a single lO-year cycle. Accuracy was neither consistently better nor worse for the projection over two five-year 60 Table 10. Mean error (bias), and standard deviation (8) of mean error for estimated trees/acre (TPA) by species for lO-year FVS projections run over a single 10- year cycle. Bias is expressed as the predicted value minus the observed value. Full 1 noDI 2 noSI,A,DI 3 Mean Mean Mean Species 11 4 Error s Error S Error 8 N. Red Oak 96 -7.4 54.7 -6.0 52.3 -6.8 51.4 Other Red Oak 57 -O.8 32.5 -1.5 35.0 -l.0 34.6 White Oak 89 13.3 60.1 14.1 74.1 14.1 741 Sugar Maple 33 0.4 110.2 4.5 109.4 3.4 107.7 Red Maple 62 -23.5 70.3 -22.9 70.5 -22.9 69.8 Basswood 16 1.4 12.6 2.3 12.4 2.3 12.3 American Beech 18 -5.1 34.4 -4.5 30.4 -4.9 30.9 All Species 142 -10.8 88.2 -6.0 90.9 -7.0 89.4 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of sample points. cycles, while precision was slightly better (Table 11). The same species were again overpredicted in this simulation. The “Full” and “non’ simulation levels provided the most accurate estimates of TPA for certain species, and the “noDI” level produced the most accurate estimates for all species combined. As in the previous analysis, mean errors of prediction for all species other than red maple were less than 15 TPA, with errors for the other red oak group once again being the smallest. Red maple mean errors were slightly improved but still exceeded 21 TPA for each of the three simulation levels. No single simulation level consistently provided more precise predictions than the other 61 Table 11. Mean error (bias), and standard deviation (s) of mean error for estimated trees/acre (TPA) by species for 10-year FVS projections run over two five- year cycles. Bias is expressed as the predicted value minus the observed value. Fulll noDI2 noSI,A,DI 3 Mean Mean Mean Species n " Error 8 Error 8 Error s N. Red Oak 96 -6.8 52.9 -5.5 51.4 -6.3 50.5 Other Red Oak 57 -1.0 33.3 -l.6 35.3 -1.2 34.9 White Oak 89 13.4 61.0 14.1 74.1 14.1 74.1 Sugar Maple 33 0.3 109.6 4.0 108.9 2.9 107.4 Red Maple 62 -22.4 64.3 -21.9 65.9 -24.5 70.6 Basswood 16 1.4 12.5 2.3 12.3 2.2 12.2 American Beech 18 -5.2 33.9 -4.7 30.1 -5.1 30.7 All Species 142 -9.0 86.0 -5.5 89.0 -6.4 87.6 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 2 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of sample points. two. As before, predictions for sugar maple were the least precise, followed by predictions for all species combined, then red maple and then white oak, while basswood predictions were again the most precise. The results of the TPA analysis for the first set of five-year simulations (1986- 1991) (Table 12) are both different from as well as similar to the results of the previous two sets of simulations. The “noDI” simulation level produced the most accurate results for four of the seven individual species (red oak, sugar maple, basswood, and beech), as well as for all species combined. The “Full” level produced the most accurate results for 62 Table 12. Mean error (bias), and standard deviation (8) of mean error for estimated trees/acre (TPA) by species for five-year F VS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value nrinus the observed value. Fulll noDI2 noSI,A,DI 3 Mean Mean Mean Species 11 4 Error 8 Error S Error S N. Red Oak 96 -6.5 72.9 -4.7 72.4 -5.2 71.7 Other Red Oak 58 -6.6 31.0 -7.3 35.6 -11.4 39.6 White Oak 90 0.8 46.2 1.1 59.9 0.6 61.2 Sugar Maple 36 -3.6 88.1 -1.9 88.1 -7.6 98.2 Red Maple 65 -13.5 43.2 -13.2 43.3 -13.0 42.6 Basswood 17 -0.4 13.8 0.0 13.7 -2.7 14.8 American Beech 20 -2.7 17.9 -2.5 15.6 -3.6 15.7 All Species 146 -16.6 84.8 -13.3 88.0 -19.7 91.2 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of sample points. the other red oak group, while the “noSI,A,DI” level produced the most accurate estimates of TPA for both white oak and red maple. FVS overpredicted TPA for a single species, white oak, across each of the three simulation levels, and underpredicted TPA for the other six species examined as well as for all species combined. Red maple and all species combined were the only groups with mean errors exceeding 7 TPA. Where the ORO group was predicted with the most accuracy in the previous two analyses, mean errors for this same group were the second largest of the seven species examined in this analysis. Again, none of the three simulation levels consistently provided the most 63 precise predictions. Levels of precision for the seven individual species varied greatly. TPA predictions for sugar maple were once again the least precise, followed by all species combined and the red oak group, while predictions for basswood and beech were again the most precise. As expected after the diameter growth analysis, the results of the TPA analysis for the second set of 5-year simulations (1991-1996) (Table 13) were markedly different than those of the previous analysis. Again, FVS overpredicted mortality for some species (ORO and white oak), and underpredicted mortality for the remainder at each of the three simulation levels. This time, however, the “noDI” simulation level provided the most accurate TPA predictions for all species but white oak, which was predicted most accurately by the “Full” level. Sugar maple and red maple each exhibited mean errors of more than 10 TPA in at least one of the three simulation levels. Red maple estimates were consistently poor, while basswood and the ORO group both had mean errors of less than 2 TPA for each of the three simulation levels. Accuracy across all species combined was more than halved for each of the three simulation levels when compared to the previous analysis. The “Full” simulation level generally provided more precise estimates of TPA than the other two simulation levels. Beech estimates, along with estimates of TPA for all species combined, were the least precise while those for basswood were again the most precise. It appears from these analyses that the “noDI” simulation level generally produces more accurate TPA estimates at the plot level for five-year simulations than the “Full” and “noSI,A,DI” levels. Both the “noDI” level and the “Full” level perform roughly equally for 10-year simulations. As in the diameter growth analysis, results indicate that a 64 Table 13. Mean error (bias), and standard deviation (s) of mean error for estimated trees/acre (TPA) by species for five-year FVS projections (1991-1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. Fulll noDI2 noSI,A,DI 3 Mean Mean Mean Species 11 ’ Error 8 Error S Error S N. Red Oak 95 -4.6 37.1 -3.7 35.5 -4.0 35.3 Other Red Oak 52 1.3 23.6 1.1 24.0 1.2 23.8 White Oak 81 7.5 45.3 8.2 48.3 8.2 48.3 Sugar Maple 32 -11.6 50.3 -6.5 48.4 -6.9 48.7 Red Maple 63 -10.3 49.7 -10.0 49.5 -10.3 49.2 Basswood 15 -1.0 10.8 -0.5 11.5 -0.5 11.4 American Beech 17 -2.3 68.5 -1.9 69.6 -2.1 74.3 All Species 136 -7.5 62.4 -4.0 62.9 -4.6 63.3 ‘ Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of sample points. single 10- year cycle projection produces slightly more accurate, if more variable, estimates than a projection composed of two consecutive 5-year cycles. The results of the two 5-year sets of simulations indicate that prediction accuracy for TPA did not improve much across individual species for the five-year period without leaf defoliator outbreaks or drought when compared to prediction results for the five-year period which experienced both. However, prediction accuracy across the aggregate of all species combined was improved. 65 In addition to an examination of plot-level TPA predictions, an analysis was also done to determine the ability of F VS to accurately predict stand-level TPA. It is important to recall that stand-level measurements for this study were composed of individual tree measurements from up to six sampled subplots. Actual stand level TPA measurements were found by averaging TPA measurements for all subplots fi'om a stand. The results of this analysis for each of the three simulation levels in each of the four different simulation sets (Table 14) appear to indicate that, contrary to the plot-level TPA analysis, slightly more accurate lO-year projections are obtained when the cycle length is shortened from ten to five years. However, TPA estimates for the 10-year projection over a single lO-year cycle were more precise. As in the plot-level TPA analysis, the “noDI” simulation level produced more accurate estimates of stand TPA than either of Table 14. Mean error (bias), and standard deviation (8) of mean error for estimated stand-level trees/acre (TPA) by F VS simulation. Bias is expressed as the predicted value minus the observed value. Full1 noDI 2 noSI,A,DI 3 Mean Mean Mean Species 11 4 Error S Error S Error S 86-96 (1 cycle) 41 -25.5 159.6 -21.5 164.3 -22.1 163.5 86-96 (2 cycles) 41 -23.7 160.2 -19.9 164.5 -20.3 163.7 86-91 (1 cycle) 42 -157 69.1 -14.1 71.7 -14.3 71.2 91-96 (1 cycle) 44 -205 174.7 -16.4 175.7 -16.8 175.3 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of stands. 66 the other two simulation levels, with those for the “Full” level being the least accurate of the three. The roles were reversed, though, for precision of estimation. With regards to the two sets of 5-year simulations, estimates were more accurate for each simulation level in the 1986-1991 set than in the 1991-1996 set. In addition, estimates in the 1986-1991 set were more than twice as precise as those in the 1991-1996 set. As with the ten-year projections, the “noDI” level produced the most accurate estimates for each of the 5-year simulations. None of the three simulation levels produced more precise estimates for these two sets of simulations, however. Basal Area Error Analysis An analysis of the BA errors of prediction can be used as an overall evaluation of the FVS simulator, indicating how well the diameter growth and mortality components work together to approximate forest growth (Holdaway and Brand 1983). The sample unit for this analysis was once again the sample point, with errors expanded to a ft2 /acre basis. As in the Dbh growth and TPA analyses, an examination of FVS’ ability to accurately predict sample point BA was made for the two different sets of ten-year projections, followed by an identical analysis for the two different five-year projections. Again, each set of simulations was composed of the same three simulation levels. The results of the BA analysis for the first set of simulations, used to project growth over a ten-year period by way of a single 10-year cycle, are presented in Table 15. FVS underpredicted BA for three species (red oak, red maple, and beech), and overpredicted BA for the other four species as well as for all species combined. No 67 Table 15. Mean error and standard deviation (8) of mean error for estimated basal area (BA) (ft2 / acre) by species for lO-year FVS projections run over a single 10- year cycle. Bias is expressed as the predicted value minus the observed value. Full1 noDI2 noSI,A,DI 3 Mean Mean Mean Species 11 ‘1 Error S Error S Error S N. Red Oak 96 -2.6 10.7 0.2 10.4 -l.7 9.3 Other Red Oak 57 0.1 8.3 -1.2 9.2 -0.8 8.9 White Oak 89 6.0 10.6 7.4 10.4 7.5 10.4 Sugar Maple 33 2.0 12.5 4.4 12.6 3.4 11.4 Red Maple 62 -1.6 6.8 -1.6 6.4 -2.2 6.1 Basswood 16 2.4 5.5 6.5 6.4 6.2 6.3 American Beech 18 -0.2 2.0 1.0 2.3 0.4 2.0 All Species 142 4.3 14.3 8.1 14.6 6.1 13.8 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. ‘1 Number of sample points. species produced mean errors greater than 8 ft2 /acre , with beech and the other red oak group having the smallest errors of prediction across the three simulation levels. The “Full” simulation level produced the most accurate BA estimates for six of the seven species, and for all species combined. The “non’ level produced the most accurate estimate of BA for red oak. BA predictions for the white oak forest type were the least accurate of any species, followed by those for basswood. The “noSI,A,DI” simulation level produced the most precise results by a small margin. Predictions for all species 68 combined as well as for sugar maple, white oak and red oak were the least precise, while those for American beech and basswood were the most precise. The second BA analysis involved the set of simulations projected over a 10-year period with a cycle length of 5 years. The results of this analysis (Table 16) had the same pattern as those of the preceding projection, with accuracy being slightly lower and precision being slightly higher. BA was consistently underpredicted for red maple and the other red oak group, with underpredictions also occurring in at least one simulation level for red oak and American beech. All other species were consistently overpredicted. Once Table 16. Mean error and standard deviation (8) of mean error for estimated basal area (BA) (ft2 lacre) by species for 10-year F VS projections run over two five- year cycles. Bias is expressed as the predicted value minus the observed value. Full1 noDI2 noSI,A,DI 3 Mean Mean Mean Species n 1 Error s Error s Error s N. Red Oak 96 -2.2 9.8 0.4 10.0 -1.5 8.9 Other Red Oak 57 -0.1 8.5 -1.3 9.2 -0.8 8.9 White Oak 89 6.2 10.5 7.6 10.4 7.7 10.4 Sugar Maple 33 2.1 12.4 4.4 12.5 3.4 11.3 Red Maple 62 -1.4 6.6 -l.5 6.2 -2.5 8.5 Basswood 16 2.4 5.4 6.4 6.4 6.1 6.2 American Beech 18 -0.1 2.0 1.0 2.3 0.4 2.1 All Species 142 5.1 13.6 8.4 14.3 6.5 13.6 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. 4 Number of sample points. 69 again, no species exhibited mean errors greater than 8 ft2 lacre , with American beech and the other red oak group again having the smallest mean errors of prediction across the three simulation levels. As in the previous analysis, the “Full” simulation level produced the most accurate BA estimates for six of the seven species, and for all species combined. The “noDI” simulation level again produced the most accurate estimate of BA for northern red oak. In addition, BA predictions for the white oak forest type were again the least accurate of any species, followed as before by those for all species combined. Unlike the previous analysis, no one simulation level was consistently more precise than the other two. However, as before, predictions for all species combined as well as for sugar maple, white oak and red oak were the least precise, while those for beech and basswood were the most precise. The third BA analysis involved the set of simulations projected over the 5-year period from 1986 to 1991, with a cycle length of 5 years. The results of this analysis(Table 17) are in some ways similar to those of the preceding analysis. The same four species as in the previous analysis were underpredicted for at least one simulation level, with red oak now being consistently underpredicted across all three levels. No species exhibited mean errors greater than 4 ft2 /acre in magnitude, with mean errors for no species being greater than 2 ft2 /acre in magnitude for the “Full” simulation level. No simulation level consistently produced the most accurate estimates of BA, with the “Full” level being more accurate for four species (white oak, other red oak, sugar maple, and red maple), the “noSI,A,DI” level being more accurate for basswood, beech, and all species combined, and the “noDI” level being the most accurate predictor of northern red 70 Table 17. Mean error (bias), and standard deviation (5) of mean error for estimated basal area (BA) (ft2 / acre) by species for five-year FVS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. Fulll noDI2 noSI,A,DI 3 Mean Mean Mean Species 11 4 Error S Error 8 Error S N. Red Oak 96 -1.9 7.6 -0.5 6.5 -1.4 5.9 Other Red Oak 58 -1.2 4.9 -1.7 5.8 -3.1 12.3 White Oak 90 1.9 4.8 2.6 5.1 2.3 6.8 Sugar Maple 36 1.6 8.4 2.9 8.2 2.1 11.1 Red Maple 65 -0.6 3.2 -0.7 3.1 -0.9 2.8 Basswood 17 1.1 3.9 2.5 4.9 0.3 5.6 American Beech 20 -0.4 1.2 0.4 1.1 -0.1 2.3 All Species 146 0.5 9.6 2.4 9.8 0.1 14.4 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. ‘1 Number of sample points. oak BA. In addition, while the “Full” simulation level produced the most precise results for the case of all species combined, no simulation level consistently provided more precise results across individual species than the other two. The results of the BA analysis for the second set of 5-year simulations (1991- 1996) (Table 18) are in many ways different than those of the previous analysis. BA for red oak, red maple and the other red oak group was again consistently underpredicted across the three simulation levels. However, sugar maple was also underpredicted for the “Full” level, while beech - previously underpredicted by both the “Full” and “noSI,A,DI” 71 Table 18. Mean error (bias), and standard deviation (8) of mean error for estimated basal area (BA) (ft2 lacre) by species for five-year FVS projections (1991-1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. Full1 noDI2 noSI,A,DI 3 Mean Mean Mean Species 11 4 Error S Error S Error S N. Red Oak 95 -1.6 6.8 -0.7 6.4 -1.5 6.1 Other Red Oak 52 -0.4 6.5 -O.7 6.8 -0.5 6.7 White Oak 81 2.5 7.3 3.9 7.7 3.9 7.7 Sugar Maple 32 -1.3 5.8 1.0 4.5 0.5 4.5 Red Maple 63 -1.0 4.9 -0.8 5.3 -1.0 4.9 Basswood 14 0.1 2.8 0.3 2.9 1.2 2.8 American Beech 18 0.2 1.0 0.7 1.2 0.5 1.2 All Species 136 0.7 9.5 3.3 9.2 2.3 9.1 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. ‘1 Number of sample points. levels - was not underpredicted in any level for this set of simulations. Once again, no species exhibited mean errors greater than 4 ft2 / acre for any of the three simulation levels, but errors for the “Full” now exceeded 2 ft2 /acre for each species. The “Full” simulation level produced more accurate estimates of BA for four of the seven species (other red oak, white oak, basswood, and American beech), as well as for all species combined. In addition, the “Full” level produced the most precise results for all individual species with the exceptions of red oak and sugar maple, and for all species 72 combined. Estimates of BA for all species were the most variable, followed by estimates for the three individual oak forest types. These analyses of the ability ofFVS to accurately predict plot-level basal area indicate that the “Full” Simulation level produced the most consistently accurate estimates of basal area for all species other than red oak over a ten-year projection. As in the diameter growth and TPA analyses, it appears that a single 10-year cycle projection produces slightly more accurate, if more variable, estimates than a projection composed of two consecutive 5-year cycles. Results for the two five-year analyses were less clear. The “Full” level did produce more accurate BA estimates for a number of individual species, as well as for all species combined. However, the species which this level predicted more accurately than the other two levels were not consistent between the two sets of five-year simulations. In addition prediction accuracy was not much improved across individual species for the period without outbreaks of leaf defoliators or drought than for the period experiencing both. As in the TPA analysis, in addition to an examination of plot-level BA prediction, an analysis was also done to determine the ability of FVS to accurately predict Stand-level BA. Actual stand-level BA measurements were found for each stand by averaging BA measurements for all subplots in each stand. The results of this analysis for each of the three simulation levels in each of the four different simulation sets (Table 19) indicate that, as in the plot-level BA analysis, slightly more accurate lO-year projections were obtained when the cycle length was lengthened from five to ten years. However, BA estimates for the 10-year projection over two 5-year cycles were more precise by a small 73 Table 19. Mean error (bias), and standard deviation (s) of mean error for estimated stand-level basal area (BA) (ft2 / acre) by FVS simulation. Bias is expressed as the predicted value minus the observed value. Full 1 noDI2 noSI,A,DI 3 Mean Mean Mean Species 11 ‘1 Error s Error 8 Error 8 86-96 (1 cycle) 41 8.8 13.2 12.0 13.1 10.4 12.5 86-96 (2 cycles) 41 9.6 12.9 12.6 13.2 11.0 12.7 86-91 (1 cycle) 42 6.1 7.6 4.0 8.5 5.3 7.0 91-96 (1 cycle) 44 2.8 13.4 0.4 13.2 2.1 13.5 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 3 Simulation run without site index and age information for each stand, and without individual tree diameter increment information. ‘1 Number of stands. margin. Also similar to the plot level BA analysis was the fact that the “Full” simulation level provided more accurate estimates of stand BA than either of the other two simulation levels for each of the two lO-year projections. The “noSI,A,DI” simulation level produced the most precise estimates, by a small margin. With regards to the two sets of 5-year simulations, estimates were more than twice as accurate for each simulation level in the 1991-1996 set than in the 1986-1991 set. However, estimates in the 1986- 1991 set were nearly twice as precise as those in the 1991-1996 set. Unlike the ten-year projections, the “noDI” level produced the most accurate estimates for each of the 5-year simulations. Measures of precision for the three levels in each of these four sets of simulations were fairly equivalent, with the “noSI,A,DI” level producing the most precise estimates in three of the four. 74 Comparison of FVS and TWIGS Prediction Accuracy The analysis of the predictive ability of FVS for different simulation types and levels of input will be followed by a comparison of the predictive ability of FVS and I'WIGS. As with FVS, the analysis will begin with a comparison of diameter growth prediction accuracy, followed by analyses of TPA and BA prediction accuracy. Diameter Growth Error Analysis Throughout the FVS diameter growth analysis, the “Full” simulation level consistently performed better with regards to accuracy than either of the other two simulation levels. In addition, accuracy was consistently higher between the two sets of 10-year projections for the projection made over a single 10-year cycle rather than over two 5-year cycles. For that reason, comparisons to 10-year TWIGS diameter growth estimates will be made with results from the “Full” simulation level in the set of 10-year projections made over a single 10-year cycle. TWIGS 5-year projections will be compared to “Full” simulation level estimates for FVS projections for the same five-year periods as the TWIGS estimates for similar reasons. The results from the first set of comparisons (Table 20) indicate that, contrary to previous results with F VS, TWIGS prediction accuracy is improved for shorter cycle lengths, as claimed by Miner et al. (1988). The TWIGS simulation run over two 5- year cycles produced more accurate results than the simulation done over a single lO-year cycle for nearly every species and size-class combination other than those for white oak, and for all species combined. Neither simulation appeared to be consistently more 75 Table 20. Mean error (bias), and standard deviation (5) of mean error for estimated dbh (in) by species and size class for lO-year FV S and lO-year TWIGS projections run over a single lO-year cycle, and the 10-year TWIGS projection run over two S-year cycles. Bias is expressed as the predicted value minus the observed value. FVS (Full 1 ) TWIGS (1x10) 2 TWIGS (2x5) 3 Species Diameter Class 11 4 Mean Error s Mean Error 8 Mean Error s N. Red Oak 3.0 - 4.9 6 N/A 3 N/A N/A N/A N/A N/A 5.0 - 10.9 170 0.22 0.30 0.34 0.31 0.32 0.31 11.0 - 16.9 322 0.19 0.42 0.35 0.42 0.33 0.42 17.0 + 193 0.08 0.58 0.18 0.50 0.13 0.51 All 692 0.18 0.45 0.29 0.43 0.29 0.43 Other Red Oak 3.0 - 4.9 1 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 65 0.13 0.48 0.05 0.35 0.01 0.35 11.0 - 16.9 114 0.34 0.50 0.17 0.38 0.16 0.38 17.0 + 40 0.45 0.81 0.12 0.50 0.10 0.50 All 220 0.29 0.57 0.12 0.40 0.09 0.40 White Oak 3.0 - 4.9 17 0.26 0.26 0.33 0.25 0.34 0.25 5.0 - 10.9 169 0.33 0.27 0.37 0.24 0.37 0.24 11.0 - 16.9 105 0.22 0.49 0.29 0.35 0.30 0.35 17.0 + 12 0.29 0.65 0.54 0.49 0.56 0.49 All 303 0.29 0.37 0.35 0.30 0.36 0.30 Sugar Maple 3.0 - 4.9 22 0.21 0.23 0.33 0.26 0.32 0.25 5.0 - 10.9 141 0.41 0.45 0.49 0.45 0.46 0.44 11.0 - 16.9 50 0.14 0.50 0.11 0.53 0.08 0.52 17.0 + 9 N/A N/A N/A N/A N/A N/A All 228 0.32 0.47 0.37 0.48 0.34 0.48 Red Maple 3.0 - 4.9 32 0.14 0.32 0.13 0.31 0.13 0.31 5.0 - 10.9 74 0.19 0.47 0.16 0.47 0.14 0.47 11.0 - 16.9 23 0.42 0.66 0.31 0.53 0.27 0.52 17.0 + 1 l 1.14 1.57 0.32 0.93 0.27 0.90 A11 142 0.23 0.68 0.19 0.50 0.17 0.49 Basswood 3.0 - 4.9 0 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 18 0.36 0.28 0.77 0.39 0.72 0.39 11.0 - 16.9 55 0.42 0.47 0.83 0.38 0.80 0.38 17.0 + 9 N/A N/A N/A N/A N/A N/A All 82 0.45 0.45 0.83 0.38 0.78 0.39 American Beech 3.0 - 4.9 3 N/A N/A N/A N/A N/A N/A 5.0 - 10.9 19 0.23 0.37 0.24 0.36 0.22 0.35 11.0 - 16.9 24 0.45 0.49 0.50 0.50 0.44 0.50 17.0 + 5 N/A N/A N/A N/A N/A N/A All 52 0.32 0.44 0.32 0.47 0.28 0.46 All Species 3.0 - 4.9 81 0.18 0.27 0.21 0.29 0.22 0.28 5.0 - 10.9 697 0.30 0.39 0.35 0.39 0.33 0.38 11.0 - 16.9 754 0.29 0.50 0.33 0.47 0.31 0.47 17.0 + 300 0.23 0.74 0.23 0.57 0.20 0.58 All 1842 0.27 0.50 0.31 0.45 0.29 0.45 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run over a single lO-year cycle. 3 Simulation run over two five-year cycles. ‘1 Number of trees. 5 Sample size was less than 10, so no analysis was run. 76 precise than the other for individual species or size-classes. TWIGS overpredicted diameter growth for each species and size-class in both simulations, as FVS had done. A comparison of TWIGS’ and FVS’ diameter prediction accuracy indicated that the division between areas where FVS produced more accurate results and where TWIGS produced more accurate results depended solely upon species, and not on size-class. F VS produced more accurate estimates of diameter growth across all species and size- class combinations but one for red oak, white oak, sugar maple, and basswood, while TWIGS produced more accurate estimates of diameter growth across all species and size-class combinations for the other red oak group, red maple, and beech. FVS also produced more accurate estimates for all but the largest size-class when all species were combined. Large differences in accuracy between the two simulators occurred in predictions of diameter growth for the other red oak group and basswood, as well as in the largest size-class for red maple. TWIGS predictions were, in general, slightly more precise than those of FVS. Diameter growth predictions for the other red oak group using TWIGS were the most accurate of any species for the two simulators. Basswood predictions using TWIGS were the least accurate. White oak growth estimates using TWIGS were the most precise, while F VS red maple predictions were the least precise. A comparison of the estimates produced by F VS and TWIGS for diameter grth between 1986 and 1991 (Table 21) displays a pattern of results similar to that of the 10- year projection comparison. As was the case in the FVS analysis, TWIGS overpredicted diameter growth for all species and size-class combinations. FVS once again produced more accurate diameter growth estimates for all size—classes of red oak, basswood, and 77 Table 21. Mean error (bias), and standard deviation (5) of mean error for estimated dbh (in.) by species and size class for 5-year FVS (Full) and TWIGS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value nrinus the observed value. rvs (Full 1 ) TWIGS Species Diameter Class 11 2 Mean Error 5 Mean Error 8 N. Red Oak 3.0 - 4.9 15 0.15 0.13 0.19 0.10 5.0 - 10.9 200 0.13 0.19 0.18 0.19 11.0 - 16.9 336 0.04 0.25 0.14 0.25 17.0 + 153 -0.04 0.39 0.05 0.33 All 706 0.07 0.28 0.14 0.26 Other Red Oak 3.0 - 4.9 9 N/A 3 N/A N/A N/A 5.0 - 10.9 85 0.14 0.26 0.08 0.21 11.0 - 16.9 111 0.22 0.29 0.10 0.20 17.0 + 39 0.25 0.40 0.09 0.27 All 244 0.18 0.30 0.09 0.21 White Oak 3.0 - 4.9 28 0.22 0.21 0.21 0.13 5.0 -10.9 189 0.19 0.17 0.19 0.15 11.0 - 16.9 107 0.10 0.23 0.14 0.21 17.0 + 12 0.34 0.48 0.34 0.27 All 338 0.17 0.22 0.18 0.18 Sugar Maple 3.0 - 4.9 23 0.09 0.19 0.14 0.19 5.0 - 10.9 159 0.27 0.27 0.29 0.26 11.0 - 16.9 52 0.20 0.27 0.16 0.24 17.0 + 5 N/A N/A N/A N/A All 244 0.24 0.26 0.24 0.25 Red Maple 3.0 - 4.9 39 0.09 0.19 0.07 0.18 5.0 - 10.9 73 0.11 0.28 0.08 0.26 11.0 - 16.9 23 0.40 0.36 0.27 0.27 17.0 + 9 N/A N/A N/A N/A All 151 0.16 0.37 0.12 0.26 Basswood 3.0 - 4.9 1 N/A N/A N/A N/A 5.0 - 10.9 18 0.21 0.19 0.42 0.20 11.0 - 16.9 56 0.23 0.22 0.41 0.20 17.0 + 6 N/A N/A N/A N/A All 81 0.24 0.23 0.41 0.20 American Beech 3.0 - 4.9 6 N/A N/A N/A N/A 5.0 - 10.9 21 0.16 0.25 0.12 0.21 11.0 - 16.9 23 0.17 0.22 0.16 0.29 17.0 + 4 N/A N/A N/A N/A All 58 0.14 0.29 0.12 0.24 All Species 3.0 - 4.9 123 0.11 0.19 0.12 0.17 5.0 - 10.9 802 0.19 0.23 0.19 0.22 11.0 - 16.9 775 0.14 0.29 0.15 0.26 17.0 + 246 0.08 0.47 0.09 0.35 All 1962 0.15 0.29 0.17 0.25 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Number of trees 3 Sample size was less than 10, so no analysis was run. 78 for all species together by a very small margin, while TWIGS once again produced more accurate estimates for the other red oak group, red maple, and beech by a small margin. Differences in accuracy between the two simulators predictions for sugar maple and white oak growth were nearly zero. Again, TWIGS estimates were slightly more precise than those of FVS for nearly every species and size-class combination. The most accurate predictions for any species were made by F VS for red oak growth. TWIGS basswood growth estimates were the least accurate of any species. White oak grth estimates using TWIGS were the most precise of any species, while FVS red maple growth estimates were the least precise. A comparison of the results of F VS and TWIGS 5-year simulations of growth for the period between 1991 and 1996 (Table 22) produced a number of different patterns from those of the previous two comparisons. FVS underpredicted diameter growth for the other red oak group, sugar maple, and red maple, while TWIGS underpredicted growth for the other red oak group alone. FVS produced more accurate growth estimates for nearly every species and size-class combination, while TWIGS produced more precise estimates for most combinations. No size-class produced consistently poor results for either simulator. FVS estimates were more accurate by a sizable margin for basswood, American beech and white oak. The highest accuracy was achieved with F VS for red maple, while the lowest was for basswood using TWIGS. Estimates for white oak growth using TWIGS were the most precise, while those for sugar maple using both FVS and TWIGS were equally the least precise. 79 Table 22. Mean error (bias), and standard deviation (5) of mean error for estimated dbh (in.) by species and size class for S-year F VS (Full) and TWIGS projections (1991—1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. FVS (Full 1 ) mos Species Diameter Class 11 2 Mean Error s Mean Error 8 N. Red Oak 3.0 - 4.9 6 N/A 3 N/A N/A N/A 5.0 - 10.9 170 0.06 0.19 0.15 0.20 11.0 - 16.9 333 0.10 0.27 0.17 0.24 17.0 + 211 0.07 0.34 0.10 0.31 All 721 0.08 0.28 0.15 0.26 Other Red Oak 3.0 - 4.9 1 N/A N/A N/A N/A 5.0 - 10.9 70 -0.04 0.18 -0.04 0.17 11.0 - 16.9 119 0.01 0.22 -0.02 0.21 17.0 + 41 -0.14 0.30 -0.16 0.29 All 231 -0.02 0.23 -0.05 0.22 White Oak 3.0 - 4.9 17 0.06 0.21 0.16 0.21 5.0 - 10.9 161 0.06 0.18 0.16 0.17 11.0 - 16.9 102 0.03 0.21 0.13 0.20 17.0 + 12 -0.03 0.31 0.17 0.27 All 292 0.05 0.20 0.15 0.19 Sugar Maple 3.0 - 4.9 17 0.05 0.11 0.18 0.11 5.0 - 10.9 148 0.02 0.26 0.16 0.24 11.0 - 16.9 59 -0.17 0.38 -0.07 0.40 17.0 + 8 N/A N/A N/A N/A All 239 -0.03 0.30 0.11 0.30 Red Maple 3.0 - 4.9 30 0.02 0.18 0.02 0.18 5.0 - 10.9 85 0.00 0.23 0.07 0.26 11.0 - 16.9 31 -0.06 0.29 0.06 0.30 17.0 + 11 -0.14 0.51 0.02 0.50 All 159 -0.01 0.26 0.06 0.28 Basswood 3.0 - 4.9 0 N/A N/A N/A N/A 5.0 - 10.9 15 0.12 0.20 0.35 0.24 11.0 - 16.9 54 0.09 0.21 0.39 0.19 17.0 + 9 N/A N/A N/A N/A All 78 0.10 0.24 0.39 0.22 American Beech 3.0 - 4.9 4 N/A N/A N/A N/A 5.0 - 10.9 24 0.07 0.18 0.23 0.22 11.0 - 16.9 28 0.17 0.33 0.32 0.32 17.0 + 5 N/A N/A N/A N/A All 62 0.1 1 0.27 0.25 0.29 All Species 3.0 - 4.9 76 0.03 0.16 0.10 0.19 5.0 - 10.9 714 0.04 0.22 0.14 0.22 11.0 - 16.9 788 0.06 0.28 0.13 0.29 17.0 + 320 0.04 0.35 0.05 0.33 All 1911 0.05 0.27 0.12 0.27 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Number of trees 3 Sample size was less than 10, so no analysis was run. 80 It appears from these comparisons of the predictive ability of FVS and TWIGS that each simulator predicts better than the other for certain species over a ten-year period. TWIGS estimates for this projection length were more precise than those of FVS, while FVS predicted diameter growth more accurately for a larger number of species. The results of the two five-year comparisons indicate that F VS consistently produces more accurate five-year projections, while TWIGS produces more precise projections for periods not experiencing defoliation and drought. Higher prediction accuracy for periods when these do occur can be achieved with either FVS or TWIGS, depending upon the species. Trees/Acre Error Analysis As previously stated, information on the predictive power of TWIGS’ and FVS’ mortality components is provided by an analysis of the simulators’ predictions of TPA. A comparative examination of the predictive abilities of F VS’ and TWIGS’ mortality components is made for the two different ten-year projections, followed by a similar comparison for the two five-year projections. Throughout the FVS TPA analysis, the “noDI” simulation level consistently performed as well or better with regards to accuracy than either of the other two levels. In addition, accuracy was consistently higher between the two sets of 10-year projections for the projection made over two S-year cycles than the projection made over a single lO-year cycle. For these reasons, comparisons to 5 and 10-year TWIGS TPA estimates will be made with results from the “non’ simulation 81 level in 5 and lO-year FVS analyses, with lO-year TWIGS projections being compared to 10-year F VS projections made over two 5-year cycles. A comparison of F VS and TWIGS 10-year TPA projections (Table 23) produced two important results. The first result was that a 'I'WIGS lO-year projection run over two 5-year cycles produced more accurate estimates of individual species TPA than a TWIGS lO-year projection run over a single 10-year cycle. The second result indicated that TWIGS produced more accurate lO-year estimates of plot—level TPA than FVS for a number of species (red oak, other red oak, red maple, and basswood), as well as for all species combined. The margin of disparity between estimates was small for nearly all Table 23. Mean error (bias), and standard deviation (8) of mean error for estimated trees/acre (TPA) by species for lO-year FVS and TWIGS projections run over two five-year cycles, and a lO-year TWIGS projection run over a single 10- year cycle. Bias is expressed as the predicted value minus the observed value. FVS(Full)1 TWIGS (1x10)2 TWIGS (2x5) 3 Mean Mean Mean Species n ‘1 Error S Error S Error 8 N. Red Oak 96 -5.5 51.4 -4.6 51.1 -4.2 50.6 Other Red Oak 57 -1.6 35.3 -0.7 34.0 -0.7 34.4 White Oak 89 14.1 74.1 14.2 74.3 14.2 74.2 Sugar Maple 33 4.0 108.9 6.2 112.0 5.7 111.6 Red Maple 62 -21.9 65.9 -20.9 65.2 -20.5 62.6 Basswood 16 2.3 12.3 2.2 12.7 2.2 12.7 American Beech 18 -4.7 30.1 -28.6 84.5 -29.2 85.2 All Species 142 -5.5 89.0 -5.1 95.0 -4.7 94.2 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run over a single lO-year cycle. 3 Simulation run over two consecutive five-year cycles. ‘1 Number of sample points. 82 species, however. Predictions for the other red oak group were the most accurate for each simulator. White oak and red maple TPA predictions were considerably less accurate than those of other species for both simulators; TWIGS predictions of beech TPA were highly inaccurate. The level of precision achieved by the two simulators varied widely across the different species groups. TWIGS TPA predictions were slightly less precise than those of FVS for the majority of the species groups examined. Basswood TPA estimates were the most precise across both simulators while estimates for sugar maple and all species combined were the least precise. A comparison of F VS and TWIGS 5-year TPA projections for the period fi'om 1986-1991 (Table 24) produced noticeably different results than those or the 10-year projections. F VS more accurately predicted plot-level T PA than TWIGS for five of seven species (red oak, sugar maple, red maple, basswood, and American beech), as well as for all species combined. TWIGS produced more accurate predictions for both the other red oak group and white oak TPA. Unlike the previous comparison, white oak predictions were no longer among the least accurate, but rather along with basswood were among the most accurate for both simulators. Red maple predictions continued to be among the least accurate, along with predictions for all species combined. Neither simulator consistently offered more precise predictions than the other across the different species, with the level of precision obtained for different species again varying widely. The precision of TWIGS’ beech TPA estimates improved dramatically over those of the previous lO-year comparison, making predictions for that species along with basswood 83 Table 24. Mean error (bias), and standard deviation (s) of mean error for estimated trees/acre (TPA) by species for five-year FVS and TWIGS projections (1986-1991) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. FVS(noDI) 1 TWIGS Species 11 2 Mean error s Mean error s N. Red Oak 96 -4.7 72.4 -6.5 72.9 Other Red Oak 58 -7.3 35.6 -6.6 31.0 White Oak 90 1.1 59.9 0.8 46.3 Sugar Maple 36 -1.9 88.1 -3.6 88.1 Red Maple 65 -13.2 43.3 -13.5 43.2 Basswood 17 0.0 13.7 -0.4 13.8 American Beech 20 -2.5 15.6 -2.7 17.9 All Species 146 -13.3 88.0 -14.1 87.9 1 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 2 Number of sample points. the most precise for each of the two simulators. Predictions for sugar maple and all species combined were once again the least precise. The results for the comparison of 5-year FVS and TWIGS’ predictions of plot- level TPA for the period from 1991-1996 (Table 25) differ in many ways from those of the previous 5-year comparison. As in the 10-year comparison, TWIGS TPA predictions were more accurate than those of FVS for the majority of the individual species groups examined, as well as for all species combined. FVS predictions were more accurate for only the white oak, other red oak, and beech forest types for both simulators. Predictions for the ORO group along with basswood were among the most accurate, whereas ORO group predictions were among the least accurate in the previous comparison analysis. 84 Predictions for all species together were no longer among the least accurate as well. Red maple predictions remained the least accurate for FVS predictions, while beech predictions reverted back to the high level of inaccuracy seen in the comparison of FVS and TWIGS lO-year predictions. Once again, neither simulator consistently provided more precise estimates than the other. Basswood predictions remained the most precise while predictions for American beech were the least precise. TWIGS’ predictions for beech had more than twice the imprecision of any other species. The results of these comparison tests of FVS’ and TWIGS’ TPA predictions indicate first that, as seen in the diameter growth comparisons, 10-year TWIGS’ simulations over shorter cycle lengths produced more accurate estimates of plot-level TPA than TWIGS predictions for the same period using longer cycle lengths. This bears Table 25. Mean error (bias), and standard deviation (s) of mean error for estimated trees/acre (TPA) by species for five-year FVS and TWIGS projections (1991-1996) run over a single five-year cycle. Bias is expressed as the predicted value minus the observed value. FVS(noDI) 1 TWIGS Species 11 2 Mean error s Mean error s N. Red Oak 95 -3.7 35.5 -2.6 33.7 Other Red Oak 52 1.1 24.0 1.6 22.5 White Oak 81 8.2 48.3 8.4 48.4 Sugar Maple 32 -6.5 48.4 -2.5 50.6 Red Maple 63 -10.0 49.5 -8.8 45.6 Basswood 15 -0.5 11.5 -0.3 11.3 American Beech 17 -1.9 69.6 -14.2 190.4 All Species 136 -4.0 62.9 -2.3 84.3 1 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 2 Number of sample points. 85 out the claim made by Miner et al. (1988). In addition, TWIGS produced more accurate estimates than FVS for most species during this same 10-year period, while F VS predictions were generally more precise. As in the diameter growth comparisons, results of the two five-year comparisons show that FVS consistently produces more accurate five-year projections. TWIGS, on the other hand, produces more precise projections for periods not experiencing outbreaks of leaf defoliators and drought. Higher accuracy of prediction for periods when these do occur can be achieved with either FVS or TWIGS, depending upon the species being projected. TWIGS had problems accurately and precisely predicting beech TPA in both 10-year projections and the second 5-year projection, while predictions for the other species had roughly equal accuracy for the two simulators. As in the F VS comparison analyses, an analysis was done to compare FVS’ and TWIGS’ abilities to accurately predict stand level TPA. The results of this analysis for the “noDI” simulation level of the different FVS simulations and the four different TWIGS simulations (Table 26) appear to indicate that, as in the plot-level TPA analysis, slightly more accurate lO-year projections are obtained when the cycle length is shortened from ten to five years. Precision was essentially equivalent for the different 10- year projections for both FVS and TWIGS. Unlike the plot level TPA analysis, the FVS “noDI” simulation level for the 10-year projection over two 5-year cycles provided more accurate estimates of stand TPA than either of the other two TWIGS 10-year projections. FVS projections were also slightly more precise. With regards to the two sets of 5-year simulations, TWIGS estimates were more accurate than those of FVS for both time 86 Table 26. Mean error (bias), and standard deviation (8) of mean error for stand-level trees/acre (TPA) by FVS and TWIGS simulation. Bias is expressed as the predicted value minus the observed value. FVS(noDI) 1 TWIGS Species 11 2 Mean error 8 Mean error s 86-96 (1 cycle) 41 -21.5 164.3 -21.1 164.6 86-96 (2 cycles) 41 —l9.9 164.5 -20.1 165.0 86-91 (1 cycle) 42 -14.1 71.7 -13.1 71.6 91-96 (1 cycle) 44 -16.4 175.7 -15.2 175.9 1 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 2 Number of stands. periods, with those for 1986-1991 being slightly more accurate than estimates for 1991- 1996. Neither simulator produced noticeably more precise estimates of stand TPA than the other. Projections for the period from 1986-1991 were more than twice as precise as those for 1991-1996 for both simulators, indicating a lack of improvement in prediction accuracy for periods not experiencing outbreaks of leak defoliators and drought as seen previously. Basal Area Error Analysis Along with diameter growth and TPA, a comparison of FVS and TWIGS BA prediction accuracy is necessary to determine how well the different components of each simulator work together to project total forest growth. Throughout the FVS BA analysis, the “Full” simulation level consistently performed as well or better with regards to accuracy than either of the other two simulation levels. In addition to this, between 87 the two lO-year projections, accuracy was slightly higher for the projection made over a single lO-year cycle than the projection made over two 5-year cycles. For these reasons, comparisons to 5 and 10-year TWIGS TPA estimates were made with results fiom the “Full” simulation level in 5 and 10-year FVS analyses, with lO-year TWIGS projections being compared to 10-year FVS projections made over a single lO-year cycle. The results of the BA comparison for lO-year FVS and TWIGS projections (Table 27) indicate that, unlike the TPA comparison analysis, neither the projection over a single lO-year cycle nor the projection over two 5-year cycles produced consistently more accurate or precise BA estimates than the other across the different species groups. Table 27. Mean error (bias), and standard deviation (s) of mean error for estimated basal area (BA) (ft2 /acre) by species for lO-year FVS and 'I'WIGS projections run over a single 10-year TWIGS and a lO-year 'I'WIGS projection run over two five-year cycles. Bias is expressed as the predicted value minus the observed value. Full1 TWIGS (1x10)2 TWIGS (2x5)3 Mean Mean Mean Species n 4 Error 8 Error S Error S N. Red Oak 96 -2.6 10.7 0.4 10.0 0.2 9.6 Other Red Oak 57 0.1 8.3 -1.0 8.9 -1.1 9.0 White Oak 89 6.0 10.6 6.8 10.3 6.8 10.3 Sugar Maple 33 2.0 12.5 4.3 13.3 4.0 13.0 Red Maple 62 -1.6 6.8 -1.6 6.0 -1.7 5.8 Basswood 16 2.4 5.5 6.2 6.6 6.0 6.5 American Beech 18 -0.2 2.0 -2.8 6.8 -3.0 7.0 All Species 142 4.3 14.3 7.2 14.4 7.0 14.2 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Simulation run over a single lO-year cycle. 3 Simulation run over two consecutive five-year cycles. 4 Number of sample points. 88 F VS predictions were, however, consistently more precise than those of either lO-year TWIGS projection for each individual species with the exception of northern red oak, as well as for all species combined. FVS estimates for the other red oak group, sugar maple, basswood, and American beech were more than twice as accurate as those of TWIGS. White oak consistently had the worst prediction accuracy of any species for both TWIGS and FVS, while other red oak was the most accurate. Neither simulator consistently produced more precise BA estimates across the different species examined. Estimates for all species combined, sugar maple, northern red oak, and white oak were the least precise. American beech and basswood BA estimates were the most precise, with standard deviations less than 6 ft2 / acre. The comparison of FVS and TWIGS 5-year TPA projections for the period from 1986-1991 (Table 28) produced very unique results. FVS and 'I'WIGS estimates of BA were essentially equally accurate for all seven species, with those for red oak differing by 0.1 ft2 lacre. Precision for these estimates were also nearly identical for all eight species. Only estimates for all species combined were noticeably dissimilar, with FVS producing more accurate predictions. BA estimates for each species group had mean errors less than 2.0 ft2 /acre , with beech and red maple BA being the most accurate. Beech estimates were also the most precise, while those for all species combined, and for sugar maple were the least precise. The results for the comparison of 5-year F VS and TWIGS’ predictions of plot- level TPA for the period from 1991-1996 (Table 29) do not follow a similar pattern. FVS produced more accurate estimates for four of seven individual species groups (white 89 Table 28. Mean error (bias), and standard deviation (8) of mean error for estimated basal area (BA) (ft2 / acre) by species for five-year FVS and TWIGS projections (1986-1991) run over a single five- year cycle. Bias is expressed as the predicted value minus the observed value. F VS(Full) 1 TWIGS Species 11 2 Mean error s Mean error s N. Red Oak 96 -1.9 7.6 -2.0 7.0 Other Red Oak 58 -1.2 4.9 -1.2 4.9 White Oak 90 1.9 4.8 1.9 4.7 Sugar Maple 36 1.6 8.4 1.6 8.4 Red Maple 65 -O.6 3.2 -0.6 3.2 Basswood 17 1.1 3.9 1.1 3.9 American Beech 20 -0.4 1.2 -0.4 1.2 All Species 146 0.5 9.6 1.8 9.5 1 Simulation run with site index and age information for each stand individual tree diameter increment information. 2 Number of sample points. oak, sugar maple, basswood and American beech) and for all species combined. TWIGS produced more accurate estimates for northern red oak, other red oak, and red maple. Mean errors of prediction for all species other than white oak were less than 2.0 ft2 / acre for both simulators, with white oak mean error being less than 4.0 ft2 /acre for both FVS and TWIGS. Neither simulator consistently produced more precise BA estimates across all species groups than the other. Predictions of BA for beech were the most precise, while those for all species combined and for the three oak species groups were the least precise across both simulators. Unlike the TPA and DBH comparisons, the results of the BA prediction comparisons do not indicate that shorter cycle lengths in 'I'WIGS projections result in 90 Table 29. Mean error (bias), and standard deviation (s) of mean error for estimated basal area (BA) (ft2 /acre) by species for five-year FVS and TWIGS projections (1991-1996) run over a single five- year cycle. Bias is expressed as the predicted value minus the observed value. FVSO2ull)l TWIGS Species 11 2 Mean error s Mean error s N. Red Oak 95 -1.6 6.8 0.3 6.3 Other Red Oak 52 -0.4 6.5 -0.2 6.4 White Oak 81 2.5 7.3 3.7 7.7 Sugar Maple 32 -1.3 5.8 2.0 5.3 Red Maple 63 -1.0 4.9 -0.6 5.2 Basswood 15 0.1 2.8 1.1 3.0 American Beech 17 0.2 1.0 -l.0 3.2 All Species 136 0.7 9.5 4.1 9.3 1 Simulation run with site index and age information for each stand and individual tree diameter increment information. 2 Number of sample points. more accurate estimates of plot-level BA for the seven species examined. Ten-year projections with FVS resulted in more accurate estimates of plot BA for nearly all species, when compared to TWIGS projections for the same period, although neither F VS nor TWIGS consistently produced more precise estimates than the other. Unlike the two previous sets of comparisons, neither simulator consistently produced more accurate or precise results across species groups for either of the two five-year periods examined. In addition, prediction accuracy did not seem to consistently increase or decrease for the period not experiencing outbreaks of leaf defoliators and drought. As in the previous TPA analysis, a comparison was done of both FVS and TWIGS abilities to predict firture stand level BA. Unlike the previous analysis, the 91 results of this analysis (Table 30) indicated that more accurate lO-year projections with TWIGS are obtained when the cycle length is extended from five to ten years, as is the case for FVS. However, precision was not consistently higher for one projection over the other. FVS projections produced more accurate estimates of 10-year BA than TWIGS for both five and 10-year cycle length projections, although TWIGS estimates had slightly higher precision. A similar situation was found for five-year projections using the two simulators. FVS five-year predictions were more accurate than those of TWIGS, but at the same time were less precise for both five-year periods. At the same time, five-year projections for the period which did not experience gypsy moth outbreak or drought were considerably more accurate, if less precise, than projections for the period which experienced both. In summary, the results of this study indicate that, for the area in question, F VS simulations including diameter increment information, along with stand site index and age information, generally produce more accurate estimates of five and ten-year individual Table 30. Mean error (bias), and standard deviation (s) of mean error for stand-level basal area (BA) (ft2 /acre) by FVS and TWIGS simulation. Bias is expressed as the predicted value minus the observed value. FVS(Full) 1 TWIGS Species 11 2 Mean error 5 Mean error s 86-96 (1 cycle) 41 8.8 13.2 11.3 12.7 86-96 (2 cycles) 41 9.6 12.9 11.2 12.7 86-91 (1 cycle) 42 4.0 8.6 5.6 7.1 91-96 (1 cycle) 44 0.4 13.2 3.6 12.9 1 Simulation run with site index and age information for each stand, but without individual tree diameter increment information. 2 Number of stands. 92 tree diameter growth than those of other FVS simulation levels. This was true for both plot and stand-level changes in BA. More accurate estimates of plot and stand level trees/acre were generally obtained using the simulation which included stand level site index and age information, but did not include individual tree diameter increment information. Results of the FVS and TWIGS comparison tests indicated that each simulator tended to produce more accurate estimates of 10-year individual tree diameter growth than the other for different species as a whole, regardless of size-class. Results for five-year grth estimates were less clear, as F VS produced consistently more accurate diameter growth estimates across the different species examined for the period not experiencing gypsy moth outbreak and drought. The period experiencing both stress factors produced results similar to those of the 10-year analysis. A comparison of TPA predictions for the two simulators failed to provide conclusive results as to which of the two simulators would consistently produce more accurate or precise estimates of plot or stand-level TPA. Results for the BA comparison were similar for five-year projections. FVS provided more accurate lO-year estimates of plot level BA for nearly every species examined. Stand-level BA estimates for FVS were more accurate for both five and ten- year projections than those of TWIGS, although less precise. A discussion of these results and possible explanations for them is contained in the following section. DISCUSSION No projection system can perfectly represent the real system being modeled, and therefore little is gained from simply showing that the projection system under examination is an inexact copy of nature (Buchman and Shifley 1983). Recognizing this fact and keeping in mind their regional calibrations, we evaluated the abilities of FVS Lake States and Lake States TWIGS to predict individual tree and stand-level growth for upland hardwoods in Michigan’s Lower Peninsula. This evaluation may be usefiil both to the people who use these models and the people who create them. However, our target audience is the district manager who can take the results presented here and apply them. As many district managers will attest, even poor estimates can be better than no estimates at all. This study compared FVS’ and TWIGS’ growth predictions, as well as the predictive abilities of different simulation levels within FVS Lake States. The results are meant to provide forest managers with specific information on the accuracy and precision of growth and yield estimates which can be attained using F VS and TWIGS for the study area. Before moving into a discussion of the results of the preceding chapter, a broader issue relating to this study must first be addressed. The projections analyzed here deal with growth over five and ten-year periods. Ten-year simulations appear to be the standard for model validation, having been used to analyze the predictive power of the 93 94 different predecessors to the TWIGS and FVS Lake States simulators. The validity of five-year growth projections, on the other hand, have been questioned due to the potential for measurement error to mask the accuracy of prediction attained by a model for periods of this length. Despite this concern, five-year growth data and projections have been used in a number of studies. The first sub-regional TWIGS validation study for the Lower Peninsula of Michigan (Guertin and Ram 1996) analyzed TWIGS ability to accurately predict five-year growth. Kowalski and Gertner (1989) and Payendeh and Papadopol (1994) also used five-year growth data and projections in their studies of TWIGS. Guertin and Ram (1996) point out that short-terrn projections are often used to compare early responses to different treatments, for updating inventories, and for evaluating models against temporary growth plots. It is for these reasons that this study was not limited to an analysis of ten-year growth projections. In addition, field procedures for this study were designed to minimize the amount of measurement error as much as possible. FVS Validation 10-Year Projection The tables in the Results section indicate that, as one may expect, the F VS simulation level using all available individual tree and stand-level information (“Full”) produced the most accurate estimates of individual tree diameter grth and plot-level BA (Figure 2). There were, however, two unexpected results. The first surprising result was the consistency with which simulations without diameter increment information, 95 1.0 25 .r r 20.. 0.75- r ' I O a a g 15 G 0.5“ 2 10- 15 O O m o O ‘5 025- O E] 5- O c: . ______________________ 3 0.0 ———————————————————————— g 0 E g -5. 025- I I- . -10‘ -0.5 I I I -15 I I I Full noDI noSI,A,DI Full noDI noSI,A,DI Simulation Level Simulation Level Figure 2. Mean and standard deviation for mean error of predicted DBH (inches) and BA (ft2 lacre) for all species combined by simulation level for projection over a single lO-year cycle (positive values are overpredictions). stand site index, or stand age information (“noSI,A,DI”) produced more accurate diameter growth and BA estimates than simulations lacking only the diameter increment information (“noDI”). This seems to indicate that site index and age information have the effect of lowering F VS’ prediction accuracy. The second surprising result was the fact that the “noDI” level produced the most accurate estimates of plot and stand-level TPA (Figure 3). This turns out to be an affect of the first surprising result. Before discussing these results, though, it is useful to first look at the roles these variables (diameter increment, age, site index) play in calculating growth estimates in FVS. As stated previously, FVS and TWIGS use the same three linked equations to predict diameter growth (equation 1): the potential growth function (equation 2), the competition modifier (equation 3), and the diameter adjustment factor (equation 4) CI. 96 100 Mean Error TPA Ol I | l | A l V l l I | Al VF l I | | Cl -100 .L .1} Full noDI noSI,A,DI Simulation Level Figure 3. Mean and standard deviation for mean error of predicted TPA over a single lO-year cycle for projection over a single lO-year cycle (positive values are overpredictions). (Miner et al. 1988). It is the potential growth fiinction (equation 2) to which we must look in order to understand the unexpected results we obtained. When diameter growth information is included in a projection for five or more trees of the same species in a stand, FVS will recalibrate its diameter growth fiinction for that species to fit the site’s growth characteristics. Keeping in mind FVS’ regional nature, it is therefore understandable why leaving this information out resulted in noticeably less accurate predictions of diameter growth. Age information is not used directly in diameter grth calculations, but rather in conjunction with site index by FVS’ height growth equations. Site index, however, does play a direct role in the potential growth firnction of F VS’ diameter growth calculations. As a result of the b 4 coefficient being positive across all 97 species, increases in site index values result in increases in diameter grth estimates. This has important implications when looking at results for the study area. Overall, FVS overpredicted diameter growth for 10-year simulations at all three simulation levels for both five and 10-year cycle lengths by an average of 0.27 to 0.39 in. (Figure 4). In addition, each individual species and size-class combination was also overpredicted for 10-year projections (Figure 5). Such consistency in overprediction can be attributed to F VS’ calibration with data at a regional level. With so much room for improvement, it was not surprising that the differences between the recalibrated diameter estimates for the “Full” simulation level and those for the “noDI” level were as large as they were (Figure 2). What was surprising was that the “noSI,A,DI” simulation level also produced more accurate estimates than the “noDI” simulation level. Again, this 1.0 0.75-1 ' r t E 0.54 E2025- 0 13 ‘3 m g 00- ----- -—— —————— -—— ----- —-- ----- 2 I. -025- ' " '0.5 I I I I I I .9. 1" .9. 1" S 1" 35 (ii _>_,< <11 35 (31 Full noDI noSI,A,DI Figure 4. Mean and standard deviation for mean error of predicted DBH (inches) for 10- year projections by cycle length and simulation level (positive values are overpredictions). Mean Error DBH Mean Error DBH Northern Red Oak 98 3.0 2.5- 2.0‘ 1.51 1.0' -O.5 ‘ -1.0 0.59 ....... 12-- __ DBH Class White Oak - q DBH Class Mean Error DBH Mean Error DBH 5.5 Other Red Oak 3.0 2.5-1 2.0-1 1.5-' 0.5m CD 0.0" ------- - - — — - I- _______ -0.5 ~» ’].0 I I I I I DBH Class Sugar Maple 3.0 2.5 . 2.0 - 1.5 e 1.0 a 0.5 - 0,0‘r--§—-— -— __________ -O.5 T -1.0 I I I I I DBH Class (Figure continued on next page) Mean Error DBH Mean Error DBH 99 Red Maple Basswood 3.0 3.0 2.5 - I' 2.5 I 2.0 - 2.0 I E 1.5 - Q 1.5 ‘ o 1g 1.04 m 1.0 - 0.5 . 5 0 5 ~ 0 . z i 0.0-~-- --- -- ---~-- --- 0.0+ --------------- --- '05 ' 13 -0.5 I '1-0 I I I I l '10 I I I I I A B C D E A B C D E DBH Class DBH Class American Beech All Species Combined 3.0 3.0 2.5 A 2.5 e 2.0 e 2.0 .. {I} 1.5 4 g 1.5 - 5' 1.0 4 l: 1.0- .- LL] 0.5 - § 054 (i E 512 O 0.02 ------- —- —————— ___. 0,04--- —-- —— ——— ------- -0.5 - -05 - .. '1-0 I I I I I -1.0 l I I 1 1 A B C D E A B C D E DBH Class DBH Class DBH Classes A : 3.0 - 4.9 inches D : 17.0 + inches B : 5.0 - 10.9 inches C: 11.0 - 16.9 inches E : All classes combined Figure 5. Mean and standard deviation for mean error of predicted DBH (inches) by DBH size-class and species for lO-year projections over a single lO-year cycle. This set of figures is indicative of the pattern of prediction observed for all 10- year projections. Only DBH Classes with more than 10 samples are shown. (Positive values are overpredictions). 100 appears to indicate that leaving out site index and age information increases, or at least does not reduce, prediction accuracy. The explanation for this lies in the way F VS Lake States predicts grth of stands for which no site index information has been supplied. Stands such as these are automatically assigned red pine as the site species, with a site index of 60 feet at a base age of 50 years. An examination of the output files for each stand shows this value to also be the assigned site index for all seven species examined in this study. Comparing this value with actual estimated site index values for the 48 stands included in the study showed this uniform assignment of 60 feet at a base age of 50 years to be, on average, eight feet less than the actual Site index values with a standard deviation of 14.6 feet. Because of the role played by site index in calculating of diameter growth, a lower mean value for site index due to poor estimation resulted in lower mean estimates of diameter growth than if actual site index had been used. In other words, true site index values for this area, by virtue of the fact that they were eight feet higher, on average, than the estimated site index values produced by F VS, had the effect of increasing the amount by which F VS overpredicted diameter growth. Users of FVS should keep this and the affect of the diameter growth information in mind when running their own projections. The consistency with which F VS overpredicted individual-tree diameter growth and underpredicted site index for the study area, and the explanation above lead to the conclusion that users predicting growth for this area should not include site index and age values in their projections. In stands outside this region FVS may or may not overpredict annual diameter growth and site index. These instances should be handled differently. Because of the relationship of site index to annual diameter growth predictions, sin cases 101 where diameter growth and site indices are both overpredicted, site indices should be included in growth projections. When diameter growth is underpredicted and site indices are overpredicted, site indices should not be used. And, in cases where both diameter growth and site indices are underpredicted, site indices should be used. In their validation of the STEMS85 model, Holdaway and Brand (1986) discussed how an average underprediction of diameter growth by 0.08 in. over 10 years produced a mean bias of 0 ft2 /acre in basal area estimation, while an average overprediction of diameter growth by 0.08 in. resulted in an overestimation of BA by 2 ft2 / acre. Therefore, a slight underprediction of diameter is preferable to a slight overprediction when accurate estimates of both diameter grth and BA are needed. Keeping this in mind, it is clear why the larger overpredictions of diameter growth produced by the “noDI” simulation level resulted in larger overpredictions of plot-level BA than either of the other two levels for projections with cycle lengths of both five and 10 years (Figure 6). FVS consistently overpredicted mortality across all lO-year growth projections, with TPA estimates being most accurate for the “noDI” simulation level (Figure 7). In order to understand why this simulation level produced more accurate estimates of TPA than the other two levels, the diameter growth function must again be examined. As explained above, the lack of diameter growth information and the inclusion of actual site index values are what made the diameter growth estimates produced by the “noDI” simulation level the least accurate, resulting in larger overpredictions than the other two 102 25 20' -- I . _ Q) 154 g 10 El 0 0 <5 (9 g 5‘ <9 (3 LL] 3 0d»-———-<——-— —————— ———- —————— ——-————— S -5-1 ‘- -10-‘ il- -15 I I I I I I 2 In 2 In 2 In .25 «’9‘ s S if <11 Full noDI noSI,A,DI Figure 6. Mean and standard deviation for mean error of predicted basal area (ft2 / acre) for lO-year projections by cycle length and simulation level (positive values are overpredictions). 150 100‘ . " 'r - SE 50- l-‘ E m 0*"‘a>"25“”‘<5"<> ‘‘‘‘ < S”>‘"‘ 8 E -50-* -100- I ~ .. 4 - -150 r r r r r 1 2 In 2 In 2 In 35 «’51 5 61 5 «11 Full noDI noSI,A,DI Figure 7. Mean and standard deviation for mean error of predicted TPA for lO-year projections by cycle length and simulation level (positive values are overpredictions). 103 simulation levels. Examining F VS’ mortality filnction (equation 5), we see that predicted annual diameter growth plays an important role in predicting annual tree survival. Due to the b 3 and b 4 coefficients being uniformly positive, larger diameter estimates result in lower probabilities of mortality. Therefore, overprediction of diameter grth led to higher survival rates than if diameter growth had been predicted more accurately. Hence the reason why the “noDI” simulation level, the least accurate diameter growth predictor, produced higher values for TPA. Recalling that F VS consistently underpredicted TPA for all simulation levels, we also see why TPA estimates for this simulation level were more accurate than those of the other two levels. In addition to a comparison of the different simulation levels, 10-year FVS projections for the same period, but with differing cycle lengths, were also compared. Projections done over a single 10-year cycle were found to produce slightly more accurate estimates of diameter growth than projections made over two five-year cycles (Figure 4). As a result, estimates of BA were slightly more accurate (Figure 6), while those of TPA were slightly less accurate (Figure 7) for the single lO-year cycle projections, for the same reasons as seen in the comparison of the different FVS levels. Differences in the final 10-year estimates produced by these two different simulations arise as a result of changing values for individual tree crown ratio and for certain factors in the competition modifier function. In order to see why this is so, it is necessary to look more closely at how FVS actually goes about “growing” trees. FVS calculates all diameter growth, regardless of projection length, using a 10- year interval. In other words, FVS takes the input values for the difl‘erent variables 104 involved in the estimation of growth and from these directly generates lO-year diameter growth. If the user specifies a shorter cycle length than 10 years, for example five years, then FVS takes these ten-year growth estimates and multiplies them by 0.5. As a result, for the case of the simulations run over a single lO-year cycle, no modifications were made to the initial 10-year growth estimates produced. However, in the case of the projections made with a cycle length of five years in this study, estimates were generated for a ten year period and then cut in half. These new estimates were then used as the input values for a second cycle lO-year projection of growth, also to be cut in half. The use of these new estimates as inputs into the second cycle was the source of the disparity between the growth estimates for the two sets of 10-year projections with different cycle lengths. One of the key variables in the potential growth function (equation 2) of FVS is crown ratio. In the case of the projection over a single lO-year cycle, potential growth for the entire 10-year period was estimated using the initial value for crown ratio. As the projection was 10 years in length, this estimate of potential growth was not altered. In the case of the two cycle projection, the initial value for crown ratio was used to estimate potential growth for 10 years. This estimate was then cut in half, and a new estimate of crown ratio at the end of the five-year projection period was created. This new value, which may or may not have been the same as the original crown ratio value, was then used in calculations for the second 10-year projection of growth. Estimates of growth for this second period were then also cut in half to produce the final lO-year growth estimates. In addition to crown ratio, there are two variables in the competition modifier 105 which may also change in value between the first and second cycles. These variables are AD, the average stand diameter, and R, the ratio of a trees diameter to the average stand diameter. As FVS simulates growth, the value of AD will increase after the first five-year growth period as individual trees are grown and their diameters change. The value of R for each tree will also change as trees of different species and size-classes are grown differently by FVS. Changes in the values of these two variables, along with any changes in value of the crown ratio variable for each tree, will lead to differences in the amount of mortality each tree is expected to experience over the course of the projection. These changes result in different final 10-year diameter growth, TPA, and BA estimates than would have been seen for projections using a single 10-year cycle. Based on these relationships and the previous study by Holdaway (1985) into the prediction accuracy of the crown ratio model, it appears that projections should be made using a single cycle in order to minimize the effect of poorly predicted crown ratio values. However, the user should be carefill when making projections over long periods (> 30 years), during which tree growth rates and crown structures may change dramatically. Further study should be done to confirm these conclusions. Five-Year Projections F ive-year predicted diameter growth accuracy using FVS was greater for the period from 1991-1996 than for the period from 1986-1991 (Figure 8). Measurements for each period were taken from the same trees on permanent research plots. Differences between the two periods appear to include a tent caterpillar outbreak in 1987, a gypsy 106 0.50 v ‘I' 0.25” I O 0 11° O D (D (P ‘5 0 120.00‘ ----- --- —————— —--« —————— ___e__-_ "-1 .. 5 o .I. E -O.25" -0.50 , , I I I I 5. 8 a 8 6? 8 \o ._'. \o ..'.. \b .1. 0° ON 00 O\ 00 0 Full noDI noSI,A,DI Figure 8. Mean and standard deviation for mean error (inches) of predicted TPA for five- year projections by projection period and simulation level (positive values are overpredictions). moth outbreak in 1990-1991, and a drought during the summer of 1988. Stands hit by gypsy moths were sprayed during the summers of 1991 and 1992, thereby reducing the effects this pest had after 1991. Guertin and Ram (1996) postulated that the three disturbances might have reduced growth rates for several of the species examined, especially the oaks, resulting in larger TWIGS overpredictions for the first five-year period. From Table 3 we see that growth rates were lower between 1986 and 1991 than between 1991 and 1996 for five of the seven species and for all species combined. Tables 9 shows that F VS prediction accuracy improved by 0.12-0.21 in. for the five species which experienced increased diameter growth rates during the second five-year period, and did not show noticeable improvement (0.01-0.03 in.) for the two species which did 107 not experience increased diameter growth rates. Further evidence to support Guertin and Ramm’s statement is given by the fact that Dbh growth for three of seven species was underpredicted for the second five-year period, with the overall mean error of prediction for all species being 0.05 inches. For the period from 1986-1991 all seven species were overpredicted, with a mean error of prediction for all species combined being 0.15 inches. The BA and TPA analyses for these two periods did not provide any clear-cut conclusions as to whether or not the drought or defoliation which occurred produced less accurate results. It does appear, however, that these three different types of stress had a combined noticeable effect on the level of accuracy achieved in predicting five-year diameter growth by F VS for five of the seven species examined. FVS and TWIGS Comparison A comparison of 10-year projections for FVS and TWIGS showed each simulator to predict diameter growth for certain species with slightly higher accuracy than the other simulator, with F VS producing more accurate estimates for all species combined (Figure 9). While they use the same equations, FVS and TWIGS differ in the way they estimate growth. TWIGS calculates diameter growth on a yearly basis, generating new values for crown ratio, AD, and R for each year in the prediction cycle, while FVS calculates diameter growth for a lO-year interval as already discussed. This difference opens the door to differences in the accuracy of growth estimates for the two simulators. The fact that FVS recalibrates its diameter growth function in the presence of diameter growth data might lead one to expect FVS estimates to be at least slightly more accurate than 108 .Amaouoaooacgo 8a 839» 03:88 $298 Seamgu 9.3 .66 gov 683 meouoo “ea mOHBH .298 30:: .2qu a .56 once 863 2558.85 m>m .momoomm Ea Lowe—saw 538m 3 maouoohoa .803: e8 Ema 38:85 me 3325 echo 588 new dosage engage. was 532 .e Rafi seam 2&2 2%: Mac 80 Ba no cam BEES OOOBmmam sauce... 8m 9&5 ads? 850 SE62 88% =< l l l l l l l l O A O A o A O A o A o A o A O A S S S S S S S S S S S S S S S S I h I I n b n P n n b n b b h P mh.OI .omd- . A . Ame- w I I ......................... II - -I- - ...... 1-1180 m c c . e e e .20 a . n .03 m . A . . . A; m .2: H AS om; 109 those of TWIGS. However, this was not consistently the case for lO-year projections. A possible explanation for this, which is supported by results from the two five-year simulations, focuses on the possible effects of the defoliation and drought which occurred between 1986 and 1991. Five-year diameter growth estimates for the period from 1986-1991 showed a pattern similar to those of the 10-year analysis (Figure 10). However, estimates for the period from 1991-1996 were very different. FVS produced more accurate estimates than did TWIGS for this second five-year period of diameter growth for all seven species examined and for all species combined (Figure 11). As in the 10-year analysis, neither simulator consistently produced more accurate TPA estimates across all species for either five-year period (Figures 12, 13). FVS estimates of BA for 1991-1996, on the other hand, were more accurate than those of TWIGS for nearly all species (Figure 14), while the two simulators performed nearly equally with respect to BA for the period from 1986-1991 (Figure 15). These results along with Tables 3-5 indicate that defoliation and drought may indeed be the reasons for the lack of a consistent improvement in FVS diameter predictions over those of TWIGS in lO-year projections. However, this hypothesis will require additional data for confirmation. As a result of these findings, it appears that FVS should be chosen over TWIGS when making projections for periods when defoliation and drought are not anticipated. In areas where such events are anticipated, it appears either TWIGS or FVS could be used, as both achieve nearly the same level of accuracy due to the decreased diameter 110 .Amaouomuoégo 2a 839» 033mg .398 §>.o>m 03% a .550 once 803 2280 FE mOEH we“ w> m .86on was 53:86 538w E 2378a 3 80:87am .30.».m .8.“ Ema 3869:“ .«o @225 .85 508 Sm 9:333 Eavgm Ba :82 .2 8:3,.“ 58m 2%: 2%: x5 #5 8m v.5 Ba 85988 voonmam 5053. Ba swam 2.23 sec gaeoz $83 5. l l .I... l l .Iw l l m d m a m d m J m d m d m d m a o A o A o A o A o A o A o A o A S S S S S S S S S S S S S S S S b Ll - n P n n P — - b n n r n - mh.Ol .03- Rd. W i. m I I ................. - II - II - II - ...... II. a ... 8. m .30 m H I .02 mud .Amqouomwefigo 98 839» 25603 .296 “margin ofiwfim a .55 gov 0.63 8038qu mOEH was m>m .88on ER 8338?. 530% 3 Goa- 5a 5 3038.85 130%.m How Ema @82on .«o 90:25 Hobo :88 5m dogged 3393 Ea S32 .2 853m fiBm 2%: 2&2 x8 x8 8m *5 8m BEES @993QO 565:2 vex Hamsm SE? 550 805.52 86on =< L l l l l l l l mg mg mg mg mg mg mg ma 9 A D A D A D A O A 9 A D A D A 1|. S S S S S S S S S S S S S S S S n p p p P n p p P - - — n p p P - WF.OI Iowo- $3- m m .................. II III - II - II - II III. a coo m -30 m H -02 mud 112 $833398? 8a 83? 328$ .298 Hack-02m 23% a .520 25v 203 8038.85 mOHB-w was m>m .86on 23 83:55 £32m 3 Cog-0% C goa8_.o.a 30>.m 8m $2. 364on we 90:25 .55 508 you nouagow vacuum 93 982 .2 warm nooom 29:2 0332 %O xwo 3m Mao wow 339800 895QO .3053 BM Emsm 833 $50 E0582 860mm =< l l l l l l L l mg mg mg mg mg mg mg mg 9 A D A D A 9 A D A 9 A D A D A S S S S S S S S S S S S S S S S n h h — P - n P p L p n r P p - WNNI .02- W $5- a m 1-4-$1-4»; ...... - .:.-4 47:4- - ...... -:.o m i a V .03 mNN 113 .AmaouoE-kego 2a 838, 03:88 .205 Hag-gm Bmfim a .5>o 25v 983 2838.85 @053. can m> m 360% 93 88:55 530% 3 3&2- SaS E538 “Ba Bum-m Ho,“ Jed-H. @88on mo 90:25 Hobo :38 How nowmgov Eng—Sm was 982 .2 oSwE €on 2932 2934 Mao SEC 3% Mao com 358500 Boa-mam quot-«Bax BM “swam 323 550 Eontoz 86on =< L L L L L L L L m a m a m a m a m a m a m a m a D A D A D A D A D A D A D A D A S S S S S S S S S S S S S S S S p P n - — - n p — h — p n p p n WNNI .02. W 13.- 9 % w Inn-c-n II: I: -:- :- lu- - I-» n n..- - lu- - -- I --.- H - m 4 4 L- L- L- L- o m .3 a V -03 mmm 114 .9:ofio€8&u>o 0.8 833 03:88 .298 Hag-95 Ewan a .85 25v 825 9838 ._-o-a mOHBH 98 m> m .86on 98 “82:86 559% 3 6mm: -Eos maouoofioa Sow-m 8,“ Aouoémub ) 550 50552 86on =< - SDIML - SAd - SSIML ' SAd ' SDIML - SAd - SDIML ” SAzl " SDIML ' SAd ' SOIML P SAJ ' SDIML - SDIML G G luuu*uu¢ns*nn*uu nu ------------------ AYnhv-uuAY uuuuuuuu Gnu--o #4- 'SAJ i ”SA-:1 ’A I U 4 V8 10113 anew 115 .Amnouomvoufigo o? 82? 03:88 .296 Hag-02m flwfim a 55 08% 0.53 8550 5.8 mOHB-H. was m>m .85on 83 53:87- 5.:on 3 Caz-was 8250.58 Rea-m 5m @53m 5