USING EXISTING INFRASTRUCTURE AS GROUND CONTROL TO SUPPORT CITIZEN SCIENTIST COASTAL UAS MONITORING PROGRAMS By Lucas Frederick Rabins A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Geography—Master of Science 2022 ABSTRACT USING EXISTING INFRASTRUCTURE AS GROUND CONTROL TO SUPPORT CITIZEN SCIENTIST COASTAL UAS MONITORING PROGRAMS By Lucas Frederick Rabins Recent publications have described the ability of citizen scientists to conduct unoccupied aerial system (UAS) flights to collect data for coastal management. Ground control points (GCPs) can be collected to georeference these data, however collecting GCPs require expensive surveying equipment not accessible to citizen scientists. Instead, existing infrastructure can be used as naturally occurring GCPs (NGCPs), although availably of NGCP placement on such infrastructure differs from published best practices of GCP placement. This study therefore evaluates the achievable accuracy of sites georeferenced with NGCPs through an analysis of 20 diverse coastal sites. At most sites NGCPs produced horizontal and vertical root mean square errors (RMSE) less than 0.060 m which are similar to those obtained using traditional GCPs. To support future UAS citizen science coastal monitoring programs, an assessment to determine the optimal NGCP quantity and distribution was conducted for six coastal sites. Results revealed that generally at least seven NGCPs collected in the broadest distribution across the site will result in a horizontal and vertical RMSE less than 0.030 m and 0.075 m respectively. However, the relationship between these placement characteristics and RMSE was poor, indicating that georeferencing accuracy using NGCPs cannot be optimized solely through ideal quantity and distribution. The results of these studies highlight the value of NGCPs to support UAS citizen science coastal monitoring programs, however they also indicate a need for an initial accuracy assessment of sites surveyed with NGCPs at the onset of such programs. Dedicated to all my family, friends, and mentors who have supported me in recent years, and to all the citizen scientists who volunteer their time to improve their community iii TABLE OF CONTENTS LIST OF TABLES ........................................................................................................................ v LIST OF FIGURES ..................................................................................................................... vi KEY TO ABBREVIATIONS ................................................................................................... viii Introduction ................................................................................................................................... 1 Study Sites, Data, and Methods ................................................................................................... 9 Site Selection and Description ................................................................................................... 9 NGCP and CP Placement ........................................................................................................ 12 Imagery Collection ................................................................................................................... 12 Structure from Motion Processing .......................................................................................... 14 Accuracy Assessment ............................................................................................................... 15 Optimal Quantity & Distribution of NGCPs ........................................................................... 16 Results .......................................................................................................................................... 19 Accuracy Assessment ............................................................................................................... 19 Optimal Quantity of NGCPs .................................................................................................... 22 Optimal Distribution of NGCPs ............................................................................................... 25 Discussion..................................................................................................................................... 28 Conclusion ................................................................................................................................... 34 APPENDIX .................................................................................................................................. 35 REFERENCES............................................................................................................................ 35 iv LIST OF TABLES Table 1. Site type, elevation range, approximate area, quantity of natural ground control points (NGCPs) & check points (CPs), and description of all 20 sites included in this study .................11 Table 2. Effect of removing easily destroyed natural ground control points (NGCPs) on check point (CP) horizontal (XY) and vertical (Z) RMSE .......................................................................22 v LIST OF FIGURES Figure 1. Example of ground control point (GCP)/check point (CP) surveyed on high contrast checkered marker prior to UAS flight ..............................................................................................4 Figure 2. Examples of all types of natural ground control points (NGCPs) and check points (CPs) included in study ....................................................................................................................7 Figure 3. Location of the six communities participating in the Great Lakes citizen science UAS monitoring program established at Michigan State University ............................................10 Figure 4. Example of accuracy assessment natural ground control point (NGCP) and check point (CP) placement scheme at Fifth Ave. Beach ........................................................................13 Figure 5. Distribution of check point (CP) error magnitudes (m) for all sites in the horizontal (orange), vertical (blue), and total (grey) directions .....................................................................19 Figure 6. Check point (CP) and natural ground control point (NGCP) errors for sites with bluff morphology: Lake Superior Roadside Park (top) and Casco Nature Preserve (Bottom) .....20 Figure 7. Natural ground control point (NGCP) quantity vs vertical (Z) RMSE (top) and horizontal (XY) RMSE (bottom) .................................................................................................... 23 Figure 8. Casco Nature Preserve Site natural ground control point (NGCP) quantity vs RMSE ........................................................................................................................................24 Figure 9. Check point (CP) and natural ground control point (NGCP) errors for replications of North Beach using 3 NGCPs ....................................................................................................27 Figure 10. Examples of natural ground control points (NGCPs) placed on objects with varying visibility .........................................................................................................................................31 Figure 11. Placement and errors of NGCPs (blue) and CPs (red) in AuSable shoreline Park, Iosco County .................................................................................................................................37 Figure 12. Placement and errors of NGCPs (blue) and CPs (red) in Bar Lake, Manistee ...........38 Figure 13. Placement and errors of NGCPs (blue) and CPs (red) in Berrien Beach, Chikaming .....................................................................................................................................38 Figure 14. Placement and errors of NGCPs (blue) and CPs (red) in Boardwalk West, Manistique......................................................................................................................................39 Figure 15. Placement and errors of NGCPs (blue) and CPs (red) in Casco Nature Preserve, South Haven ...................................................................................................................................39 vi Figure 16. Placement and errors of NGCPs (blue) and CPs (red) in Cherry Beach, Chikaming ......................................................................................................................................40 Figure 17. Placement and errors of NGCPs (blue) and CPs (red) in Fifth Ave. Beach, Manistee .........................................................................................................................................40 Figure 18. Placement and errors of NGCPs (blue) and CPs (red) in First St. Beach, Manistee .........................................................................................................................................41 Figure 19. Placement and errors of NGCPs (blue) and CPs (red) in Harbert Beach, Chikaming ......................................................................................................................................41 Figure 20. Placement and errors of NGCPs (blue) and CPs (red) in Lake Superior Roadside Park, Marquette .............................................................................................................................42 Figure 21. Placement and errors of NGCPs (blue) and CPs (red) in North Beach, South Haven ...................................................................................................................................42 Figure 22. Placement and errors of NGCPs (blue) and CPs (red) in North Beach Access, Manistee .........................................................................................................................................43 Figure 23. Placement and errors of NGCPs (blue) and CPs (red) in Oscoda Beach Park, Iosco ...............................................................................................................................................43 Figure 24. Placement and errors of NGCPs (blue) and CPs (red) in Pine St. Lakeshore Blvd., Marquette .......................................................................................................................................44 Figure 25. Placement and errors of NGCPs (blue) and CPs (red) in South Beach, South Haven ...................................................................................................................................44 Figure 26. Placement and errors of NGCPs (blue) and CPs (red) in South Beach Park, Marquette .......................................................................................................................................45 Figure 27. Placement and errors of NGCPs (blue) and CPs (red) in Tawas City Shoreline Park, Iosco .....................................................................................................................................45 Figure 28. Placement and errors of NGCPs (blue) and CPs (red) in Tawas Point Beach, Iosco ...............................................................................................................................................46 Figure 29. Placement and errors of NGCPs (blue) and CPs (red) in Town Line Beach, Chikaming ......................................................................................................................................46 Figure 30. Placement and errors of NGCPs (blue) and CPs (red) in Township Park, Manistique......................................................................................................................................47 vii KEY TO ABBREVIATIONS CP Check Point DEM Digital Elevation Model GCP Ground Control Point GSD Ground Sampling Distance NGCP Natural Ground Control Point RMSE Root Mean Square Error RTK-GPS Real Time Kinematic Global Positioning System SFM Structure from Motion UAS Unoccupied Aerial System XY Horizontal (Error, RMSE, etc.) XYZ Horizontal and vertical (Error, RMSE, etc.) Z Vertical (Error, RMSE, etc.) viii Introduction Coastal environments are broadly used by humans to host infrastructure, provide ecosystem services, and supply areas for housing and recreation among other things (de Groot et al., 2012; Mehvar et al., 2018). Unfortunately these uses can be disrupted by natural system dynamics such as erosion and sediment transport resulting from storms and changing water levels (Phillips & Jones, 2006; Theuerkauf & Braun, 2021; Zhang et al., 2004). Effective coastal management must therefore balance these human uses with natural system dynamics as well as develop proactive strategies that help build resilience to future system dynamics (Dastgheib et al., 2018; Gopalakrishnan et al., 2016; Masselink & Lazarus, 2019; Molino et al., 2020; Rumson et al., 2020). Developing effective coastal management and coastal resilience strategies both rely on high spatio-temporal resolution coastal morphology data (Huang et al., 2021; Nichols et al., 2019; Rumson et al., 2020). These data are needed to assess vulnerability to existing infrastructure (Bove et al., 2020), delineate property ownership (Morton & Speed, 1998), design and evaluate shoreline management actions such as armoring or beach nourishment (Gares et al., 2006), track general coastal morphology changes (Theuerkauf et al., 2021; Turner et al., 2016), as well as a myriad of other uses. While the need and utility for such datasets is well articulated both in the scientific literature as well as in community and beach master plans, the complexity of gathering, analyzing, and implementing the data can create a barrier for communities who lack the personnel or financial resources to collect high-resolution data. Citizen science-based mapping programs that utilize new technologies such as unoccupied aerial systems (UAS) present an opportunity to broaden participation in data-driven management to such communities, however, the accuracy and utility of such datasets must be critically evaluated. 1 Aerial imagery and digital elevation models (DEMs) are two of the primary datasets coastal managers can utilize to develop data-driven management plans and document coastal changes (Huang et al., 2021). Prior to the use of UAS, aerial imagery used to visualize shoreline change was collected by occupied aircrafts and satellites. Unfortunately, low-cost or publicly available satellite imagery such as that from the Sentinel-2 satellite may lack the desired resolution (e.g. 10 m GSD; Sentinel-2 Data Access and Products Fact Sheet, 2015) necessary for coastal management, while the high cost of occupied aircraft imagery limits the collection frequency. Historically coastal elevation data were collected along profiles using the rod and level technique (Emery, 1961). As more advanced methods such as differential Real-Time Kinematic GPS (RTK-GPS) deployed through backpack or ATV surveys became established, researchers have been able to frequently generate DEMs over large spatial extents (Harley et al., 2011; Lee et al., 2013). RTK-GPS surveys require interpolation between mapped transects however, limiting accuracy over complex topography (e.g., Theuerkauf & Rodriguez, 2012). With the advent of airborne LiDAR, researchers can collect elevation models at high resolutions over large and complex areas (Jakubowski et al., 2013; Kidner et al., 2004), although high cost similarly limits its use for frequent surveys. Terrestrial LiDAR can be deployed more frequently (O’Dea et al., 2019) but is limited by the spatial extent of the surveys. UAS represent the latest advancement in coastal surveying, as such technology are capable of generating aerial imagery and DEMs through a single survey and are more cost and time-efficient to deploy than previous survey methods (Mancini et al., 2013; Turner et al., 2016). Geospatial datasets generated from UAS are derived by implementing photogrammetric structure-from-motion (SFM) algorithms on a series of overlapping offset images collected by UAS. These algorithms match common features in overlapping images to create a network of tie 2 points used to reconstruct objects in 3-dimensional space (James et al., 2017; Westoby et al., 2012). Unlike traditional photogrammetry however, SFM does not require targets of known location to reconstruct the scene. Instead, through the process of tie point matching, the geometry of the scene is reconstructed by estimating the position and orientation of the camera at each photo, as well as internal camera parameters such as focal length and lens distortion coefficients (Jin-Si R. Over et al., 2021; Westoby et al., 2012). This network of tie points can then be used to generate data products including DEMs and aerial imagery comprised of a mosaic of the original drone images (known as an orthomosaic). These data products are initially created in an arbitrary coordinate system however and require additional information to be accurately georeferenced, which is necessary to compare these data products to existing datasets. Georeferencing of UAS-SFM surveys is commonly conducted using ground control points (GCPs). In most cases, these points are physical markers with a high contrast checkered pattern (Figure 1) placed around the scene and surveyed directly prior to the flight using RTK- GPS. The user identifies the location and coordinates of the GCPs in the images which allows the SFM software to optimize the estimates of the camera position, orientation, and internal parameters. This process defines a coordinate system and scale of the tie point network, as well as reducing systematic errors across the site (James et al., 2017). In addition to GCPs, check points (CPs) are surveyed and input into the SFM software in the same manner as GCPs but are not used in the error reduction and georeferencing process. Instead, the CPs are used as independent reference points to estimate the accuracy of the scene reconstruction at a given location. Simply put, GCPs are used in the georeferencing process, whereas CPs are used to assess the accuracy of the scene reconstruction. 3 Figure 1. Example of ground control point (GCP)/check point (CP) surveyed on high contrast checkered marker prior to UAS flight. While GCPs can greatly increase the accuracy of UAS-SFM derived data products, GCP effectiveness largely depends on their quantity and distribution throughout the site (see following examples). As a result, there is a growing body of literature providing guidance on the ideal placement characteristics of GCPs. For sites less than 40 ha, studies have found that maximum achievable horizontal (XY) and vertical (Z) root mean square errors (RMSE) of 0.02 – 0.05 m and 0.04 – 0.10 m were reached with 5 – 20 and 9 – 20 GCPs respectively, although XY and Z RMSEs of 8 cm or less were reached with 5-7 and 7-12 GCPs respectively (Agüera-Vega et al., 2017; Manfreda et al., 2019; Martínez-Carricondo et al., 2018; Meinen & Robinson, 2020; Oniga et al., 2018; Santana et al., 2021; Yu et al., 2020). These recommendations are likely site specific however, as environmental variables such as surface texture and morphology may effect survey 4 accuracy (Eltner et al., 2015; Westoby et al., 2012). Two specific studies of coastal areas found similar results on ideal GCP quantity, indicating that a XY RMSE less than 0.05 m and Z RMSE less than 0.06 was reached with 6 - 7 and 11-12 GCPs respectively (Yu et al., 2020; Zimmerman et al., 2020). Studies evaluating the spatial distribution of GCPs found that at a minimum, GCPs should be placed around the perimeter of the site to minimize error (Martínez-Carricondo et al., 2018). This error can then be further decreased, especially in the vertical direction by including GCPs evenly distributed throughout the site (Martínez-Carricondo et al., 2018; Santana et al., 2021). Zimmerman et al. tested distributions in a coastal environment and found GCPs placed at the corners of the rectangular study area, with adequate cross shore coverage, and at the minimum and maximum elevations minimized errors throughout the site. Conversely, studies have found that GCPs without adequate coverage can lead to inflated errors (Martínez- Carricondo et al., 2018; Santana et al., 2021). For example, concentrating all GCPs in the center or in one corner of the site. The ease of UAS surveys, as well as the growing body of literature supporting their accuracy using GCPs, have led to widespread use of UAS for coastal monitoring. Casella et al., (2020) reviewed the current state of UAS for coastal monitoring and found 47 publications where UAS were used for coastal data collection between 2015 and 2019. Of these publications, at least 20 presented a time series of UAS-derived data, reflecting the repeatability of coastal UAS surveys. Specific uses of UAS-derived data in the coastal literature are numerous and include studying processes of coastal geomorphic change in beach and bluff environments (Krueger et al., 2020; Theuerkauf et al., 2021), documenting coastal habitat loss in response to changing water levels (Theuerkauf & Braun, 2021), evaluating elevation and vegetation characteristics in salt marshes (Pinton et al., 2021), and developing citizen science based coastal 5 change monitoring programs (Pucino et al., 2021; Theuerkauf et al., 2022). The wide range of applications supports the value of UAS-derived data for coastal management, however, the expertise needed to both design and implement a monitoring program as well as process, analyze, and interpret the data often does not exist in small, resource-limited communities. Recent studies have provided a gateway for resource limited and/or small communities to leverage UAS for coastal monitoring through networks of citizen scientists and institutional partnerships (Pucino et al., 2021; Theuerkauf et al., 2022). These citizen science networks train volunteer UAS pilots to periodically survey coastal areas in their communities. The image sets they collect are sent to institutions with the capacity and expertise to process the imagery into data products that can be used by the communities for coastal management. This approach is made possible using automated flight planning software, allowing citizen scientists to execute repeatable preplanned flight missions optimized for SFM data collection. Pucino et al., (2021) and Theuerkauf et al., (2022) both found that the imagery collected is of sufficient quality to be integrated into a SFM workflow and can be used to track shoreline changes in communities. While clearly beneficial for scientists and communities alike, these networks pose a challenge to traditional GCP collection as citizen scientists are not equipped with RTK-GPS and cannot survey GCPs immediately prior to their flights. Pucino et al., (2021) addresses this challenge by equipping each citizen scientist pilot with a set of “Smart GCPs” which have a high accuracy differential GPS system built into the survey marker. While effective, these “Smart GCPs” are costly and require significant additional effort on behalf of the citizen scientists to effectively deploy, retrieve, and maintain. Additionally, the monitoring sites typically desired by coastal communities are often popular public beaches, making it challenging to place these “Smart GCPs” in areas where they will not be disturbed for the duration of the survey. 6 Figure 2. Examples of all types of natural ground control points (NGCPs) and check points (CPs) included in study. A) NGCP, B) NGCP surveyed on concrete block in the backshore that could become unusable if covered by sediment deposition, C) Semipermanent NGCP mounted on a metal pole, D) Semipermanent NGCP spraypainted onto concrete, E) CP surveyed on a temporary checkered high contrast marker. Red circles indicate the location of the NGCP where applicable. Theuerkauf et al. addresses this challenge by utilizing existing infrastructure such as corners of park benches, parking stripes, manhole covers, or other static objects as GCPs (Figure 2A). These naturally occurring ground control points or NGCPs, can be surveyed at the onset of a monitoring program and used in lieu of traditional GCPs for the extent of the monitoring period. This method reduces project cost and survey effort compared to “Smart GCPs”, thus allowing for broader participation in the monitoring network for communities in a range of socio-economic settings. The accuracy of sites surveyed with NGCPs must be critically evaluated however, as the opportunities for NGCP placement are often restricted to stable upland structures, failing to adequately cover the dynamic coastal area where monitoring is needed. As a result, opportunities to place these NGCPs may drastically differ from the quantity and/or distribution of GCPs recommended in the literature, which may present a challenge for deriving accurate data products for coastal research and management. 7 Given the challenges with NGCPs placement, the purposes of this study are to: 1) Evaluate the ability of NGCPs to generate high accuracy data through an accuracy assessment of 20 coastal sites ranging in size and morphology and surveyed using NGCPs. 2) Contribute to the literature on optimal GCP quantity and distribution in the case of NGCPs in coastal environments in a six-site subsample of these 20 sites. Through these investigations, we seek to demonstrate the utility of NGCPs for citizen science surveys and provide guidance for future UAS citizen science monitoring networks utilizing NGCPs. 8 Study Sites, Data, and Methods Site Selection and Description All sites included in this study are located within the six communities participating in the Great Lakes citizen science UAS monitoring program described in Theuerkauf et al., 2022 (Figure 3). This program established at Michigan State University currently supports seven citizen scientist pilots across six communities to survey a total of 20 coastal sites using UAS every two months. The communities participating in this program were selected based on a combination of coastal and/or socioeconomic vulnerability indicators described in (Mack et al., 2020). In this study high coastal vulnerability was defined as areas experiencing high rates of coastal recession while high socioeconomic vulnerability was identified as areas with low per capita income. The resulting communities selected therefore have a need for high quality coastal data to inform coastal management decisions as well as a potential lack of resources to collect these data. Over these 6 communities, the 20 sites were selected based on the recommendation of community stakeholders as well as in areas of frequent coastal change as determined through an analysis of historical images and shoreline change data (e.g. Mack et al. 2020). At each of these 20 sites, a field survey was completed to establish NGCPs throughout the site. The sites ranged in size from approximately 0.1 ha to 3.0 ha and represent a range of morphologies from flat beaches varying only 1.25 meters in elevation to steep bluffs varying over 20 meters in elevation. In addition, these sites contain a variety of coastal infrastructure representing various opportunities for NGCP placements. The quantity and diversity of sites as well as the diversity of NGCP placement opportunities in this study represent an ideal opportunity to rigorously test the accuracy of NGCPs for georeferencing in coastal environments. 9 Figure 3. Location of the six communities participating in the Great Lakes citizen science UAS monitoring program established at Michigan State University. Communities include: Chikaming Township, City of South Haven, City of Manistee, City of Manistique, City of Marquette, and Iosco County. Among these 20 sites, several had unique morphology and NGCP placement characteristics that are referred to in this publication. Sites with bluff morphology (Lake Superior Roadside Park & Casco Nature Preserve) are bluff sites that only had NGCP placement opportunities at the top of the bluff landward of the crest, with no NGCPs placement locations lakeward of the bluff toe. Sites with backshore NGCP placement only (Cherry Beach & Townline Beach) refer to sites at the bottom of bluffs where the backshore makes up the entire site with no possible structures to survey as NGCPs. The resulting NGCPs at these sites needed to be mounted on poles and placed at the bluff toe where they were partially obscured by overhanging vegetation. 10 Table 1. Site type, elevation range, approximate area, quantity of natural ground control points (NGCPs) & check points (CPs), and description of all 20 sites included in this study. Elevation range is calculated from the difference between the highest and lowest NGCP/CP within the site. Approximate area excludes excess area outside the boundaries of the site captured by NGCPs/CPs. #CP and # NGCP refer to the quantity of check points and natural ground control points surveyed at the site. Starred sites indicate those used for the analysis of NGCP quantity and distribution. Elevation Approximate Site Site Type Range (m) Area (ha) #CPs #NGCPs Description AuSable Shoreline Park Primary 1.3 0.6 11 6 Sandy beach abutting river jetty Bar Lake Primary 2.4 0.5 10 8 Sandy beach with rip rap spanning the length of site Berrien Beach Primary 5.1 0.5 11 4 Sandy beach with adjacent foredune Sandy beach and adjacent foredune abutting river jetty in western 1/2 Boardwalk West* Primary 3.6 2.7 16 10 of site, rip rap extending to waters edge in eastern 1/2 of site Casco Nature Preserve* Bluff Morphology 21.3 1.0 13 5 Steep consolidated bluff Backshore NGCP Cherry Beach 1.9 0.5 12 3 Sandy beach at toe of bluff with overhanging vegetation placement only Fifth Ave. Beach Primary 6.2 2.3 18 8 Sandy beach with adjacent foredune in eastern 1/3 of site First St. Beach Primary 5.9 1.1 10 7 Sandy beach with adjacent foredune in southern 1/4 of site Sandy beach below steep consolidated bluff with large staircase Harbert Beach Primary 8.5 0.2 10 5 connecting bluff top to bluff toe Sandy beach below unconsolidated sandy bluff with rip rap in western Lake Superior Roadside Park* Bluff Morphology 8.5 1.9 28 12 1/2 of site North Beach* Primary 1.9 3.0 17 10 Sandy beach abutting river jetty North Beach Access* Primary 8.4 0.7 22 11 Sandy beach with adjacent foredune Oscoda Beach Park* Primary 2.9 2.1 20 8 Sandy beach Pebble beach with adjacent foredune in southern 1/2 of site, rip rap Pine St. Lakeshore Blvd. Primary 2.7 2.0 16 7 extending to water's edge in northern 1/2 of site South Beach Primary 12.2 2.6 12 13 Sandy beach with adjacent foredune in southern 1/2 of site Flat vegetated area with trees and shrubs in northern 1/2 of site, dense South Beach Park Primary 1.8 2.6 9 7 grass in southern 1/2 of site Flat grassy park with rip rap extending to water's edge in western 1/2 Tawas City Shoreline Park Primary 1.4 1.5 13 11 of site, sandy beach in northern 1/2 of site Tawas Point Beach Primary 2.5 1.6 20 7 Sandy beach surrounding lake inlet on 3 sides Backshore NGCP Town Line Beach 2.9 0.1 11 3 Sandy beach at toe of bluff with overhanging vegetation placement only Sparsely vegetated dunes in eastern 1/2 of site, sandy beach with Township Park Primary 5.6 1.1 18 8 adjacent foredune in western 1/2 of site 11 NGCP and CP Placement For each site, 3-12 NGCPs were surveyed using either a Trimble R10-2 or R12 RTK GPS system with 0.012 m (SD = 0.004 m) horizontal precision and 0.020 m (SD = 0.007 m) vertical precision. NGCPs were placed on any permanent objects throughout the site that could be easily identified in the UAS imagery. When possible, NGCPs were placed on objects in the broadest possible horizontal and vertical distributions as well as evenly distributed throughout the site in agreement with current literature on optimal GCP placement (Martínez-Carricondo et al., 2018; Santana et al., 2021; Zimmerman et al., 2020). At several of our sites where there were either no opportunities to place NGCPs, or where the opportunities did not include a large portion of the site, we installed supplemental semipermanent NGCPs (Figure 2). These semipermanent NGCPs were either spray painted survey markers on exposed concrete (Figure 2D), or a painted survey maker affixed to a 4-to-6-foot metal pole driven into the ground (Figure 2C). For this study we refer to both semipermanent NGCPs and NGCPs surveyed on objects already in the site as NGCPs unless explicitly stated otherwise. To assess errors across the site we additionally surveyed 9-20 CPs depending on the size and topography of the site. These check points were placed strategically to capture the maximum and minimum heights of the site, the furthest lakeward extent of the site, around the perimeter of the site, the tops and bottoms of any berms of bluffs, and otherwise evenly distributed throughout the site (Figure 4). Imagery Collection UAS surveys were conducted with either a DJI Phantom 4 Pro V2.0 or an Autel Evo II Pro quadcopter. The camera mounted on these UAS are similar with one inch, 5472x3648 20- megapixel sensors, and lenses with a similar f/2.8-f/11 aperture range, although with slightly different focal lengths. Since differing focal lengths result in varying spatial resolutions for a 12 given flight height, flights were standardized by the ground footprint of the pixel, hereby referred to as the ground sampling distance (GSD) rather than by flight height. A GSD of 0.015 m was chosen, giving us the ability to confidently pick out the location of NGCPs. Recorded GSD varied between 0.013 and 0.020 m except for Township Park and AuSable Shoreline Park which had a GSD of 0.007 and 0.008 m respectively, as these sites were surveyed before a GSD of 0.015 m was chosen. All surveys were conducted with 80% front and side overlap as recommended by Dandois et al., 2015, as well as nadir camera angle and auto settings for exposure, ISO, and aperture. Figure 4. Example of accuracy assessment natural ground control point (NGCP) and check point (CP) placement scheme at Fifth Ave. Beach. CPs are shown as red points and natural NGCPs are shown as blue points. Pole mounted semipermanent NGCPs are shown as a blue point containing “P”. Portion of survey area where NGCPs can be surveyed are outlined in blue while portion of survey area where no NGCPs can be found are outlined in red. 13 Structure from Motion Processing All SFM models were created using Agisoft Metashape Professional Edition (Agisoft Metashape, 2021). Initial tie point matching was performed with the high accuracy setting and default parameters of 40,000 key point limit and 4,000 tie point limit. Adaptive model fitting was used which allows Metashape to automatically select which internal camera parameters are estimated based on their reliability estimates (Agisoft Metashape User Manual - Professional Edition, Version 1.6). Generally, more internal camera parameters can be reliably estimated when stronger geometry exists in the scene such as a building or bluff (Agisoft Metashape User Manual - Professional Edition, Version 1.6). Selecting which internal camera parameters to include is typically done on a site-specific basis, however due to the diversity of morphologies among our sites and the necessity of a streamlined workflow, the adaptive camera model fitting option was effective and efficient. The locations of the NGCPs and CPs in each photo were selected with an estimated placement precision of one pixel (referred to as image coordinate marker accuracy in Metashape). The default image coordinate marker accuracy in Metashape is 0.5 pixels, however this value was increased to one pixel to account for the difficulty in identifying NGCP locations over traditional GCPs surveyed on checkered markers. Precision of the RTK-GPS for each NGCP and CP was set to the mean precision of all the RTK-GPS markers for a given survey site on a given day, which ranged from 0.007 – 0.025 m in the horizontal direction and 0.016 – 0.034 m in the vertical direction. The camera locations and internal parameters were optimized using all NGCPs with the adaptive camera model fitting parameter selected. Metashape then produces an estimate of the location of each CP based on the optimized tie point network which can be 14 compared to the surveyed RTK-GPS locations to calculate a residual error (in meters) for each CP (Equation 1). Accuracy Assessment Accuracy of each of the 20 sites was determined by calculating the horizontal (XY), vertical (Z), and total XYZ error for each CP, as well as the horizontal, vertical, and total root mean square errors (RMSE) for each site using the following equations: Equation 1: Calculation of horizontal (A), vertical (B), and total (C) error for each CP. (A) 𝐸𝑟𝑟𝑜𝑟𝑋𝑌 = √(𝑋𝑀𝑜𝑑 – 𝑋𝐺𝑃𝑆 )2 + (𝑌𝑀𝑜𝑑 – 𝑌𝐺𝑃𝑆 )2 (B) 𝐸𝑟𝑟𝑜𝑟𝑍 = √(𝑍𝑀𝑜𝑑 – 𝑍𝐺𝑃𝑆 )2 (C) 𝐸𝑟𝑟𝑜𝑟𝑋𝑌𝑍 = √(𝑋𝑀𝑜𝑑 – 𝑋𝐺𝑃𝑆 )2 + (𝑌𝑀𝑜𝑑 – 𝑌𝐺𝑃𝑆 )2 + (𝑍𝑀𝑜𝑑 – 𝑍𝐺𝑃𝑆 )2 Equation 2: Calculation of horizontal (A), vertical (B), and total (C) RMSE for all CPs in a site. ∑𝑛 2 2 𝑖=1[(𝑋𝑀𝑜𝑑 – 𝑋𝐺𝑃𝑆 ) +(𝑌𝑀𝑜𝑑 – 𝑌𝐺𝑃𝑆 ) ] (A) 𝑅𝑀𝑆𝐸𝑋𝑌 = √ 𝑛 ∑𝑛𝑖=1(𝑍𝑀𝑜𝑑 – 𝑍𝐺𝑃𝑆 ) 2 (B) 𝑅𝑀𝑆𝐸𝑍 = √ 𝑛 ∑𝑛 2 2 2 𝑖=1[(𝑋𝑀𝑜𝑑 – 𝑋𝐺𝑃𝑆 ) +(𝑌𝑀𝑜𝑑 – 𝑌𝐺𝑃𝑆 ) + (𝑍𝑀𝑜𝑑 – 𝑍𝐺𝑃𝑆 ) ] (C) 𝑅𝑀𝑆𝐸𝑋𝑌𝑍 = √ 𝑛 where (𝑋, 𝑌, 𝑍)𝑀𝑜𝑑 are the locations of the CPs derived from the SFM model and (𝑋, 𝑌, 𝑍)𝐺𝑃𝑆 are the locations collected through the RTK-GPS survey. Several additional tests were conducted to evaluate NGCPs under various possible scenarios. For example, the destruction of semipermanent NGCPs may be possible, as the 15 spraypainted points can be washed away and pole mounted points can be moved by either human or natural disturbances. Additionally, several NGCPs were surveyed on concrete blocks on the backshore (backshore NGCPs) that may become unusable if covered by sediment deposition (Figure 2B). To assess the effect of the removal of the two types of semipermanent NGCPs as well as backshore NGCPs on site accuracy, we conducted an accuracy assessment on sites georeferenced excluding these three types of points. Additionally at bluff sites, we tested the inclusion of either one or two GCPs at the base of the bluff in the center and at either end respectively. This scenario evaluates how the accuracies of these sites may be improved beyond the accuracies achievable with solely NGCPs located at the bluff top. The results of this accuracy assessment across the 20 sites are reported both in the horizontal and vertical directions as understanding how these accuracies differ informs how the derived data products can be used for coastal management. For example, data products that are accurately georeferenced in both the horizontal and vertical directions can be used to answer coastal management questions that require accurate elevations such as establishing patterns of sediment erosion and accretion, or delineating elevation based regulatory lines. However, data products that are only accurate in the horizontal directions may only be suitable to answer coastal management questions that do not require vertical elevations such as evaluating coastal areas for recreation or generating bluff recession rates by tracking the horizontal position of the bluff crest over time. Optimal Quantity & Distribution of NGCPs To determine the effect of NGCP quantity and distribution on the CP RMSE, a series of tests were conducted using different quantities and distributions of NGCPs on a subsample (n=6) of the total sites (n=20). These sites represent a variety of coastal morphologies and elevation 16 ranges including low relief beaches (North Beach & Oscoda Beach Park), moderate relief foredunes (North Beach Access & Boardwalk West), and high relief bluffs (Casco Nature Preserve & Lake Superior Roadside Park) as described in (Table 1). For each of the six subsampled sites, we randomly selected five different combinations of three NGCPs (the minimum required for georeferencing; James & Robson, 2012) from the total amount of NGCPs used for the site to be included in the camera optimization procedure. We then repeated this process for 50%, 75%, and 100% of the total quantity of NGCPs available for a total of 16 replications per site. The total number of NGCPs available to use at a given site for these tests varied from 8-11 depending on site size and structures available to be used for NGCPs. For each replication of NGCPs, we included any NGCPs not randomly selected to be included in the replication as additional CPs. A regression analysis comparing the quantity of NGCPs to the XY and Z RMSE for all sites combined was conducted with the results of these replications. For the purposes of the regression analysis, outliers greater than three times the interquartile range above the upper quartile or below the lower quartile were excluded. This removed six replications from the Z RMSE regression and three replications from the XY RMSE regression. An exception to this procedure was implemented at the Casco Nature Preserve site which is a high relief bluff with five NGCPs and five CPs along the stable land at the bluff top (landward of the bluff crest) and eight CPs at the bluff bottom (lakeward of the bluff toe). Because of the lack of NGCPs available to be used for replications, we included the five CPs at the top of the bluff as additional NGCPs for a total of 10 possible NGCPs to be used in the replications. The CPs at the top of the bluff were more accurate than those at the bottom of the bluff due to their relatively high proximity to the NGCPs at the bluff top. Therefore, increasing 17 NGCPs removed these higher accuracy CPs, leading to a change in the proportion of CPs located at the bluff top vs bluff bottom. To remove the effect of this varying proportion of higher accuracy CPs, we calculated CP RMSE using only the eight CPs located at the bluff bottom for all NGCP quantities, resulting in a more meaningful comparison of error between replications. 18 Results Accuracy Assessment Figure 5. Distribution of check point (CP) error magnitudes (m) for all sites in the horizontal (orange), vertical (blue), and total (grey) directions. Boxes represent the interquartile range (IQR), whiskers represent points <1.5*(IQR) above or below the IQR, and dots represent points >1.5*(IQR) above or below the IQR. The left most 16 sites (Bar lake - Harbert St. Beach) plotted using the left Y axis from 0.00 - 0.15 m while the 4 sites on the right (Lake Superior Roadside Park - Casco Nature Preserve) are plotted using the right Y axis from 0.0 - 1.0 m. Figure 5 shows the results of the accuracy assessment for all 20 sites both in the horizontal, vertical, and total direction. Of these 20 sites, those with either backshore NGCP placement only (Town Line Beach and Cherry Beach) or with bluff morphology (Lake Superior Roadside Park and Casco Nature Preserve) are shown in the panel to the right, while the 16 remaining sites (referred to as primary sites) are shown to the left. At these sixteen primary sites, Z RMSE was less than 0.060 m and in 14 out of the 16 sites, XY RMSE was less than 0.025 m. The exceptions are in Harbert Beach and Berrien Beach which had an XY RMSE at 0.067 & 0.058 m respectively. For the two sites with backshore NGCP placement only, Z RMSE was 19 notably higher at 0.153 and 0.302 m while XY RMSE was like the primary sites, at 0.050 and 0.046 m. For the sites with bluff morphology, Lake Superior Roadside Park had a Z RMSE higher than the primary sites at 0.107 m, but similar XY RMSE at 0.027 m. For Casco Nature Preserve, Z RMSE was notably higher than any other site in the study at 0.743 m, and XY Figure 6. Check point (CP) and natural ground control point (NGCP) errors for sites with bluff morphology: Lake Superior Roadside Park (top) and Casco Nature Preserve (Bottom). Red points indicate CP locations and blue points indicate NGCP locations. Semipermanent spraypainted NGCPs and semipermanent NGCPs mounted on poles are represented with an “S” and “P” respectively. Upper point labels show Horizontal (XY) error and lower labels show vertical (Z) error in cm. 20 RMSE was slightly larger than all other sites at 0.079 m. For both these sites, errors were concentrated at the base of the bluff, while errors at the top of the bluff were similar in magnitude to those found in the primary sites (Figure 6). For the two sites with bluff morphology, the addition of one or two additional GCPs located at the bottom of the bluff was evaluated. At Lake Superior Roadside Park, including one GCP placed at the bottom of the bluff decreased Z RMSE from 0.107 to 0.037 m and XY RMSE from 0.027 to 0.019 m with two GCPs producing a negligible additional decrease in XY and Z RMSE. At Casco Nature Preserve, one GCP at the bottom of the bluff decreased Z RMSE from 0.743 m to 0.228 m and slightly increased XY RMSE from 0.079 to 0.111 m. Including two GCPs at the base of the bluff further decreased Z RMSE to 0.143 and XY RMSE to 0.045 m. At several sites, semipermanent NGCPs installed with either spray paint or metal poles as well as the easily covered backshore points were removed from the analyses (Table 2). Of the 17 cases of reduced combinations of NGCPs, in eight cases removing these points had no effect (< 0.025 m change) in either XY or Z RMSE. In seven cases, the removal of these points caused Z RMSE to increase, however it had little effect on the XY RMSE. The exception to these findings were the Township Park cases where removing these points caused an increase in XY RMSE and no effect on Z RMSE. At this site however, the semipermanent NGCPs increase the longshore spread of the NGCPs from roughly 25% to nearly 100%, while not contributing at all to the vertical spread of the NGCPs. Therefore, removing these semipermanent NGCPs caused a notable reduction in horizontal NGCP distribution without effecting vertical NGCP distribution. 21 Table 2. Effect of removing easily destroyed natural ground control points (NGCPs) on check point (CP) horizontal (XY) and vertical (Z) RMSE. Cases of removed NGCPS include no backshore points (NB), no spraypainted points (NS), and no points mounted on metal poles (NP). Changes to RMSE <0.025 m are noted as “no effect” while all other changes are noted as “original value → new value”. Mean changes are calculated including the cases where removing points had no effect. All values are shown in meters. Site and Case #NGCPs XY RMSE Z RMSE Pine St. Lakeshore Blvd. (NB) 7→6 no effect no effect Lake Superior Roadside Park (NS) 12 → 8.. no effect no effect North Beach Access (NP) 11 →10. no effect no effect Boardwalk West (NS) 10 → 5.. no effect no effect South Beach (NP) 13 → 11 no effect 0.041 → 0.087 5th Ave. Beach (NBP) 8→7 no effect no effect 5th Ave. Beach (NP) 8→6 no effect 0.029 → 0.083 5th Ave. Beach (NB, NP) 8→5 no effect 0.029 → 0.081 North Beach (NB) 10 → 9.. no effect 0.060 → 0.129 North Beach (NS) 10 → 8.. no effect no effect North Beach (NB, NS) 10 → 7.. no effect 0.060 → 0.086 1st St. (NB) 7→6 no effect 0.043 → 0.122 1st St. (NP) 7→6 no effect no effect 1st St. (NB, NP) 7→5 no effect 0.043 → 0.091 Township Park (NP) 8→7 no effect no effect Township Park (NS) 8→7 0.042 → 0.129 no effect Township Park (NP, NS) 8→6 0.021 → 0.144 no effect Mean Change 0.019 0.024 Optimal Quantity of NGCPs Through the detailed analysis of NGCP quantity at the six subsampled sites, it was evident that the relationship between NGCP quantity and CP RMSE was different for the Casco Nature Preserve site compared to the other five sites (Boardwalk West, Lake Superior Roadside Park, North Beach, North Beach Access, and Oscoda Beach Park), thus Casco Nature Preserve was removed from the regression analysis (Figures 5 & 6). This analysis revealed the relationship between XY and Z RMSE and quantity of NGCPs can be described using the following equations depicted by the blue lines in Figure 7. 22 Equation 3: Relationship between CP Z RMSE in meters and quantity of NGCPs (#NGCP). 𝑍 𝑅𝑀𝑆𝐸 (𝑚) = 0.3468(#𝑁𝐺𝐶𝑃)−.8889 , 𝑅 2 = 0.217, 𝑃 − 𝑉𝑎𝑙𝑢𝑒 = < 0.01 Equation 4: Relationship between CP XY RMSE in meters and quantity of NGCPs (#NGCP). 𝑋𝑌 𝑅𝑀𝑆𝐸 (𝑚) = 0.0488(#𝑁𝐺𝐶𝑃)−.4308 , 𝑅 2 = 0.194, 𝑃 − 𝑉𝑎𝑙𝑢𝑒 = < 0.01 Figure 7. Natural ground control point (NGCP) quantity vs vertical (Z) RMSE (top) and horizontal (XY) RMSE (bottom). Each point represents the RMSE of a single replication for a given quantity of NGCPs, while point colors represent the site from which the RMSE is derived. Boxes represent the interquartile range (IQR), whiskers represent points <1.5*(IQR) above or below the IQR, and dots represent points >1.5*(IQR) above or below the IQR. Blue lines represent trend lines with the form 𝑦 = 𝑎𝑥 𝑏 . Outliers greater than 3 times the interquartile range above the upper quartile of each NGCP quantity are not shown (top n=6, bottom n= 3). 23 In these five sites RMSE decreases in both the vertical and horizontal directions with the inclusion of additional NGCPs. Furthermore, the range of RMSE values for a given NGCP quantity is the greatest using only three NGCPs and generally decreases approaching the total number of NGCPs used. In the vertical direction in most cases, a Z RMSE less than 0.075 m was achieved using seven NGCPs or greater. The exception is in Lake Superior Roadside Park (high relief bluff) which reached a Z RMSE of 0.080 – 0.128 m with six NGCPs and did not improve with additional NGCPs. In the horizontal direction, in most cases an XY RMSE of 0.045 m was Figure 8. Casco Nature Preserve Site natural ground control point (NGCP) quantity vs RMSE. Purple points represent vertical (Z) RMSE, and orange points represent horizontal (XY) RMSE. Each point represents the RMSE of a single replication for a given quantity of NGCPs. Boxes represent the interquartile range (IQR), whiskers represent points <1.5*(IQR) above or below the IQR, and dots represent points >1.5*(IQR) above or below the IQR. 24 achieved using only four NGCPs or greater, and in all cases an XY RMSE less than 0.030 m was achieved using seven NGCPs or greater. In both the horizontal and vertical directions, we observe a minimum achievable RMSE regardless of the number of NGCPs used at roughly 0.013 m and 0.025 respectively. While the mean RMSE across all replications decreases with additional NGCPs, for the majority of NGCP quantities used, some combinations of NGCPs achieve this minimum possible error. Contrary to other sites, at Casco Nature Preserve (high relief bluff) which was not included in the regression analysis, Z RMSE increased from three to five NGCPs, and stabilized at a Z RMSE of 0.875 – 0.966 m with five or more NGCPs. In the horizontal direction, XY RMSE was largely unaffected by an increase in number of NGCPs, ranging from 0.038 to 0.111 m for all but one replication (Figure 8). Optimal Distribution of NGCPs For each of the sites tested above, the distributions of NGCPs that produced the lowest and highest XYZ RMSE for each number of NGCPs tested above were also evaluated. The results varied by site with no clear relationship to site morphology. At Boardwalk West (moderate relief foredune) and Lake Superior Roadside Park (high relief bluff), for all quantities of NGCPs the distribution with the lowest XYZ RMSE had a broader or roughly equivalent horizontal spread of NGCPS than that of the distribution with the highest XYZ RMSE. The placement of NGCPS throughout these two sites did not allow us to isolate the effects of vertical vs horizontal distributions however, as the NGCPs either lacked vertical variation (Lake Superior Roadside Park), or the NGCPs with the greatest vertical variation also had the greatest horizontal variation (Boardwalk West). Similar findings were observed at North Beach Access (low relief beach), although this site highlighted the importance of a broad horizontal as well as vertical distribution of NGCPs. This is especially evident when looking at the NGCP replications with 25 the minimum number of NGCPs (3). In these replications, the distribution with broad Z spread and limited XY spread had a Z & XY RMSE of 0.037 and 0.100 m respectively, while the distribution with limited Z spread and broad XY spread had a Z & XY RMSE of 0.193 and 0.041 m respectively. This is contradicted however with the replications using 75% of the NGCPs (8). In these replications the XY & Z RMSEs between the two distributions was nearly identical at 0.019 – 0.025 m and 0.035 – 0.040 m respectively, but the vertical spread for the two distributions varied by over four meters, a substantial portion of the 8.4 m total elevation range for the site. In contrast, at two of the sites the smallest XYZ RMSE was not always achieved with the broadest NGCP distribution. For the minimum number of NGCPs (3) in Oscoda Beach Park (low relief beach), the less distributed NGCPs produced an XYZ RMSE of 0.237 m while the broadest distribution of NGCPs possible produced an XYZ RMSE of 2.310 m. With 75% of the NGCPs (6) however, the broadest distribution did produce the lowest XYZ errors. At North Beach (low relief beach), across all NGCP quantities the broadest distribution of NGCPs did not produce the lowest XYZ RMSE. This is evident in the replications using the minimum quantity of NGCPs (3) where the points clustered in one corner of the site produced an XYZ error of 0.064 m while the NGCPs spanning across the site produced an XYZ RMSE of 0.264 m (Figure 9). For replications using both 50% (5) and 75% (8) NGCPs, there was little difference in the spread between the points, however the XYZ RMSEs varied substantially from 0.039 - 0.294 m and 0.053 - 0.233 m respectively. 26 Figure 9. Check point (CP) and natural ground control point (NGCP) errors for replications of North Beach using 3 NGCPs. Top panel shows the distribution of NGCPs that resulted in the lowest CP total RMSE and bottom panel shows the distribution of NGCPs that resulted in the highest CP total RMSE. Red points indicate CP locations and blue points indicate NGCP locations. Spraypainted NGCPs and backshore NGCPs are represented with an “S” and “B” respectively. Upper point labels show horizontal (XY) error and lower labels show vertical (Z) error in cm. 27 Discussion UAS surveys deployed by citizen scientists represent an opportunity to expand high quality coastal data to small, resource limited communities (Pucino et al., 2021; Theuerkauf et al., 2022). The ability to georeference these sites using NGCPs reduce survey cost and effort, however the use of NGCPs may introduce errors reducing the utility of the derived data products for coastal management. This study critically evaluated the accuracy of surveys georeferenced using NGCPs in coastal settings for the first time through an assessment of 20 real world examples of sites surveyed using NGCPs. Through this assessment, it was determined that for almost all sites, using NGCPs produced vertical and horizontal errors comparable or slightly higher than errors obtained using traditional GCPs in coastal environments (Casella et al., 2020; Yu et al., 2020; Zimmerman et al., 2020). Unlike previous studies focusing only on 1-3 sites, the inclusion of 20 sites in this study give further confidence to these findings, as a range of coastal morphologies and opportunities for NGCP placement were evaluated. At these accuracies, the data products generated from these citizen scientist networks can be used to support the myriad of data driven management activities in resource limited communities. The analysis conducted in this study to determine the optimal quantity and distribution of NGCPs to minimize RMSE of surveys helps to support the expansion of the citizen science-UAS coastal monitoring programs. Through this analysis it was found that the accuracy of sites is only partially dependent on these placement characteristics. The effect of the quantity of NGCPs throughout sites can be modeled with equations 3 & 4 although the low R2 squared values of these regressions indicate they should not be solely relied on to determine site error for a given quantity of NGCPs. In fact, the optimum achieved RMSE using all NGCPs available at a site can be achieved with as few as three NGCPs in some cases, highlighting the influence of other 28 factors apart from NGCP quantity on check point RMSE. Furthermore, the relationship between NGCP quantity and RMSE is not consistent and appears to be partially influenced by site morphology. At bluff sites, for example, no additional change in accuracy was achieved with greater than 5-6 NGCPs, and in the case of one site (Casco Nature Preserve), more than three NGCPs counterintuitively resulted in an RMSE increase. In order to minimize error, our findings support previous work (Martínez-Carricondo et al., 2018; Santana et al., 2021; Zimmerman et al., 2020) that indicates NGCPs should be placed around the perimeter of the site in the broadest distribution possible in both the horizontal and vertical directions. In several cases notable deviations from this pattern were observed however, with the broadest distributions of NGCPs resulting in the higher errors compared to cases with more clustered NGCPs. While these findings together provide basic recommendations about optimal NGCP quantity and distribution, it is clear there are other factors impacting site accuracy that cannot be described solely through these NGCP placement characteristics. One possible factor is the varying ability of NGCPs to be identified in the UAS images. Unlike traditional GCPs placed on identical checkered markers, objects used as NGCPs are placed on a variety of structures, leading to differences in image coordinate marker accuracy among the points. For example, at a 0.015 m GSD, misplacing the NGCP by two pixels could result in up to a 0.03 m variation in NGCP image coordinate marker accuracy. It is therefore possible that certain combinations of NGCPs contain points with higher or lower image coordinate marker accuracy, leading to the variations in CP error among different NGCP combinations. Taking these findings into account, best practices for minimizing errors across sites surveyed by citizen scientists and georeferenced using NGCPs are presented. If both horizontal and vertical accuracy is desired, surveying at least seven NGCPs is recommended and will likely 29 produce an Z RMSE of less than 0.075 m and XY RMSE less than 0.030 m. If only horizontal accuracy is required, the quantity of NGCPs can be reduced to four to obtain an RMSE less than 0.045 m. Surveying additional NGCPs is recommended however, as this may further decrease both horizontal and vertical accuracy as well as provide a buffer if any NGCPs are disturbed. These NGCPs should be as broadly distributed around the site as possible in both the horizontal and vertical directions, but only if they can be sufficiently identified in photos. In other words, broadly placed NGCPs that are difficult to accurately identify may not be as useful as more clustered NGCPs that are easier to identify. Additionally, adding an abundance of NGCPs clustered in one portion of the survey area is discouraged, especially if this area is geographically dissimilar from the rest of the site, for example at the top of a bluff. Of the NGCPs used in this study, it is evident that consistent identification is easiest for those NGCPs with high color contrast placed on flat ground, for example the corner of a white crosswalk on black asphalt (Figure 10D, E). Conversely, placement of NGCPs on elevated geometric edges such as the top corner of a fence post or corner of a stair step should be avoided. In these cases, the varying angle of the object relative to the camera position will make it challenging to consistently identify the point (Figure 10A). If placing points on geometric edges is unavoidable, structures with high color contrast between the vertical and horizontal surfaces of the structure are recommended. Without this contrast, when the vertical edge of the structure faces the camera, the geometric structure of the object can be difficult to resolve making NGCP identification inconsistent (figure 10B). If need be, shadows can be utilized as contrast considering time of day of the surveys. For example, if a citizen scientist expects to be conducting surveys in the morning with the sun in the southeast, the northwest corner of the 30 structure is the optimal choice for an NGCP, allowing a shadow to be cast across the vertical side of the structure, increasing contrast (Figure 10E). Figure 10. Examples of natural ground control points (NGCPs) placed on objects with varying visibility. Top panels show an NGCP surveyed on the corner of a post visible from the ground (A), visible from a UAS image with the sun illuminating the corner of the post used as an NGCP (B), and visible from a UAS with the sun illuminating the opposite side of the post used as an NGCP. Bottom row shows an NGCP surveyed on the corner of a painted crosswalk from the ground (D), and from a UAS (E). Red circles indicate approximate locations of the NGCPS. Placing an abundance of easy to identify NGCPs widely distributed throughout the survey site is often not possible. Therefore, an accuracy assessment using CPs placed throughout the site at the onset of a project should be conducted by the supporting institution for all citizen science UAS monitoring efforts. Without an accuracy assessment, the undocumented spatial errors of the data products could result in misleading data and less effective, cost-efficient, or even counterproductive coastal management actions. Conducting such an accuracy assessment will also allow a research team to update the check point errors across the site, should some 31 NGCPs be destroyed. In this case, the georeferencing process can be repeated excluding the destroyed NGCPs, allowing researchers to determine the resulting change in accuracy and if a replacement effort for the lost NGCPs is necessary. Challenges to georeferencing sites using NGCPs stem largely from sites either with bluff morphology, backshore NGCP placement only, or when NGCPs are destroyed. In these cases, vertical error may be higher than desired, potentially limiting their use for coastal management actions requiring accurate vertical elevation data such as volumetric calculations or delineating elevation-based jurisdictional boundaries (e.g., elevation-based ordinary high-water mark). However, in these cases our results indicate that horizontal error largely remains at levels comparable to the primary sites, allowing the data products from these challenging sites to be used for actions requiring only horizontal accuracy. For example, in many coastal states, erosion rates are used to evaluate erosion hazards associated with new and existing infrastructure (EGLE, NYS-DEC). These rates are typically determined by comparing the horizontal position of a coastal erosion line over time, only requiring accuracy in the horizontal dimension. Additionally at bluff sites, these vertical inaccuracies can be reduced with the inclusion of even a single GCP placed at the bottom of the bluff. While this may not be possible using NGCPs, a single RTK enabled “smart GCP” marker could be used similar to those in Pucino et al. (2021) if access to the base of the bluff is possible for citizen scientists. While the inclusion of “smart GCPs” increases project cost and citizen science effort if used exclusively, a single smart GCP used for this purpose may provide a middle ground for maximizing accuracy while minimizing cost and effort. While in these challenging cases the absolute accuracy of the data products may not be optimal, the fact that a consistent set of NGCPs are being used may facilitate high relative 32 accuracy ideal for multitemporal change detection (Eltner et al., 2015). Future research is needed to fully understand the persistence of errors over time however, which can be effected by varying environmental conditions such as lighting, texture, and morphology change in response to sediment movement or coastal development. (Eltner et al., 2015; Westoby et al., 2012). Additionally, prior literature has found that check point error increases with greater distance to the nearest GCP (Gindraux et al., 2017; Sanz-Ablanedo et al., 2018). Therefore, if a coastal area expands, a previously conducted accuracy assessment may fail to account for increased errors at the water’s edge. Such expansion is common in freshwater environments such as the Great Lakes due to fluctuating water levels, or in marine environments where sediment supply outpaces sea level rise (Hanrahan et al., 2010; Quinn, 2002; van IJzendoorn et al., 2021). In most marine environments however, predicted coastal recession due to global sea level rise (Leatherman et al., 2000; Nicholls & Cazenave, 2010) may decrease the distance from NGCPs to the water’s edge, reducing check point errors at the shoreline. 33 Conclusion In this study, the accuracy of UAS-SFM models georeferenced using NGCPs was evaluated at 20 coastal sites. At these sites, the use of NGCPs generally produced errors similar to those observed using traditional GCPs in similar environments. Higher vertical errors are observed however when NGCP placement is limited either by the morphology of the site or when previously surveyed NGCPs are moved or destroyed, reducing the ability of the derived data products to be used in management decisions requiring accurate elevations. In these cases, higher horizontal error was not observed thus preserving their utility for management decisions that only require horizontal accuracy. A detailed investigation on the optimal quantity and distribution of NGCPs needed to minimize error revealed that at most sites seven NGCPs were needed to consistently achieve vertical accuracies of less than 0.075 m Z RMSE and 0.030 m XY RMSE. If only horizontal accuracy is required is required, in most sites four NGCPs can be used to consistently obtain less than 0.045 m XY RMSE. In some sites however, the optimal accuracy can be obtained with far fewer GCPs, while in other sites additional NGCPs may increase error if they are restricted to areas morphologically dissimilar from the rest of the site. Generally, the broadest distribution of NGCPs both in the horizontal and vertical directions produced the lowest errors. Several notable exceptions were observed however were clustered NGCPs produced lower errors than broadly distributed NGCPS. While these findings together provide general guidelines to the optimal NGCP placement, the exceptions to these guidelines indicate that the accuracy of a site cannot be described purely through the quantity and distribution of NGCPs. Conducting an accuracy assessment of sites surveyed using NGCPs is therefore essential, as the lack of such an assessment can obscure errors throughout the site, leading to misinformed and less effective management decisions. 34 The use of NGCPs in citizen-science based UAS coastal monitoring networks reduce survey cost and effort which facilitates the participation of smaller, resource limited communities in the networks. With the high-quality data generated from these studies, these communities are afforded the opportunity to conduct effective and timely data-driven management decisions. 35 APPENDIX 36 NGCP and CP Placement and Errors for All Sites Included in Study Figure 11. Placement and errors of NGCPs (blue) and CPs (red) in AuSable shoreline Park, Iosco County. 37 Figure 12. Placement and errors of NGCPs (blue) and CPs (red) in Bar Lake, Manistee. Figure 13. Placement and errors of NGCPs (blue) and CPs (red) in Berrien Beach, Chikaming. 38 Figure 14. Placement and errors of NGCPs (blue) and CPs (red) in Boardwalk West, Manistique. Figure 15. Placement and errors of NGCPs (blue) and CPs (red) in Casco Nature Preserve, South Haven. 39 c Figure 16. Placement and errors of NGCPs (blue) and CPs (red) in Cherry Beach, Chikaming. Figure 17. Placement and errors of NGCPs (blue) and CPs (red) in Fifth Ave. Beach, Manistee. 40 Figure 18. Placement and errors of NGCPs (blue) and CPs (red) in First St. Beach, Manistee. Figure 19. Placement and errors of NGCPs (blue) and CPs (red) in Harbert Beach, Chikaming. 41 Figure 20. Placement and errors of NGCPs (blue) and CPs (red) in Lake Superior Roadside Park, Marquette. Figure 21. Placement and errors of NGCPs (blue) and CPs (red) in North Beach, South Haven. 42 Figure 22. Placement and errors of NGCPs (blue) and CPs (red) in North Beach Access, Manistee. Figure 23. Placement and errors of NGCPs (blue) and CPs (red) in Oscoda Beach Park, Iosco. 43 Figure 24. Placement and errors of NGCPs (blue) and CPs (red) in Pine St. Lakeshore Blvd., Marquette. Figure 25. Placement and errors of NGCPs (blue) and CPs (red) in South Beach, South Haven. 44 Figure 26. Placement and errors of NGCPs (blue) and CPs (red) in South Beach Park, Marquette. Figure 27. Placement and errors of NGCPs (blue) and CPs (red) in Tawas City Shoreline Park, Iosco. 45 Figure 28. Placement and errors of NGCPs (blue) and CPs (red) in Tawas Point Beach, Iosco. Figure 29. Placement and errors of NGCPs (blue) and CPs (red) in Town Line Beach, Chikaming. 46 Figure 30. Placement and errors of NGCPs (blue) and CPs (red) in Township Park, Manistique. 47 REFERENCES 48 REFERENCES Agisoft Metashape (1.6.5). (2021). [Computer software]. Agisoft Metashape User Manual—Professional Edition, Version 1.6. (2020). 172. Agüera-Vega, F., Carvajal-Ramírez, F., & Martínez-Carricondo, P. (2017). Assessment of photogrammetric mapping accuracy based on variation ground control points number using unmanned aerial vehicle. Measurement, 98, 221–227. https://doi.org/10.1016/j.measurement.2016.12.002 Bove, G., Becker, A., Sweeney, B., Vousdoukas, M., & Kulp, S. (2020). A method for regional estimation of climate change exposure of coastal infrastructure: Case of USVI and the influence of digital elevation models on assessments. Science of The Total Environment, 710, 136162. https://doi.org/10.1016/j.scitotenv.2019.136162 Casella, E., Drechsel, J., Winter, C., Benninghoff, M., & Rovere, A. (2020). Accuracy of sand beach topography surveying by drones and photogrammetry. Geo-Marine Letters, 40(2), 255–268. https://doi.org/10.1007/s00367-020-00638-8 Dandois, J. P., Olano, M., & Ellis, E. C. (2015). Optimal Altitude, Overlap, and Weather Conditions for Computer Vision UAV Estimates of Forest Structure. Remote Sensing, 7(10), 13895–13920. https://doi.org/10.3390/rs71013895 Dastgheib, A., Jongejan, R., Wickramanayake, M., & Ranasinghe, R. (2018). Regional Scale Risk-Informed Land-Use Planning Using Probabilistic Coastline Recession Modelling and Economical Optimisation: East Coast of Sri Lanka. Journal of Marine Science and Engineering, 6(4), 120. https://doi.org/10.3390/jmse6040120 de Groot, R., Brander, L., van der Ploeg, S., Costanza, R., Bernard, F., Braat, L., Christie, M., Crossman, N., Ghermandi, A., Hein, L., Hussain, S., Kumar, P., McVittie, A., Portela, R., Rodriguez, L. C., ten Brink, P., & van Beukering, P. (2012). Global estimates of the value of ecosystems and their services in monetary units. Ecosystem Services, 1(1), 50– 61. https://doi.org/10.1016/j.ecoser.2012.07.005 EGLE - High Risk Erosion Areas: Program and Maps. (n.d.). Retrieved August 9, 2021, from https://www.michigan.gov/egle/0,9429,7-135-3311_4114-344443--,00.html Eltner, A., Kaiser, A., Castillo, C., Rock, G., Neugirg, F., & Abellan, A. (2015). Image-based surface reconstruction in geomorphometry – merits, limits and developments of a promising tool for geoscientists [Preprint]. Cross-cutting themes: Digital Landscapes: Insights into geomorphological processes from high-resolution topography and quantitative interrogation of topographic data. https://doi.org/10.5194/esurfd-3-1445- 2015 49 Emery, K. O. (1961). A SIMPLE METHOD OF MEASURING BEACH PROFILES. Limnology and Oceanography, 6(1), 90–93. https://doi.org/10.4319/lo.1961.6.1.0090 Gares, P. A., Wang, Y., & White, S. A. (2006). Using LIDAR to Monitor a Beach Nourishment Project at Wrightsville Beach, North Carolina, USA. Journal of Coastal Research, 22(5 (225)), 1206–1219. https://doi.org/10.2112/06A-0003.1 Gindraux, S., Boesch, R., & Farinotti, D. (2017). Accuracy Assessment of Digital Surface Models from Unmanned Aerial Vehicles’ Imagery on Glaciers. Remote Sensing, 9(2), 186. https://doi.org/10.3390/rs9020186 Gopalakrishnan, S., Landry, C. E., Smith, M. D., & Whitehead, J. C. (2016). Economics of Coastal Erosion and Adaptation to Sea Level Rise. Annual Review of Resource Economics, 8(1), 119–139. https://doi.org/10.1146/annurev-resource-100815-095416 Hanrahan, J. L., Kravtsov, S. V., & Roebber, P. J. (2010). Connecting past and present climate variability to the water levels of Lakes Michigan and Huron. Geophysical Research Letters, 37(1). https://doi.org/10.1029/2009GL041707 Harley, M. D., Turner, I. L., Short, A. D., & Ranasinghe, R. (2011). Assessment and integration of conventional, RTK-GPS and image-derived beach survey methods for daily to decadal coastal monitoring. Coastal Engineering, 58(2), 194–205. https://doi.org/10.1016/j.coastaleng.2010.09.006 How are coastal areas regulated by the CEHA Permit Program? - NYS Dept. Of Environmental Conservation. (n.d.). Retrieved August 9, 2021, from https://www.dec.ny.gov/lands/86541.html Huang, X., Song, Y., & Hu, X. (2021). Deploying Spatial Data for Coastal Community Resilience: A Review from the Managerial Perspective. International Journal of Environmental Research and Public Health, 18(2), 830. https://doi.org/10.3390/ijerph18020830 Jakubowski, M. K., Guo, Q., & Kelly, M. (2013). Tradeoffs between lidar pulse density and forest measurement accuracy. Remote Sensing of Environment, 130, 245–253. https://doi.org/10.1016/j.rse.2012.11.024 James, M. R., & Robson, S. (2012). Straightforward reconstruction of 3D surfaces and topography with a camera: Accuracy and geoscience application. Journal of Geophysical Research: Earth Surface, 117(F3). https://doi.org/10.1029/2011JF002289 James, M. R., Robson, S., d’Oleire-Oltmanns, S., & Niethammer, U. (2017). Optimising UAV topographic surveys processed with structure-from-motion: Ground control quality, quantity and bundle adjustment. Geomorphology, 280, 51–66. https://doi.org/10.1016/j.geomorph.2016.11.021 50 Jin-Si R. Over, Andrew C. Ritchie, Christine J. Kranenburg, Jenna A. Brown, Daniel Buscombe, Tom Noble, Christopher R. Sherwood, jonathan A. Warrick, & Phillipe A. Wernette. (2021). Processing Coastal Imagery With Agisoft Metashape Professional Edition, Version 1.6—Structure From Motion Workflow Documentation (Open-File Report, pp. 2021–1039) [Open-File Report]. Kidner, D. B., Thomas, M. C., Leigh, C., Oliver, J. R., & Morgan, C. G. (2004). Coastal monitoring with LiDAR: Challenges, problems, and pitfalls (M. Ehlers, F. Posa, H. J. Kaufmann, U. Michel, & G. De Carolis, Eds.; p. 80). https://doi.org/10.1117/12.565648 Krueger, R., Zoet, L. K., & Rawling, J. E. (2020). Coastal Bluff Evolution in Response to a Rapid Rise in Surface Water Level. Journal of Geophysical Research: Earth Surface, 125(10). https://doi.org/10.1029/2019JF005428 Leatherman, S. P., Zhang, K., & Douglas, B. C. (2000). Sea level rise shown to drive coastal erosion. Eos, Transactions American Geophysical Union, 81(6), 55–57. https://doi.org/10.1029/00EO00034 Lee, J.-M., Park, J.-Y., & Choi, J.-Y. (2013). Evaluation of Sub-aerial Topographic Surveying Techniques Using Total Station and RTK-GPS for Applications in Macrotidal Sand Beach Environment. Journal of Coastal Research, 65 (10065), 535–540. https://doi.org/10.2112/SI65-091.1 Mack, E. A., Theuerkauf, E., & Bunting, E. (2020). Coastal Typology: An Analysis of the Spatiotemporal Relationship between Socioeconomic Development and Shoreline Change. 17. Mancini, F., Dubbini, M., Gattelli, M., Stecchi, F., Fabbri, S., & Gabbianelli, G. (2013). Using Unmanned Aerial Vehicles (UAV) for High-Resolution Reconstruction of Topography: The Structure from Motion Approach on Coastal Environments. Remote Sensing, 5(12), 6880–6898. https://doi.org/10.3390/rs5126880 Manfreda, S., Dvorak, P., Mullerova, J., Herban, S., Vuono, P., Arranz Justel, J. J., & Perks, M. (2019). Assessing the Accuracy of Digital Surface Models Derived from Optical Imagery Acquired with Unmanned Aerial Systems. Drones, 3(1), 15. https://doi.org/10.3390/drones3010015 Martínez-Carricondo, P., Agüera-Vega, F., Carvajal-Ramírez, F., Mesas-Carrascosa, F.-J., García-Ferrer, A., & Pérez-Porras, F.-J. (2018). Assessment of UAV-photogrammetric mapping accuracy based on variation of ground control points. International Journal of Applied Earth Observation and Geoinformation, 72, 1–10. https://doi.org/10.1016/j.jag.2018.05.015 Masselink, G., & Lazarus, E. D. (2019). Defining Coastal Resilience. Water, 11(12), 2587. https://doi.org/10.3390/w11122587 51 Mehvar, S., Filatova, T., Dastgheib, A., de Ruyter van Steveninck, E., & Ranasinghe, R. (2018). Quantifying Economic Value of Coastal Ecosystem Services: A Review. Journal of Marine Science and Engineering, 6(1), 5. https://doi.org/10.3390/jmse6010005 Meinen, B. U., & Robinson, D. T. (2020). Mapping erosion and deposition in an agricultural landscape: Optimization of UAV image acquisition schemes for SfM-MVS. Remote Sensing of Environment, 239, 111666. https://doi.org/10.1016/j.rse.2020.111666 Molino, G. D., Kenney, M. A., & Sutton-Grier, A. E. (2020). Stakeholder-defined scientific needs for coastal resilience decisions in the Northeast U.S. Marine Policy, 118, 103987. https://doi.org/10.1016/j.marpol.2020.103987 Morton, R. A., & Speed, F. M. (1998). Evaluation of Shorelines and Legal Boundaries Controlled by Water Levels on Sandy Beaches. Journal of Coastal Research, 14(4), 1373–1384. Nicholls, R. J., & Cazenave, A. (2010). Sea-Level Rise and Its Impact on Coastal Zones. Science, 328(5985), 1517–1520. https://doi.org/10.1126/science.1185782 Nichols, C. R., Wright, L. D., Bainbridge, S. J., Cosby, A., Hénaff, A., Loftis, J. D., Cocquempot, L., Katragadda, S., Mendez, G. R., Letortu, P., Le Dantec, N., Resio, D., & Zarillo, G. (2019). Collaborative Science to Enhance Coastal Resilience and Adaptation. Frontiers in Marine Science, 6. https://www.frontiersin.org/article/10.3389/fmars.2019.00404 O’Dea, A., Brodie, K. L., & Hartzell, P. (2019). Continuous Coastal Monitoring with an Automated Terrestrial Lidar Scanner. Journal of Marine Science and Engineering, 7(2), 37. https://doi.org/10.3390/jmse7020037 Oniga, V.-E., Breaban, A.-I., & Statescu, F. (2018). Determining the Optimum Number of Ground Control Points for Obtaining High Precision Results Based on UAS Images. Proceedings, 2(7), 352. https://doi.org/10.3390/ecrs-2-05165 Phillips, M. R., & Jones, A. L. (2006). Erosion and tourism infrastructure in the coastal zone: Problems, consequences and management. Tourism Management, 27(3), 517–524. https://doi.org/10.1016/j.tourman.2005.10.019 Pinton, D., Canestrelli, A., Wilkinson, B., Ifju, P., & Ortega, A. (2021). Estimating Ground Elevation and Vegetation Characteristics in Coastal Salt Marshes Using UAV-Based LiDAR and Digital Aerial Photogrammetry. Remote Sensing, 13(22), 4506. https://doi.org/10.3390/rs13224506 Pucino, N., Kennedy, D. M., Carvalho, R. C., Allan, B., & Ierodiaconou, D. (2021). Citizen science for monitoring seasonal-scale beach erosion and behaviour with aerial drones. Scientific Reports, 11(1), 3935. https://doi.org/10.1038/s41598-021-83477-6 52 Quinn, F. H. (2002). Secular Changes in Great Lakes Water Level Seasonal Cycles. Journal of Great Lakes Research, 28(3), 451–465. https://doi.org/10.1016/S0380-1330(02)70597-2 Rumson, A. G., Garcia, A. P., & Hallett, S. H. (2020). The role of data within coastal resilience assessments: An East Anglia, UK, case study. Ocean & Coastal Management, 185, 105004. https://doi.org/10.1016/j.ocecoaman.2019.105004 Santana, L. S., Ferraz, G. A. E. S., Marin, D. B., Barbosa, B. D. S., Santos, L. M. D., Ferraz, P. F. P., Conti, L., Camiciottoli, S., & Rossi, G. (2021). Influence of flight altitude and control points in the georeferencing of images obtained by unmanned aerial vehicle. European Journal of Remote Sensing, 54(1), 59–71. https://doi.org/10.1080/22797254.2020.1845104 Sanz-Ablanedo, E., Chandler, J. H., Rodríguez-Pérez, J. R., & Ordóñez, C. (2018). Accuracy of Unmanned Aerial Vehicle (UAV) and SfM Photogrammetry Survey as a Function of the Number and Location of Ground Control Points Used. Remote Sensing, 10(10), 1606. https://doi.org/10.3390/rs10101606 Sentinel-2 Data Access and Products Fact Sheet. (2015). https://sentinel.esa.int/documents/247904/1848117/Sentinel- 2_Data_Products_and_Access Theuerkauf, E., & Braun, K. (2021). Rapid water level rise drives unprecedented coastal habitat loss along the Great Lakes of North America. Journal of Great Lakes Research, 47(4), 945–954. https://doi.org/10.1016/j.jglr.2021.05.004 Theuerkauf, E., Bunting, E., Mack, E., & Rabins, L. (2022). Initial insights into the development and implementation of a citizen-science drone-based coastal change monitoring program in the Great Lakes region. Journal of Great Lakes Research, S0380133022000260. https://doi.org/10.1016/j.jglr.2022.01.011 Theuerkauf, E., Mattheus, C. R., Braun, K., & Bueno, J. (2021). Patterns and processes of beach and foredune geomorphic change along a Great Lakes shoreline: Insights from a year- long drone mapping study along Lake Michigan. Shore & Beach, 46–55. https://doi.org/10.34237/1008926 Theuerkauf, E., & Rodriguez, A. B. (2012). Impacts of Transect Location and Variations in Along-Beach Morphology on Measuring Volume Change. Journal of Coastal Research, 28(3), 707–718. https://doi.org/10.2112/JCOASTRES-D-11-00112.1 Turner, I. L., Harley, M. D., & Drummond, C. D. (2016). UAVs for coastal surveying. Coastal Engineering, 114, 19–24. https://doi.org/10.1016/j.coastaleng.2016.03.011 van IJzendoorn, C. O., de Vries, S., Hallin, C., & Hesp, P. A. (2021). Sea level rise outpaced by vertical dune toe translation on prograding coasts. Scientific Reports, 11(1), 12792. https://doi.org/10.1038/s41598-021-92150-x 53 Westoby, M. J., Brasington, J., Glasser, N. F., Hambrey, M. J., & Reynolds, J. M. (2012). ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology, 179, 300–314. https://doi.org/10.1016/j.geomorph.2012.08.021 Yu, J. J., Kim, D. W., Lee, E. J., & Son, S. W. (2020). Determining the Optimal Number of Ground Control Points for Varying Study Sites through Accuracy Evaluation of Unmanned Aerial System-Based 3D Point Clouds and Digital Surface Models. 19. Zhang, K., Douglas, B. C., & Leatherman, S. P. (2004). Global Warming and Coastal Erosion. Climatic Change, 64(1/2), 41–58. https://doi.org/10.1023/B:CLIM.0000024690.32682.48 Zimmerman, T., Jansen, K., & Miller, J. (2020). Analysis of UAS Flight Altitude and Ground Control Point Parameters on DEM Accuracy along a Complex, Developed Coastline. Remote Sensing, 12(14), 2305. https://doi.org/10.3390/rs12142305 54