APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT 7 Results The following Section describes the results achieved for this project and the conclusions reached regarding the quality of the provided LIDAR data. The results are grouped based on geographic area, which represent different environments, scenarios, and datasets from various LIDAR systems and operating parameters. Mecklenburg County, NC (16 feet pixel) Automatic determination of the LAG, HAG and elevation of TBF was performed for 39 tiles. CCS extracted 414,105 objects from a total area of 1,000 km2. Nearly 80% of these objects were homes and other buildings. Each tile measured 18,000 ft (x-axis) by 16,000 ft (y-axis). Typical samples of extractions are shown in Figures 26, 27, and 28 for tile E08. Figure 26 — Z_minimal from Last Returns for Tile E08 Mecklenburg County, NC Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 27 — Footprints of 13,076 Structures Extracted Automatically for Highly Populated Tile E08 Mecklenburg County, NC. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 28 — Boundaries of 13,067 Structures Automatically Extracted for Tile E08 Mecklenburg County, NC. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 29 — Small Area of Tile E08 Mecklenburg County, NC. Left: Z_minimal from Last Returns Right: Footprints of Structures that were Extracted Automatically. (1-pixel Objects (mostly trees) Deleted as Noise) For each tile of Mecklenburg County, NC, CCS generated a data file that lists all structures where each entry describes a building. Format of file: Home # Coordinates of Central Points Area Perimeter Average Building Height Max Building Height Number X, Y ft2 ft ft ft 13 1456227.00 , 521090.00 3072.00 288.00 19.02 23.50 LAG for Z_Min LAG for Z_ground HAG for Z_ground TBF ft ft ft ft 655.61 655.61 664.65 669.11 Heights of homes are relative (average or maximal) heights in the last returns up to minimal heights of the ground in a footprint of a home. There are two LAG approaches for calculation (in most cases they are the same): using Z_min from last return or using Z of ground (bare Earth). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Home #13188 on the edge on deep canal Control home #6,7 and structure #13252 Control home #4 Control home #8 Control home #5 Figure 30 — Small area of Tile E09 Mecklenburg County, NC. Left: Z_minimal from last returns; Right: footprints of structures automatically extracted including control homes #4, 5, 6, 7, and 8. For control homes, LAG, TBF and HAG (in some cases) can be compared between control parameters and parameters extracted from the LIDAR data. Control homes #6 and 7 were extracted as one merged structure #13,252 from CCS' list due to intermediate tree. Control homes #6 and #7 and extracted structure #13252 have the following parameters: Home #6 Home #7 Structure #13252 LAG 633.1 ft 639.9 ft 635.01 ft HAG 634.7 ft 644.1 ft 644.73 ft TBF 634.6 ft 646.0 ft 644.73 ft The CCS software looks for the minimal elevation on the boundary of the merged structure #13252 as LAG (635.01 ft) and this LAG must be close to minimal LAG of both homes #6 and #7 (633.1ft). The CCS software looks for maximal elevation on the boundary of merged structure #13252 as HAG (644.73 ft) and this HAG must be close to maximal HAG of both homes #6 and #7 (644.1ft). Due to merging of these homes, CCS has a large error for the LAG of one house and large errors for the HAG and TBF of another house. CCS cannot consider the LAG and HAG of merged homes as reliable determinations. As seen in Figure 30, structure #13188 was located on the edge of a deep canal (part of the home actually overlapped this canal). As a result, pixels for the boundary of the home included the area at the bottom of the canal. Automatically determined LAG Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT shows the depth of the lowest part of the canal near the home. The actual LAG in such cases is a function of architectural solution, and can be close to the bottom of the canal, or the top of the edge of the canal (or ground around). The difference between these possibilities is 15 ft. This case can be considered a possible source of large error for the LAG, an example of where human intervention is needed. Control home #12 Control home #13 Control home #11 Control home #14 Figure 31 — Control homes #11, 12, 13 and 14 from Dewberry's list. Left: Z_minimal from last returns; Right: extracted footprints. Figure 32 — Control homes #1, 2, 3, 9, 10 and 15 from Dewberry's list. Left: Z_minimal from last returns; Right: extracted footprints. Footprint of home N1 missed. Control home #10 Control home #9 Control home #3 Control home #2 Control home #1 Control home #15 Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 33 — Control homes #16, 17 and 18 from Dewberry's list. Left: Z_minimal from last returns; Right: extracted footprints. Control home #17 Control home #16 Control home #18 The following table lists the results of extraction compared with control data (error shows extracted data minus control data; negative errors mean that the extracted data is lower than the control data): Accuracy of LAG, HAG and TBF for 18 homes (Tile E09, Mecklenburg, NC) *) #N X_c Y_c #N X_d Y_d r-min d~LAG d~HAG d~TBF CCS (coord) Dew- (coord.) ft errors in ft list in pixel berry ========================================================== 800 1045 67 2 1044 68 25.9 1.0 -0.6 -1.5 895 1055 76 3 1054 75 27.0 1.3 1.4 0.9 13647 77 917 4 76 917 19.4 -3.8 2.3 0.8 ** 12442 78 908 5 77 908 20.6 -1.3 5.3 5.3 *** 13252 74 896 6 77 896 48.0 1.9 10.0 10.1 **** 13252 74 896 7 73 899 42.1 -4.9 0.6 -1.3 **** 13010 67 883 8 65 881 45.8 0.0 4.6 4.2 1037 1070 84 9 1071 85 11.3 0.8 0.5 0.7 1063 1075 86 10 1075 88 24.7 0.1 1.1 -0.1 12936 644 878 11 653 879 139.7 0.0 - 9.1 *** 13104 661 889 12 664 884 98.9 -0.7 - 3.0 13333 662 906 13 664 898 125.2 -1.4 - 1.9 13246 673 894 14 674 892 27.7 -0.3 - 3.4 782 1053 66 15 1052 67 27.6 -0.9 - 3.7 6486 496 425 16 495 425 13.2 -1.3 --3.6 6444 492 422 17 491 423 21.5 0.6 --2.4 6349 492 417 18 490 416 32.6 0.5 --1.8 Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Notes related to results above: * Building N1 was not extracted due to small gradient on the edge. ** CCS can not find a reason for such a large LAG error (the home was extracted properly and LAG has good agreement with last returns of LIDAR data around home). Two sources for the error are possible: 1) error in control data (elevation certificate); 2) large slope of ground surface near home (4-5 feet per 16 feet pixel; no signature because of large natural gradient around home, but large artificial gradient in boundary pixels is possible). *** Large error in TBF reflects a large error in HAG that was determined as the first approximation HAG only. HAG_1 (and TBF) overestimated as a result of capturing part of the trees near the home (see Fig.4). Second approximation of HAG that was developed later can fix this error and HAG and TBF will be close to the actual value. **** Homes #6 and #7 were not extracted properly due to an intermediate tree (extracted as merged structure #13252 in CCS list). See Figure 30 and comments in text. Average errors (exclude homes #6 and #7) for LAG: 0.93 ft or 28 cm HAG: 1.98 ft or 59 cm TBF: 2.83 ft or 85 cm Quality of extraction of homes in Mecklenburg County, NC is near 80% (for example, 15 homes from 18 control homes were extracted properly). About 20% of the missing structures are likely due to large pixels, merging, complex shape etc. Approximately 2030% of the objects from the list of extracted structures can be considered false alarms (due to dense clusters of trees). Results of processing Mecklenburg County LIDAR data show that this LIDAR data has reasonable penetration in foliage and good Z-accuracy. The main problem with this data is the low statistics (wide point spacing) and, as a result, large pixel size (16 feet). Sometimes extracted buildings were merged with other buildings and trees. The array of Z_minimal from last return provides a means from minimizing the effects of noise attributed from foliage. Using this type of data results in a decreased size of structures up to 1 pixel from each side: boundary pixels consist of returns from edge of roofs and ground simultaneously and considered in this data as objects with minimal heights or ground. For 16 ft pixels, some buildings with a size smaller than 32 ft can be unintentionally omitted. Comparing the results of automatically extracted footprints of homes and automatically determined LAG, HAG and TBF (methods of extractions described in previous charts) shows that LIDAR data for Mecklenburg County provide: Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT 1. Reasonably good quality for the extraction of structures with average level of false alarms, and good determination of structure features such as: central point, size, relative and absolute heights. 2. Accurate determination for LAG (about 1 feet average error). 3. Reasonably accurate estimation for HAG, which is more sensitive to objects around the structure (trees, cars, etc). (2 ft average error). First approximation of HAG has large errors (5-10 ft) in nearly 20% of the cases. 4. Fair estimation for elevation of TBF as a function of LAG and HAG and average parameters of home (2.5-3.0 ft average error). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Prince George's County, MD (8 feet pixel) Automatic determination of the LAG, HAG and TBF elevation was performed for six tiles 209NE04, 209NE05, 208NE04, 208NE05, 206NE03 and 205NE03 where CCS extracted 6,172 structures. The tile size was 6400 ft x 4400 ft or 2.5 km2. The total area processed was 15 km2. Figures 34 and 35 show Z_minimal from last returns and extracted footprints for tile 208NE04. Figure 34 — Z_minimal from last returns for tile 208NE04, Prince George's County, MD. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 35 — Footprints of 1,220 Objects and Structures Automatically Extracted for Tile 208NE04 of Highly Populated Prince George's County, MD. Merged close buildings Trees extracted as homes Figure 36 — Small Area of tile 208NE04. Left: Z_minimal from Last Returns Right: Footprints of Extracted Objects (two typical problems: merging closest homes and false alarms from large trees) Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT About 80-90% of the homes were extracted correctly in Prince Georges County, MD; 10-20% of the structures were merged with another or with trees or not extracted properly due to low laser light reflection. Nearly 20% of extracted objects were false alarms. Control data (ECs) for 20 homes shows high quality of automatic determination of LAG, HAG and TBF. All control homes were extracted properly (case of home #17 will be discussed below). The LAG calculations were very accurate. Positive errors in the HAG indicate that the extracted HAG was higher than the control HAG. The main reason for large positive errors is that the extracted HAG was the first approximation for the HAG. The second approximation for the HAG was developed later (for Beaufort County data). The HAG for two cases (#1 and #17) will be discussed below. The estimation for TBF elevation can be considered of good quality and accuracy. The largest error for TBF was building #17, which will be discussed below. Accuracy of LAG, HAG and TBF for 20 homes Prince George's County, MD N Number of home N in Image of Accuracy (in cm) in control list CCS building LAG HAG TBF (# tile) list 1. 24 (209NE04) #98 25.pdf -8.7 +66.0 +9.3 * 2. 41 (209NE04) #54 60.pdf -10.2 +7.5 -16.2 3. 43 (209NE04) #38 105.pdf -12.9 +16.2 -18.9 4. 45 (209NE04) #45 102.pdf +15.0 +2.1 +51.0 5. 46 (209NE04) #53 100.pdf +9.9 +9.0 +6.9 6. 49 (209NE04) #11 101.pdf +14.7 +7.8 -15.3 7. 53 (208NE04 #1175 108.pdf +11.7 +4.5 +20.7 8. 55 (208NE04) #1162 112.pdf +4.2 +5.4 +4.2 9. 58 (208NE04) #1087 89.pdf +15.9 +1.8 +9.9 10. 63 (208NE04) #1200 113.pdf +14.7 +12.3 +23.7 11. 74 (209NE04) #55 35.pdf -0.9 +1.8 - 12. 76 (209NE04) #120 29.pdf -12.3 +38.7 +14.7 13. 78 (209NE05) #26 56.pdf -6.9 +62.1 - 14. 84 (208NE04) #977 4.pdf -10.8 +82.5 - 15. 85 (209NE04) #152 23.pdf -3.0 +37.8 -27.0 16. 96 (209NE04) #243 72.pdf 0.0 +9.6 -12.0 17. 111 (209NE04) #238 37.pdf +7.5 +88.5 -153.9 ** 18. 5 (208NE04) #1163 9.pdf +12.6 -15.6 19. 16 (208NE04) #1165 135.pdf +12.0 +17.4 -3.0 20. 36 (209NE04) #144 26.pdf +3.6 +11.7 +9.6 Average error: Without building 17 (#111): 9.4 24.9 24.8 (cm) 21.6 16.2 (cm) * See Figure 37 and comments ** See Figure 38 and comments Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Compare control and extracted parameters for home #1 (24), 25.pdf Control Extracted Error LAG: 58.05 ft 57.76 ft -0.29 ft or -8.7 cm HAG: 58.75 ft 60.95 ft 2.2 ft or 66.0 cm TBF: 61.45 ft 61.76 ft 0.31 ft or 9.3 cm According to the elevation certificate, the control HAG is higher than the LAG by 21 cm and the TBF is higher than the HAG by 81 cm. According to the extracted data, the LIDAR-based HAG is higher than the LAG by 96 cm, and the TBF is higher than the HAG by 24 cm. Imagery of the home in Figure 37 shows that the HAG is closer to the TBF, not the LAG. These results indicate that the LIDAR-based determination of LAG, HAG and TBF can be used to check for blunders in existing ECs. The following is a comparison of control and extracted parameters for home #17 (111), 37.pdf Control Extracted Error LAG: 46.45 ft 46.70 ft +7.5 cm HAG: 48.10 ft 51.05 ft +88.5 cm TBF: 56.18 ft 51.05 ft -153.9 cm According to the elevation certificate, the control TBF is higher than the LAG by 9.73 ft or 292 cm. From the imagery, CCS cannot find a reason for such differences, but prior experience indicates errors in the control data. Another lesson from this imagery is that LIDAR-based determination of LAG, HAG and TBF considers any long building (cluster of townhomes, shopping mall) as a single structure with one set of LAG, HAG and TBF. CCS can not extract separate LAG, HAG and TBF for each townhome or building with individual postal address but physically merged with other buildings. This is the source for large errors in LAG, HAG and TBF. Human intervention cannot help in this situation (at least not in all cases). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Elevation of TBF LAG Elevation of TBF HAG Figure 37 — Front (top) and rear (bottom) images of the home #1 (24), 25.pdf. From imagery, CCS can estimate that HAG is close to the TBF and much higher than LAG. Imagery supports the LIDAR-based estimation and confirms a probable blunder in the EC. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT TBF according to Elevation Certificate (56.18 ft or 9.73 ft higher than LAG) LAG according to Elevation Certificate (46.45 ft) Figure 38 — Front (top) and rear (bottom) images of the home #17 (111), 37.pdf. From imagery CCS can estimate that the TBF must be very close to the LAG, but the elevation certificate shows that the TBF is much higher than the LAG. This, too, appears to be a blunder in the EC. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT The results of processing the data for Prince George's County shows that this LIDAR data has average quality of foliage penetration, good Z-accuracy, and high horizontal resolution (8 feet pixel). The main problem with the data is that the sensitivity of LIDAR is not so high and misses data returns from dark asphalt or dark roofs (as a result, parts of buildings were lost). The following summarizes the results for calculation in Prince George's County: 1. LAGs can be extracted from LIDAR data with accuracy close to the accuracy of the LIDAR data (average error <10 cm) for all buildings. 2. HAGs can be extracted from LIDAR data with good accuracy (average error ~20 cm) for buildings that were properly extracted. The main source of error for the HAG is the quality of the extraction and use of the second approximation. Errors of automated extraction can be fixed by human intervention for the most difficult cases. 3. TBFs can be extracted from LIDAR data with good accuracy (~20 cm) using a reasonable set of rules for determination of TBF. Source of error for TBF is a deviation from these rules from area to area. Customization of these rules can be done by human intervention using a larger set of control data and imagery. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Harris County, TX (5 feet pixel) Description of LIDAR data for Harris County (information from vendor): “The Harris Co, TX Full Feature Surface Database consists of x, y, and z point data derived from an Airborne LIDAR Topographic Mapping System (ALTMS). The x, y, and z values are stored in space delimited ASCII text files. "Purpose: Data was created so that it could be used as highly accurate, inexpensive way to create digital topographic vector and raster files for implementation in Geographic Information Systems (GIS) and used for the Tropical Storm Allison Recovery Project. "Supplemental Information: The data points are nominally spaced at 1.5-meter intervals with approximately a 0.50 meter horizontal accuracy. The 1.5-meter spacing may vary in areas not reflective to laser pulses, such as water bodies, dark asphalt roofs, and some types of glass or fiberglass construction. "Surface elevation value accuracy is better than 15 centimeters. Flight altitude is approximately 915 meters, creating a data swath of approximately 550 meters. "The elevation data provided is for the earth's surface and includes vegetation, such as trees and shrubs, as well as the built environment.” Note: This is the LIDAR first return of the top surface, rather than the last return normally used to generate bare- earth digital terrain models. Specific information for tile q29095f52: Beginning_Date: 2001.10.07 Ending_Date: 2001.11.05 Bounding_Coordinates: West_Bounding_Coordinate: 3058995 East_Bounding_Coordinate: 3079695 North_Bounding_Coordinate: 13836960 South_Bounding_Coordinate: 13813455 Automatic determination of LAG, HAG and elevation of TBF was performed for 19 tiles in Harris County, TX (each tile 11.34 km2, total area 215 km2). The main problem with this data was the low sensitivity of the LIDAR receiver. As a result, buildings with dark roofs were not captured in the data (see Figures 39-43). The low sensitivity of the LIDAR equipment resulted in single returns and very poor penetration of foliage (see Figures 39 and 40). As a result, there is systematic merging of buildings and large tree clusters. The CCS software that had good results in Mecklenburg and Prince George's Counties could extract only large clusters in forested areas (see Figure 41). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT The quality of extraction is estimated as <50% for CCS' usual approach and 50% to 70% after additional and time-consuming human intervention for foliage filtering (see Figure 44). Figure 39 — Z_minimal for part of tile q30095a24, Harris County, TX before filling holes. All dark areas are areas without LIDAR data. These dark areas include most homes in this tile. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 40 — Distribution of LIDAR shots in tile q30095a24. Vertical strips show data swaths according to direction of flights. Water bodies and dark roofs have no reflection. White strips show overlapped LIDAR data (area with larger statistics, i.e., dense point spacing because of double collection in overlap areas). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Areas containing homes with dark roofs Figure 41 — Footprints of structures extracted in tile q30095a24. As result, there is poor penetration of laser beams in foliage, most buildings in forested area are merged. Forested areas look like huge clusters versus individual stands of tree. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Buildings with Dark or Asphalt Roofs Figure 42 — No LIDAR data collected for certain buildings with dark roofs (i.e., tar or asphalt). Tile q29095g44. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Highway Parking lots Figure 43 — Z_minimal for part of tile q29095f52 (center of bottom part), Harris County, TX before filling holes. All dark areas are areas without LIDAR data. These dark areas include the part of highway, parking lots and homes in this tile. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 44 — Top: Z_minimal for tile q30095g44. Left top part of area was additionallyfiltered from foliage. Bottom: extracted footprints. Quality extraction in filtered area is better than in other areas, but is far from the quality of the extraction for data in Prince George's County, MD. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Comparison of data (LAG and HAG) for five control homes shows additional problems: a. All control points were shifted from the homes to sidewalks (Figure #45). b. Control data for LAG and HAG has a serious difference from LIDAR-based estimations and LIDAR data for this area (Figures #46-48). Figure 45 — Five control homes for Harris County, TX (tile q 29095h32). All control points are close to sidewalks or streets and far from homes. Two horizontal lines show profile of elevations (see Figure 47 and 48). Evaluation of Alternatives in Obtaining Structural Elevation Data 355 Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Geological shift of surface of 2-2.5 feet Figure 46 — Ground for area with five control homes for Harris County, TX (tile q29095h32). Ground in this area is very flat and varies near 1 foot only. Only one vertical shift of 2-2.5 ft was found. Ditches near road have 1-1.5 feet of depth. Control LAG (ft) HAG (ft) 5923 Glen Lee Dr 80.01 80.12 6018 Woodview Dr 76.02 78.62 5947 Marwood Dr 74.30 74.60 5939 Glen Grove 78.35 78.36 5930 Morningdale St 78.37 78.41 The top profile of elevations in Figure 47 shows the elevation near control points on Morningsdale St and Marwood Dr. The profile shows an elevation for all areas near 72 ft. Compared with the control LAG and HAG for these points, the control points are much higher (2 - 6 ft) than the LIDAR profile. The reason for such a difference is not clear. Bad calibration of LIDAR data can be responsible for the systematic shift of elevation LIDAR profile, but this hypothesis does not explain why control homes in the flat area have LAG and HAG variance up to 6 ft. An example of this is the control point on Marwood Dr. which is lower than the control point on Morningdale St. by 4.07 ft, but the LIDAR profile (Figure 47) shows that these two points must be very close to each Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT other (within 1 ft). CCS sees the same situations with all control points: they are higher than LIDAR data and have suspicious variability in the LAG and HAG. Profile of elevation for y=1958, tile q29095h32 Z (feet) 120 100 80 60 40 20 0 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101106111116121126131 Pixels (5 feet each) Two 1-ft ditches near road Home Series1 Series2 Figure 47 — Profile of elevation between two control points shows flat area (72 - 73 ft) with variation near 1 ft. The second profile of elevation (Figure 48) shows heights near the control point on Glen Lee Dr. The elevation varies between 74 - 75 ft and the control LAG and HAG is greater than 80 ft, which is not realistic from the LIDAR data. The LIDAR data shows that the control points on Glen Lee Dr. and Woodview Dr. must be within 1 ft of each other, but the control LAG for Woodview Dr. is less than Glen Lee Dr. point by 4 ft. CCS cannot exclude bad calibration of the LIDAR as a possible cause, but more likely is that the control data for the LAG and HAG has serious errors (up to 6 ft in elevation). In this situation, it does not make sense to compare the control data with data extracted from LIDAR. In addition, there are large horizontal shifts in the control points that cannot provide robust identification between control points and specific homes. Evaluation of Alternatives in Obtaining Structural Elevation Data 357 Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Profile of elevation for Y=1762, tile q29095h32 Z (feet) 120 100 80 60 40 20 0 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 126 131 136 141 146 151 Pixels (5 feet) Homes with elevation of edge of roofs near 86-87 ft Series1 Series2 Figure 48 — Profile of elevations near Glee Lee Dr shows five homes with elevation of roof edges near 86 - 87 ft and flat ground (74 - 75 ft) with variation near 1 ft. The control LAG and HAG for this area is more than 80 ft, which cannot be correct. Based on the LIDAR data statistics for Harris County, CCS expected good results for construction of 3-D building geometries. Unfortunately, the results were not as expected, because of foliage obscurations, low resolution of building features, and abnormal reflections from roof materials (see Figure 49). Of the seven buildings constructed in 3-D, only one was of reasonably good quality and clearly shows an attached deck (Figure 49). Evaluation of Alternatives in Obtaining Structural Elevation Data 358 Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 49 — Extraction of 3-D geometry of seven buildings adversely affected by surrounding trees. Only one of the seven structures had good quality for the shape and the detailed features (e.g., second floor deck). Conclusion: LIDAR data for Harris County, TX had poor foliage penetration and lacked receiver sensitivity to take measurements of buildings with dark roof material. Moreover, this data shows minimal ability to meet FEMA goals. Even though the data statistics allowed CCS to choose a small pixel size for processing, it did not compensate for the lack of LIDAR system sensitivity. The quality of home extraction was also poor. Comparisons of building extractions with five control homes show strange and large (up to 6 ft) differences between elevation measurements and control data. CCS believes that in this case, the control data was wrong and elevation from LIDAR data is correct. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Beaufort County, SC (4 feet pixel) Automatic determination of the LAG, HAG and TBF was performed for 18 tiles from Beaufort County, SC (each tile 1.47 km2, total area 26.5 sq.km). Nearly 2000 buildings were extracted and processed. The quality of the LIDAR data (high statistics, penetration on foliage, robust reflection from dark surfaces) was very good (see Figures 50 and 51) and quality of building extraction was also very good (see Figures 52 and 53). Figures 54, 55 and 56 show samples of heavily forested tiles. Power lines Figure 50 — Z_maximal from last return, tile 2056_144. Z_maximal reflection from the tops of trees showing that Beaufort County is a heavy vegetated area. Detection of power lines in the last returns of LIDAR data indicates very sensitive LIDAR equipment was used. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 51 — Z_minimal from the last return, tile 2056_144. Good penetration of laser beams in foliage allows software filtering of most forested areas and creates a condition for high quality extraction of buildings. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 52 — Footprints of extracted buildings in Beaufort County, SC (Tile 2056_144). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Control building #261 Control building #247 Figure 53 — Footprints of extracted buildings (Tile 2056_144). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 54 — Part of heavily vegetated tile 2040_180, Beaufort County, SC. Z_minimal from last returns. Very good penetration by laser beams in foliage. The small pixel size (4 ft) provides good conditions for high quality building extraction. Figure 55 — Footprints of all objects extracted in this tile, including buildings and tree trunks. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 56 — Footprints for extracted buildings (Tile 2040_180). Results of processing LIDAR data for Beaufort County, SC: 1. The high quality of LIDAR data (small pixel, good penetration of laser beams in foliage for heavy vegetated areas) allowed quality extraction of structures. CCS was able to automatically extract 90 - 95% of the buildings. The number of false alarms (mostly due to dense clusters of trees) after automated extraction was 1020%. 2. Analysis of the control data shows that many control points have large horizontal shifts: up to 70 - 80 feet. In two cases CCS could not find homes for control points in tile 2052_168 (control point in forest where the nearest home was 600 feet) and in tile 2064_136 where the control point was in water. Control points in area 2040_184 and 2044_136 appear to be too far from homes (>100 feet for 2040_184). Typically, a control point is not in the center of home, but near the building perimeter. 3. Comparison between control heights for five buildings and LIDAR data is shown below: Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Tile: N home: LAG LAG Error HAG TBF TBF Error (LIDAR) (control) (LIDAR) (LIDAR) (control) ft ft ft (cm) ft ft ft ft (cm) 2056_144 247 15.14 10.98 4.16 (125)* 15.60 19.14 13.09 6.05 (182) 2056_144 261 11.39 11.78 -0.39 (-12) 14.10 15.39 14.21 1.18 (35) 2060_148 26 11.51 11.88 -0.37 (-11) 13.22 15.51 13.68 1.83 (55) 2060_148 43 7.41 8.38 -0.97 (-29) 12.12 12.21 13.28 -1.07 (-32) 2056_148 186 7.12 9.78 -2.66 (-80)* 9.33 11.12 15.08 -3.96 (-119) * Large errors for buildings # 247 and #186 is not clear (see analyses below). 4. High quality LIDAR data provided sufficient information for accurate generation of 3-D buildings and visual verification of the LAG and HAG. Direct determination of TBF could not be done in most cases, but reasonable estimates for the TBF were possible using information about building shape, roof heights, and LAG and HAG (see Figures 57 - 62). Figure 57 — 3D shape of building #73 (tile 2056_144, see Figures 50-53) constructed fromLIDAR data shows all basic building features and the roof. APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT LAG HAGTBF Figure 58 — Tree contours (levels) show LAG, HAG and TBF after automated extraction. Building #73 (tile 2056_144). Figure 59 — Large building #261 from tile 2056_144 (right top corner on Figures 50-53). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 60a HAG . Figure 60 — Visual verification of HAG (a) and LAG (b) for building 261 from tile 2056_144. Figure 60b LAG As can be seen in Figures 58 and 60, the 3-D model of a building and ground can verify the quality of automated determination of the LAG and HAG. LAG and HAG for buildings #73 and #261 were determined correctly using the surrounding ground surface. There were some large differences between control data and extracted data for buildings #247 and #186. Figures 61 through 63 show that the LAG and HAG for building #247 extracted from LIDAR data was correct. There are two possible reasons for such large errors between extracted and control data: 1) error in the LIDAR data, or 2) error with the elevation certificate (possibly the wrong home). After the LIDAR data was collected, the measurements were validated using many control points, which gave Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT good accuracy for at least two other control buildings (see Figure 60 for building #261). We believe that large errors in the LIDAR data is unlikely and that the errors lie within the elevation certificate for building #247. LIDAR shots Part of trees Figure 61 — Control building #247 (tile 2056_144) with large errors between control data and LIDAR results. The sparse foliage around the building would not obscure or alter the shape of the building. Blue dots show the raw LIDAR shots. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Distribution of elevation for building #247 0 50 100 150 200 250 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 Z (feet) Number of pixelsSeries1 Figure 62 — Distribution of heights of LIDAR shots shows that ground around home #247 have level near 15-16 feet. Control LAG (10.98) must be 4 feet lower than the ground. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 63 — Extracted LAG (15.14 ft), HAG (15.6 ft) and TBF (19.14 ft) for building #247. Control LAG elevation of 10.98 or 4.16 ft lower than the visible ground level or extracted LAG and HAG. The control data is likely not accurate. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figure 64 — Other side of building #247. Level of LAG and HAG (15-15.5 ft) is very close to ground level. The same situation occurs with control home N 186 in 2056_148. As can be seen from Figures 65 and 66, the LAG correctly shows the lowest point of contact between the building and ground. In such cases, there is greater confidence in results where there are more numerous LIDAR points around homes than a single point from an elevation certificate. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT LAG HAG Figure 65 — Control building 186 (tile 2056_148) with large error between control data and LIDAR results. Level of LAG (lowest line, 7.12 ft) shows lowest parts of buildings correctly and close to ground Top panel: building only Bottom panel: home and ground surface. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT HAG Figure 66 — Opposite side of building #186. Line of extracted HAG (9.33ft) is higher than the ground in many cases. Control LAG (9.78 ft) must be higher than both extracted HAG (9.33 ft) and LAG (7.12 ft). LAG HAG (TBF) Figure 67 — Control building #43 (tile 2060_148). Extracted LAG (7.41 ft) and HAG (=TBF) (12.12 ft) agree with LIDAR data. Control LAG is 8.38 ft, one foot larger than the extracted LAG. g 374 Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Figures 68 and 69 show samples of buildings with decks and second floor levels. Such details can help determine the TBF more accurately. Figure 68 — Building #44 from 2048_108. Level of second floor is visible. Deck Figure 69 — Building #101 from 2068_140. Level of second floor deck is visible (possible base for manual correction of TBF level). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT LIDAR data for Beaufort County, SC has good penetration in foliage and high spatial resolution (4 feet per pixel), meets FEMA goals, and provides enough information for robust determination of the LAG and HAG of buildings and other parameters: central points, footprints, shape of buildings (and roof elevation). Estimation of the TBF elevation can be done by computer or human analysis using information about the building shape, heights of the roofs, LAG and HAG. Direct determination of the TBF can be done, in some cases, using the LIDAR returns from a deck and other details of building. CCS analyzed five control homes. The LAG for two homes had accuracies close to the vertical accuracy of the LIDAR data (close to 10 cm). The control data for the three other homes was not accurate. As CCS saw earlier, most sets of control data had serious problems with accuracy. The use of LIDAR data for determining LAG and HAG of elevation certificates is accurate in most cases as well as the coordinates for the central points of homes. In addition, the use of LIDAR allows the generation of a 3-D image for a building (or information about distribution of elevations points in and near a building), which can be included in an elevation certificate to provide additional information. CCS results show that automated processing of LIDAR data, when complimented by human inspection and intervention for unique scenarios, can meet the FEMA requirements for: • Robust buildings extractions, • Accurate determination of LAG, HAG and TBF, and • Efficient characterization of building parameters. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT 8 Summary and Recommendations. The goal of this project was to investigate the use of LIDAR data and intelligent software to determine structural elevation data: LAG, HAG and TBF. In addition, the investigation included an assessment of capabilities to determine central point coordinates of buildings, the heights and shapes of roofs, and other building characteristics. The process and results discussed in Sections 5, 6, and 7 definitively show that LIDAR data has direct application to determining elevation data required for elevation certificates. The information content, statistics, and measurement accuracy of the provided LIDAR data varied substantially from system to system, which affected the ability to detect and automatically extract the structures, identify and discriminate buildings from other objects, resolve closely spaced buildings, and identify building features leading to indirect determination of the TBF. After completing this project, CCS has determined that no universal set of software parameters can be employed. Varying environments require some level of software customization to achieve optimal results when used to process LIDAR data. CCS scientists and analysts have concluded that automated software alone will not provide the required results for 100% of the cases; however, automated, intelligence software used to process LIDAR data is a significant improvement over traditional methods. Additional details follow. The first step in the process was to determine which LIDAR returns were associated to buildings and those from the ground adjacent to buildings. CCS achieved this by extracting building footprints and boundaries. The primary influence on the footprint and boundary extraction is the penetration of laser beams in foliage, which directly affects the ability to separate buildings from trees in LIDAR raster images. The quality of automated extraction marked in Table 2 can be improved with additional software development and improvements. To achieve 100% extraction without false alarms, manual verification and validation is required using tools such as ArcView. Table 2. Results of automated extraction of footprints of buildings. Dataset Pixel size Laser penetration through foliage Returns from “dark” roof materials Percent of extracted footprints* Percent of false alarms Mecklenburg County, NC 16 ft good good ~80% ~20-30% Prince George's County, MD 8 ft good good ~80-90% ~20% Harris County, TX 5 ft poor poor <50% >50% Beaufort County, SC 4 ft very good very good >90-95% ~10-20% * 80% of extracted footprints mean that 80% of separated structures of area extracted properly (cluster of town homes considered as single or separated structure). Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT When a building footprint database is provided by a County Government, CCS considers that data to be 100% complete with no false alarms and no manual/human intervention would be required to improve the automated extraction results. The second step focused on analyzing LIDAR returns within the extracted building footprints, boundaries of footprints and surrounding ground. The operating parameters of the LIDAR systems had a significant impact on the data and results. The number of LIDAR returns for typical residential homes varied from approximately 10 returns per home for Mecklenburg County, NC (16 feet pixel, 1-2 shots per pixel) to hundreds of returns per building for Beaufort County, SC (4 feet pixel, 1-2 shots per pixel). CCS results show that we can use automatic methods to determine the LAG with high accuracy and reliability (average error < 0.5 - 1 ft or 15-30 cm for 95% of properly extracted buildings). More accurate estimations are possible after analyzing larger sets of control data with good accuracy. Accuracy and reliability of the automatically determined HAG was reasonably good (1 - 2 feet, see Table 3). Determination of the LAG and HAG can be directly determined from the LIDAR data with a simple algorithm. The accuracy of LAG and HAG calculations depend mostly on the quality of the LIDAR data and selected size of pixel. Determination of the TBF cannot be done using direct measurements in most cases; however, the TBF elevation can be indirectly estimated using a set of rules based on LAG and HAG elevations, area of the homes, average and maximal elevation of homes. Using these indirect methods, the average error for automated calculations of the TBF was 1 - 3 feet (see Table 3). Table 3. Results of automated determination of LAG, HAG and TBF *. Average Errors Number of Control Homes Number of Acceptable Footprints Number of Questionable Control Points Dataset LAG HAG TBF Mecklenburg County, NC 28 cm 59 cm 85 cm 18 15 - Prince Georges County, MD w/ good control data 9.4 cm 24.9 cm 24.8 cm 20 20 2 9.4 cm 19.1 cm 16.2 cm 18 Harris County, TX 51.4 cm -82.6 cm 5 5 3 w/ good control data 11.5 cm -40. cm 2 * Average error of control data for Harris County, TX was close to 3 ft from our estimation and cannot be used for comparison with LIDAR data that have accuracy of 10-20 cm. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry APPENDIX J — LIDAR AUTOMATED DATA EXTRACTION REPORT Human intervention is necessary for verifying and improving the LAG, HAG and TBF. Results of this project show that LIDAR data does provide enough information for reliable determination of LAG and HAG using: -profiles of elevations of home in different directions; -distribution of pixels of homes and adjacent grounds by elevation; -3D-shape of building, reconstructed from LIDAR data. The project results show that in many cases the LIDAR-based determination of LAG and HAG are more accurate than control data that had numerous errors and poor accuracy. CCS attributes the control data errors to human procedures or blunders, the employed equipment, restricted views of homes in large or forested lots, or privacy restrictions related to private property data. CCS believes that human intervention and validation can improve the average error for the LAG to approximately 0.5 – 1.0 ft, and the error in the HAG to about 1 foot with a maximum error of about 2 feet for 95% of the cases. An experienced person can improve the automated results of the TBF determination using the LIDAR determined LAG and HAG and knowledge of local building architecture and LIDAR elevation measurements from decks, balconies, edges of roofs and other elements of homes. CCS estimates that this human intervention can improve the accuracy of TBF estimates to 1 - 2 feet with maximum errors of 2 - 3 feet for 95% of the cases. Results of this project and previous experience of CCS experts involved in a CENSUS project for determining center points of structures from LIDAR data, shows that in all cases where building footprints are extracted properly, the building central points are calculated with an accuracy of 2 - 4 meters. The average heights of roofs, maximal heights of roofs, area and length of perimeter of home can also be determined with high accuracy. Using data about these building features, CCS can classify extracted structures as one- level, two-levels home, three-levels, garage, or large building such as shopping centers. The addition of confidence metrics for the LAG, HAG, TBF and other parameters of homes will be useful for focusing of human efforts for the most suspicious cases. Evaluation of Alternatives in Obtaining Structural Elevation Data Dewberry