Appeal Brief | Appeal Letter | Appeal Analysis | Back
Second Appeal Analysis
PA ID# 063-99063-00; Livingston Parish
PW ID# PW 54; Debris Removal
High winds and heavy rains caused by Hurricane Gustav damaged and downed trees resulting in widespread vegetative debris throughout Livingston Parish (Applicant), blocking the Applicant’s roadways and creating a threat to public health and safety. The Applicant contracted with International Equipment Distributors, Inc. (IED) for debris removal and Professional Engineering Consultants Corporation (PEC) to monitor the debris removal. The Applicant participated in the Pilot Program, and FEMA reviewed and approved its Debris Management Plan, providing the Applicant an additional 5 percent federal cost share for eligible debris removal.
FEMA prepared PW 54 Version 0 for $2,892,893for the removal of 289,405 cubic yards (CY) of debris removed from the right-of-way (ROW) from September 9, 2008, to November 12, 2008 and direct administrative costs. FEMA awarded PW 54 on December 24, 2008, at the increased federal cost share of 95 percent.
Between September 22, 2008, and December 18, 2008, IED removed hazardous trees and hazardous hanging limbs. FEMA did not include this work in PW 54 Version 0 at the time of obligation because the Applicant had not provided sufficient documentation.
FEMA’s Initial Validation of Leaners and Hangers
On July 21, 2009, the Applicant submitted the contract cost documentation for the work to remove hazardous trees and hanging limbs. FEMA personnel from the Louisiana Recovery Office (LRO) performed a validation review of the eligibility of the work and claimed costs. The documentation provided by the Applicant included over 4,300 tree (load) tickets and 108,000 photographs of the trees prior to and following the work performed. These included photographs of limbs and stumps used to document their size. The applicant claimed the removal of 88,892 hazardous hanging limbs at a cost of $16,252,085 and 1,349 hazardous trees for $396,100.
Between July 28 and August 13, 2009, FEMA reviewed a 20 percent sample of the documentation supplied by the Applicant’s contractors and performed site inspections. FEMA validated those tree tickets that had photographs documenting the work performed. FEMA evaluated 17,653 of the claimed hazardous hanging limbs and 285 of the claimed hazardous trees. After completing site inspections, FEMA determined that a significant portion of the work that the Applicant performed was ineligible under the PA program. FEMA calculated that only 16 percent of the claimed hazardous hanging limbs and 34 percent of the claimed hazardous leaning trees were eligible for funding.
PW 54, Version 1
Following the validation process, FEMA prepared Version 1 of PW 54 to expand the scope and increase the cost estimate by $4,732,680 for a total amount obligated of $7,625,573. Of the additional $4,732,680, FEMA approved $2,735,007 for hazardous hanging limbs and hazardous trees based on the eligibility percentages determined from the validation process. The remaining additional funding was for additional eligible ROW debris, Temporary Debris Storage and Reduction Site management, and removal of hazardous stumps.
FEMA obligated PW 54 Version 1 on October 13, 2009. Because it was obligated outside the timeframe of the Pilot Program, PW 54 Version 1 was funded at a Federal cost share of 90 percent, rather than the 95 percent used for PW 54 Version 0 under the Pilot Program.
Second Validation Using GOHSEP Sample
Following obligation of Version 1, the Applicant and the Governor’s Office of Homeland Security and Emergency Preparedness (GOHSEP) asserted that FEMA’s sampling methodology was not a true random sample because it only included load tickets with photographs. GOHSEP then selected a statistically random sample of tickets from the entire population of the load tickets, without regard to whether photographs were available. On November 19, 2009, GOHSEP submitted the new sample and FEMA used it to conduct a second eligibility analysis. FEMA determined that only 13 percent of the hanging limbs and 19 percent of hazardous trees were eligible. Because FEMA had already written and obligated PW 54 Version 1 for the 16 percent and 34 percent eligible amounts, FEMA agreed to accept the original and higher funding amount as the basis for payment.
In its first appeal, the Applicant contested two issues: (1) FEMA’s decision to fund PW 54 Version 1 at 90 percent federal cost share rather than 95 percent under the Pilot Program, and (2) FEMA’s methodology for evaluating the eligibility of the debris removal work performed. FEMA determined that since PW 54 was originally awarded on December 24, 2008, prior to the Pilot Program deadline, that PW 54 Version 1 was also eligible for the 95 percent Federal cost share; therefore, FEMA approved that part of the first appeal.
FEMA also determined that its review of the Applicant’s documentation was adequate to determine the eligible amount of work performed, and that the ineligible work performed was accurately documented in the preparation of PW 54 Version 1. The Regional Administrator determined that FEMA had appropriately reviewed all documentation, including load tickets, photographs, and daily reports from the Applicant’s monitoring contractor, and conducted site inspections of trees and stumps to validate eligible work for reimbursement.
In its second appeal, the Applicant maintained its initial claim that it was held to an unfair documentation standard and that FEMA made data entry, sampling, and analysis errors that led to inappropriate reductions in funding. The Applicant further claimed that FEMA’s methodology to derive a percentage of eligible debris removal was fundamentally flawed and that 89 percent of the debris removal as claimed was eligible. Disaster Assistance Fact Sheet 9580.204, Documenting and Validating Hazardous Trees, Limbs, and Stumps, dated August 2, 2009, states, “FEMA will approve 100 percent funding for the applicable scope of work if it validates at least 80 percent of the sample of work items.” The Applicant claimed that 89% of the work is eligible, and requested reimbursement of $16,626,340 to cover the full cost of debris removal for leaners and hangers.
In the review of this second appeal, FEMA evaluated the specific issues raised by the Applicant. These issues included:
· Methodology employed by FEMA to sample the documentation and compliance with FEMA policy.
· Accuracy of FEMA eligibility percentage calculations.
· Specific eligibility determinations made by FEMA as part of the sample.
Documentation Review and Sampling Methodology
In its second appeal, the Applicant argued that FEMA applied an unfair standard for documentation by only including tickets with corresponding photographs in the eligibility sample. The Applicant and GOHSEP repeatedly pointed out that photographs are not required as documentation of eligibility, but are merely listed as one of several ways that eligibility can be documented.
When FEMA calculated the percentage of eligible debris removal based on the eligible amount in the 20 percent sample, FEMA did not consider any items ineligible due to a lack of photographs. Rather, they simply were not included in the sample population. Therefore, tickets that could not be associated with photographs did not influence the eligible percentage. Using the photographs provides more information from which an eligibility determination could be made, and the photographs would be as likely to show eligible work as ineligible work. While excluding a category of items from a sample (i.e. tickets without photographs) would result in a sample that is no longer technically random, FEMA found no evidence to suggest that excluding the tickets from the sample would skew the results. The Applicant did not present any evidence to suggest that a higher percentage of eligible work would have been found in the tickets without photographs.
This conclusion is also supported by the second sampling, which was performed at the request of GOHSEP. For this sampling, GOHSEP created a statistically random sample of 20 percent of the load tickets. FEMA then reviewed the eligibility of these load tickets without excluding any tickets for any reason. No tickets were removed from this sample even if photographs associated with the tickets were unavailable. Using this sample population, FEMA found only 13 percent of the hangers and 19 percent of the hazardous tree to be eligible, lower percentages than the original sample.
While photographs are not required in order to document eligibility, in this case it was appropriate for FEMA to use the photographs submitted as a basis for determining eligibility for the sample. Other forms of documentation, such as debris load tickets, also can be used to document eligibility. However the information provided by the Applicant is contradictory as to how its monitoring contractors documented eligibility. For example, a letter to the Louisiana Legislative Auditor from Mr. Tony Arikol, President of PEC, indicates that debris load tickets were used to document all work performed rather than only eligible work; however, PEC’s contract with the Applicant states “[t]he primary function of the Loading Site Monitors is to issue debris load tickets for eligible debris cleared and to provide documentation as required by FEMA reimbursement requirements.” Furthermore, the Applicant’s Tree Monitor Reports include multiple examples that call into question the eligibility of the work performed, including the removal of dead trees and trees on private property. Given the discrepancies in how and when debris monitors were documenting eligible work, the load tickets cannot be relied upon to accurately document eligibility in this case.
In the second appeal, the Applicant cited several examples of work that FEMA had found ineligible during the review, and claimed that the evidence supported the eligibility of these items. However, as part of the second appeal analysis, an independent review of the photographs and Daily Hazardous Tree Monitor Reports submitted by the Applicant supported FEMA’s original eligibility determinations. For example, many photographs of limbs, taken to demonstrate the diameter of branches cut, clearly show an entire limb with no breaks, and therefore would not have been “hazardous hangers.” Other photographs clearly show limbs cut down for reasons unrelated to hurricane damage, such as rot. None of the photographs examined as part of the second appeal demonstrated eligible work beyond that already identified by FEMA in the initial review.
In its second appeal, the Applicant claimed that FEMA made multiple data entry mistakes and analysis errors in its Validation Results spreadsheet. The claimed data entry errors consisted of 23 duplicated tree tickets and a number of limb cuts that did not match the number of limb cuts listed on tree tickets or the contractor’s invoice summary spreadsheet.
In support of its appeal, the Applicant included a spreadsheet comparing FEMA’s Validation Results spreadsheet with the Invoice Summary spreadsheet submitted by the Applicant’s contractor. The comparison spreadsheet detailed a number of discrepancies between the contractor and FEMA spreadsheets. The Applicant’s review assumed that all tree cuts identified on each tree ticket were validated by FEMA and that the number of cut limbs listed on the FEMA’s spreadsheet was recorded directly from the corresponding tree ticket. However, FEMA’s methodology differed from the Applicant’s understanding. FEMA attempted to validate the total number of eligible cuts on a given tree rather than on an entire ticket. In many instances, tree tickets recorded cuts from multiple trees. In other cases, the monitor’s record of the number of cuts from a particular tree began on a particular ticket and carried over onto subsequent ticket(s). Where this occurred, FEMA validated all cuts for a particular tree recorded on multiple tree tickets. On its spreadsheet, FEMA attributed the total number of cuts to the first tree ticket used to document the cuts for that tree. Similarly, if a selected tree ticket showed cuts from a tree that had been carried over into the sample tree ticket, those cuts were not included in the review.
For this reason, the total number of cuts validated under a particular ticket number might be either more or less than the number of cuts recorded on that ticket. Some of the discrepancies noted by the Applicant also appear to represent data entry errors by FEMA. Given that the total hanger count identified by the Applicant differed from FEMA’s count by only 25 hangers out of more than 17,000 total hangers, a difference of approximately 0.1 percent, the impact of any potential data entry errors is negligible.
In preparing the second appeal, the Applicant used a version of the Validation Results spreadsheet dated August 27, 2009; however, in preparing PW 54 Version 1, FEMA used a version of the spreadsheet dated September 10, 2009. The August 27, 2009, document included 23 duplicate tree tickets as identified by the Applicant. It appears that this version of the spreadsheet was a working document and not the final document used to determine the eligibility percentage in PW 54 Version 1. The final Validation Results spreadsheet dated
September 10, 2009, does not contain the identified duplicate tree ticket entries.
Based on a comprehensive and independent review and analysis of the data submitted with the appeal, FEMA has determined that the Applicant’s contractor completed ineligible debris removal work. This determination is documented by photographs, daily reports, and FEMA site visit logs. FEMA’s methodology to review the Applicant’s documentation was appropriate and accurate, and is further supported by the additional random sampling performed at GOHSEP’s request. Therefore, the second appeal is denied.