Website Monitoring

Websites are the primary means by which federal agencies communicate with the public. Changes to the presentation of and access to website information can therefore directly impact the public’s ability to participate in environmental governance. Since January 2017, EDGI’s website monitoring team has been tracking changes to tens of thousands of federal agency environmental webpages. We document, analyze, and report on these changes; assess the quality and accessibility of information; and provide recommendations to agencies for improving the provision and preservation of their public information. 

Our work revealed systematic suppression of information relating to environmental rules and regulations under the Trump administration, and the imperative for a series of improvements to website governance, as described in Access Denied: Federal Web Governance Under the Trump Administration. We also detailed widespread censorship of climate change language across federal agencies during the Trump administration in The New Digital Landscape, and elucidated patterns of censorship in Visualizing changes to US federal environmental agency websites 2016-2020. During the Biden administration, we are pressing the agencies to improve web resources and build websites that can foster scientific literacy and learning, as well as opportunities for public oversight and agency accountability. A key focus area for us is the availability and utility of web information to help the public understand and comment on proposed regulations. We keep a public record of website changes that we observe in our Federal Environmental Web Tracker (updated quarterly) and support journalists and other environmental advocacy groups in conducting their own analyses. 

Our body of work consists of three types of technical reports, white papers, and public comments:

For more information on EDGI’s website monitoring work, please contact us through Gretchen Gehrke at gretchen.gehrke@envirodatagov.org or  edgi.websitemonitoring@protonmail.com. You can also explore how development on our monitoring platform, Scanner, is doing at https://github.com/edgi-govdata-archiving/web-monitoring. 

Web Governance Assessment Reports

1) Web Governance Analysis of Changes to the EPA’s Unconventional Oil and Gas Extraction Effluent Guidelines Webpage  (screenshots)

Environmental Protection Agency (EPA) | Report written: December 10, 2019 | This report covers changes made to the EPA’s Unconventional Oil and Gas Extraction (UOG) Effluent Guidelines webpage between July 1 and July 5, 2019, including the removal of information relevant to UOG extraction wastewater management and the final rule’s enforcement. The webpage provides contextual information regarding the promulgation of EPA’s final rule prohibiting the discharge of UOG extraction wastewater into Publicly Owned Treatment Works (POTWs) and the extension of the implementation deadline of this rule for some UOG operators in Pennsylvania. However, as of the date of this report, the webpage does not contain sufficient information for a public audience to understand EPA’s regulatory actions. EDGI recommends that government websites such as this one preserve the historical train of regulatory developments and track changes to their webpages such that the public can view the evolution of the information available online. EDGI further recommends that agencies provide ladders of information on their websites, tailoring content at broad and specific levels for both the general public and audiences more versed in the subject matter at hand.

Access Assessment Reports

13) Department of the Interior Deletes Budget Documents From Website

Department of the Interior (DOI) | Report written: July 28, 2020 |There has been a significant reduction in the financial transparency of the Department of the Interior (DOI). Between March and May 2020, DOI removed the budget justifications for all years prior to FY2020 that had been hosted on its DOI.gov website. The URLs for those budget justifications now register as “Page Not Found.” Additionally, budget request highlights (called “Budget in Briefs”) and budget process testimonies have also been
removed and now register as either “Access Denied” or “Page Not Found.” Through these removals, DOI has significantly impaired public understanding of the financial trajectory of the Department, and impeded public and Congressional oversight

12) Website Actions by Federal Environmental Agencies During the Government Shutdown, Including Broad Restriction of Access on NOAA Websites  (screenshots)

National Oceanic and Atmospheric Administration (NOAA) | Report written: February 25, 2019 |Different federal agencies managed their web resources differently during the recent  35-day partial government shutdown. Many agencies, such as the Environmental  Protection Agency (EPA) placed a banner at the top of webpages to inform the public that  websites would not be updated, and did not update public datasets. Another agency, the  National Oceanic and Atmospheric Administration (NOAA), blocked access to many web  resources by redirecting URLs to a simple government shutdown notice. The lack of  maintenance of websites during government shutdowns has historical precedent, such as  during the 16-day shutdown in 2013, but we cannot quantify its extent. In the most recent  government shutdown, nearly one-third of the NOAA webpages that EDGI monitors (1,045  out of 3,275) redirected to the shutdown notice, including informational pages and data  portals for archival climate, atmospheric, oceanic, and environmental data. However,  some essential real-time operational data, such as that produced by the National Centers for  Environmental Prediction (NCEP), remained available and were regularly updated during  the shutdown.   Federal agencies have built a web infrastructure in  order to deliver data and display important environmental information, but a lack of  infrastructural maintenance, such as what transpired during this past government shutdown, has demonstrated how vulnerable  that infrastructure can be. In this report we document ways in which different federal agencies managed their web  resources during the most recent government shutdown, with particular focus on NOAA. At the end of the report we discuss implications and recommendations for data access and  management during future funding lapses.

  • EDGI Blog –

11) DOI Further Restricts Access to Climate Change Web Pages (screenshots)

Department of the Interior (DOI) | Report written: February 28, 2019 | Since Summer 2018, the Department of the Interior (DOI) has restricted access to its primary climate change webpage: https://www.doi.gov/climate. This change extends previous content removals from the page reported on by EDGI in 2017. DOI has also removed “Climate Change” from its linked menu of “Our Priorities” on the agency’s home page and restricted access to: https://www.doi.gov/climate/carbonfootprint.

10) Reduction in Access to Sage Grouse Conservation Information and Resources on BLM Websites (screenshots)

Bureau of Land Management (BLM) | Report written: December 18, 2018 | The U.S. Department of the Interior’s Bureau of Land Management (BLM) removed and altered web resources regarding conservation efforts for the sage grouse during the period between March and December 2018, when sage grouse conservation plans have been under revision by BLM.

9) EPA Discontinues Updates to Climate Change Websites (screenshots)

Environmental Protection Agency (EPA) | Report written: October 31, 2018 | Since April 2017, ​EPA has substituted their climate change subdomain​ with a splash page  stating that these sites were being updated to reflect the views of the Trump  administration. In October 2018, EPA modified that splash page to remove any mention of  “updating” and to simply state, “We want to help you find what you are looking for.” EPA  also removed links to the outgoing administration snapshot (January 19, 2017 snapshot) of  the climate change website main page and to the press release about the initial removal of  the climate change websites.

8) Removals of Migratory Bird Treaty Act and Incidental Take Resources from the DOI’s U.S. Fish and Wildlife Service Website (screenshots)

U.S. Fish and Wildlife Service (FWS) | Report written: September 25, 2018 | The U.S. Department of Interior’s (DOI) U.S. Fish and Wildlife Service (FWS) has removed  links and content related to the Migratory Bird Treaty Act and to “Incidental Take” from  three webpages in the FWS domain. The FWS “Incidental Take” page itself has also been  removed. Additionally, a “public involvement initiative” website of FWS, birdregs.org, that  focused on the incidental take of migratory birds, is no longer accessible. The website  changes described in this report occurred between December 28, 2017 and April 6, 2018.

7) Removal of EPA’s “International Priorities” and “International Grants and Cooperative Agreements” pages, as well as corresponding links, from the International Cooperation Website (screenshots)

Environmental Protection Agency (EPA) | Report written: April 23, 2018 | The EPA has removed its “International Priorities” page, which listed “Strong Environmental Institutions,” “Climate Change,” “Air Quality,” “Clean Water,” “Toxic Chemicals,” and “E-Waste” as priority areas. Likewise, the “International Grants and Cooperative Agreements” page and links to information about priorities and grant applications were removed from EPA’s International Cooperation website.

6) Removal of Climate Action Plans and Links from the National Park Service’s Climate Friendly Parks Program Website (screenshots)

Department of the Interior (DOI) | Report written: December 21, 2017 | 92 documents regarding national parks’ response to climate change have been removed from the National Park Service (NPS) “Climate Friendly Parks Program” website. NPS is part of the Department of the Interior. To become a Climate Friendly Parks (CFP) member, parks are required to meet four milestones, which include developing a baseline of greenhouse gas emissions for park operations and completing a park climate action plan. Links to the parks’ action plan documents and to two webpages detailing aspects of two particular parks’ plans have been removed from a list of the Program’s member national parks. No public NPS Web archive has yet been identified, but instructions to email NPS for access to the parks’ action plans have been added.

5) Assessment ​of ​Removals ​and ​Changes ​in ​Access ​to ​Resources ​on the ​EPA’s ​“Climate ​and ​Energy ​Resources ​for ​State, ​Local, ​and Tribal ​Government” ​Website (screenshots)

Environmental Protection Agency (EPA) | Report written: October 18, 2017 | On ​April ​28, ​2017, ​the ​EPA ​removed ​its ​website ​titled ​“Climate ​and ​Energy ​Resources ​for State, ​Local, ​and ​Tribal ​Governments.” ​Approximately ​three ​months ​later, ​a ​new ​website titled ​“Energy ​Resources ​for ​State, ​Local, ​and ​Tribal ​Governments” ​was ​launched ​in ​its place, ​with ​fewer ​pages ​and ​omitting ​Web ​resources ​relating ​to ​climate ​and ​climate ​change. All ​Web ​tools ​and ​certain ​energy ​resources ​appear ​to ​have ​been ​moved ​from ​the ​previous website ​to ​the ​new ​one, ​although ​a ​comprehensive ​comparison ​has ​not ​yet ​been conducted. ​The ​launch ​of ​the ​new ​“Energy Resources” ​website ​is ​the ​first ​example ​where ​the ​EPA ​has ​returned ​content ​following those ​April ​28th ​removals.

4) Removal of Substantial Portion of Content and Reduction in Access to Resources on the USDA’s Climate Hubs Website (screenshots)

US Department of Agriculture (USDA) | Report written: August 1, 2017 | Between the evenings of July 27 and July 28, USDA removed substantial portions of its Climate Hubs website. The web content was removed for approximately 5 days without notice or explanation and was returned following media inquiries made with the USDA based on the release of the report above.

3) Missing Environmental Protection Agency Endangerment Finding Web Resources

Environmental Protection Agency (EPA) | Report written: July 12, 2017 | Web resources were removed from the “Endangerment and Cause or Contribute Findings for Greenhouse Gases under the Section 202(a) of the Clean Air Act” ​page​ on the EPA’s website. The lack of access to these resources likely originated from mismanagement of web resources during the EPA’s climate change website overhaul, which occurred on April 28, 2017, and not a more recent removal of content.

2) Removal of Information and Reduction in Access to Resources on the EPA’s Clean Water Rule Website (screenshots)

Environmental Protection Agency (EPA) | Report written: June 29, 2017 | The EPA overhauled its website pertaining to the Clean Water Rule. The previous website, which extensively described the ​Clean Water Rule and its benefits​, was replaced by a website providing information about the EPA’s review of the Rule.

1) Change in Access to the EPA’s “A Student’s Guide to Global Climate Change” Website (screenshots)

Environmental Protection Agency (EPA) | Report written: May 5, 2017 | As a result of the EPA’s Climate Change website overhaul, the “A Student’s Guide to Global Climate Change” subdomain is no longer accessible via the main EPA website and is incorrectly stated as captured in the EPA’s January 19 snapshot.

Content Change Reports

25) Evolution of the Department of the Interior’s Tagline, Mission, and Priorities

Department of the Interior (DOI) | Report written: June 1, 2020 | In March 2020, the U.S. Department of the Interior (DOI) changed its tagline, highlighted in every DOI webpage (www.doi.gov/) footer, from “Protecting America’s Great Outdoors and Powering Our Future” to “Stewarding Conservation and Powering Our Future.” This represents a trend in the Department’s shifting public image. In 2018, DOI changed the wording of its mission from “…protects and manages…” to “…conserves and manages…,” discarding “protects” after at least 20 years as part of its mission statement. DOI has also shifted its stated priorities in recent years, removing priorities regarding youth engagement, integrated conservation approaches, and climate change, and replacing them with priorities regarding jobs, regulatory reform, protecting the border, fulfilling internal agency goals, and recreation.

24) DOI Adds “Recreation” Priority and Web Resources

Department of the Interior (DOI) | Report written: May 4, 2020 | In March 2019, the Department of the Interior (DOI) updated the dropdown list of “Priorities” on its webpages to include a new priority of “Recreation.” The landing page of the new priority included links to recreational resources at four of DOI’s bureaus: National Park Service, Bureau of Land Management, Fish and Wildlife Service, and Bureau of Reclamation. The new priority reflected recent actions by DOI and lawmakers to highlight
recreational uses of public lands.

23) Changes to EPA’s “Natural Gas Extraction – Hydraulic Fracturing” Webpage (screenshots)

Environmental Protection Agency (EPA) | Report written: October 9, 2018 | The Environmental Protection Agency’s (EPA’s) former “Natural Gas Extraction – Hydraulic Fracturing” webpage has been updated to entirely remove the page title and change it to “Unconventional Oil and Natural Gas Development.” A section detailing EPA stakeholder outreach called “Promoting Transparency and Conducting Outreach” was removed. A section called “Convening Stakeholders” was added that highlights EPA partnerships with oil and natural gas sectors. Content and links related to EPA guidance and compliance material on hydraulic fracturing were removed.

22) Removal of Climate Change Mentions, Links, and Information from the CDC’s National Institute for Occupational Safety and Health Website (screenshots)

Department of Health and Human Services (HHS) | Report written: July 2, 2018 | In late 2016, the Centers for Disease Control and Prevention’s (CDC) National Institute for Occupational Safety and Health (NIOSH) webpages on climate change were altered to remove mentions of the impact of climate change on occupational safety and health. Climate change mentions and information, including many sentences and a chart detailing the relationship between climate change and workplace health and safety, were removed from pages. An entire page linking to government and academic publications addressing the impact of climate change on worker health and safety was removed.

21) Removal of a Climate Change Webpage from and Shift in Language on the Bureau of Land Management’s Website (screenshots)

Department of the Interior (DOI) | Report written: January 9, 2018 | DOI’s Bureau of Land Management (BLM) removed its climate change webpage and altered language on several webpages describing BLM’s mission, management of public lands, and national priorities.

20) Removal from the Greening EPA Website of a Climate Change Adaptation Web Resource, Links to Resources, and Mentions of EPA’s Own Greening Performance Goals (screenshots)

Environmental Protection Agency (EPA) | Report written: December 6, 2017 | The EPA’s Greening ​EPA ​website ​has ​been ​altered ​to remove ​access ​to ​Web ​resources ​on ​climate ​change ​adaptation, ​and ​to ​remove ​wording related ​to ​EPA’s ​own ​goals ​for ​climate ​change ​resilience ​and ​adaptation. ​An ​EPA ​statement on ​climate ​change ​adaptation ​and ​a ​link ​to ​the ​Statement ​have ​been ​entirely ​removed. Links ​to ​EPA’s ​Climate ​Change ​Adaptation ​Plans ​have ​been ​removed ​from ​at ​least ​two pages ​on ​the ​EPA ​website. ​References ​to ​EPA’s ​federal ​leadership ​and ​goals ​to ​cover ​100 percent ​of ​its ​own ​electricity ​use ​nationwide ​through ​purchasing ​renewable ​energy ​have also ​been ​removed.

19) Confirmation ​of ​Changes ​to ​the ​Federal ​Emergency ​Management Agency’s ​Hurricane ​Maria ​Webpage (screenshots)

Federal Emergency Management Agency (FEMA) | Report written: October 5, 2017 |The ​Federal ​Emergency ​Management ​Agency ​(FEMA) ​removed ​statistics ​from ​the ​“Federal Response ​Updates” ​section ​on ​its ​“Hurricane ​Maria” ​webpage. ​Certain ​subsections ​and bullet ​points ​reporting ​statistical ​metrics, ​quantifying ​access ​to ​electricity ​and ​drinking water, ​were ​removed. ​Additional ​statistics, ​descriptive ​bullet ​points, ​and ​images ​were ​also updated. ​This ​report ​confirms ​the ​changes ​to ​FEMA’s ​“Hurricane ​Maria” ​webpage ​identified by The ​Washington ​Post article below.

18) Changes to Language on the Environmental Protection Agency’s SmartWay Program Website  (screenshots)

Environmental Protection Agency (EPA) | Report written: September 19, 2017 | The ​EPA ​changed ​language ​on ​the ​SmartWay Program ​website, ​reducing ​mentions ​of ​carbon, ​greenhouse ​gasses, ​and ​climate change. ​Terms ​like ​“sustainability” ​and ​“emissions” ​replaced ​mentions ​of ​“carbon,” and ​emphasis ​on ​international ​SmartWay ​and ​other ​climate ​efforts ​were ​reduced.

17) Removals​ ​of​ ​Climate​ ​Change​ ​Mentions​ ​and​ ​Links​ ​from​ ​the National​ ​Institute​ ​of​ ​Environmental​ ​Health​ ​Sciences​ ​Website (screenshots)

Department of Health and Human Services (HHS) | Report written: August 20, 2017 | The​ ​National​ ​Institute​ ​of​ ​Environmental​ ​Health​ ​Sciences​ ​(NIEHS)​ ​has​ ​altered​ ​climate change​ ​language,​ ​updated​ ​climate​ ​change​ ​references,​ ​and​ ​reduced​ ​access​ ​to​ ​a​ ​Web resource​ ​with​ ​information​ ​on​ ​climate​ ​change​ ​and​ ​human​ ​health​ ​across​ ​several webpages.​ ​On​ ​the​ ​Global​ ​Environmental​ ​Health​ ​pages,​ ​the​ ​term​ ​“climate​ ​change”​ ​has been​ ​changed​ ​to​ ​“climate”​ ​on​ ​side​ ​menus​ ​and​ ​page​ ​titles.​ ​Links​ ​to​ ​an​ ​educational fact​ ​sheet​ ​about​ ​climate​ ​change​ ​were​ ​removed,​ ​reducing​ ​access​ ​to​ ​the​ ​resource.

16) Changes to the Department of Energy’s Office of Technology Transitions Website (screenshots)

Department of Energy (DOE) | Report written: May 20, 2017 | The DOE’s Office of Technology Transitions’ (OTT) Clean Energy Investment Center page has been updated to entirely remove the phrase “clean energy.” The Center’s name was changed from “Clean Energy Investment Center” to “Energy Investor Center” and links to pages on clean energy resources were removed from the OTT website.

15) Confirmation of Changes to Department of the Interior’s Climate Change Page (screenshots)

Department of the Interior (DOI) | Report written: April 28, 2017 |The DOI made changes to its climate change page, removing and re-writing descriptive paragraphs and entire sections on the DOI’s role in addressing climate change.

14) Changes to DOE’s Office of Energy Efficiency & Renewable Energy Bioenergy Technologies Office Web Pages (screenshots)

13) Changes to DOE’s Office of Energy Efficiency & Renewable Energy Vehicle Technologies Office Web Pages (screenshots)

12) Changes to DOE’s Office of Energy Efficiency & Renewable Energy Wind Energy Technologies Office Web Pages (screenshots)

Note: Reports 12-14 were co-released

Department of Energy (DOE) | Reports written: April 19, 2017 | DOE’s Office of Energy Efficiency & Renewable Energy (EERE) has made extensive changes to pages pertaining to the Bioenergy Technologies Office, Wind Energy Technologies Office and Vehicles Technologies Office. Stated office priorities were changed to decrease emphasis on renewable fuels as a replacement for fossil fuels and increase emphasis on US jobs and economic growth.

11) Rewording of Language, Domain Names, and Titles on the Department of Transportation’s Federal Highway Administration Environment Pages (screenshots)

Department of Transportation (DOT) | Report written: April 13, 2017 | The DOT’s Federal Highway Administration ​(FHWA) has changed language and titles across multiple pages relating to environmental effects of transportation, such as replacing “climate change” and “greenhouse gases” with terms like “sustainability” and “emissions”.

10) Removal and Change of Language and Content on the Government Accountability Office’s Explanation of Risk in the Management of Federal Oil and Gas Resources (screenshots)

Government Accountability Office (GAO) | Report written: April 4, 2017 | The GAO page on managing federal oil and gas resources has been edited to change the framework within which natural gas production is discussed. Content on the environmental and public health risks of shale oil and natural gas production was also removed.

9) Change of Language and Content for the EPA’s Climate Ready Water Utilities Program and Web Page (screenshots)

Environmental Protection Agency (EPA) | Report written: March 28, 2017 | The EPA’s Climate Ready Water Utilities program was renamed to Creating Resilient Water Utilities. This change was reflected on the program’s page – all mentions of the word “climate” were removed and terms like “extreme weather” and “resilience” were emphasized.

8) Changes to Office of Science and Technology Policy Mission Statement (screenshots)

White House (WH) | Report written: March 22, 2017 | The description and mission statement of the WH’s Office of Science and Technology policy was changed, and major site reorganization occurred.

7) Removal of Language and Links on the Bureau of Land Management’s Hydraulic Fracturing and Methane Rules (screenshots)

Department of the Interior (DOI) | Reports written: February 13, 2017 | The DOI’s Bureau of Land Management’s language about the purpose of the 2015 Hydraulic Fracturing Rule and a link to that rule from a page on regulations for onshore energy production were removed. Additionally, a section on, and link to, the proposed Methane Waste Prevention Rule was removed.

6) Changes to the State Department’s Office of Global Change Description (screenshots)

Department of State | Report written: February 10, 2017 (Updated: April 12, 2017) | The description paragraph for the Office of Global Change has been significantly changed. In particular, the terms “adaptation” and “sustainable landscapes” have been added, while the term “greenhouse gas” has been removed. Several links, including one to the Climate Action Report, were removed.

5) Changes to DOE’s Energy Information Administration Kids Educational Pages (addendum; screenshots)

Department of Energy (DOE) | Reports written: February 8, 2017 | On the DOE’s Energy Information Administration (EIA) website, language and content on the Energy Kids educational pages describing environmental impacts of various energy sources has been changed and removed to downplay the harms of nonrenewable energy sources.

4) Changes to EPA’s Page About Office of Science and Technology within Office of Water (screenshots)

Environmental Protection Agency (EPA) | Report written: January 31, 2017 | The description of the role of the Office of Water’s Office of Science and Technology, in the EPA, removed any mention of science, and replaced a mission of developing scientific foundations for protecting people with developing economically and technologically feasible industry performance standards.

3) Changes to the EPA’s Page on Federal Collaborations on Climate Change (screenshots)

Environmental Protection Agency (EPA) | Reports written: January 29, 2017 | Links to and descriptions of environmental and climate change programs which establish collaborations with international, federal, interagency, and tribal partners were removed from the EPA’s Federal Partner Collaborations page.

2) Removal of the Department of Energy’s Climate Action Plan Page (screenshots)

1) Removal of the State Department’s Climate Action Reports and Changes in Climate Office Descriptions (screenshots)

Note: Reports 1 and 2 were co-released

Department of State and Department of Energy (DOE) | Reports written: January 26, 2017 | On the Department of State’s website, the descriptions of the Office of Global Change and the Office of the Special Envoy for Climate Change were changed. In addition, pages relating to climate change agreements, such as a page about the Climate Action Report, were removed. On the DOE’s website, the Climate Action Plan page was removed.

White Papers

2) The New Digital Landscape: How the Trump Administration Has Undermined Federal Web Infrastructures for Climate Information

1) Changing the Digital Climate: How Climate Change Web Content is Being Censored Under the Trump Administration

Public Comments

8) EDGI’s Comment on FWS’ Draft Environmental Impact Statement for the Proposed Regulation of the Migratory Bird Treaty Act

7) EDGI’s Comment on EPA’s Supplemental Notice of Proposed Rulemaking for the Strengthening Transparency in Regulatory Science Rule

6) EDGI’s Comment on FWS’ Proposed Revisions to the Migratory Bird Treaty Act

5) EDGI’s Comment on CEQ’s Proposed Changes to the National Environmental Policy Act

4) EDGI’s Comment on OSTP’s Draft Desirable Characteristics of Repositories for Managing and Sharing Data Resulting From Federally Funded Research

3) EDGI’s Comment on EPA’s Waters of the United States Proposed Rule

2) EDGI’s Comment on EPA’s Strengthening Transparency in Regulatory Science Proposed Rule

1) EDGI’s Comment on EPA’s Clean Power Plan Proposed Repeal

Disclaimer for all EDGI Reports: The information and images within the report are for general information purposes only. The scope of this report is limited to version monitoring information of publicly available websites. EDGI has no control over the nature, content, or sustained availability of the websites we monitor. While EDGI works to assure that the information in this report is correct, that information is subject to the limitations of version tracking software, and is provided “as is.” EDGI makes no representations or warranties of any kind, express or implied, about the completeness or reliability of this information, nor does EDGI intend to assess any agency or entity’s intentions or rationale for the demonstrated changes to any webpages or other online content that appear in this report. Do not rely on the information in this report as predictive, or ascribe intent not presented within the report. In no event will EDGI or any of its members be liable for the use or misuse of the information in this report. Please consult with an appropriately qualified expert if you require qualitative evaluation of or advice about the content of this report.