About the Data System (Purposes, Audience):
The Visibility Information Exchange Web System (VIEWS) is an online system of data and tools designed to support the EPA's Regional Haze Rule by providing states, tribes, scientists, researchers, planners, students, and others with easy online access to a wide variety of air quality datasets. The VIEWS team maintains a comprehensive database of air quality data from over two dozen monitoring networks and is constantly acquiring new data sets and adding new networks. The VIEWS website offers many tools and resources, including the Database Query Wizard, an air quality trends tool, an aerosol species composition tool, a monitoring site metadata browser, a dynamic contour-mapping tool, general-purpose charting and graphing tools, visibility photographs, Class I Area webcams, and more. The VIEWS website currently has over 1100 registered users from over 100 different countries and hosts thousands of visitors each month. The VIEWS team also develops and maintains the IMPROVE website, the WRAP Technical Support System (TSS), and the Air Toxics Data Archive (ATDA), all of which utilize the foundational database and software architecture developed for VIEWS. Ongoing development and maintenance of VIEWS is conducted by Colorado State University's Cooperative Institute for Research in the Atmosphere (CIRA) in Fort Collins, Colorado. Visitors to VIEWS are encouraged to register and provide comments, questions, and feedback to the VIEWS team regarding all aspects of the website and database.
In 2002, a steering committee composed of representatives from the EPA and the five Regional Planning Organizations (RPOs) – Western Regional Air Partnership (WRAP), Central Regional Air Planning Association (CENRAP), Midwest Regional Planning Organization (MRPO), Mid-Atlantic/Northeast Visibility Union (MANE-VU) and Visibility Improvement State and Tribal Association of the Southeast (VISTA) – decided to extend the contract between CIRA and the Western Governors Association (WGA) in order to develop and expand the existing WRAP online ambient monitoring database into a national system called the Visibility Information Exchange Web System (VIEWS). As a result, the VIEWS development team designed and implemented an improved database management system and a new website to serve as the foundational elements of VIEWS, based upon requirements and feedback from the VIEWS steering committee. Over the next few years the user base for VIEWS grew significantly, as did user requests for additional datasets and new functionality. Based largely upon its popularity and success within the general air quality community, VIEWS was selected by the WRAP in 2005 to serve as the foundational architecture and starting point for the development of the WRAP’s Technical Support System (TSS), a high-level planning and decision and support system intended to aid the development of state and tribal Implementation Plans in the WRAP as required by the Regional Haze Regulations. The TSS was built upon the database and software infrastructure of VIEWS, and the first version was launched in early 2007. The TSS is designed to be a portal to the technical data and analytical results prepared by WRAP Forums and Workgroups in support of the air quality planning needs of western state and tribes. These resources are maintained and updated to support the implementation of regional haze plans and other air quality analysis and management activities.
RPOs, NPS, WRAP, CIRA
List of Publications, Papers, Presentations:
Data System Scope
|AQS Fine Mass FRM (D)||Daily||01/01/1999||12/31/2003||798317||04/13/2004|
|AQS Fine Mass FRM (H)||Hourly||01/01/1999||12/31/2003||7028023||04/13/2004|
|AQS Fine Speciation (D)||Daily||02/09/2000||12/31/2006||6839576||05/24/2007|
|AQS PM10 - Daily||Daily||01/01/1994||12/31/2003||1121497||07/30/2004|
|CASTNet Dry Chemistry||Weekly||01/06/1987||03/23/2005||793290||11/23/2005|
|CASTNet Visibility Chemistry||Daily||10/25/1993||12/27/2001||166950||03/11/2004|
|IMPROVE Aerosol (Preliminary)||Daily||8/1/2006||2/26/2007||660492||08/31/2007|
|IMPROVE Aerosol (Raw)||Daily||03/02/1988||7/31/2006||10804852||08/31/2007|
|IMPROVE Coarse Mass Speciation Study||Daily||03/01/2003||07/29/2003||444|
|IMPROVE Aerosol (RHR1)||Daily||01/01/1988||12/31/2004||7220634||12/12/2005|
|IMPROVE Aerosol (RHR2) (New Algorithm)||Daily||01/01/1988||12/31/2005||4861358||03/09/2006|
|NADP/Mercury Deposition Network (MDN)||Weekly||11/22/1995||09/28/2004||109820||03/30/2005|
|NADP/Nation Trends Network (NTN)||Weekly||07/05/1978||02/03/2004||4824603|
|Navaho Generating Station (NGS)||Daily||03/04/1992||05/30/1992||701||04/02/2004|
|NPS SFU Aerosol||Daily||07/27/1979||11/13/1993||850126||03/02/2004|
|SEARCH All Variables||Daily||05/01/1998||3/31/2005||438650||02/13/2006|
|SEARCH Best Estimate||Daily||05/01/1998||3/31/2005||153416||02/13/2006|
[[DataSystemParam::The integrated VIEWS database contains data for over 250 air quality variables and parameters. These include surface observations, modeled parameters, and emissions data. For a complete list of the parameters maintained for each individual dataset, please visit the VIEWS Query Wizard and note the information on the "Select Parameters" tab as you select various datasets.]]
Spatial - Temporal Coverage:
[[DataSystemCoverage::Currently, the datasets in VIEWS are mostly relevant to the United States and neighboring countries such as Canada and Mexico. For complete information on the geographical coverage of the various datasets in VIEWS, please visit the VIEWS Site Browser, which is an online tool that displays maps of the monitoring sites for each network. Future plans include the addition of various international datasets and global satellite data.]]
[[DataSystemAppHealth::Many of the datasets maintained by VIEWS are frequently used for health research and epidemiological studies. In particular, university researchers and state departments of health frequently visit VIEWS to download and/or analyze the various PM2.5 and Hazardous Air Pollutant (HAP) datasets maintained in the integrated database. A recent addition to VIEWS is the dataset from the NADP Mercury Deposition Network (MDN), and data from the NPS Ozone network is currently being added as well. The VIEWS team also developed and currently maintains the Air Toxics Data Archive (ATDA) for the EPA, which contains HAP, criteria pollutant, and other air toxics data.]]
Forecasting and Reanalysis:
[[DataSystemAppFcstReAnaly::The Regional Haze Rule (RHR) goals include achieving natural visibility conditions at 156 Federally mandated Class I areas by 2064. In more specific terms, that RHR goal is defined as (1) visibility improvement toward natural conditions for the 20% of days that have the worst visibility and (2) no worsening in visibility for the 20% of days that have the best visibility. One component of the states’ demonstration to EPA that they are making reasonable progress toward this 2064 goal is the comparison of modeled visibility projections for the first milestone year of 2018 with what is termed a uniform rate of progress (URP) goal. The 2018 URP goal is obtained by constructing a “linear glide path” (in deciviews) that has at one end the observed visibility conditions during the mandated five-year (2000-2004) baseline period and at the other end natural visibility conditions in 2064; the visibility value that occurs on the glide path at year 2018 is the URP goal. The VIEWS infrastructure has been used to develop two online tools for the WRAP's Technical Support System (TSS) that are designed to help states and tribes forecast future visibility conditions and analyze their progress toward achieving natural visibility conditions by 2064. The Monitoring Glide Slope Tool allows users to view the glide slope information, and the Visibility Projection Tool displays RHR glide slope information in addition to model-based visibility projections for the 2018 reasonable progress milestone. Additional information on how the data in the VIEWS integrated database is used by the TSS tools to project future visibility conditions and assess reasonable progress towards future visibility goals can be found in the TSS Air Quality Modeling document.]]
[[DataSystemAppModelEval::The TSS Model Performance Tool can be used to evaluate model performance on a site-by-site basis by comparing the 2002 modeling results to the IMPROVE monitoring data collected in 2002. A detailed description of how the TSS data and tools can be used for model evaluation can be found in the TSS Air Quality Modeling document.]]
Characterization, Trends, Accountability:
Data System IT
Primary/Official Store for Some data:
[[PrimaryDataStorage::VIEWS is the primary online source for data collected by the IMPROVE (Interagency Monitoring of Protected Visual Environments) aerosol and optical networks. VIEWS is also the primary source for the official Regional Haze Rule data that includes the baseline IMPROVE data with additional substituted data values and various data aggregations that have been calculated according to the EPA's Regional Haze Rule Guidance documents.]]
VIEWS employs an advanced data acquisition and import system to integrate data from several air quality data centers into a single, highly-optimized data warehouse. Ground-based measurements from dozens of monitoring networks, air quality modeling results, and detailed emissions inventories are imported and updated on a regular basis using a generalized, uniform data model and carefully standardized metadata. Names, codes, units, and quality flags from the source datasets are carefully mapped to a unified standard, and native formats and organizations are transformed into a common, normalized database schema. This design enables users to explore, merge, and analyze datasets of widely-varying origin in a consistent, unified manner with a common set of tools and web services. This degree of interoperability allows decision-makers to analyze diverse datasets side-by-side and focus on high-level planning strategies without having to contend with the details of data management and manipulation.
Providing Data Access to users/externals:
The VIEWS website provides easy online access to the integrated database through a wide variety of tools and applications. Users are encouraged to register with the website, but no account or login is required to use any of the VIEWS resources. The entire contents of the integrated database are freely and openly available for ad hoc querying, display, and download. In addition, users are encouraged to submit custom requests for data and to provide additional relevant datasets for inclusion in the database.
The VIEWS data import system accepts data in a variety of formats from various providers and can also automatically download and extract data from known online sources. The import system extracts the data and metadata from its source, examines it for basic integrity issues like duplicate records and mismatched data types, performs various data conversions, transforms the data into an integrated schema (described above), calculates checksums for numeric fields, and validates the resultant data against its source before transferring it to the production data warehouse. Many other processing operations – such as filtering, aggregation, averaging, formatting, renaming, recoding, and outputting – can be done on-the-fly by using the various data access and analysis tools on the website.
A wide variety of general-purpose visualization and analysis tools are available on the VIEWS website for exploring the data and producing graphs, charts, maps, and formatted data files. Specialized tools allow users to browse and analyze time series, visibility trends, spatial patterns, temporal aerosol profiles, chemical species composition, light extinction budgets, back trajectories, estimated natural conditions, visibility projections, glide slope plots, emissions inventories, weighted emissions potentials, and model performance evaluations. All tools have been designed with common user interface components and are being re-factored with a Service Oriented Architecture approach to allow them to function as web services.
Decision Support (e.g. some integration into user business process):
VIEWS/TSS users are typically asking questions of “What pollutants are impacting a given area?” and “Where are these pollutants coming from?” The answers depend upon accurate assessments of aerosol loading and source attribution, and FLMs and states are occupied on an ongoing basis with these goals. States are further mandated to answer the question of “What can be done to reduce these impacts?”, because the Regional Haze Rule requires states and tribes to develop implementation plans for reducing emissions and demonstrating reasonable progress towards doing so, and these plans must provide for an improvement during the 20% worst visibility days while also ensuring no degradation during the 20% best visibility days. To accomplish this, users must identify the pollutants, quantify their amounts, and determine the sources of anthropogenic emissions that contribute to this pollution on both the “best” and the “worst” visibility days in a given area. They must then determine available control measures for each source and evaluate these measures on the basis of costs, time, energy and environmental impacts, and the remaining life of the source. Planners then employ these analyses to make decisions about what controls to implement, to estimate projected improvements, and to track their progress in reaching these goals. The resulting decisions have obvious ecological impacts, but can also have important political and economic impacts in the sense that deciding which sources to control is a politically-significant issue and the process of controlling emissions and tracking progress costs money and takes time. VIEWS and the TSS have been designed with these decision needs in mind from the outset, and input and feedback from primary decision-makers is solicited on a routine basis through workshops and training sessions. Such input is carefully factored back into the design process to improve existing tools and design new ones in an ongoing process of progressive evolution.
The acquisition, import, verification, transformation, integration, presentation, analysis, and dissemination of data are all handled by the various subsystems of VIEWS. In addition, the regular solicitation and incorporation of user feedback ensures that this end-to-end integration is further supplemented by the input of the decision makers who comprise the VIEWS user base.
Other DS Values:
Data Access and/or Output Interoperability:
Raw data and metadata can be obtained from VIEWS in a variety of output mediums: Delimited or fixed width ASCII text files, Microsoft Excel workbooks, static HTML tables, and/or interactive HTML "data grids". In addition, the user can select the specific fields that are included in the output results and can specify a variety of output options, such as "wide" or "skinny" format, date format, missing data placholder, quoted text, etc. The VIEWS team is also working to provide the option to output data in XML and as XML SOAP responses.
Reusable Tools and Methods:
[[DataSystemArchToolsMethods::Several VIEWS tools have been developed as web services, and many existing tools are currently being re-factored to function as web services as well. In addition, several VIEWS web pages have been designed to allow developers of other web sites to "embed" and "brand" them on their own web sites with a minimum of effort. For an example of this, please visit the VIEWS Site Browser Development page.]]
Security Barriers and Solutions:
All data, tools, and other resources on VIEWS are freely accessible. No account is required to use VIEWS, but users are encouraged to register. Direct, read-only access to the integrated VIEWS database is also available via direct MS SQL Server connection, OLEDB, or ODBC.
User Feedback Approach:
[[DataSystemArchUserFeedbck::User feedback is facilitated through a variety of methods: the online Feedback form, direct email to the VIEWS team, or via the contact information published on VIEWS. The VIEWS team handles many feedback responses throughout a given week, and are typically able to respond within a day or two of receving a request or suggestion. User feedback is also actively solicited via periodic workshops and training sessions, or via conference calls with the steering committee and end users.]]