White Paper on Systems Assessment
Back to <Data Summit Workspace
Back to <Community Air Quality Data System Workspace
- Mintz:White paper on Systems Assessment Word Doc - Updated 2008-05-08
Focused Assessment of Air Quality Community Information Systems DRAFT White Paper
Background
EPA hosted an Air Quality Data Summit in February 2008 to bring together partners in the air quality data systems community. The purpose of the summit was three-fold:
- to convene organizations and individuals with key roles in retrieving, storing, disseminating, and analyzing air quality data in order to learn about and explore efficient means of leveraging the numerous individual efforts underway;
- to assist EPA/OAQPS in honing its role in the larger air quality data community;
- to begin to establish a community-wide strategy responsive to user defined needs.
A key outcome of the summit was a community air quality data systems strategy which lists several activities for carrying forward the principles from the summit. One of the activities in this strategy is an assessment of systems. This paper describes the planned assessment: its purpose and scope, how it will be conducted, and how it will feed into the other activities listed in the strategy.
Purpose and Scope of Assessment
Before developing a comprehensive, strategic plan for data systems, we first need to assess existing major systems. If we could go back 15 years and start from scratch, knowing what we know now, we likely would be able to grow a very efficient network of systems. But the reality today is that the Air Quality Community has a conglomerate of data systems and applications with relatively little forethought of standardization, compatibility, or extensibility. Since we can’t go back, the question is: “How do we start from where we are as a community and plan for the future?”
The purpose of this assessment is to systematically determine and document the status of several information systems in the Air Quality Community, focusing on EPA systems and a few external systems which are closely linked to the EPA systems, at least potentially compete for federal resources with the EPA systems, and/or handle very similar (if not actually the same) types of data as the EPA systems. The assessment will consider the value of these systems to the community as a whole, particularly to state and local governments, industry and the research community, and not just EPA. We need to understand the roles, technical capabilities, costs, compatibility, and sustainability of each system as well as how they relate to one another now in order to plan appropriately for the future. The assessment will not reconsider the role or technical approach of AQS and AirNow as primary data intake and archive systems, but will consider their connections to other data systems and their data analysis features. The assessment will fully examine the features of the identified systems that deal with ambient air concentrations measured at ground stations and with emissions estimates. Because so few multi-user systems now house air quality modeling outputs and because wider data user needs for this type of data are not yet clear, the assessment will not pay much if any attention to features specific to air quality model outputs. The effort spent on satellite data will fall somewhere between that spent on the former and latter types of data.
The following systems are being considered initially:
- AQS & AQS Data Mart
- AirQuest
- EIS
- AIRNow & AIRNow Tech
- DataFed
- HEI Air Quality Database
- VIEWS (including any other IMPROVE-related data system not formally part of VIEWS)
- RSIG
Goals of Assessment
The primary goal of this assessment is to provide critical information to assist EPA/OAQPS in planning the future direction and missions of existing data systems, both those operated internally and those that are significantly funded or influenced by EPA. The secondary goal is to provide information for subsequent development of a strategic plan, a “roadmap for the future” for the larger community. This information will consist of identified strengths and shortcomings, redundancies, unmet needs, etc. relative to the strategic vision. The assessment will start with developing a list of attributes based on the data summit that address desired data formatting and interface standards, data visualization and analysis tools, and yet to be determined measures of success, such as development cost to value ratios. Ideally, the assessment will make recommendations (including tangible, realistic milestones) that could help transition from our current state to a desired future state.
Conducting the Assessment
EPA/OAQPS will engage a contractor to conduct the assessment relying largely on document reviews and interviews aimed at answering the questions set forth below. Since the goal is to inform future planning, we will focus on questions regarding cost, function, data integrity, governance, and sustainability. In addition, the assessment should acknowledge from the summit the common principles of a desired future state (“more economically efficient”, “fewer redundancies”, etc.) Below is a list of questions to help focus the assessment on the aforementioned principles.
Cost/Value
1. What are the total annual costs to operate the system? How much is being invested annually in system enhancements, beyond status quo operation?
2. What is the consequence if the system became unavailable (permanently, temporarily)?
Sustainability
3. What factors favor or work against the sustainability of the system (and underlying data) over the next 5 to 10 years?
Function
4. What function(s) does it serve? Main audience/clients? Regulatory purpose?
Governance
5. Who owns (makes final decisions about) the system, the data, the application?
Data Integrity
6. How are the data quality assured, particularly the accuracy and completeness of data that is copied or transformed from another data system?
Accessibility
7. To whom is the data/system available? Are there firewall/security issues that prev
ent access for certain audiences?
Interoperability
8. What data standards are used, and how do those coordinate or conflict with standards in the other data systems particularly the systems that provide primary data intake and archiving?
9. How easily can the system be adapted to a GEO or Exchange Network interoperability framework?
10. Does this system copy and store data from another source?
Timeline and Final Product
EPA/OAQPS is requesting that the contractor complete the assessment and provide an interim report by July 15, 2008 and a final report by July 31, 2008. The report will include an integrated summary of the assessment, as well as recommendations (tangible, realistic milestones) for transitioning from our current state to a desired future state. This information will help with subsequent development of a strategic plan.
White Paper on Systems AssessmentCommunity Air Quality Data System Workspace Data Summit Workspace2008-05-08White Paper on Systems AssessmentMintz
