Group A2 - Enabling Data Access

Attendees

Ben Domenico
Beth Hassett-Sipple
Bill Lamason
Brad Johns
Bryan Hubbell
Carol Meyer
Chet Wayland
David McCabe
Doug Solomon
Ed Lillis
Frank Lindsay
Gary Lear
Gary Walter
Jeff West
Joe Tikvart (Notes2)
Nancy Ritchey
Nick Mangus (Notes1)
Phil Dickerson
Rob Willis (Facilitator)
Sharon Leduc
Shawn McClure
Stefan Falke
Susan Lundquist
Tim Dye

Data System Connectivity Matrix

The connectivity matrix indicates data system connections ranging from downloading data to standards-based interoperability.

The yellow symbols <-- and --> indicate one-way connectivity, green <--> indicates bi-directional connectivity.

This matrix is incomplete and in flux as the connections among systems are identified and logged. Please add connections or make corrections.

To edit the click on the link to 'Data System Connectivity Matrix' above the matrix and then login as aqcluster PW: aqcluster
NOTE: To login, select the 'sign up now' button in the top right. Before editing, make sure that you see a 'Save' button in the toolbar.

type in the arrow in the appropriate cell
color the cell either yellow or green
describe the connection in the appropriate on the Data System Connectivity Matrix

Please do not copy and paste cells. Each cell contains a hyperlink to its corresponding section of the wiki page.

Data System Profiles

	Data Access	Primary Data Storage]	Data Access Interoperability
3D-AQS	Not Given	Not Given	Not Given
AIRNow	AIRNow itself provides AQI data to the public. Raw, numerical data is available to stakeholders via AIRNowTech. AIRNow Gateway is a new addition that provides data streams to a variety of customers, utilizing a web service infrastructure.	AIRNow is the primary store for national real-time air quality data, however the raw data exists at the State, local, or tribal agency level. It is also the store for the official air quality forecast that is issued by state/local agencies. AQS is the official store for this same data.	Data is available in a variety of common formats – KML, ASCII text, and web services.
CASTNET	All data is provided on the CASTNET website after it is QA’ed. Ozone data is provided to the public through the CASTNET website and AirNow	Not Given	Data is available within the EPA firewall via SQL*Net and ODBC drivers. Data are available via the Web as prepackaged data sets and a data query wizard.
CMAQ	Not Given	Not Given	Not Given
DataFed		DataFed is a mediator of data flow between providers and users. It does not primary/official data.
EMF	Not a current goal of the project. Internal EPA only.	National Computing Center (NCC) (HPC side). Not available outside of EPA firewall	Have data interchange with EIS using formats specified (initial design work in progress).
EPA AIRQuest Data Warehouse	Not Given	None	Not Given
EPA AQS	AQS provides standard and ad hoc report access to users. By policy, only state, local, tribal and federal agency employees and contractors are allowed an AQS account. External access is handled by the AQS Data Mart. The AQS Data Mart is the dissemination piece of AQS. It contains all of the data reported to AQS from 1980 to the present (even AQS does not keep that much on-line). It is updated nightly with the data added to AQS during the business day. Access requires registration but is open to anyone and there is a guest account. Access is via 5 SOAP web services or a “Direct Interface” query builder (which is just a GUI on top of the SOAP web services).	Station based air quality measurements from the national ambient air quality monitoring network(s).	The AQS database can be accessed by end users through the AQS application or Oracle Discoverer. The AQS Data Mart can be accessed via SOAP web services, a query interface on the EPA TTN, or via an SQL connection for EPA users and applications.
ESIP	Promote the integration and flow of Earth science data from collection and analysis to end-use.	No. Although many of its members provide that service.	ESIP promotes the use of standards for data interoperability. The ESIP semantic web cluster is exploring new technologies for enhancing data access and interoperability.
GIOVANNI	Giovanni provides access to the original and their subsets in various formats: HDF, netCDF, ASCII, kml/kmz (for Google Earth)	OMI, MLS, HIRDLS, AIRS, TRMM, TOMS	SOA, WMS, WCS, OPeNDAP, custom arrangements
GeoWeb	All of the files indexed by the Geoweb Service provide metadata so that the user can understand the content, and are copied from their source to the user’s workstation when a visualization is requested. The visualization is performed on the user’s workstation within the World Wind Geo tool, within Google Earth, etc. Some of the files indexed by the Geoweb Service are served by OPeNDAP servers (Open-source Project for a Network Data Access Protocol) operated by NASA, NOAA, etc. Web browsers can work with OPeNDAP servers to identify a subset of the data, based on the variables in the file; and then download just that subset of data to the user’s workstation.	Not Given	Not Given
HEI	Yes, using csv files	Not Given	Not Given
NARSTO	This is the primary purpose of the NARSTO Permanent Data Archive	The Langley DAAC, http://eosweb.larc.nasa.gov/PRODOCS/narsto/table_narsto.html	Data are located via the Web and are downloaded by FTP. Most users open the data files in a spreadsheet program.
NASA Atmospheric Science Data Center	Several custom order tools and subset tools.	Store next near-online tape silo, online data poo	Metadata managed in PostGres POSTGIS open source database
NEISGEI	Standards based access to emissions and related data. Uses DataFed for data registration and access.	NEISGEI provides standardized interfaces to emissions data stores.	OGC standards (WMS, WCS, WFS)
RSIG	Yes. RSIG integrates incoming proprietary dataset formats into standard formats the user can save to their PC. A user can save the data or visualization--or both--to their local computer in such standard formats as portable binary, ASCII, NetCDF, binary XDR, and MPEG. The user can then export the selected datasets from RSIG into other applications--such as GIS tools--for further analysis. Also, developers can bypass the Java client totally by using OGC-WCS/WMS-compliant server scripts to programmatically capture RSIG-subsetted data for use in their applications. Includes numerous CMAQ conveniences: enables comparison (difference, absolute difference, percent difference, ratio, after regridding) of Ozone, AOD, PM25 from MODIS, FAQSD, GASP, MOZAIC, Airnow and AQS to CMAQ; displays intersection of CALIPSO and MOZAIC 4D path in CMAQ grid (that is, extract CMAQ data for each lon-lat-elevation-time of the other data).	No. RSIG generally accesses data from other sources (and subsets it at the source).	RSIG uses Open Geospatial Consortium (OGC) Web Mapping Service (WMS) and Web Coverage Service (WCS) standards for specifying data subsets and a supports a variety of standard output formats (ASCII, NetCDF, KMZ, etc.) requested by users to enable data to be saved and input into other applications.
Unidata IDD Data System	The main goal of the Unidata data systems is to make the data available to users.	Not Given	Unidata has been a leader in the integration of weather data into the world of OGC and ISO standards via projects such as THREDDS (THematic Real-time Environmental Distributed Data Services) and GALEON (Gateway to Air, Land, Environment, Ocean NetCDF). Unidata also hosted an Interoperability Day at last September's OGC Technical Committee meetings.
VIEWS	The VIEWS website provides easy online access to the integrated database through a wide variety of tools and applications. Users are encouraged to register with the website, but no account or login is required to use any of the VIEWS resources. The entire contents of the integrated database are freely and openly available for ad hoc querying, display, and download. In addition, users are encouraged to submit custom requests for data and to provide additional relevant datasets for inclusion in the database.		Raw data and metadata can be obtained from VIEWS in a variety of output mediums: Delimited or fixed width ASCII text files, Microsoft Excel workbooks, static HTML tables, and/or interactive HTML "data grids". In addition, the user can select the specific fields that are included in the output results and can specify a variety of output options, such as "wide" or "skinny" format, date format, missing data placholder, quoted text, etc. The VIEWS team is also working to provide the option to output data in XML and as XML SOAP responses.