Difference between revisions of "Air Quality Data Network"

From Earth Science Information Partners (ESIP)
Line 5: Line 5:
  
 
Air quality data from mandated and scientific observations and numerical model results have long been gathered, analyzed and distributed through various channels. Data providers and their "customers" have established working procedures allowing them to exchange the data they need to "get their job done". However, as could be demonstrated in the case of individual air pollution events, it is rather difficult to obtain additional data from other sources when needed. Historically, the existing networks have developed regionally and there are as many different procedures to provide and exchange data as there are data networks. With interoperability as a guiding principle, the GEO AQ CoP tries to enable connections across regions and institutional borders. When fully implemented, a truly interoperable data network will make it much easier for the users of AQ data to find relevant data sets, to obtain the portion of these data they are interested in, and to analyze different data sets in a coherent fashion.
 
Air quality data from mandated and scientific observations and numerical model results have long been gathered, analyzed and distributed through various channels. Data providers and their "customers" have established working procedures allowing them to exchange the data they need to "get their job done". However, as could be demonstrated in the case of individual air pollution events, it is rather difficult to obtain additional data from other sources when needed. Historically, the existing networks have developed regionally and there are as many different procedures to provide and exchange data as there are data networks. With interoperability as a guiding principle, the GEO AQ CoP tries to enable connections across regions and institutional borders. When fully implemented, a truly interoperable data network will make it much easier for the users of AQ data to find relevant data sets, to obtain the portion of these data they are interested in, and to analyze different data sets in a coherent fashion.
 +
 +
== What is needed to achieve the goal of interoperability? ==
 +
 +
The most fundamental change towards interoperable data exchange is the replacement of 1:1 provider-user relationships with user-driven machine-to-machine connections. Just as when visiting foreign countries, this requires some agreement about the "language" in which information is exchanged. In the data network, these are the "protocols" (e.g. WCS, WFS, WMS, SOS, etc.) and the data formats (e.g. netcdf, csv, etc.). A client application and a data server must support the same protocol so that it is possible to obtain information about the data that are stored on the server and to formulate specific requests for data extraction. Once the data arrive, the client must be able to handle the data format in which the content is packaged.
 +
 +
Beyond the basic data access described above, there are other important technical and non-technical elements without which interoperability cannot be achieved. In order to find relevant data sets (the first step in each analysis), data catalogues must be established and agreement must be sought with respect to the metadata facets which describe the actual data sets. Related to this is the definition of a common vocabulary (for example species names, platform or instrument names, etc.). Furthermore, individual data providers may have specific requirements concerning the traceability of data, descriptions of ownership, tracing of data users or data access restrictions. There are different ways to define a "network topology" consisting of client and catalogue applications, data servers, mediators, web processing tools, etc.. Some agreement must be reached how this topology may look like in the AQ world in order to achieve the goal of integrating data and to avoid the creation of a new splinter network.
  
 
In August 2011, a meeting was organized upon invitation by the GEO AQ CoP in order to take stock of the existing AQ data network and its interoperability status (see [[Air_Quality_Data_Network_Solta_2011 || Solta workshop 2011]]) and to discuss the  challenges and potential solutions on the technical and networking level. From the set-up of this workshop, a number of specific topics were identified. The links below lead to summaries of the workshop discussions and serve as a basis for further elaboration which shall proceed in this Wiki.  
 
In August 2011, a meeting was organized upon invitation by the GEO AQ CoP in order to take stock of the existing AQ data network and its interoperability status (see [[Air_Quality_Data_Network_Solta_2011 || Solta workshop 2011]]) and to discuss the  challenges and potential solutions on the technical and networking level. From the set-up of this workshop, a number of specific topics were identified. The links below lead to summaries of the workshop discussions and serve as a basis for further elaboration which shall proceed in this Wiki.  
  
 
----
 
----
 +
== Interoperable AQ network status ==
 +
 
{| width="100%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;"  
 
{| width="100%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;"  
 
|- valign="top" bgcolor="pink"
 
|- valign="top" bgcolor="pink"

Revision as of 01:54, September 4, 2011

< Back to AQ CoP.png | Workshops | Air Quality Data Network


What is the AQ data network?

Air quality data from mandated and scientific observations and numerical model results have long been gathered, analyzed and distributed through various channels. Data providers and their "customers" have established working procedures allowing them to exchange the data they need to "get their job done". However, as could be demonstrated in the case of individual air pollution events, it is rather difficult to obtain additional data from other sources when needed. Historically, the existing networks have developed regionally and there are as many different procedures to provide and exchange data as there are data networks. With interoperability as a guiding principle, the GEO AQ CoP tries to enable connections across regions and institutional borders. When fully implemented, a truly interoperable data network will make it much easier for the users of AQ data to find relevant data sets, to obtain the portion of these data they are interested in, and to analyze different data sets in a coherent fashion.

What is needed to achieve the goal of interoperability?

The most fundamental change towards interoperable data exchange is the replacement of 1:1 provider-user relationships with user-driven machine-to-machine connections. Just as when visiting foreign countries, this requires some agreement about the "language" in which information is exchanged. In the data network, these are the "protocols" (e.g. WCS, WFS, WMS, SOS, etc.) and the data formats (e.g. netcdf, csv, etc.). A client application and a data server must support the same protocol so that it is possible to obtain information about the data that are stored on the server and to formulate specific requests for data extraction. Once the data arrive, the client must be able to handle the data format in which the content is packaged.

Beyond the basic data access described above, there are other important technical and non-technical elements without which interoperability cannot be achieved. In order to find relevant data sets (the first step in each analysis), data catalogues must be established and agreement must be sought with respect to the metadata facets which describe the actual data sets. Related to this is the definition of a common vocabulary (for example species names, platform or instrument names, etc.). Furthermore, individual data providers may have specific requirements concerning the traceability of data, descriptions of ownership, tracing of data users or data access restrictions. There are different ways to define a "network topology" consisting of client and catalogue applications, data servers, mediators, web processing tools, etc.. Some agreement must be reached how this topology may look like in the AQ world in order to achieve the goal of integrating data and to avoid the creation of a new splinter network.

In August 2011, a meeting was organized upon invitation by the GEO AQ CoP in order to take stock of the existing AQ data network and its interoperability status (see | Solta workshop 2011) and to discuss the challenges and potential solutions on the technical and networking level. From the set-up of this workshop, a number of specific topics were identified. The links below lead to summaries of the workshop discussions and serve as a basis for further elaboration which shall proceed in this Wiki.


Interoperable AQ network status

WCS Distributors - Current

CIERALogo.png-CIRALogo.jpeg-Datafed.png-DLRIcon.jpeg-GiovanniIcon.png-FZJuelichIcon.png-NILUIcon.jpeg

WCS Distributors - Interested

AirNow.jpg- EEA.jpg - Aerocom.png -RSIG.png -AQDataPortal.png

Catalogs

AQ Comm Cat.png<-->GI-cat.png<-->Clearinghouse.png

Clients

FZJuelichIcon.pngDatafed.png

WCS Comm Protocol.png
200px-WCS netCDF Wrapper.png

Standards and Conventions

WCS, netCDF, CF, ISO

WCS Server Software

Grid data, Example | Point data, Example

WCS Tools and Methods
Network Background | Data Hubs

CORE uFIND.png

Browse Catalog and Data

FanIn-FanOut.png
InteropStack.png
GEOAQCoP.png

Network Users

Applications: Decision Support | Science Teams | Facilitators


GEO AQ Community of Practice

Purpose: Connecting | Enabling
Contributors:DataPortals | ScienceTeams | DecisionsSystems | Facilitators
Beneficiaries:DataPortals | ScienceTeams | DecisionsSystems | Facilitators




Browse Catalog and Data
ADN Scraps