|
|
| (85 intermediate revisions by 9 users not shown) |
| Line 1: |
Line 1: |
| | <noinclude>{{AQ CoP Solta2011 Backlinks}} | | <noinclude>{{AQ CoP Solta2011 Backlinks}} |
| | + | ====<center><big>[[ADN About| About]] | [[ADN Interoperability| Interoperability]] | [[ADN Standards|Standards]] | [[ADN Technical Issues| Servers]] | [[ ADN Catalogs|Catalogs]] | [[ ADN Clients| Clients]] | [[ADN Network| Network]] |[[ADN Participants|Participants]]</big> </center>==== |
| | + | <br> |
| | + | ---- |
| | + | <big><big><big>[[ADN Goals and Process|Goal]]:</big></big></big> |
| | + | <font color= "green"><big>A <big><b>Data Network</b></big> to <big><b>Find, Access,</b></big> understand and merge air quality observations, emissions and models</big></font><br> |
| | | | |
| | + | <big><big><big>[[ADN Goals and Process|How]]:</big></big></big> |
| | + | <font color= "green"><big>By enabling existing <big><b>Data Portals, Decision</b></big> and <big><b>Science</b></big> projects to create and to make use the Network</big></font><br> |
| | ---- | | ---- |
| − | == What is the AQ data network? ==
| + | <br> |
| | | | |
| − | Air quality data from mandated and scientific observations and numerical model results have long been gathered, analyzed and distributed through various channels. Data providers and their "customers" have established working procedures allowing them to exchange the data they need to "get their job done". However, as could be demonstrated in the case of individual air pollution events, it is rather difficult to obtain additional data from other sources when needed. Historically, the existing networks have developed regionally and there are as many different procedures to provide and exchange data as there are data networks. With interoperability as a guiding principle, the [[GEO_AQ_CoP | GEO AQ CoP]] tries to enable connections across regions and institutional borders. When fully implemented, a truly interoperable data network will make it much easier for the users of AQ data to find relevant data sets, to obtain the portion of these data they are interested in, and to analyze different data sets in a coherent fashion.
| + | {|Align="center" width="30%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;" |
| − | | + | |- valign="top" |
| − | == What is needed to achieve the goal of interoperability? == | + | |bgcolor="#CCCCCC" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="100%"| |
| − | | + | <center><b><big><big>'''Network Status'''</big></big></b></center> |
| − | The most fundamental change towards interoperable data exchange is the replacement of 1:1 provider-user relationships with user-driven machine-to-machine connections. Just as when visiting foreign countries, this requires some agreement about the "language" in which information is exchanged. In the data network, these are the "protocols" (e.g. WCS, WFS, WMS, SOS, etc.) and the data formats (e.g. netcdf, csv, etc.). A client application and a data server must support the same protocol so that it is possible to obtain information about the data that are stored on the server and to formulate specific requests for data extraction. Once the data arrive, the client must be able to handle the data format in which the content is packaged.
| + | |} |
| − | | + | {|Align="center" width="30%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;" |
| − | Beyond the basic data access described above, there are other important technical and non-technical elements without which interoperability cannot be achieved. In order to find relevant data sets (the first step in each analysis), data catalogues must be established and agreement must be sought with respect to the metadata facets which describe the actual data sets. Related to this is the definition of a common vocabulary (for example species names, platform or instrument names, etc.). Furthermore, individual data providers may have specific requirements concerning the traceability of data, descriptions of ownership, tracing of data users or data access restrictions. There are different ways to define a "network topology" consisting of client and catalogue applications, data servers, mediators, web processing tools, etc.. Some agreement must be reached how this topology may look like in the AQ world in order to achieve the goal of integrating data and to avoid the creation of a new splinter network.
| + | |- valign="top" bgcolor="pink" |
| − | | + | |bgcolor="#D5F9DF" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="100%"| |
| − | In August 2011, a meeting was organized upon invitation by the [[GEO_AQ_CoP | GEO AQ CoP]] in order to take stock of the existing AQ data network and its interoperability status (see [[Air_Quality_Data_Network_Solta_2011 || Solta workshop 2011]]) and to discuss the challenges and potential solutions on the technical and networking level. From the set-up of this workshop, a number of specific topics were identified. The links below lead to summaries of the workshop discussions and serve as a basis for further elaboration which shall proceed in this Wiki.
| + | [[image:ADN Distributors.png|300px|link=http://wiki.esipfed.org/index.php/WCS_Distributors]]<br> |
| | + | [http://webapps.datafed.net/CORE.uFIND Access catalogue] |
| | + | |} |
| | | | |
| | + | <br> |
| | ---- | | ---- |
| − | == Interoperable AQ network status == | + | <br> |
| − | | + | {| width="100%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;" |
| − | '''Network Topology''' (ToBeDone) | + | |- valign="top" bgcolor="#FFFFFF" |
| − | | + | |bgcolor="#CCCCCC" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="100%"| |
| − | | + | <center><b><big><big>'''Network Components'''</big></big></b></center> |
| | + | |} |
| | {| width="100%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;" | | {| width="100%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;" |
| − | |- valign="top" bgcolor="pink" | + | |- valign="top" bgcolor="#FFFFFF" |
| − | |bgcolor="#D5F9DF" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="30%"| | + | |bgcolor="FFFFFF" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="30%"| |
| − | <center>'''[[WCS Distributors|WCS Distributors ]]- Current'''</center> | + | <center>'''[[ADN_Distributor_List | WCS Distributors]] - Current'''</center> |
| − | [[File:CIERALogo.png|50px|link=http://webapps.datafed.net/Core.uFIND?distributor=ciera]]-[[File:CIRALogo.jpeg|40px|link=http://webapps.datafed.net/Core.uFIND?distributor=cira]]-[[File:Datafed.png|45px|link=http://webapps.datafed.net/Core.uFIND?distributor=datafed]]-[[File:DLRIcon.jpeg|30px|link=http://webapps.datafed.net/Core.uFIND?distributor=dlr]]-[[File:GiovanniIcon.png|30px|link=http://webapps.datafed.net/Core.uFIND?distributor=giovanni]]-[[File:FZJuelichIcon.png|50px|link=http://webapps.datafed.net/Core.uFIND?distributor=fzjuelich%2cfz%2bjuelich]]-[[File:NILUIcon.jpeg|20px|link=http://webapps.datafed.net/Core.uFIND?distributor=nilu]]<br> | + | [[File:CIERALogo.png|50px|link=http://webapps.datafed.net/Core.uFIND?distributor=ciera]]-[[File:CIRALogo.jpeg|40px|link=http://webapps.datafed.net/Core.uFIND?distributor=cira]]-[[File:Datafed.png|45px|link=http://webapps.datafed.net/Core.uFIND?distributor=datafed]]-[[File:DLRIcon.jpeg|30px|link=http://webapps.datafed.net/Core.uFIND?distributor=dlr]]-[[File:NASAIcon.png|25px|link=http://webapps.datafed.net/Core.uFIND?distributor=nasa_acdisc]]-[[File:FZJuelichIcon.png|50px|link=http://webapps.datafed.net/Core.uFIND?distributor=fzjuelich%2cfz%2bjuelich]]-[[File:NILUIcon.jpeg|30px|http://webapps.datafed.net/Core.uFIND?distributor=NILU]]<br> |
| | <center>'''WCS Distributors - Interested'''</center> | | <center>'''WCS Distributors - Interested'''</center> |
| − | [[Image:AirNow.jpg|50px]]- [[Image:EEA.jpg|30px]] - [[image:Aerocom.png|30px]] -[[File:RSIG.png|30px]] -[[image:AQDataPortal.png|50px]]<br> | + | [[Image:AirNow.jpg|50px]]- [[Image:EEA.jpg|30px]] - [[image:Aerocom.png|30px]]-[[image:AQDataPortal.png|50px]]<br> |
| − | |bgcolor="#D5F9DF" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="30%"| | + | |bgcolor="FFFFFF" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="30%"| |
| − | ====<center> Catalogs</center>==== | + | ====<center> [[ADN_Catalogs|Catalogs]]</center>==== |
| − | <center>[[Image:AQ Comm Cat.png|40px]]<-->[[Image:GI-cat.png|40px]]<-->[[image:Clearinghouse.png|100px]] </center><br> | + | <center>[[Image:AQ Comm Cat.png|40px|link=http://webapps.datafed.net/CORE.uFIND]]<-->[[Image:GI-cat.png|40px|link=http://essi-lab.eu/cgi-bin/twiki/view/GIcat]]<-->[[image:Clearinghouse.png|100px|link=http://www.earthobservations.org/gci_gci.shtml]] </center><br> |
| − | <center>'''[http://webapps.datafed.net/CORE.uFIND Browse Catalog and Data]'''</center>
| + | |bgcolor="FFFFFF" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="30%"| |
| − | |bgcolor="#D5F9DF" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="30%"| | + | ==== <center>[[ADN_Clients|Clients]]</center> ==== |
| − | ==== <center>Clients</center> ==== | |
| − | | |
| | [[Image:FZJuelichIcon.png|60px]][[Image:Datafed.png|60px]] | | [[Image:FZJuelichIcon.png|60px]][[Image:Datafed.png|60px]] |
| | |} | | |} |
| − |
| |
| − |
| |
| − | == Technical Issues ==
| |
| − |
| |
| − | Implementing a fully interoperable data network requires agreements about a number of issues on the technical level. This includes for example the selection of [[http://www.opengeospatial.org | OGC]] standard(s) for geographical information exchange, the packaging of the data content (data format), and the metadata description of individual data sets and data catalogues. Eventually, all of these issues must be realized in computer code, and such code needs to be maintained and adapted to changing requirements.
| |
| − |
| |
| − | The GEO AQ CoP is providing an open source community WCS server software which has demonstrated its functionality over the past years in the [[http://datafed.net/ DataFed]] and [[http://ogc-interface.icg.kfa-juelich.de:50080 FZ Jülich]] environments. At the [[Air_Quality_Data_Network_Solta_2011 | Solta 2011]] this code was adopted as reference implementation for the AQ data network, and several participants pledged to install the software at their data nodes or to contribute to the further development of the server functionality.
| |
| − |
| |
| − | The sections below briefly summarize the main technical issues related to functioning of the AQ data network and the community WCS server tool. The section headings link to more detailed discussions about each of these issues.
| |
| − |
| |
| − | === [[Standards_and_Conventions_for_WCS_Server | Standards and Conventions to enable the AQ Data Network]] ===
| |
| − |
| |
| − | * [[http://www.opengeospatial.org | OGC]] '''[[Standards_and_Conventions_for_WCS_Server#WCS_Standard|Web Coverage Service (WCS)]]'''
| |
| − | * [[http://www.unidata.ucar.edu/software/netcdf/ | Network Common Data Format (netCDF)]] '''[[Standards_and_Conventions_for_WCS_Server#netCDF_Data_Model|data model]]'''
| |
| − | * [[http://cf-pcmdi.llnl.gov/ | Climate and Forecasting (CF)]] '''[[Standards_and_Conventions_for_WCS_Server#CF_Conventions|metadata conventions]]'''
| |
| − | * [[http://www.iso.org/iso/catalogue_detail.htm?csnumber=26020 | ISO 19115]] '''[[Standards_and_Conventions_for_WCS_Server#ISO_19115l|metadata requirements]]'''
| |
| − |
| |
| − | === [[WCS Server Software|WCS Server Software]] ===
| |
| − |
| |
| − | * '''[[WCS_Server_Software#WCS_Server_for_CF-netCDF_Grid_Data_Type| Grid data]]''' ([[Example_CF-NetCDF_for_Satellite|Example]])
| |
| − | * '''[[WCS_Server_Software#WCS_Server_for_Station-Point_Data_Type|Point data]]''' ([[Example_SQL_for_Stations|Example]])
| |
| − | * '''[[WCS_Tools_and_Methods|WCS Tools and Methods]]'''
| |
| − |
| |
| − |
| |
| − | ==[[GEO_AQ_CoP_Communities|Network Users]]==
| |
| − |
| |
| − | * [[GEO AQ CoP Decision Support|Decision Support]]
| |
| − | * [[GEO AQ CoP Science Teams|Science Teams]]
| |
| − | * [[GEO AQ CoP Facilitators|Facilitators]]
| |
| − | ----
| |
| − |
| |
| | __NOEDITSECTION__ | | __NOEDITSECTION__ |
| | __NOTOC__ | | __NOTOC__ |
| | | | |
| − | | + | [[Category:AQ Data Network]] [[category:AQWG]] |
| − | | |
| − | [[ADN Scraps]]
| |
| − | [[HTAP Data Network: Background, Status|Network Background]] | [[HTAP_Data_Network:_Background,_Status#HTAP_Network_Pilot_Nodes| Data Hubs]]<br>
| |
| − | [[File:FanIn-FanOut.png|thumb|40px]][[Image:InteropStack.png|thumb|40px]][[File:GEOAQCoP.png|thumb|40px]]
| |
| − | | |
| − | [[Category:AQ Data Network]] [[category:AQ_CoP]] | |
| Workshops | Air Quality Data Network
