Difference between revisions of "Rational for WCS and WFS Combination for point data"

From Earth Science Information Partners (ESIP)
 
(9 intermediate revisions by one other user not shown)
Line 3: Line 3:
 
[http://sourceforge.net/p/aq-ogc-services/home/ Project on SourceForge]
 
[http://sourceforge.net/p/aq-ogc-services/home/ Project on SourceForge]
  
== Objective ==
+
== Objective and Current Experience ==
  
While WCS was designed for gridded data, we have found it to be a good delivery standard for station data also. With a minimal convention it's possible to use the combination of WCS and WFS for the purposes that they were designed for, and create an interoperable station data convention.  
+
While WCS was designed for gridded data, we have found it to be a good delivery standard for station data also. With a minimal metadata encoding conventions it's possible to use the combination of WCS and WFS for the purposes that they were designed for, and use existing standards for interoperable data delivery system.  
  
== Experience with Station Point Data ==
+
The Web Coverage Service was designed to deliver continuous or discrete coverages. The basic dimensions are geographic boundingbox and time. In general, each coverage point has little metadata about itself. The Web Feature Service was designed to deliver rich metadata about stationary objects. While the feature has a boundingbox, the lat and lon are not dimensions. They are attributes of features.
  
We have used the combination of WCS and WFS at Datafed for station data for six years. Here are a few observations:
+
By combining those two it is possible to deliver complete metadata of a station network.
  
* Typical station data looks like 3D, but is in reality 2D.
+
== Dimensions covering geographic area ==
  
** In cube data, you have typically at least 4 dimensions, lat, lon and time.
+
* Typical station network looks like 2D, but is in reality 1D.
** In point data, you have typically 2 dimensions, location and time.
 
  
The station has 2 attributes: lat and lon, but it's still one dimension. This is an important distinction which was not in the design requirements for the WCS standard. Fortunately, the combination of WCS and WFS works great.
+
** In cube data, you have typically at least 2 dimensions, lat and lon.
 +
** In point data, you have typically 1 dimension, location.
  
== Web Coverage and Feature Services ==
+
The location has 2 attributes: lat and lon, but it's still one dimension. This is an important distinction and WCS 1.1 supports such dimensions.
 
 
The Web Coverage Service was designed to deliver continuous coverages. The basic dimensions are geographic boundingbox and time. In general, each coverage point has little metadata about itself.
 
 
 
The Web Coverage Service was designed to deliver rich metadata about stationary objets. While the feature has a boundingbox, the lat and lon are not dimensions. They are attibutes of features.
 
  
 
== A Coverage can be Irregular ==
 
== A Coverage can be Irregular ==
Line 28: Line 24:
 
The WCS DescribeCoverage supports both regular and irregular coverages.
 
The WCS DescribeCoverage supports both regular and irregular coverages.
  
[Regular Data](http://data1.datafed.net:8080/NRL?service=WCS&version=1.1.2&request=DescribeCoverage&Identifiers=NAAPS_press)
+
[http://data1.datafed.net:8080/NRL?service=WCS&version=1.1.2&request=DescribeCoverage&Identifiers=NAAPS_press Regular Data]
 +
 
 +
Excerpt:
  
 
     <SpatialDomain>
 
     <SpatialDomain>
Line 46: Line 44:
 
The GridBaseCRS element manifests the projection: "WGS 84"; type "2dSimpleGrid" origin -179.5, -89.5 and increments 1.0 and 1.0 degrees. In this case, the grid has regular lat and lon dimensions.
 
The GridBaseCRS element manifests the projection: "WGS 84"; type "2dSimpleGrid" origin -179.5, -89.5 and increments 1.0 and 1.0 degrees. In this case, the grid has regular lat and lon dimensions.
  
[Irregular Data](http://data1.datafed.net:8080/AQS?service=WCS&version=1.1.2&request=DescribeCoverage&Identifiers=AQS_D)
+
[http://data1.datafed.net:8080/AQS?service=WCS&version=1.1.2&request=DescribeCoverage&Identifiers=AQS_D Irregular Data]
 +
 
 +
Excerpt:
  
 
     <SpatialDomain>
 
     <SpatialDomain>
Line 55: Line 55:
 
       </SpatialDomain>
 
       </SpatialDomain>
  
This is a description of irregular space dimensions. The data can still be an irregular grid, bot example fine grid along the equator and sparse on the poles. On in this case, the coverage is not a grid at all, but a station network.
+
This is a description of irregular space dimensions. The data can still be an irregular grid, for example fine grid along the equator and sparse on the poles. On point data, the coverage is not a grid at all, but a station network.
  
== More Dimension Metadata in DescribeCoverage ==
+
== Additional Dimensions in DescribeCoverage ==
  
The regular lat, lon, and time dimensions are described in the DescribeCoverage document
+
This is an elevation dimension. Irregular dimensions are expressed as axises, with some standard metadata like datatype and Units Of Measure. The WCS adds some CF-NetCDF metadata into the AxisMetadata node. This is still all according to the standard,
  
 
     <Axis identifier="elev">
 
     <Axis identifier="elev">
Line 79: Line 79:
 
     </Axis>
 
     </Axis>
  
This is an elevation dimension. Irregular dimensions are expressed as axises, with some standard metadata like datatype and Units Of Measure. The WCS adds some CF-NetCDF metadata into the AxisMetadata node. This is still all according to the standard,
 
  
== Missing Metadata in DescribeCoverage ==
+
== Metadata Limits in DescribeCoverage ==
  
 
While the extra dimensions in gridded data can be expressed with WCS 1.1, for station data the location table causes trouble. Here is a typical location table:
 
While the extra dimensions in gridded data can be expressed with WCS 1.1, for station data the location table causes trouble. Here is a typical location table:
Line 90: Line 89:
 
     ....
 
     ....
  
 +
There is no good way to express a table in the WCS, but WFS fits to the job perfectly. It's a collection of stationary located objects with varying amount of metadata.
 +
 +
The solution is to combine WCS and WFS. The WCS framework enumerates the location dimension keys, loc_codes, in the AvailableKeys node. Then the real rich location table is delivered via the [http://data1.datafed.net:8080/AQS?service=WFS&Version=1.0.0&Request=GetFeature&typename=AQS_D&filter=field:PB&outputFormat=text/csv WFS link].
  
 
     <Axis identifier="location">
 
     <Axis identifier="location">
 
         <ows11:Title>Location Table</ows11:Title>
 
         <ows11:Title>Location Table</ows11:Title>
         <AvailableKeys><Key>loc_code</Key></AvailableKeys>
+
         <AvailableKeys>
         <ows11:Metadata xlink:type="simple" xlink:href="http://data1.datafed.net:8080/EEA_AirBase?service=WFS&Version=1.0.0&Request=GetFeature&typename=AirBase_D&filter=field:TSP&outputFormat=text/csv" />
+
            <Key>010730025</Key>
 +
            <Key>010730028</Key>
 +
            ...
 +
        </AvailableKeys>
 +
         <ows11:Metadata xlink:type="simple" xlink:href="http://data1.datafed.net:8080/AQS?service=WFS&Version=1.0.0&Request=GetFeature&typename=AQS_D&filter=field:PB&outputFormat=text/csv" />
 
     </Axis>
 
     </Axis>
  
== Combining WCS and WFS ==
+
(In the actual implementation AvailableKeys does not enumerate loc_codes, because it's redundant. We're just showing it's possible.)
 +
 
 +
== Location Metadata ==
  
== Necessary Location Metadata ==
+
For a generic client, the necessary location metadata attributes are loc_code, lat and lon.
  
== Optional Location Metadata ==
+
It is useful to add more metadata, like loc_name and elev. All these names are just a convention, and need to be documented as a best practice.
  
 
== Custom Location Metadata ==
 
== Custom Location Metadata ==
 +
 +
Many networks display a large collection of different kind of metadata that is very provider-specific, and therefore impossible to standardize. For example the AQS network:
 +
 +
[http://webapps.datafed.net/datafed.aspx?wcs=http://data1.datafed.net:8080/AQS&coverage=AQS_D&field=pm10&loc_code=380171004 AQS_D pm10 at location 380171004]
 +
 +
If you press the small "m" button next to the parameter list, you'll see the [http://webapps.datafed.net/metadata.aspx?wcs=http://data1.datafed.net:8080/AQS&coverage=AQS_D&field=pm10&loc_code=380171004&view_id=metadata Rich Metadata View]
 +
 +
It's impossible to standardize such information, so we should just agree how to encode the URL that pops up the HTML page containing human readable information about the location.

Latest revision as of 23:07, August 23, 2011

Back to WCS Wrapper

Project on SourceForge

Objective and Current Experience

While WCS was designed for gridded data, we have found it to be a good delivery standard for station data also. With a minimal metadata encoding conventions it's possible to use the combination of WCS and WFS for the purposes that they were designed for, and use existing standards for interoperable data delivery system.

The Web Coverage Service was designed to deliver continuous or discrete coverages. The basic dimensions are geographic boundingbox and time. In general, each coverage point has little metadata about itself. The Web Feature Service was designed to deliver rich metadata about stationary objects. While the feature has a boundingbox, the lat and lon are not dimensions. They are attributes of features.

By combining those two it is possible to deliver complete metadata of a station network.

Dimensions covering geographic area

  • Typical station network looks like 2D, but is in reality 1D.
    • In cube data, you have typically at least 2 dimensions, lat and lon.
    • In point data, you have typically 1 dimension, location.

The location has 2 attributes: lat and lon, but it's still one dimension. This is an important distinction and WCS 1.1 supports such dimensions.

A Coverage can be Irregular

The WCS DescribeCoverage supports both regular and irregular coverages.

Regular Data

Excerpt:

   <SpatialDomain>
       <ows11:BoundingBox crs="urn:ogc:def:crs:OGC:2:84" dimensions="2">
           <ows11:LowerCorner>-179.5000 -89.5000</ows11:LowerCorner>
           <ows11:UpperCorner>+179.5000 +89.5000</ows11:UpperCorner>
       </ows11:BoundingBox>
       <GridCRS>
       <GridBaseCRS>urn:ogc:def:crs:OGC:2:84</GridBaseCRS>
           <GridType>urn:ogc:def:method:WCS:1.1:2dSimpleGrid</GridType>
           <GridOrigin>-179.5000 -89.5000</GridOrigin>
           <GridOffsets>+1.0000 +1.0000</GridOffsets>
           <GridCS>urn:ogc:def:cs:OGC:0.0:Grid2dSquareCS</GridCS>
       </GridCRS>
   </SpatialDomain>

The GridBaseCRS element manifests the projection: "WGS 84"; type "2dSimpleGrid" origin -179.5, -89.5 and increments 1.0 and 1.0 degrees. In this case, the grid has regular lat and lon dimensions.

Irregular Data

Excerpt:

   <SpatialDomain>
       <ows11:BoundingBox crs="urn:ogc:def:crs:OGC:2:84" dimensions="2">
           <ows11:LowerCorner>-126.0000 +24.0000</ows11:LowerCorner>
           <ows11:UpperCorner>-65.0000 +52.0000</ows11:UpperCorner>
       </ows11:BoundingBox>
     </SpatialDomain>

This is a description of irregular space dimensions. The data can still be an irregular grid, for example fine grid along the equator and sparse on the poles. On point data, the coverage is not a grid at all, but a station network.

Additional Dimensions in DescribeCoverage

This is an elevation dimension. Irregular dimensions are expressed as axises, with some standard metadata like datatype and Units Of Measure. The WCS adds some CF-NetCDF metadata into the AxisMetadata node. This is still all according to the standard,

   <Axis identifier="elev">
       <ows11:Title>air_pressure</ows11:Title>
       <AvailableKeys>
           <Key>1000.0</Key>
           <Key>975.0</Key>
           ...
           <Key>10.0</Key>
       </AvailableKeys>
       <ows11:DataType>double</ows11:DataType>
       <ows11:UOM>mbar</ows11:UOM>
       <ows11:Metadata>
       <cf:AxisMetadata>
           <cf:axis positive="down">Z</cf:axis>
           <cf:standard_name>air_pressure</cf:standard_name>
       </cf:AxisMetadata>
       </ows11:Metadata>
   </Axis>


Metadata Limits in DescribeCoverage

While the extra dimensions in gridded data can be expressed with WCS 1.1, for station data the location table causes trouble. Here is a typical location table:

   loc_code,  lat,   lon,    elev
   010730025, 33.55, -86.78, 54
   010730028, 33.53, -86.85, 51
   ....

There is no good way to express a table in the WCS, but WFS fits to the job perfectly. It's a collection of stationary located objects with varying amount of metadata.

The solution is to combine WCS and WFS. The WCS framework enumerates the location dimension keys, loc_codes, in the AvailableKeys node. Then the real rich location table is delivered via the WFS link.

   <Axis identifier="location">
       <ows11:Title>Location Table</ows11:Title>
       <AvailableKeys>
           <Key>010730025</Key>
           <Key>010730028</Key>
           ...
       </AvailableKeys>
       <ows11:Metadata xlink:type="simple" xlink:href="http://data1.datafed.net:8080/AQS?service=WFS&Version=1.0.0&Request=GetFeature&typename=AQS_D&filter=field:PB&outputFormat=text/csv" />
   </Axis>

(In the actual implementation AvailableKeys does not enumerate loc_codes, because it's redundant. We're just showing it's possible.)

Location Metadata

For a generic client, the necessary location metadata attributes are loc_code, lat and lon.

It is useful to add more metadata, like loc_name and elev. All these names are just a convention, and need to be documented as a best practice.

Custom Location Metadata

Many networks display a large collection of different kind of metadata that is very provider-specific, and therefore impossible to standardize. For example the AQS network:

AQS_D pm10 at location 380171004

If you press the small "m" button next to the parameter list, you'll see the Rich Metadata View

It's impossible to standardize such information, so we should just agree how to encode the URL that pops up the HTML page containing human readable information about the location.