Difference between revisions of "Rational for WCS and WFS Combination for point data"

From Earth Science Information Partners (ESIP)
 
(13 intermediate revisions by one other user not shown)
Line 3: Line 3:
 
[http://sourceforge.net/p/aq-ogc-services/home/ Project on SourceForge]
 
[http://sourceforge.net/p/aq-ogc-services/home/ Project on SourceForge]
  
== Experience with Station Point Data ==
+
== Objective and Current Experience ==
  
We have used the combination of WCS and WFS at Datafed for station data for six years. Here are a few observations:
+
While WCS was designed for gridded data, we have found it to be a good delivery standard for station data also. With a minimal metadata encoding conventions it's possible to use the combination of WCS and WFS for the purposes that they were designed for, and use existing standards for interoperable data delivery system.  
  
* Typical station data looks like 3D, but is in reality 2D.
+
The Web Coverage Service was designed to deliver continuous or discrete coverages. The basic dimensions are geographic boundingbox and time. In general, each coverage point has little metadata about itself. The Web Feature Service was designed to deliver rich metadata about stationary objects. While the feature has a boundingbox, the lat and lon are not dimensions. They are attributes of features.
  
** In cube data, you have typically at least 4 dimensions, lat, lon and time.
+
By combining those two it is possible to deliver complete metadata of a station network.
** In point data, you have typically 2 dimensions, location and time.
 
  
The station has 2 attributes: lat and lon, but it's still one dimension. This is an important distinction which was not in the design requirements for the WCS standard. Fortunately, the combination of WCS and WFS works great.
+
== Dimensions covering geographic area ==
  
== Web Coverage and Feature Services ==
+
* Typical station network looks like 2D, but is in reality 1D.
  
The Web Coverage Service was designed to deliver continuous coverages. The basic dimensions are geographic boundingbox and time. In general, each coverage point has little metadata about itself.
+
** In cube data, you have typically at least 2 dimensions, lat and lon.  
 +
** In point data, you have typically 1 dimension, location.
  
The Web Coverage Service was designed to deliver rich metadata about stationary objets. While the feature has a boundingbox, the lat and lon are not dimensions. They are attibutes of features.
+
The location has 2 attributes: lat and lon, but it's still one dimension. This is an important distinction and WCS 1.1 supports such dimensions.
  
== A Coverage can be irregular ==  
+
== A Coverage can be Irregular ==
  
== Existing metadata in DescribeCoverage ==
+
The WCS DescribeCoverage supports both regular and irregular coverages.
  
== Missing Metadata in DescribeCoverage ==
+
[http://data1.datafed.net:8080/NRL?service=WCS&version=1.1.2&request=DescribeCoverage&Identifiers=NAAPS_press Regular Data]
  
== Combining WCS and WFS ==
+
Excerpt:
  
== Necessary Location Metadata ==
+
    <SpatialDomain>
 +
        <ows11:BoundingBox crs="urn:ogc:def:crs:OGC:2:84" dimensions="2">
 +
            <ows11:LowerCorner>-179.5000 -89.5000</ows11:LowerCorner>
 +
            <ows11:UpperCorner>+179.5000 +89.5000</ows11:UpperCorner>
 +
        </ows11:BoundingBox>
 +
        <GridCRS>
 +
        <GridBaseCRS>urn:ogc:def:crs:OGC:2:84</GridBaseCRS>
 +
            <GridType>urn:ogc:def:method:WCS:1.1:2dSimpleGrid</GridType>
 +
            <GridOrigin>-179.5000 -89.5000</GridOrigin>
 +
            <GridOffsets>+1.0000 +1.0000</GridOffsets>
 +
            <GridCS>urn:ogc:def:cs:OGC:0.0:Grid2dSquareCS</GridCS>
 +
        </GridCRS>
 +
    </SpatialDomain>
  
== Optional Location Metadata ==
+
The GridBaseCRS element manifests the projection: "WGS 84"; type "2dSimpleGrid" origin -179.5, -89.5 and increments 1.0 and 1.0 degrees. In this case, the grid has regular lat and lon dimensions.
 +
 
 +
[http://data1.datafed.net:8080/AQS?service=WCS&version=1.1.2&request=DescribeCoverage&Identifiers=AQS_D Irregular Data]
 +
 
 +
Excerpt:
 +
 
 +
    <SpatialDomain>
 +
        <ows11:BoundingBox crs="urn:ogc:def:crs:OGC:2:84" dimensions="2">
 +
            <ows11:LowerCorner>-126.0000 +24.0000</ows11:LowerCorner>
 +
            <ows11:UpperCorner>-65.0000 +52.0000</ows11:UpperCorner>
 +
        </ows11:BoundingBox>
 +
      </SpatialDomain>
 +
 
 +
This is a description of irregular space dimensions. The data can still be an irregular grid, for example fine grid along the equator and sparse on the poles. On point data, the coverage is not a grid at all, but a station network.
 +
 
 +
== Additional Dimensions in DescribeCoverage ==
 +
 
 +
This is an elevation dimension. Irregular dimensions are expressed as axises, with some standard metadata like datatype and Units Of Measure. The WCS adds some CF-NetCDF metadata into the AxisMetadata node. This is still all according to the standard,
 +
 
 +
    <Axis identifier="elev">
 +
        <ows11:Title>air_pressure</ows11:Title>
 +
        <AvailableKeys>
 +
            <Key>1000.0</Key>
 +
            <Key>975.0</Key>
 +
            ...
 +
            <Key>10.0</Key>
 +
        </AvailableKeys>
 +
        <ows11:DataType>double</ows11:DataType>
 +
        <ows11:UOM>mbar</ows11:UOM>
 +
        <ows11:Metadata>
 +
        <cf:AxisMetadata>
 +
            <cf:axis positive="down">Z</cf:axis>
 +
            <cf:standard_name>air_pressure</cf:standard_name>
 +
        </cf:AxisMetadata>
 +
        </ows11:Metadata>
 +
    </Axis>
 +
 
 +
 
 +
== Metadata Limits in DescribeCoverage ==
 +
 
 +
While the extra dimensions in gridded data can be expressed with WCS 1.1, for station data the location table causes trouble. Here is a typical location table:
 +
 
 +
    loc_code,  lat,  lon,    elev
 +
    010730025, 33.55, -86.78, 54
 +
    010730028, 33.53, -86.85, 51
 +
    ....
 +
 
 +
There is no good way to express a table in the WCS, but WFS fits to the job perfectly. It's a collection of stationary located objects with varying amount of metadata.
 +
 
 +
The solution is to combine WCS and WFS. The WCS framework enumerates the location dimension keys, loc_codes, in the AvailableKeys node. Then the real rich location table is delivered via the [http://data1.datafed.net:8080/AQS?service=WFS&Version=1.0.0&Request=GetFeature&typename=AQS_D&filter=field:PB&outputFormat=text/csv WFS link].
 +
 
 +
    <Axis identifier="location">
 +
        <ows11:Title>Location Table</ows11:Title>
 +
        <AvailableKeys>
 +
            <Key>010730025</Key>
 +
            <Key>010730028</Key>
 +
            ...
 +
        </AvailableKeys>
 +
        <ows11:Metadata xlink:type="simple" xlink:href="http://data1.datafed.net:8080/AQS?service=WFS&Version=1.0.0&Request=GetFeature&typename=AQS_D&filter=field:PB&outputFormat=text/csv" />
 +
    </Axis>
 +
 
 +
(In the actual implementation AvailableKeys does not enumerate loc_codes, because it's redundant. We're just showing it's possible.)
 +
 
 +
== Location Metadata ==
 +
 
 +
For a generic client, the necessary location metadata attributes are loc_code, lat and lon.
 +
 
 +
It is useful to add more metadata, like loc_name and elev. All these names are just a convention, and need to be documented as a best practice.
  
 
== Custom Location Metadata ==
 
== Custom Location Metadata ==
 +
 +
Many networks display a large collection of different kind of metadata that is very provider-specific, and therefore impossible to standardize. For example the AQS network:
 +
 +
[http://webapps.datafed.net/datafed.aspx?wcs=http://data1.datafed.net:8080/AQS&coverage=AQS_D&field=pm10&loc_code=380171004 AQS_D pm10 at location 380171004]
 +
 +
If you press the small "m" button next to the parameter list, you'll see the [http://webapps.datafed.net/metadata.aspx?wcs=http://data1.datafed.net:8080/AQS&coverage=AQS_D&field=pm10&loc_code=380171004&view_id=metadata Rich Metadata View]
 +
 +
It's impossible to standardize such information, so we should just agree how to encode the URL that pops up the HTML page containing human readable information about the location.

Latest revision as of 23:07, August 23, 2011

Back to WCS Wrapper

Project on SourceForge

Objective and Current Experience

While WCS was designed for gridded data, we have found it to be a good delivery standard for station data also. With a minimal metadata encoding conventions it's possible to use the combination of WCS and WFS for the purposes that they were designed for, and use existing standards for interoperable data delivery system.

The Web Coverage Service was designed to deliver continuous or discrete coverages. The basic dimensions are geographic boundingbox and time. In general, each coverage point has little metadata about itself. The Web Feature Service was designed to deliver rich metadata about stationary objects. While the feature has a boundingbox, the lat and lon are not dimensions. They are attributes of features.

By combining those two it is possible to deliver complete metadata of a station network.

Dimensions covering geographic area

  • Typical station network looks like 2D, but is in reality 1D.
    • In cube data, you have typically at least 2 dimensions, lat and lon.
    • In point data, you have typically 1 dimension, location.

The location has 2 attributes: lat and lon, but it's still one dimension. This is an important distinction and WCS 1.1 supports such dimensions.

A Coverage can be Irregular

The WCS DescribeCoverage supports both regular and irregular coverages.

Regular Data

Excerpt:

   <SpatialDomain>
       <ows11:BoundingBox crs="urn:ogc:def:crs:OGC:2:84" dimensions="2">
           <ows11:LowerCorner>-179.5000 -89.5000</ows11:LowerCorner>
           <ows11:UpperCorner>+179.5000 +89.5000</ows11:UpperCorner>
       </ows11:BoundingBox>
       <GridCRS>
       <GridBaseCRS>urn:ogc:def:crs:OGC:2:84</GridBaseCRS>
           <GridType>urn:ogc:def:method:WCS:1.1:2dSimpleGrid</GridType>
           <GridOrigin>-179.5000 -89.5000</GridOrigin>
           <GridOffsets>+1.0000 +1.0000</GridOffsets>
           <GridCS>urn:ogc:def:cs:OGC:0.0:Grid2dSquareCS</GridCS>
       </GridCRS>
   </SpatialDomain>

The GridBaseCRS element manifests the projection: "WGS 84"; type "2dSimpleGrid" origin -179.5, -89.5 and increments 1.0 and 1.0 degrees. In this case, the grid has regular lat and lon dimensions.

Irregular Data

Excerpt:

   <SpatialDomain>
       <ows11:BoundingBox crs="urn:ogc:def:crs:OGC:2:84" dimensions="2">
           <ows11:LowerCorner>-126.0000 +24.0000</ows11:LowerCorner>
           <ows11:UpperCorner>-65.0000 +52.0000</ows11:UpperCorner>
       </ows11:BoundingBox>
     </SpatialDomain>

This is a description of irregular space dimensions. The data can still be an irregular grid, for example fine grid along the equator and sparse on the poles. On point data, the coverage is not a grid at all, but a station network.

Additional Dimensions in DescribeCoverage

This is an elevation dimension. Irregular dimensions are expressed as axises, with some standard metadata like datatype and Units Of Measure. The WCS adds some CF-NetCDF metadata into the AxisMetadata node. This is still all according to the standard,

   <Axis identifier="elev">
       <ows11:Title>air_pressure</ows11:Title>
       <AvailableKeys>
           <Key>1000.0</Key>
           <Key>975.0</Key>
           ...
           <Key>10.0</Key>
       </AvailableKeys>
       <ows11:DataType>double</ows11:DataType>
       <ows11:UOM>mbar</ows11:UOM>
       <ows11:Metadata>
       <cf:AxisMetadata>
           <cf:axis positive="down">Z</cf:axis>
           <cf:standard_name>air_pressure</cf:standard_name>
       </cf:AxisMetadata>
       </ows11:Metadata>
   </Axis>


Metadata Limits in DescribeCoverage

While the extra dimensions in gridded data can be expressed with WCS 1.1, for station data the location table causes trouble. Here is a typical location table:

   loc_code,  lat,   lon,    elev
   010730025, 33.55, -86.78, 54
   010730028, 33.53, -86.85, 51
   ....

There is no good way to express a table in the WCS, but WFS fits to the job perfectly. It's a collection of stationary located objects with varying amount of metadata.

The solution is to combine WCS and WFS. The WCS framework enumerates the location dimension keys, loc_codes, in the AvailableKeys node. Then the real rich location table is delivered via the WFS link.

   <Axis identifier="location">
       <ows11:Title>Location Table</ows11:Title>
       <AvailableKeys>
           <Key>010730025</Key>
           <Key>010730028</Key>
           ...
       </AvailableKeys>
       <ows11:Metadata xlink:type="simple" xlink:href="http://data1.datafed.net:8080/AQS?service=WFS&Version=1.0.0&Request=GetFeature&typename=AQS_D&filter=field:PB&outputFormat=text/csv" />
   </Axis>

(In the actual implementation AvailableKeys does not enumerate loc_codes, because it's redundant. We're just showing it's possible.)

Location Metadata

For a generic client, the necessary location metadata attributes are loc_code, lat and lon.

It is useful to add more metadata, like loc_name and elev. All these names are just a convention, and need to be documented as a best practice.

Custom Location Metadata

Many networks display a large collection of different kind of metadata that is very provider-specific, and therefore impossible to standardize. For example the AQS network:

AQS_D pm10 at location 380171004

If you press the small "m" button next to the parameter list, you'll see the Rich Metadata View

It's impossible to standardize such information, so we should just agree how to encode the URL that pops up the HTML page containing human readable information about the location.