Questions and Comments about CF-1.6 Station Data Convention

Back to WCS Wrapper

The objective of this page is to promote discussion about CF-netCDF formats for station point data.

CSV, Comma Separated Values: Its Uses and Limitations

CSV format is easy, compact and practical for many uses.It can be fed to any spreadsheet and consumed easily with custom software.

Example of WCS GetCoverage query returning an envelope with uri to the result CSV file

   loc_code,lat,lon,datetime,pm25nr, etc...
   350610008,34.81,-106.74,2007-05-12T00:00:00,3.8, etc...
   350439004,35.62,-106.72,2007-05-12T00:00:00,20.9, etc...
   350610008,34.81,-106.74,2007-05-12T01:00:00,6.9, etc...

Unfortunately, the format makes embedding metadata in a CF-like convention difficult.

• Location dimension:
  • Incompleteness: No idea what else is known about the stations than loc_code, lat and lon.
  • Stations with no data may be present with NULL value, or may not be present at all. No indication what's the case.
  • Repetition of the same latitude and longitude values.

• Time dimension:
  • What's the periodicity? The software needs to guess, that the sample query coverage AQS_H actually is hourly data.
  • Are all the locations in the same periodicity, or do the locations have individual recording times? Again a guess.
  • What was the requested time min + max, and what's the real returned time range.

There's field pm25nr but there's more about it. It's PM 2.5 Non-Reference Method, units are ug/m3, source is from EPA Air Quality Network.

CF-netCDF for the rescue.

Existing CF-netCDF Conventions

At datafed, we have supported station data before CF-1.5 and WCS 1.1.0 were official. This was implemented in the older WCS 1.0.0 standard and has not been incorporated into the WCS 1.1 service yet.

Sample Query to AQS_H returning CF-netCDF station time series

Same as above, but returning only the CDL header file without data

The above implementation is still lacking what we really need, so let's use it just as a proof of concept: it is possible to pack station data into a CF-NetCDF and it's more expressive than CSV. It's also created from an obsolete CF-1.5 draft, so it's incompatible with the real CF-1.5 document.

CF 1.5 and Station Data

The section 5.4. Timeseries of Station Data shows a more recently accepted coding of station data.

That section shows almost what datafed is producing. Simplifying the data by removing the pressure dimension:

   dimensions:
       station = 10 ;  // measurement locations
       time = UNLIMITED ;
   variables:
       float humidity(time,station) ;
       humidity:long_name = "specific humidity" ;
       humidity:coordinates = "lat lon" ;
       double time(time) ;
       time:long_name = "time of measurement" ;
       time:units = "days since 1970-01-01 00:00:00" ;
       float lon(station) ;
       lon:long_name = "station longitude";
       lon:units = "degrees_east";
       float lat(station) ;
       lat:long_name = "station latitude" ;
       lat:units = "degrees_north" ;

Properties of the data:

• Stations:
  • Stations are indexed along station dimension.
  • Each station has lat and lon.
• Time:
  • Measurement time is enumerated and is consistent across the locations.
  • Time is the unlimited dimension, so new measurements can be appended.
• Data
  • Only the data is stored, very efficient transport mode for densely packed data.
  • Times and stations are not stored physically with each data point, since they can be computed from the index.
  • Missing data could be indicated by adding missing_value attribute.

CF 1.6 Draft and Station Data

The section H.2. Time Series Data shows a more recently accepted coding of station data.

   dimensions:
       station = 10 ;  // measurement locations
       name_strlen = 15 ; // location name
       time = UNLIMITED ;
   variables:
       float humidity(station,time) ;
       humidity:standard_name = "specific humidity" ;
       humidity:coordinates = "lat lon alt" ;
   double time(time) ; 
       time:standard_name = "time";
       time:long_name = "time of measurement" ;
       time:units = "days since 1970-01-01 00:00:00" ;
   float lon(station) ; 
       lon:standard_name = "longitude";
       lon:long_name = "station longitude";
       lon:units = "degrees_east";
   float lat(station) ; 
       lat:standard_name = "latitude";
       lat:long_name = "station latitude" ;
       lat:units = "degrees_north" ; 
   float alt(station) ;
       alt:long_name = "vertical distance above the surface" ;
       alt:standard_name = "height" ;
       alt:units = "m";
       alt:positive = "up";
       alt:axis = "Z";
   char station_name(station, name_strlen) ;
       station_name:long_name = "station name" ;
       station_name:cf_role = "timeseries_id";
   attributes:
       :featureType = "timeSeries";

Properties of the data:

• Stations:
  • Same as above, with addition
  • Each station has station_name.
  • Each station has alt for altitude with units and direction.
• Time:
  • Same as above, with standard_name attribute
• Data
  • Same as above, with standard_name attribute

CF-netCDF Data Model extension specification (Draft)

This is an OGC draft paper edited by Ben Domenico and Stefano Nativi. It differs a little bit from CF-1.5 and 1.6

It contains similar timeSeries convention: a series of data points at the same spatial location with monotonically increasing times.

The data has the same two dimensions as above: station and time. But time also has the same two dimensions. This means, hat every station may have it's own time values, Time for observation at index 34 may change from station to station.

The example given is not consistent: the time variable has only one dimension.

   dimensions:
       time = UNLIMITED; // (5 currently)
       station = 10;
       nv = 2;
   variables:
       float pressure(time,station);
       pressure:long_name = "pressure";
       pressure:units = "kPa";
       pressure:cell_methods = "time: point";
   float maxtemp(time,station);
       maxtemp:long_name = "temperature";
       maxtemp:units = "K";
       maxtemp:cell_methods = "time: maximum";
   float ppn(time,station);
       ppn:long_name = "depth of water-equivalent precipitation";
       ppn:units = "mm";
       ppn:cell_methods = "time: sum";
   double time(time);
       time:long_name = "time";
       time:units = "h since 1998-4-19 6:0:0";
       time:bounds = "time_bnds";
   double time_bnds(time,nv);

Requirements Suggested by Datafed

All of the below are suggestions that are already in use at Datafed. Comments are welcome.

The station table should have any number of fields

The station (location) table at datafed has

• Minimum 3 fields: loc_code, lat, lon
• Optional standard fields: loc_name, elev
• Any number of extended fields: state_code, county_code, etc...

Time dimension should have periodicity encoded

todo

History shoule be more than single comment

todo