Difference between revisions of "Glossary"

From Earth Science Information Partners (ESIP)
Line 119: Line 119:
 
== Location Table ==
 
== Location Table ==
  
TODO: Purpose and naming
+
The location table describes the location dimension for point data.
 +
 
 +
The fields that datafed uses are:
 +
 
 +
* Mandatory fields:
 +
 
 +
** loc_code: A unique text field, used to identify a location.
 +
** lat: Latitude of the location in degrees_north
 +
** Lon: Elevation of the location in degrees_east
 +
 
 +
* Optional datafed fields:
 +
 
 +
** loc_name: Reasonably short text describing location.
 +
** elev: elevation in meters.
 +
 
 +
* data specific fields:
 +
** Any field with any name
 +
 
 +
Good loc_codes are short abbreviations like ACAD and YOSE for Acadia and Yosemite National Parks. Completely numeric loc codes are possible, but more difficult to recognize and since leading zeros are significant, tools like excel may think they're numbers and cut them off.
 +
 
 +
If the loc_codes are long, like 9 characters, it's useful to generate a numeric 16-bit primary key for the location table and use it for joining the data tables with the location table. This may help in indexing and speed things up quite a bit.
 +
 
 +
Example: [http://128.252.202.19:8080/CIRA?service=WFS&version=1.0.0&request=GetFeature&typename=SO4f&outputformat=text/csv CIRA/VIEWS location table]
  
 
== Metadata ==
 
== Metadata ==

Revision as of 11:44, September 1, 2010

Back to WCS Wrapper

Glossary for Common Terms and Standard Names in Datafed WCS Wrapper Framework

AQ_uFIND

A front end to GEOSS Clearinghouse. Currently can be used to find WMS services.

Example use: AQ_uFIND.aspx?datatype=point

Capabilities Processor

This component creates the standard XML documents for WCS and WFS services.

It operates on Metadata and data configuration. From Metadata the capabilities document gets Title, Abstract, Keywords, Contact person etc... From data configuration the processor gets full information of each coverage.

WCS Wrapper Configuration for Cubes

WCS Wrapper Configuration for Point Data

Coverage Processor

The Coverage Processor is a component that performs three different activities:

  • WCS Query Parser. The syntax is checked and output is binary object of all the query elements.
  • Subsetter. This component finds the desired coverage and applies filters to read a subset of the data:
    • Fields: A client querying wind data may be interested in speed and direction, but reject air pressure.
    • Bounding Box: restrict response to certain geographical area.
    • Time: Default time, One time, list of times, periodic range of times.
    • Grid size and interpolation: High resolution data can be interpolated to lower resolution.
    • By dimension: Select only one or some wavelengths,elevations,locations
  • Formatter. The binary data is returned in desired format. Currently supported are NetCDF-CF for cubes and CSV, Comma Separated Values for points.

NetCDF-CF based processor is completely generic for any compatible netCDF-CF file.

SQL processors can be either configured to supported DB schema types, or custom written for esoteric databases.

By writing a custom processor, anything can be used as a data source.

Cube Data Configuration

For standard netCDF-CF files, the configuration is automatic. Each file becomes a coverage, and each variable becomes a field. This is by far the easiest way to create a WCS service. Examples are testprovider which comes with the installation package, and NASA which serves some datasets downloaded from NASA.

For daily netCDF-CF files it is possible to create a service without compiling them into single file. See Serving data from periodic collection of NetCDF files as an example.

By creating a custom handler, it is possible to store data anywhere.

Datafed Browser

TODO: describe classic browser

TODO: describe browsing WCS without df catalog http://webapps.datafed.net/datafed.aspx?wcs=http://128.252.202.19:8080/CIRA&coverage=VIEWS&param_abbr=SO4f

TODO: Describe GE Plugin browser

Feature Processor

Web Feature Service, WFS, is good in publishing geographic information that does not change by time.

With datafed WCS it is used to publish the location table for point data, because WCS DescribeCoverage Document does not support such rich dimensions well and location tables are static geographic information.

The component that performs three different activities:

  • WFS Query Parser. The syntax is checked and output is binary object of all the query elements.
  • Subsetter.
    • Each field may have different location table. If a the data is sparse, some fields have data only in a few locations, it makes sense to return only those locations.
    • Locations may also be filtered by geographic bounding box.
    • Other WFS filters are not implemented.
  • Formatter. The data is returned in desired format. Currently the only supported is CSV, Comma Separated Values

GEOSS Clearinghouse

The Clearinghouse is a component in the GEOSS Common Infrastructure. One of it's functions is the GEOSS Components and Services Registry

Google Earth

http://earth.google.com/

TODO: describe standalone and plugin

describe images and static points

describe dynamic points

ISO 19115 Metadata

Description of a service, with strictly defined XML presentation. Contains service URL's and metadata about the service.

ISO 19115 Maker

A public service to create an ISO 19115 record from a WCS or WMS service.

If the Capabilities document contains necessary keywords, the document can be created automatically: ISO 19115 for AIRNOW pmfine WMS.

Without keywords in the URL, the metadata can be passed via URL parameters.

KML Keyhole Markup Language

KML is the way to describe content in Google Earth and Google Maps. KML documentation is hosted by google.

KML Maker

Datafed tools produce KML directly out of data, which can be produced with WCS or WMS services.

KML from a CIRA/VIEWS showing SO4f and direct link

KML from NASA giovanni WMS and direct link

Precompiled examples:

Point Demo

Gridded Demo

Location Table

The location table describes the location dimension for point data.

The fields that datafed uses are:

  • Mandatory fields:
    • loc_code: A unique text field, used to identify a location.
    • lat: Latitude of the location in degrees_north
    • Lon: Elevation of the location in degrees_east
  • Optional datafed fields:
    • loc_name: Reasonably short text describing location.
    • elev: elevation in meters.
  • data specific fields:
    • Any field with any name

Good loc_codes are short abbreviations like ACAD and YOSE for Acadia and Yosemite National Parks. Completely numeric loc codes are possible, but more difficult to recognize and since leading zeros are significant, tools like excel may think they're numbers and cut them off.

If the loc_codes are long, like 9 characters, it's useful to generate a numeric 16-bit primary key for the location table and use it for joining the data tables with the location table. This may help in indexing and speed things up quite a bit.

Example: CIRA/VIEWS location table

Metadata

Abstract, Contact Information, Keywords and any other such documentation that is needed in classifying or finding the service. The metadata is accessible for the user via capabilities and coverage description documents.

NetCDF-CF

TODO: describe

http://wiki.esipfed.org/index.php/Creating_NetCDF_CF_Files

Point Data Configuration

Programmed instructions for the framework how to access data. This includes but is not limited to

  • For Cube Coverages:
    • Automatic: Information of variables and dimensions extracted from netCDF-CF files.
    • Manual: Hand-edited python dictionaries describing the netCDF files, their variables and dimensions.


  • For Point Coverages:
    • Hand-edited python dictionaries describing Names and Columns of Location and Data tables.
    • Custom modules for databases that are too esoteric configure in pure declarative manner.

Cube: Configuring NetCDF based Cube Data


Point:

Location Table Configuration

Data Table Configuration

Point Location Configuration

Programmed instructions for the framework how to access data. This includes but is not limited to

  • For Cube Coverages:
    • Automatic: Information of variables and dimensions extracted from netCDF-CF files.
    • Manual: Hand-edited python dictionaries describing the netCDF files, their variables and dimensions.


  • For Point Coverages:
    • Hand-edited python dictionaries describing Names and Columns of Location and Data tables.
    • Custom modules for databases that are too esoteric configure in pure declarative manner.

Cube: Configuring NetCDF based Cube Data


Point:

Location Table Configuration

Data Table Configuration

SQL Database for Points

Currently the datafed WCS for points supports one kind of point data: Fixed locations and regular intervals.

Storing Point Data in a Relational Database

WCS Capabilities Document

The document contains all the high level information about a service

The Document contains:

  • Description of the Service
  • Machine Readable and Human Readable Name.
  • Keywords
  • Contact Information
  • HTTP access information
  • List of coverages in the service
    • Machine Readable and Human Readable Name.
    • Keywords
    • Latitude and Longitude bounds.
    • Time range in version 1.0.0

Example Version 1.1.2

Example Version 1.0.0

WCS Describe Coverage Document

The document describes the coverage in detail, so that the user knows what the data is and what are the dimensions of the data.

  • Description of the Coverage
  • Machine Readable and Human Readable Name.
  • Keywords
  • Latitude and Longitude bounds.
  • Grid bounds in the projection of the data, if applicable
  • Grid size in the projection of the data, if applicable
  • Time dimension.
  • Supported Coordinate Systems
  • Supported Formats
  • Supported Interpolations
  • Fields of Coverage in versions 1.1.x
    • Name
    • Units
    • Other dimensions, like elevation or wavelength, if applicable
    • Reference to location dimension, if applicable

Example Version 1.1.2

Example Version 1.0.0

WCS GetCoverage Query

The main query to get data from a WCS

GetCoverage for points

TODO: samples

WFS Capabilities Document

The document contains all the high level information about a service

TODO: samples

WFS DescribeFeatureType

ASOS The document contains all the high level information about a service

TODO: samples


WFS GetFeature Query

The main query to get data from a WFS


TODO: samples sample