Difference between revisions of "Attribute Convention for Data Discovery 1-3"

From Earth Science Information Partners (ESIP)
Line 24: Line 24:
 
The NetCDF User Guide [http://www.unidata.ucar.edu/software/netcdf/docs/netcdf.html (NUG)] provides basic recommendations for creating NetCDF files; the NetCDF Climate and Forecast Metadata Conventions [http://cfconventions.org/ (CF)]  provides more specific guidance. The ACDD builds upon and is compatible with these conventions; it may refine the definition of some terms in those conventions, but does not preclude the use of any attributes defined by the NUG or CF.  
 
The NetCDF User Guide [http://www.unidata.ucar.edu/software/netcdf/docs/netcdf.html (NUG)] provides basic recommendations for creating NetCDF files; the NetCDF Climate and Forecast Metadata Conventions [http://cfconventions.org/ (CF)]  provides more specific guidance. The ACDD builds upon and is compatible with these conventions; it may refine the definition of some terms in those conventions, but does not preclude the use of any attributes defined by the NUG or CF.  
  
The NUG does not require any global attributes, though it recommends and defines three, title, history, and Conventions. CF specifies more: institution, source, references, comment, and featureType. ACDD 1.3 adopts all CF 1.7 global attributes. In a change from ACDD 1.2, we adopt the NUG recommendation to supply all conventions in the single Conventions attribute.
+
The NUG does not require any global attributes, though it recommends and defines three, title, history, and Conventions. CF specifies more: institution, source, references, comment, and featureType. ACDD 1.3 adopts all CF 1.6 global attributes. In a change from ACDD 1.2, we adopt the NUG recommendation to supply all conventions in the single Conventions attribute.
  
 
== Attribute Crosswalks ==  
 
== Attribute Crosswalks ==  

Revision as of 09:20, December 8, 2014

       PROPOSED - Ready for review

Context

Document

This is the Attribute Convention for Data Discovery (ACDD).

Version and Status

This version is designated as Version 1.3.4. It is a version proposed for review by the full Documentation Cluster. This version includes attribute changes as of Adjudication meeting #3, and (in bold/strikeout) some minor or discussed changes since then, including at the Nov ESIP Documentation meeting. (Once approved, this paragraph will be replaced by 'This is version 1.3 of the ACDD; it is the latest released version.')

The page Attribute Convention for Data Discovery (ACDD) always points to the current released version of the Convention. The version number at the top of the resulting page will show the current version.

See the ACDD category page for information on the history and maintenance of this convention.

Development

For development versions of the ACDD, please see the page Attribute Convention for Data Discovery Working.

Overview

This document describes attributes recommended for describing a NetCDF dataset to discovery systems such as Digital Libraries. THREDDS and other tools can use these attributes for extracting to extract metadata from datasets, and exporting to Dublin Core, DIF, ADN, FGDC, ISO 19115 and other metadata formats. This will help systems and users locate and use data efficiently.

Alignment with NetCDF and CF Conventions

The NetCDF User Guide (NUG) provides basic recommendations for creating NetCDF files; the NetCDF Climate and Forecast Metadata Conventions (CF) provides more specific guidance. The ACDD builds upon and is compatible with these conventions; it may refine the definition of some terms in those conventions, but does not preclude the use of any attributes defined by the NUG or CF.

The NUG does not require any global attributes, though it recommends and defines three, title, history, and Conventions. CF specifies more: institution, source, references, comment, and featureType. ACDD 1.3 adopts all CF 1.6 global attributes. In a change from ACDD 1.2, we adopt the NUG recommendation to supply all conventions in the single Conventions attribute.

Attribute Crosswalks

Many of these attributes correspond to general discovery metadata content, so they are availablesimilar terms exist in many metadata standards. This Attribute_Convention_for_Data_Discovery_(ACDD)_Mappings page includes a crosswalk to THREDDS, OGC CSW, ISO 19115-2 and Rubric Categories.

Additional Metadata: metadata_link attribute

The netCDF metadata model is focused on providing "use metadata" for the data included in the file (or granule). Other metadata dialects (i.e. Documents using other metadata specifications (e.g., ISO 19115) can provide additional information about collections and more details about the dataset. If additional metadata exists, you can make users aware of it by adding a global attribute named "metadata_link" to the netCDF file. The value of this attribute is a URL that gives the location of the more complete metadata.

Definitions: Data and Metadata

In several ACDD attribute names or definitions, the terms 'data' and 'metadata' are used. In the context of NetCDF files, these refer specifically to the values within the file, and the attributes of the file, respectively.

Maintenance of Metadata

ACDD attributes, like all NetCDF attributes, characterize the data they are associated with.their containing (parent) granules. As NetCDF data are processed (e.g., through subsetting or other algorithms), these characteristics can be altered. The software or user processor is responsible for updating to update these attributes as part of the processing, but unfortunately some software processes and user practices leave them unchanged. This affects both consumers and producers of these files, which comprises three including these roles:

  • developers of software tools that process NetCDF files;
  • users that create new NetCDF files from existing ones; and
  • end users of NetCDF files.

NetCDF file creators (the first two roles) should ensure that the attributes of output files accurately represent those files, and specifically should not "pass through" any source attribute in unaltered form, unless it is known to remain accurate. NetCDF file users (all three roles) should verify critical attribute values, and understand how the source data and metadata were generated, to be confident the source metadata is current.

The ACDD geospatiotemporal attributes present a special case, as this information is already fully defined by the CF coordinate variables (the redundant attributes are recommended to simplify access). Errors in these attributes will create an inconsistency between the metadata and associated data. of the granule or file data. The risk of these 'inconsistency errors' is highest for files that are aggregated into longer or larger products, or subset into shorter or smaller products, such as files from numerical forecast models and gridded satellite observations. For this reason, some providers of those data types may choose to omit the ACDD geospatiotemporal attributes from their files. If the ACDD geospatiotemporal attributes are present, checking them against the CF coordinate variables can serve as a partial test of the metadata's validity.

Attribute Content Guidance

Date and Time: ISO 8601 Recommended Formats

The ACDD specifies ISO 8601:2004 date and time formats for its temporal attributes. ACDD strongly encourages the use of the 'extended' format date-time, in the form

YYYY-MM-DDThh:mm:ss<zone>

(although ss, mm, and hh can be omitted, and <zone> can be Z, ±hh:mm, or ±hh). Per the standard, the shortened or basic format, which omits the - and : separators, "should be avoided in plain text."

For duration attributes, again the extended form is strongly encouraged for readability:

P[YYYY]-[MM]-[DD]T[hh]:[mm]:[ss]

If for some reason the strongly encouraged formats can not be used, other ISO 8601-compatible formats are acceptable, but may not be handled by some processing software.

Comma-Separated Lists

Several attributes explicitly allow the entry of multiple entities as comma-separated values. The entities in such lists which contain a comma must be enclosed in straight double quotation marks ("), which will not be considered part of the entity.

Spaces (ASCII character 32) Blanks between the entities are recommended for readability, but not required.

The same protocol may be used within free-text attributes, but is only recommended in cases where the attribute is being populated with structured data (rather than unconstrained text).

Free Text Formatting: Structured Text Considerations

In some attributes, it may be desirable to use structured text to support computer parsing of human-readable content. Because netCDF files are often translated between binary and text-based encodings like ncML (e.g., by the netCDF command line tools nco), such structures should tolerate changes in white space, including end-of-line characters, that may occur during translation.

Global Attributes

Highly Recommended

Attribute Description
title A short phrase or sentence describing the dataset. In many discovery systems, the title will be displayed in the results list from a search, and therefore should be human readable and reasonable to display in a list of such names. This attribute is also recommended by the NetCDF Users Guide and the CF conventions.
summary A paragraph describing the dataset, analogous to an abstract for a paper.
keywords A comma-separated list of key words and/or phrases. Keywords may be common words or phrases, terms from a controlled vocabulary (GCMD is often used), or URIs for terms from a controlled vocabulary (see also "keywords_vocabulary" attribute).
Conventions A comma-separated list of the conventions that are followed by the dataset. For files that follow this version of ACDD, include the string 'ACDD-1.3'. (This attribute is described in the NetCDF Users Guide.)

Recommended

Attribute Description
id An identifier for the data set, provided by and unique within its naming authority. The combination of the "naming authority" and the "id" should be globally unique, but the id can be globally unique by itself also. IDs can be URLs, URNs, DOIs, meaningful text strings, a local key, or any other unique string of characters. The id should not include white space characters blanks.
naming_authority The organization that provides the initial id (see above) for the dataset. The naming authority should be uniquely specified by this attribute. We recommend using reverse-DNS naming for the naming authority; URIs are also acceptable. Example: 'edu.ucar.unidata'.
cdm_data_type The organization of the data, as derived from the Common Data Model's Scientific Data layer and understood by THREDDS (this is a THREDDS "dataType"). One of point, profile, section, station, station_profile, trajectory, grid, image, or swath. Please note that this is different from the CF NetCDF attribute 'featureType' that indicates a Discrete Sampling Geometry file—for guidance on those terms, please see the NODC guidance.
history Provides an audit trail for modifications to the original data. This attribute is also in the NetCDF Users Guide: 'This is a character array with a line for each invocation of a program that has modified the dataset. Well-behaved generic netCDF applications should append a line containing: date, time of day, user name, program name and command arguments.' To include a more complete description you can append a reference to an ISO Lineage entity; see NOAA EDM ISO Lineage guidance.
source The method of production of the original data. If it was model-generated, source should name the model and its version. If it is observational, source should characterize it. This attribute is defined in the CF Conventions. Examples: 'temperature from CTD #1234'; 'world model v.0.1'.
processing_level A textual description of the processing (or quality control) level of the data.
comment Miscellaneous information about the data, not captured elsewhere. This attribute is defined in the CF Conventions.
acknowledgement A place to acknowledge various types of support for the project that produced this data.
license Provide the URL to a standard or specific license, enter "Freely Distributed" or "None", or describe any restrictions to data access and distribution in free text.
standard_name_vocabulary The name and version of the controlled vocabulary from which variable standard names are taken. Example: 'CF Standard Name Table v27'
date_created The date on which this version of the data was created. (Modification of values implies a new version, hence this would be assigned the date of the most recent values modification.) Metadata changes are not considered when assigning the date_created. The ISO 8601:2004 extended date format is recommended, as described in the Attribute Content Guidance section.
creator_name The name of the person (or other creator type specified by the creator_type attribute) principally responsible for creating this data.
creator_email The email address of the person (or other creator type specified by the creator_type attribute) principally responsible for creating this data.
institution The name of the institution principally responsible for originating this data. This attribute is recommended by the CF convention.
project The name of the project(s) principally responsible for originating this data. Multiple projects can be separated by commas, as described under Attribute Content Guidelines. Examples: 'PATMOS-X', 'Extended Continental Shelf Project'.
publisher_name The name of the person (or other entity specified by the publisher_type attribute) responsible for publishing the data file or product to users, with its current metadata and format.
publisher_email The email address of the person (or other entity specified by the publisher_type attribute) responsible for publishing the data file or product to users, with its current metadata and format.
publisher_url The URL of the person (or other entity specified by the publisher_type attribute) responsible for publishing the data file or product to users, with its current metadata and format.
geospatial_bounds Describes the data's 2D or 3D geospatial extent in OGC's Well-Known Text (WKT) Geometry format (reference the OGC Simple Feature Access (SFA) specification). The meaning and order of values for each point's coordinates depends on the coordinate reference system (CRS). The ACDD default is 2D geometry in the EPSG:4326 coordinate reference system. The default may be overridden with geospatial_bounds_crs and geospatial_bounds_vertical_crs (see those attributes). EPSG:4326 coordinate values are latitude (decimal degrees_north) and longitude (decimal degrees_east), in that order. Longitude values in the default case are limited to the [-180, 180) range. Example: "POLYGON ((40.26 -111.29, 41.26 -111.29, 41.26 -110.29, 40.26 -110.29, 40.26 -111.29))”. Describes geospatial extent using any of the geometric objects (2D or 3D) supported by the Well-Known Text (WKT) format.
geospatial_bounds_crs The coordinate reference system (CRS) of the point coordinates in the geospatial_bounds attribute. This CRS may be 2-dimensional or 3-dimensional, but together with geospatial_bounds_vertical_crs, if that attribute is supplied, must match the dimensionality, order, and meaning of point coordinate values in the geospatial_bounds attribute. If geospatial_bounds_vertical_crs is also present then this attribute must only specify a 2D CRS. EPSG CRSs are strongly recommended. If this attribute is not specified, the CRS is assumed to be EPSG:4326. Examples: "EPSG:4979" (the 3D WGS84 CRS), "EPSG:4047”.
geospatial_bounds_vertical_crs The vertical coordinate reference system (CRS) for the Z axis of the point coordinates in the geospatial_bounds attribute. This attribute cannot be used if the CRS in geospatial_bounds_crs is 3-dimensional; to use this attribute, geospatial_bounds_crs must exist and specify a 2D CRS. EPSG CRSs are strongly recommended. There is no default for this attribute when not specified. Examples: "EPSG:5829" (instantaneous height above sea level), "EPSG:5831" (instantaneous depth below sea level), or "EPSG:5703" (NAVD88 height).
geospatial_lat_min Describes a simple lower latitude limit; may be part of a 2- or 3-dimensional bounding region. Geospatial_lat_min specifies the southernmost latitude covered by the dataset.
geospatial_lat_max Describes a simple upper latitude limit; may be part of a 2- or 3-dimensional bounding region. Geospatial_lat_max specifies the northernmost latitude covered by the dataset.
geospatial_lon_min Describes a simple longitude limit; may be part of a 2- or 3-dimensional bounding region. geospatial_lon_min specifies the westernmost longitude covered by the dataset. See also geospatial_lon_max.
geospatial_lon_max Describes a simple longitude limit; may be part of a 2- or 3-dimensional bounding region. geospatial_lon_max specifies the easternmost longitude covered by the dataset. Cases where geospatial_lon_min is greater than geospatial_lon_max indicate the bounding box extends from geospatial_lon_max, through the longitude range discontinuity meridian (either the antimeridian for -180:180 values, or Prime Meridian for 0:360 values), to geospatial_lon_min; for example, geospatial_lon_min=170 and geospatial_lon_max=-175 incorporates 15 degrees of longitude (ranges 170 to 180 and -180 to -175).
geospatial_vertical_min Describes the numerically smaller vertical limit; may be part of a 2- or 3-dimensional bounding region. See geospatial_vertical_positive and geospatial_vertical_units.
geospatial_vertical_max Describes the numerically larger vertical limit; may be part of a 2- or 3-dimensional bounding region. See geospatial_vertical_positive and geospatial_vertical_units.
geospatial_vertical_positive One of 'up' or 'down'. If up, vertical values are interpreted as 'altitude', with negative values corresponding to below the reference datum (e.g., under water). If down, vertical values are interpreted as 'depth', positive values correspond to below the reference datum. Note that if geospatial_vertical_positive is down ('depth' orientation), the geospatial_vertical_min attribute specifies the data's vertical location furthest from the earth's center, and the geospatial_vertical_max attribute specifies the location closest to the earth's center.
time_coverage_start Describes the time of the first data point in the data set. Use the ISO 8601:2004 date format, preferably the extended format as recommended in the Attribute Content Guidance section.
time_coverage_end Describes the time of the last data point in the data set. Use ISO 8601:2004 date format, preferably the extended format as recommended in the Attribute Content Guidance section.
time_coverage_duration Describes the duration of the data set. Use ISO 8601:2004 duration format, preferably the extended format as recommended in the Attribute Content Guidance section.
time_coverage_resolution Describes the targeted time period between each value in the data set. Use ISO 8601:2004 duration format, preferably the extended format as recommended in the Attribute Content Guidance section.

Suggested

Attribute Description
creator_url The URL of the of the person (or other creator type specified by the creator_type attribute) principally responsible for creating this data.
creator_type Specifies type of creator with one of the following: 'person', 'group', 'institution', or 'position'. If this attribute is not specified, the creator is assumed to be a person.
creator_institution The institution of the creator; should uniquely identify the creator's institution. This attribute's value should be specified even if it matches the value of publisher_institution, or if creator_type is institution.
publisher_type Specifies type of publisher with one of the following: 'person', 'group', 'institution', or 'position'. If this attribute is not specified, the publisher is assumed to be a person.
publisher_institution The institution that presented the data file or equivalent product to users; should uniquely identify the institution. If publisher_type is institution, this should have the same value as publisher_name.
program The overarching program(s) of which the dataset is a part. A program consists of a set (or portfolio) of related and possibly interdependent projects that meet an overarching objective. Examples: 'GHRSST', 'NOAA CDR', 'NASA EOS', 'JPSS', 'GOES-R'.
contributor_name The name of any individuals, projects, or institutions that contributed to the creation of this data. May be presented as free text, or in a structured format compatible with conversion to ncML (e.g., insensitive to changes in whitespace, including end-of-line characters).
contributor_role The role of any individuals, projects, or institutions that contributed to the creation of this data. May be presented as free text, or in a structured format compatible with conversion to ncML (e.g., insensitive to changes in whitespace, including end-of-line characters). Multiple roles should be presented in the same order and number as the names in contributor_names.
geospatial_lat_units Units for the latitude axis described in "geospatial_lat_min" and "geospatial_lat_max" attributes. These are presumed to be "degree_north"; other options from udunits may be specified instead.
geospatial_lat_resolution Information about the targeted spacing of points in latitude. Recommend describing resolution as a number value followed by the units. Examples: '100 meters', '0.1 degree'
geospatial_lon_units Units for the longitude axis described in "geospatial_lon_min" and "geospatial_lon_max" attributes. These are presumed to be "degree_east"; other options from udunits may be specified instead.
geospatial_lon_resolution Information about the targeted spacing of points in longitude. Recommend describing resolution as a number value followed by units. Examples: '100 meters', '0.1 degree'
geospatial_vertical_units Units for the vertical axis described in "geospatial_vertical_min" and "geospatial_vertical_max" attributes. The default is EPSG:4979 (height above the ellipsoid, in meters); other vertical coordinate reference systems may be specified. Note that the common oceanographic practice of using pressure for a vertical coordinate, while not strictly a depth, can be specified using the unit bar. Examples: 'EPSG:5829' (instantaneous height above sea level), 'EPSG:5831' (instantaneous depth below sea level).
geospatial_vertical_resolution Information about the targeted vertical spacing of points. Example: '25 meters'
date_modified The date on which the data was last modified. Note that this applies just to the data, not the metadata. The ISO 8601:2004 extended date format is recommended, as described in the Attributes Content Guidance section.
date_issued The date on which this data (including all modifications) was formally issued (i.e., made available to a wider audience). Note that these apply just to the data, not the metadata. The ISO 8601:2004 extended date format is recommended, as described in the Attributes Content Guidance section.
date_metadata_modified The date on which the metadata was last modified. The ISO 8601:2004 extended date format is recommended, as described in the Attributes Content Guidance section.
keywords_vocabulary If you are using a controlled vocabulary for the words/phrases in your "keywords" attribute, this is the unique name or identifier of the vocabulary from which keywords are taken. If more than one keyword vocabulary is used, each may be presented with a prefix (e.g., "CF:NetCDF COARDS Climate and Forecast Standard Names") and a following comma, so that keywords may optionally be prefixed with the controlled vocabulary key. Example: 'GCMD:GCMD Keywords, CF:NetCDF COARDS Climate and Forecast Standard Names'.
platform Name of the platform(s) that supported the sensor data used to create this data set or product. Platforms can be of any type, including satellite, ship, station, aircraft or other. Indicate controlled vocabulary used in platform_vocabulary.
platform_vocabulary Controlled vocabulary for the names used in the "platform" attribute.
instrument Name of the contributing instrument(s) or sensor(s) used to create this data set or product. Indicate controlled vocabulary used in instrument_vocabulary.
instrument_vocabulary Controlled vocabulary for the names used in the "instrument" attribute.
metadata_link A URL that gives the location of more complete metadata. A persistent URL is recommended for this attribute.
references Published or web-based references that describe the data or methods used to produce it. Recommend URIs (such as a URL or DOI) for papers or other references. This attribute is defined in the CF conventions.

Highly Recommended Variable Attributes

Attribute Description
long_name A long descriptive name for the variable (not necessarily from a controlled vocabulary). This attribute is recommended by the NetCDF Users Guide, the COARDS convention, and the CF convention.
standard_name A long descriptive name for the variable taken from a controlled vocabulary of variable names. We recommend using the CF convention and the variable names from the CF standard name table. This attribute is recommended by the CF convention.
units The units of the variable's data values. This attribute value should be a valid udunits string. The "units" attribute is recommended by the NetCDF Users Guide, the COARDS convention, and the CF convention.
coverage_content_type An ISO 19115-1 code to indicate the source of the data (image, thematicClassification, physicalMeasurement, auxiliaryInformation, qualityInformation, referenceInformation, modelResult, or coordinate).

Deprecated

The following terms and definitions are still recognized, but are no longer recommended for use by ACDD.

Metadata_Convention: removed in favor of 'Conventions'

Conformance Test

Conformance tests are available for verson 1.1. A conformance test for this version will be linked from this page when it is available.

Additional Materials

These materials are not normative and may not be in alignment with this version of ACDD.