Difference between revisions of "Date Documentation"

From Earth Science Information Partners (ESIP)
Line 262: Line 262:
 
[[Category: Metadata Implementation]]
 
[[Category: Metadata Implementation]]
  
 +
<div id="Dates">
 
=Crosswalks=
 
=Crosswalks=
 +
Citations in metadata provide can important information about how users should cite the documented resource. These citations are an important part of the identification information and, as such, they are in a special location in the record. In addition, some dialects include citation to other resources and relevant materials that can not be included in the metadata. General citations to external resources that can occur anywhere in the record are described here.
 +
<table xmlns:xsl="http://www.w3.org/1999/XSL/Transform" width="95%" border="1" cellpadding="3" cellspacing="3" style="border-collapse: collapse;" id="citation"><tr><th width="15%">Concept</th><th width="25%">Description</th><th width="60%">Dialect (Fit) Paths</th></tr><tr><td>[[Concepts_Glossary#Publication_Date|Publication Date]]</td><td>Date of publication of the cited resource</td><td style="word-break:break-all;"><b>DIF</b> /dif9:DIF/dif9:Data_Set_Citation/dif9:Dataset_Release_Date<br><b>DIF</b> /dif9:DIF/dif9:Reference/dif9:Publication_Date<br><b>DCAT</b> /dct:issued<br><b>Dryad</b> dcterms:dateSubmitted<br><b>ECHO</b> /*/echo:InsertTime<br><b>ECS</b> /*ecs:CitationforExternalPublication<br><b>EML</b> /eml:dataset/eml:pubDate/eml:text<br><b>FGDC</b> /fgdc:metadata/fgdc:idinfo/fgdc:citation/fgdc:citeinfo/fgdc:pubdate<br><b>ISO</b> //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='publication']/gmd:date/gco:Date<br><b>ISO</b> //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='publication']/gmd:date/gco:DateTime<br><b>ISO-1</b> //cit:CI_Citation/cit:date/cit:CI_Date[cit:dateType/cit:CI_DateTypeCode)='publication']/cit:date/gco:DateTime<br></td></tr><tr><td>[[Concepts_Glossary#Revision_Date|Revision Date]]</td><td>Date of revision of the cited resource</td><td style="word-break:break-all;"><b>ISO</b> //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='revision']/gmd:date/gco:Date<br><b>ISO</b> //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='revision']/gmd:date/gco:DateTime<br><b>ISO-1</b> //cit:CI_Citation/cit:date/cit:CI_Date[normalize-space(cit:dateType/cit:CI_DateTypeCode)='revision']/cit:date/gco:Date<br><b>ISO-1</b> //cit:CI_Citation/cit:date/cit:CI_Date[normalize-space(cit:dateType/cit:CI_DateTypeCode)='revision']/cit:date/gco:DateTime<br></td></tr><tr><td>[[Concepts_Glossary#Creation_Date|Creation Date]]</td><td>Date of creation of the cited resource</td><td style="word-break:break-all;"><b>DIF</b> /dif9:DIF_Creation_Date<br><b>ISO</b> //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='creation']/gmd:date/gco:Date<br><b>ISO</b> //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='creation']/gmd:date/gco:DateTime<br><b>ISO-1</b> //cit:CI_Citation/cit:date/cit:CI_Date[normalize-space(cit:dateType/cit:CI_DateTypeCode)='creation']/cit:date/gco:DateTime<br><b>SERF</b> /serf:SERF_Creation_Date<br><b>netCDF</b> /nc:netcdf/nc:attribute[@nc:name=date_created]/@nc:value<br></td></tr></table></div>
 +
''xPath Note:'' The xPaths included in this table use several wildcards. // means
 +
any path, so //gmd:CI_ResponsibleParty indicates a gmd:CI_ResponsibleParty anywhere
 +
in an XML file. /*/ indicates a single level with several possible elements. This
 +
usually indicates one of several concrete realizations of an abstract object. For
 +
example /*/gmd:identificationInfo could be gmd:MD_Metadata/gmd:identificationInfo
 +
or gmi:MI_Metadata/gmd:identificationInfo and gmd:identificationInfo//*/gmd:descriptiveKeywords
 +
could be gmd:identificationInfo/gmd:MD_DataIdentification/gmd:descriptiveKeywords
 +
or gmd:identificationInfo/srv:SV_ServiceIdentification/gmd:descriptiveKeywords.
 +
 
=Notes=
 
=Notes=
  
 
[[Category:Metadata Implementation]]
 
[[Category:Metadata Implementation]]

Revision as of 10:18, September 21, 2015

I need to document different types of dates.

Overview

Documenting the when of data is often more complicated than the date the resource was created or modified. It is also important to know when the data collection took place and how long the project lasted. Dates are contained in different dialects on different ways.

Implementation

Content Standard for Digital Geospatial Metadata (CSDGM)

Structure

Publish Date <pre<noinclude></noinclude>><citation>

   <citeinfo>
       <origin>/metadata/idinfo/citation/citeinfo/origin</origin>
       <pubdate>/metadata/idinfo/citation/citeinfo/pubdate</pubdate>
       <pubtime>/metadata/idinfo/citation/citeinfo/pubtime</pubtime>
       <title>/metadata/idinfo/citation/citeinfo/title</title>
       <edition>/metadata/idinfo/citation/citeinfo/edition</edition>
       <geoform>/metadata/idinfo/citation/citeinfo/geoform</geoform>
       <serinfo>
           <sername>/metadata/idinfo/citation/citeinfo/serinfo/sername</sername>
           <issue>/metadata/idinfo/citation/citeinfo/serinfo/issue</issue>
       </serinfo>
       <pubinfo>
           <pubplace>/metadata/idinfo/citation/citeinfo/pubinfo/pubplace</pubplace>
           <publish>/metadata/idinfo/citation/citeinfo/pubinfo/publish</publish>
       </pubinfo>
       <othercit>/metadata/idinfo/citation/citeinfo/othercit</othercit>
       <onlink>/metadata/idinfo/citation/citeinfo/onlink</onlink>
       <lworkcit>
           <citeinfo/>
       </lworkcit>
   </citeinfo>

</citation>

Range of Dates <pre<noinclude></noinclude>> <timeinfo>

   <rngdates>
       <begdate>20000101</begdate>
       <enddate>20000202</enddate>
   </rngdates>

</timeinfo>

Single Date <pre<noinclude>> <timeinfo>

   <sngdate>
       <caldate>44440404</caldate>
   </sngdate>

</timeinfo>

Usage

UsageDescription and Xpath
Date of publication for resource being documented/metadata/idinfo/citation/citeinfo/pubdate
Range of dates during which the data was collected/metadata/idinfo/timeperd/timeinfo/rngdates/begdate And /metadata/idinfo/timeperd/timeinfo/rngdates/enddate
Single date during which the data was collected/metadata/idinfo/timeperd/timeinfo/sngdate/enddate

The time information dates can be used to describe the dataset in idinfo/timeperd, the lineage of the dataset in dataqual/lineage and cross references in idinfo/crossref The publish date in the citation can occur wherever citations can occur in a CSDGM record.

Directory Interchange Format (DIF)

Structure

Creation / Revision Date <pre<noinclude></noinclude>><Data_Set_Citation>

   <Dataset_Creator>/DIF/Data_Set_Citation/Dataset_Creator</Dataset_Creator>
   <Dataset_Editor>/DIF/Data_Set_Citation/Dataset_Editor</Dataset_Editor>
   <Dataset_Title>/DIF/Data_Set_Citation/Dataset_Title</Dataset_Title>
   <Dataset_Series_Name>/DIF/Data_Set_Citation/Dataset_Series_Name</Dataset_Series_Name>
   <Dataset_Release_Date>/DIF/Data_Set_Citation/Dataset_Release_Date</Dataset_Release_Date>
   <Dataset_Release_Place>/DIF/Data_Set_Citation/Dataset_Release_Place</Dataset_Release_Place>
   <Dataset_Publisher>/DIF/Data_Set_Citation/Dataset_Publisher</Dataset_Publisher>
   <Version>/DIF/Data_Set_Citation/Version</Version>
   <Issue_Identification>/DIF/Data_Set_Citation/Issue_Identification</Issue_Identification>
   <Data_Presentation_Form>/DIF/Data_Set_Citation/Data_Presentation_Form</Data_Presentation_Form>
   <Other_Citation_Details>/DIF/Data_Set_Citation/Other_Citation_Details</Other_Citation_Details>
   <Dataset_DOI>/DIF/Data_Set_Citation/Dataset_DOI</Dataset_DOI>
   <Online_Resource>/DIF/Data_Set_Citation/Online_Resource</Online_Resource>

</Data_Set_Citation>

Dataset Range of Dates <pre<noinclude></noinclude>><Temporal_Coverage>

   <Start_Date>/DIF/Temporal_Coverage/Start_Date</Start_Date>
   <Stop_Date>/DIF/Temporal_Coverage/Stop_Date</Stop_Date>

</Temporal_Coverage>

Paleo Dates <pre<noinclude></noinclude>><Paleo_Temporal_Coverage>

   <Paleo_Start_Date>/DIF/Paleo_Temporal_Coverage/Paleo_Start_Date</Paleo_Start_Date>
   <Paleo_Stop_Date>/DIF/Paleo_Temporal_Coverage/Paleo_Stop_Date</Paleo_Stop_Date>
   <Chronostratigraphic_Unit uuid="uuid11">
       <Eon>/DIF/Paleo_Temporal_Coverage/Chronostratigraphic_Unit/Eon</Eon>
       <Era>/DIF/Paleo_Temporal_Coverage/Chronostratigraphic_Unit/Era</Era>
       <Period>/DIF/Paleo_Temporal_Coverage/Chronostratigraphic_Unit/Period</Period>
       <Epoch>/DIF/Paleo_Temporal_Coverage/Chronostratigraphic_Unit/Epoch</Epoch>
       <Stage>/DIF/Paleo_Temporal_Coverage/Chronostratigraphic_Unit/Stage</Stage>
       <Detailed_Classification>/DIF/Paleo_Temporal_Coverage/Chronostratigraphic_Unit/Detailed_Classification</Detailed_Classification>
   </Chronostratigraphic_Unit>

</Paleo_Temporal_Coverage>

Usage

UsageDescription and Xpath
Date of publication for resource being documented/DIF/Data_Set_Citation/Dataset_Release_Date
Range of dates during which the data was collected/DIF/Temporal_Coverage/Start_Date And /DIF/Temporal_Coverage/Stop_Date
Paleo Dates/DIF/Paleo_Temporal_Coverage/Paleo_Start_Date And /DIF/Paleo_Temporal_Coverage/Paleo_Stop_Date

EOS Clearinghouse (ECHO)

Structure

Usage

International Organization for Standardization 19115 (ISO19115)

There are many dates included in the ISO Metadata Standards and they have several different types - each with its own characteristics. This page has information about valid formats for those dates.  Ron Lake's blog includes a helpful [description] of time in GML.

Structure

gco:DateTime <pre<noinclude>> <gmd:dateTime>

 <gco:DateTime>2001-01-01T00:00:00</gco:DateTime>

</gmd:dateTime>

gmd:CI_Date <pre<noinclude>> <gmd:CI_Date>

 <gmd:date>
   <gco:Date>2000-01-01</gco:Date>
 </gmd:date>
 <gmd:dateType>
               <gmd:CI_DateTypeCode
                 codeList="http://www.isotc211.org/2005/resources/Codelist/gmxCodelists.xml#CI_DateTypeCode"
                 codeListValue="creation">creation</gmd:CI_DateTypeCode>
             </gmd:dateType>

</gmd:CI_Date>

On-Going Datasets <pre<noinclude>> <gmd:date>

 <gmd:CI_Date>
   <gmd:date gco:nilReason="inapplicable"/>
   <gmd:dateType>
     <gmd:CI_DateTypeCode codeList="http://www.isotc211.org/2005/resources/Codelist/gmxCodelists.xml#CI_DateTypeCode"
       codeListValue="publication">publication</gmd:CI_DateTypeCode>
   </gmd:dateType>
 </gmd:CI_Date>

</gmd:date> ... <gmd:status>

 <gmd:MD_ProgressCode codeListValue="onGoing" codeList="http://www.isotc211.org/2005/resources
   /codeList.xml#MD_ProgressCode">onGoing</gmd:MD_ProgressCode>

</gmd:status> ... <gmd:temporalElement>

 <gmd:EX_TemporalExtent id="boundingTemporalExtent">
   <gmd:extent>
     <gml:TimePeriod gml:id="d1e50">
       <gml:beginPosition>2003-01-18T00:23:00.000Z</gml:beginPosition>
       <gml:endPosition indeterminatePosition="now"/>
     </gml:TimePeriod>
   </gmd:extent>
 </gmd:EX_TemporalExtent>

</gmd:temporalElement>

Relative Times <pre<noinclude>> <gmd:temporalElement>

 <gmd:EX_TemporalExtent>
   <gmd:extent>
     <gml:TimePeriod gml:id="id">
       <gml:duration>P1D</gml:duration>
       <gml:endPosition indeterminatePosition="now"/>
     </gml:TimePeriod>
   </gmd:extent>
 </gmd:EX_TemporalExtent>

</gmd:temporalElement>

Usage

UsageDescription and Xpath
gco:Date/metadata/idinfo/citation/citeinfo
gco:DateTime/metadata/idinfo/crossref/citeinfo
gmd:CI_Date/metadata/dataqual/lineage/srcinfo/srccite/citeinfo

gco:Date


 Date: gives values for year, month and day. Character encoding of a date is a string which shall follow the format for date specified by [ISO 8601]. A full date is formatted as YYYYMMDD or YYYY-MM-DD. This type is used in the following fields:

  • MI_Metadata.dateStamp - The ISO definition of this field is "date that the metadata was created". Common usage seems to be evolving towards "date that the metadata was created or updated" because it seems reasonable that, once the metadata have been updated, the date of that update is more interesting than the original creation date. Also, the update actually creates the current metadata, so this usage seems consistent with the ISO definition.  Note: ISO 19115-1 replaces the datestamp/Date field with the dateinfo/CI_Date object, which enables the type of date to be specified.  ISO 19115-1 expands the date type codelist from 3 values to 16.
  • MD_MaintenanceInformation.dateOfNextUpdate
  • CI_Citation.editionDate - The ISO edition field is synonymous with the concept of version, so this is the release date for the version of the resource being cited.

gco:DateTime


 DateTime: combination of a date and a time type (given by an hour, minute and second). Character encoding of a DateTime shall follow [ISO 8601]. Combined dates and times should be formatted as YYYYMMDDThh:mm:ss, YYYYMMDDThhmmss, or YYYY-MM-DDThh:mm:ss. These representations include no TimeZone indicator, so they are assumed to be local time. YYYYMMDDThh:[mm:ssZ] would indicate universal time.

  • MD_Usage.usageDateTime
  • DQ_Element.dateTime
  • LI_ProcessStep.dateTime
  • MD_StandardOrderProcess.plannedAvailableDateTime

Note: This element was incorrectly defined in the ISO 19139 nschema as an [xs:dateTime]. That type does not allow all of the ISO 8601 options. Specifically, it does not allow the specification of a time range. It will likely be deprecated in the revision of the standard and replaced with stepDateTime.

gmd:CI_Date


This type is used only in the CI_Citation and is the only date type that includes a code from the CI_DateTypeCode codelist. Valid values from the CI_DateTypeCode CodeList are: creation (001), publication (002), and revision (003).  Note: ISO 19115-1 adds an additional 13 date type code list values to the code list.  In figure, see values highlighted in red.

On-Going Datasets


 Many datasets are collected continuously through time. In these cases, a clear publication date does not exist. This situation can be indicated by a combination of "inapplicable" for the citation date, a status code = onGoing, and an endPosition with indeterminatePosition=now:

Relative Times


 In some situations it is necessary to express a time in terms of a fixed period before the present. For example, on-going data processing systems might use observation sets and products from the previous day in the creation of a product for today. In these cases the duration and the end of the time period are known, so the XML looks like:

See [ISO 8601] for information on durations.

Uncertain Times


 The GML elements for describing time positions (beginPosition and endPosition) include the indeterminatePosition attribute that can have values of before, after, now, and unknown. This attribute can be used with a time value to express uncertain times. For example, a dataset that began sometime before 1980 and continues to the present could be described as above.

=Notes

The FGDC Standard allows a variety of date representations including "Free Date" in many locations. A number of assumptions must be made in order to translate these dates into the more structured formats required by ISO. If using XSLT's to translate FGDC date content to ISO (such as the NOAA transforms), logic must be added to achieve the highest level of interoperability and and highest rate of valid ISO input. These assumptions might be based on the length of the FGDC date string:

4 characters = YYYY
6 characters = YYYYMM or bcYYYY
7 characters = 'unknown'
8 characters = YYYYMMDD or bcYYYYMM
10+ characters = bcYYYYMMDD or ccYYYYYYY... or cdYYYYYYY...

This DECODE query is used in selecting dates dates with YYYYMMDD, YYYYMM and YYYY formats from Oracle (other formats are invalid):

decode(LENGTH(DATETIME),6,'<gml:Date>'||SUBSTR(DATETIME,1,4)||'-'||SUBSTR(DATETIME,5,2)||'</gml:Date>',
8,'<gml:Date>'||DATETIME||'</gml:Date>',4,'<gml:Date>'||DATETIME||'</gml:Date>',
'<gml:Date>"'||DATETIME||'" is an invalid date format</gml:Date>')DATETIME

Other assumptions must be made as well such as

 - accounting for non-standard dates
 - accounting for various capilatizations such as 'Unknown' or 'unknown' or 'UNKNOWN' 
 - accounting for common misspellings such as 'unkown', 'unknow' ,'unkwon', etc.
 - accounting for extra spaces
 - accounting for input dates that start with bc, cc, or cd

The indeterminatePosition can be used in a couple of cases if the date is unknown or unpublished

<gml:timePosition indeterminatePosition="unknown">
<gml:timePosition indeterminatePosition="now">

If the date can not be recognized, set it as a reason for having no date:

<gmd:dateTime gco:nilReason="Date Content"/>. 

Of course, many of these will not be valid values for nilReason so they will be identified as problems during record validation. For the NOAA transforms, if a date is missing or cannot not be determined, gco:nilReason will be set to 'unknown' by default. Ending dates of 'present' and 'unknown' (also taking into consideration capitalizations, misspelling, and extra spaces) will use indeterminatePosition.

Crosswalks

Citations in metadata provide can important information about how users should cite the documented resource. These citations are an important part of the identification information and, as such, they are in a special location in the record. In addition, some dialects include citation to other resources and relevant materials that can not be included in the metadata. General citations to external resources that can occur anywhere in the record are described here.

ConceptDescriptionDialect (Fit) Paths
Publication DateDate of publication of the cited resourceDIF /dif9:DIF/dif9:Data_Set_Citation/dif9:Dataset_Release_Date
DIF /dif9:DIF/dif9:Reference/dif9:Publication_Date
DCAT /dct:issued
Dryad dcterms:dateSubmitted
ECHO /*/echo:InsertTime
ECS /*ecs:CitationforExternalPublication
EML /eml:dataset/eml:pubDate/eml:text
FGDC /fgdc:metadata/fgdc:idinfo/fgdc:citation/fgdc:citeinfo/fgdc:pubdate
ISO //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='publication']/gmd:date/gco:Date
ISO //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='publication']/gmd:date/gco:DateTime
ISO-1 //cit:CI_Citation/cit:date/cit:CI_Date[cit:dateType/cit:CI_DateTypeCode)='publication']/cit:date/gco:DateTime
Revision DateDate of revision of the cited resourceISO //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='revision']/gmd:date/gco:Date
ISO //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='revision']/gmd:date/gco:DateTime
ISO-1 //cit:CI_Citation/cit:date/cit:CI_Date[normalize-space(cit:dateType/cit:CI_DateTypeCode)='revision']/cit:date/gco:Date
ISO-1 //cit:CI_Citation/cit:date/cit:CI_Date[normalize-space(cit:dateType/cit:CI_DateTypeCode)='revision']/cit:date/gco:DateTime
Creation DateDate of creation of the cited resourceDIF /dif9:DIF_Creation_Date
ISO //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='creation']/gmd:date/gco:Date
ISO //gmd:CI_Citation/gmd:date/gmd:CI_Date[normalize-space(gmd:dateType/gmd:CI_DateTypeCode)='creation']/gmd:date/gco:DateTime
ISO-1 //cit:CI_Citation/cit:date/cit:CI_Date[normalize-space(cit:dateType/cit:CI_DateTypeCode)='creation']/cit:date/gco:DateTime
SERF /serf:SERF_Creation_Date
netCDF /nc:netcdf/nc:attribute[@nc:name=date_created]/@nc:value

xPath Note: The xPaths included in this table use several wildcards. // means any path, so //gmd:CI_ResponsibleParty indicates a gmd:CI_ResponsibleParty anywhere in an XML file. /*/ indicates a single level with several possible elements. This usually indicates one of several concrete realizations of an abstract object. For example /*/gmd:identificationInfo could be gmd:MD_Metadata/gmd:identificationInfo or gmi:MI_Metadata/gmd:identificationInfo and gmd:identificationInfo//*/gmd:descriptiveKeywords could be gmd:identificationInfo/gmd:MD_DataIdentification/gmd:descriptiveKeywords or gmd:identificationInfo/srv:SV_ServiceIdentification/gmd:descriptiveKeywords.

Notes