Difference between revisions of "Example CF-NetCDF for Satellite"

From Earth Science Information Partners (ESIP)
Line 5: Line 5:
 
== Total Ozone Mapping Spectrometer (TOMS) satellite ==
 
== Total Ozone Mapping Spectrometer (TOMS) satellite ==
  
This data has been collected by three different satellites from 1979 to present. It is downloadable in text files, linear rectangular 288 * 180 grid.  
+
This data has been collected by three different satellites from 1979 to present. It is downloadable in text files, linear rectangular grid.  
  
 
Nimbus satellite:
 
Nimbus satellite:
Line 12: Line 12:
 
* First 1978-11-01: ftp://toms.gsfc.nasa.gov/pub/nimbus7/data/aerosol/Y1978/L3_aersl_n7t_19781101.txt
 
* First 1978-11-01: ftp://toms.gsfc.nasa.gov/pub/nimbus7/data/aerosol/Y1978/L3_aersl_n7t_19781101.txt
 
* Last 1993-05-06: ftp://toms.gsfc.nasa.gov/pub/nimbus7/data/aerosol/Y1993/L3_aersl_n7t_19930506.txt
 
* Last 1993-05-06: ftp://toms.gsfc.nasa.gov/pub/nimbus7/data/aerosol/Y1993/L3_aersl_n7t_19930506.txt
 +
 +
Grid size 288 * 180
  
 
EPTOMS satellite:
 
EPTOMS satellite:
Line 18: Line 20:
 
* First 1996-07-22: ftp://toms.gsfc.nasa.gov/pub/eptoms/data/aerosol/Y1996/L3_aersl_ept_19960722.txt
 
* First 1996-07-22: ftp://toms.gsfc.nasa.gov/pub/eptoms/data/aerosol/Y1996/L3_aersl_ept_19960722.txt
 
* Last 2007-12-31: ftp://toms.gsfc.nasa.gov/pub/eptoms/data/aerosol/Y2005/L3_aersl_ept_20051231.txt
 
* Last 2007-12-31: ftp://toms.gsfc.nasa.gov/pub/eptoms/data/aerosol/Y2005/L3_aersl_ept_20051231.txt
 +
 +
Grid size 288 * 180
 +
 +
As you can see, there is a gap between 1993-05-06 and 1996-07-22.
  
 
OMI satellite:
 
OMI satellite:
Line 24: Line 30:
 
* Year 2011, growing: ftp://toms.gsfc.nasa.gov/pub/omi/data/aerosol/Y2011
 
* Year 2011, growing: ftp://toms.gsfc.nasa.gov/pub/omi/data/aerosol/Y2011
  
As you can see, there is a gap between 1993-05-06 and 1996-07-22.
+
Grid size 360 * 180
  
 
The python module AI_data.py contains the templates for ftp data urls, like
 
The python module AI_data.py contains the templates for ftp data urls, like

Revision as of 12:24, July 6, 2011

Back to WCS Access to netCDF Files

A Real life example how to download and store satellite data.

Total Ozone Mapping Spectrometer (TOMS) satellite

This data has been collected by three different satellites from 1979 to present. It is downloadable in text files, linear rectangular grid.

Nimbus satellite:

Grid size 288 * 180

EPTOMS satellite:

Grid size 288 * 180

As you can see, there is a gap between 1993-05-06 and 1996-07-22.

OMI satellite:

Grid size 360 * 180

The python module AI_data.py contains the templates for ftp data urls, like

   template_path_omi = '/pub/omi/data/aerosol/Y%Y/L3_aersl_omi_%Y%m%d.txt'
   first_omi_datetime = datetime.datetime(2004, 9, 6)

The %Y %m %d are python format codes, 4-digit year, 2-digit month, 2-digit day.

a programmer can now get the url:

   AI_data.determine_ftp_path(datetime.datetime(2010, 3, 24)

returns '/pub/omi/data/aerosol/Y%Y/L3_aersl_omi_20100324.txt'

Creating an Empty CF-NetCDF File

NetCDF Markup Language (NCML)

NCML documentation

We describe AerosolIndex.nc CF-NetCDF file in xml file AerosolIndex.ncml

The first line is the root element and namespace

   <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">


The explicit means, that all the metadata is here. Alternatively, readMetadata would require an existing source netcdf file.

       <explicit />

Global attributes. Notice, that NCML does not require CF conventions. Therefore, you have to declare the convention yourself.

       <attribute name="title" type="string" value="NASA TOMS Project" />
       <attribute name="comment" type="string" value="NASA Total Ozone Mapping Spectrometer Project" />
       <attribute name="Conventions" type="string" value="CF-1.0" />

Declare dimensions. This is a 3-dimensional grid, with time as the unlimited dimension.


       <dimension name="time" length="0" isUnlimited="true" />
       <dimension name="lat" length="180" />
       <dimension name="lon" length="288" />


       <variable name="time" type="int" shape="time">
           <attribute name="standard_name" type="string" value="time" />
           <attribute name="long_name" type="string" value="time" />
           <attribute name="units" type="string" value="days since 1979-01-01" />
           <attribute name="axis" type="string" value="T" />
       </variable>
       <variable name="lat" type="double" shape="lat">
           <attribute name="standard_name" type="string" value="latitude" />
           <attribute name="long_name" type="string" value="latitude" />
           <attribute name="units" type="string" value="degrees_north" />
           <attribute name="axis" type="string" value="Y" />
           <values start="-89.5" increment="1" />
       </variable>
       <variable name="lon" type="double" shape="lon">
           <attribute name="standard_name" type="string" value="longitude" />
           <attribute name="long_name" type="string" value="longitude" />
           <attribute name="units" type="string" value="degrees_east" />
           <attribute name="axis" type="string" value="X" />
           <values start="-179.375" increment="1.25" />
       </variable>
       <variable name="AI" type="float" shape="time lat lon">
           <attribute name="long_name" type="string" value="Aerosol Index" />
           <attribute name="units" type="string" value="fraction" />
           <attribute name="_FillValue" type="float" value="NaN" />
           <attribute name="missing_value" type="float" value="NaN" />
       </variable>
   </netcdf>

Create Script AI_create.py

   from datafed import cf1
   
   def create():
       cf1.create_ncml22('AerosolIndex.nc', 'AerosolIndex.ncml', '64bitoffset')
   
   if __name__ == "__main__":
       create()

Download Procedure

Make URL from the Template

Download the Text File

Compile the Text File into an rectangular array

Append the time slice