Making Science Data Easier to Use with OPeNDAP
From Earth Science Information Partners (ESIP)
Using Earth Science Data: Why So Difficult?
Complicated data require complicated format
- Network Common Data Form (netCDF)
- Hierarchical Data Format (HDF)
Complicated formats require good tools
Tools, Tools, Tools
EOS HDF Data Readability (without OPeNDAP)
OPeNDAP to the Rescue
Remote Access to Data--at a sub-file level
Data Access Protocol (DAP)
- Presents data in (mostly) format-independent model
- BUT some formats are more equal than others
- NetCDF/CF1
- HDF5/HDF-EOS5, HDF4/HDF-EOS2*
- Tools can use coordinate information from more-equal formats
Examples
N.B.: Do not paste these OPeNDAP URLs into a browser!
In general, this will result in downloading the original file in its entirety. The URLs are meant to be pasted into an OPeNDAP client.
Panoply Examples
- Tropospheric NO2 from OMI for July 7 2010: http://acdisc.gsfc.nasa.gov/opendap/HDF-EOS5/Aura_OMI_Level3/OMNO2e.003//2010/OMI-Aura_L3-OMNO2e_2010m0707_v003-2010m0708t161138.he5
(Does not work in GrADS)
Ozone algorithm comparison
- http://acdisc.gsfc.nasa.gov/opendap/HDF-EOS5/Aura_OMI_Level3/OMTO3e.003/2010/OMI-Aura_L3-OMTO3e_2010m0101_v003-2010m0108t061829.he5
- http://acdisc.gsfc.nasa.gov/opendap/HDF-EOS5/Aura_OMI_Level3/OMTO3d.003/2010/OMI-Aura_L3-OMTO3d_2010m0101_v003-2010m0106t053032.he5
CO2 comparison
- June 2010: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3C2M.005/2010/AIRS.2010.06.01.L3.CO2Std030.v5.6.1.85.X10189104304.hdf
- June 2003: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3C2M.005/2003/AIRS.2003.06.01.L3.CO2Std030.v5.4.12.67.X09261104526.hdf
IDV Examples
Carbon Monoxide Plume from California wildfires, seen in AIRS Daily Level 3 (AIRX3STD)
- 29 Aug: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3STD.005/2009/AIRS.2009.08.29.L3.RetStd001.v5.2.2.0.G09243131454.hdf
- 30 Aug: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3STD.005/2009/AIRS.2009.08.30.L3.RetStd001.v5.2.2.0.G09244141238.hdf
- 31 Aug: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3STD.005/2009/AIRS.2009.08.31.L3.RetStd001.v5.2.2.0.G09245132524.hdf
- 01 Sep: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3STD.005/2009/AIRS.2009.09.01.L3.RetStd001.v5.2.2.0.G09246131354.hdf
- 02 Sep: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3STD.005/2009/AIRS.2009.09.02.L3.RetStd001.v5.2.2.0.G09247131733.hdf
- 03 Sep: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3STD.005/2009/AIRS.2009.09.03.L3.RetStd001.v5.2.2.0.G09248131356.hdf
Hint: Use ascending:CO_Total_Column_A, set range to 1E18 to 2.3E18.
Level 2 AIRS (AIRX2RET) for 01 Sep:
- http://airspar1u.ecs.nasa.gov/opendap/Aqua_AIRS_Level2/AIRX2RET.005//2009/244/AIRS.2009.09.01.202.L2.RetStd.v5.2.2.0.G09245202547.hdf
- http://airspar1u.ecs.nasa.gov/opendap/Aqua_AIRS_Level2/AIRX2RET.005//2009/244/AIRS.2009.09.01.092.L2.RetStd.v5.2.2.0.G09245130401.hdf
- http://airspar1u.ecs.nasa.gov/opendap/Aqua_AIRS_Level2/AIRX2RET.005//2009/244/AIRS.2009.09.01.091.L2.RetStd.v5.2.2.0.G09245130312.hdf
(The seam between granules 91 and 92 (latter two) is because it plots only half the pixel at the edges.)
Ferret/GrADS Example
CO2 and the Jetstream, based on Chahine et al., 2008, Fig.3 Links:
- Monthly CO2 from AIRS for July 2003: http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3C2M.005/2003/AIRS.2003.07.01.L3.CO2Std031.v5.4.12.67.X09261105833.hdf
- Geopotential height from NCEP Reanalysis]: 500 mbar = level 12, July 2003 = time 667: http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis.derived/pressure/hgt.mon.mean.nc
Ferret Commands:
set region/x=0:360 set data "http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3C2M.005/2003/AIRS.2003.07.01.L3.CO2Std031.v5.4.12.67.X09261105833.hdf" shade/lev=(.000364,.000382,.000001) MOLE_FRACTION_OF_CARBON_DIOXIDE_IN_FREE_TROPOSPHERE set data "http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis.derived/pressure/hgt.mon.mean.nc" contour/k=12/L=667/overlay HGT go land
GrADS Commands
sdfopen http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/AIRX3C2M.005/2003/AIRS.2003.07.01.L3.CO2Std031.v5.4.12.67.X09261105833.hdf q file set gxout shaded set clevs 0.000362 .000364 .000366 .000368 .000370 .000372 .000374 .000376 .000378 .000380 set csmooth on d mole_fraction_o run cbarm.gs sdfopen http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis.derived/pressure/hgt.mon.mean.nc q file 2 set gxout contour set cint 100 d hgt.2(t=667,z=6)
Where To Get Tools
- Panoply: http://www.giss.nasa.gov/tools/panoply/
- IDV (Integrated Data Viewer): http://www.unidata.ucar.edu/software/idv/ (Registration required)
- McIDAS-V (IDV sibling): http://www.ssec.wisc.edu/mcidas/software/v/
- Ferret: http://www.ferret.noaa.gov/Ferret/
- GrADS (Gridded Analysis and Display System): http://www.iges.org/grads/
Acknowledgments
- Kent Yang, Joe Lee and the folks at The HDF Group for developing the enhanced HDF handlers
- OPeNDAP.org for developing the OPeNDAP protocol and server
- Tool developers at Goddard Institute for Space Studies (Panoply), Unidata (IDV), U. Wisconsin (McIDAS-V), Pacific Marine Environmental Laboratory (Ferret) and Institute of Global Environment and Society (GrADS)
- Fan Fang at GES DISC for deploying and testing the enhanced HDF handlers