Unidata IDD Data System

<Back to Data Summit Workspace <All Data Systems
Edit with Form or Submit Word Doc

General

Contact

Data System Name: Unidata Internet Data Distribution (IDD) System
Data System URL: http://www.unidata.ucar.edu/software/idd/ and http://www.unidata.ucar.edu/software/ldm/
Contact Person: Ben Domenico
Contact e-mail: ben@unidata.ucar.edu

About the Data System (Purposes, Audience)

[[About::The Unidata community of over 150 universities has built a system for disseminating real-time earth observations via the Internet. Unlike other systems, which are based on data centers where the information can be accessed, the Unidata IDD is designed so a university can request that certain data sets be delivered to computers at their site as soon as they are available from the observing system. The IDD system also allows any site with access to specialized observations to inject the dataset into the IDD for delivery to other interested sites. Many of the 250 participating sites are receiving about 2GB of real-time data per hour via the IDD -- including weather forecasts, weather station obs, data from all NEXRADS, GOES satellite imagery, lightning strike data, aircraft-borne measurements, etc. The IDD recently surpassed FTP in terms of the amount of data delivered on the Internet 2. These datasets range from huge NCEP model ouputs run on an hourly basis to data from NEXRAD radars which come in at a rate of dozens of products per second. Early experiments are underway with the DataFed group for using the underlying Unidata Local Data Manager (LDM)to deliver air quality data in real time]]

Presentation

http://www.unidata.ucar.edu/staff/mohan/Unidata Presentations/Unidata Members Meeting 2006.ppt , http://www.unidata.ucar.edu/staff/mohan/OU NWC%20Seminar.pdf

History

The Unidata IDD has been in operation on a 24x7 basis since 1995 when it replaced a real-time data delivery system that had been used a sideband on satellite TV broadcasts that were decoded by special receivers connectd to C-band receivers at user sites. The switch to IDD did away with the subscription fee which made it possible for many more sites to receive a greatly expanded menu data streams. In the last few years, the reach of the IDD has been expanded internationally and it is now being used to ship real time data in all directions among Central and South America, Europe, and a few sites in the Far East. The underlying LDM technology has been adopted by a number of different organizations to collect and disseminate their own data. Among them are the US National Weather Service which has established a collection system for the level II and level III radar data from all the NEXRAD sites. This replace the old tape-based system, greatly reducing the cost while dramatically increasing the reliability and cutting the collection time at NCDC to virtually zero. Using Unidata LDM technology , the Spanish and South Korean Weather Services have established their own internal IDD systems for internal data distribution. The THORPEX TIGGE groups are using this technology to move data among archive centers in China, Europe, and the US. The TIGGE ensemble forecasts amount to about 20 GB per hour moving on a regular basis among these centers.

Agencies

NSF, NOAA, and possibly EPA

Datasets Served

Several different types with many options within each type. Please see: http://www.unidata.ucar.edu/data/ for more details

<tbody></tbody>

Real-time Model Data

<tbody> </tbody> <a href=""><img src="" alt="NCEP data displayed in GEMPAK" style="border: 1px solid rgb(0, 0, 0);" border="0" height="83" width="105"></a> <tbody> </tbody> • <a href="http://www.unidata.ucar.edu/data/ncep_data_2.html">NCEP</a>: National Centers for Environmental Prediction • High Resolution <a href="http://www.unidata.ucar.edu/data/conduit_data_2.html">CONDUIT</a> output • <a href="http://www.unidata.ucar.edu/data/fnmoc_data_2.html">FNMOC</a>: Fleet Numerical Meteorology and Oceanography Center • <a href="http://www.unidata.ucar.edu/data/cmc_data_2.html">CMC</a>: Canadian Meteorological Centre

<a name="image"></a>Real-time Remote Sensing Image Data

<tbody> </tbody> <a href=""><img src="" alt="Satellite data displayed in GEMPAK" style="border: 1px solid rgb(0, 0, 0);" border="0" height="83" width="105"></a> <tbody> </tbody> <a href="http://www.unidata.ucar.edu/data/satellite_data_2.html">Satellite data</a> • GOES-West • GOES-East   <a href=""><img src="" alt="Radar data displayed in GEMPAK" style="border: 1px solid rgb(0, 0, 0);" border="0" height="83" width="105"></a> <tbody> </tbody> <a href="http://www.unidata.ucar.edu/data/radar_data_2.html">Radar data</a> • <a href="http://www.unidata.ucar.edu/data/levelII_data_2.html">Level II</a> (formerly <a href="http://www.unidata.ucar.edu/software/craft/">CRAFT</a>) • <a href="http://www.unidata.ucar.edu/data/levelIII_data_2.html">Level III (NIDS) Radar Products</a>

<a name="point"></a>Real-time Point Data

<tbody> </tbody> <a href=""><img src="" alt="Lightning data displayed in GEMPAK" style="border: 1px solid rgb(0, 0, 0);" border="0" height="83" width="105"></a> <tbody> </tbody> <a href="http://www.unidata.ucar.edu/data/lightning_data_2.html">Lightning</a>   <a href=""><img src="" alt="Wind profiler data displayed in Garp" style="border: 1px solid rgb(0, 0, 0);" border="0" height="83" width="105"></a> <tbody> </tbody> <a href="http://www.unidata.ucar.edu/data/profiler_data_2.html">Wind Profiler</a>   <a href=""><img src="" alt="ACARS data displayed in GEMPAK" style="border: 1px solid rgb(0, 0, 0);" border="0" height="83" width="105"></a> <tbody> </tbody> <a href="http://www.unidata.ucar.edu/data/acars_data_2.html">ACARS</a> Aircraft Communications Addressing and Reporting System   <a href=""><img src="" alt="SuomiNet data displayed in GEMPAK" style="border: 1px solid rgb(0, 0, 0);" border="0" height="83" width="105"></a> <tbody> </tbody> <a href="http://www.unidata.ucar.edu/data/suomi_data_2.html">SuomiNet</a> An array of GPS sensors collecting real time precipitable water vapor and total electron content of the ionosphere

<a name="text"></a>Real-time Text Data

<tbody></tbody> <a href=""><img src="" alt="METAR data" style="border: 1px solid rgb(0, 0, 0);" border="0" height="83" width="105"></a> <tbody></tbody> • <a href="http://www.unidata.ucar.edu/data/text_data_2.html">More about text data</a> • Non-exhaustive <a href="http://www.unidata.ucar.edu/data/text_prods_list.html">list of textual products available</a> • Text data examples: • <a href="">Operational Forecast Discussions</a> • <a href="">NWS Severe Weather Watches & Warnings</a> • <a href="">Aviation Products</a> • <a href="">Metars</a> • <a href="">Maritime Data</a>

Parameters

Not Given

Spatial - Temporal Coverage

Not Given

---

Health

Not Given

Forecasting and Reanalysis

Not Given

Model/Emissions Evaluation

Not Given

Characterization, Trends, Accountability

Not Given

Other

Not Given

Data System IT

Primary/Official Store for Some data

Not Given

Data Consolidation/integration

Not Given

Providing Data Access to users/externals

Not Given

Data Processing

Not Given

Visualization/Analysis

Not Given

Decision Support (e.g. some integration into user business process)

Not Given

End-to-End Integration

Not Given

Other DS Values

Not Given

Data Access and/or Output Interoperability

Not Given

Reusable Tools and Methods

Not Given

Security Barriers and Solutions

Not Given

User Feedback Approach

Not Given

Other Architecture

Not Given