Property:DataSystemValueProcess
From Earth Science Information Partners (ESIP)
N
Data in the Data Exchange Standard format are checked for various aspects of formatting and some content QA, and issues are resolved prior to archiving. Consistent data and metadata reporting requirements. +
C
N
Full high-level processing for CERES, MISR and CALIPSO data sets. +
U
Many Unidata sites have elaborate processing systems, but the central Unidata dissemination facilities deliver the data in its original form. +
G
E
Not Given +
3
C
E
A
Processing is done at the AIRNow Data Management Center, using Oracle and other custom code. QA/QC and data integration are the primary processing steps at the DMC. +
E
G
V
The VIEWS data import system accepts data in a variety of formats from various providers and can also automatically download and extract data from known online sources. The import system extracts the data and metadata from its source, examines it for basic integrity issues like duplicate records and mismatched data types, performs various data conversions, transforms the data into an integrated schema (described above), calculates checksums for numeric fields, and validates the resultant data against its source before transferring it to the production data warehouse. Many other processing operations – such as filtering, aggregation, averaging, formatting, renaming, recoding, and outputting – can be done on-the-fly by using the various data access and analysis tools on the website. +
D
The processing of raw data is performed by reusable web-service components, which include filtering, aggregation, and data fusion services. Data processing applications are created by chaining services using workflow software. +
E
Uses the Sparse Matrix Operator Kernel Emissions (SMOKE) modeling system to process emission inventories to the resolutions needed by the air quality models and other data summaries. Uses the Spatial Surrogate Tool (based on the MIMS Spatial Allocator) to create Spatial Surrogates that are an input to SMOKE. Uses the Speciation Tool to create chemical speciation factors for VOC, needed as an input to SMOKE. +
N
R
Yes. RSIG does subsetting, reformatting, conversion, and augmenting of data to facilitate visualization and subsequent analysis with other datasets. Examples include: aggregating across multiple files covering the user-specified date/time range, subsetting to a lon-lat box, striding over dense data points for fast visualization rasterization, decoding integer data to reals, lossless compression/decompression to speed-up transfer over the network, units conversion to SI and model-common units, data reordering to match model-common conventions such as ground-up, augmentation with longitude, latitude and elevation for each data value, and projecting and aggregating into model grid cells. +
H
