View source for IGACO Framework Architecture

==[ftp://ftp.wmo.int/Documents/PublicWeb/arep/gaw/gaw159.pdf IGACO Report]: 5.1 A Global System is Required==
'''''GR1 Establishment: An Integrated Global Atmospheric Chemistry Observation System
(IGACO) should be established for a target list
of atmospheric chemistry variables and
ancillary meteorological data.'''''

The Integrated Global Atmosphere Chemistry
Observation System (IGACO) proposed in this report
consists of three measurement components (groundbased,
aircraft and satellite), and a modelling component
to integrate the observations into a global picture
of atmospheric composition and its trends. In addition,
there are the crucial additional elements of qualityassurance,
data distribution and data archiving/
analysis. The proposed system is summarised in
Section 5.2.

The proposed system is similar to that used by the
meteorological community, which has a long history:
the meteorological system was developed by the
national hydrological and meteorological services and
space agencies, coordinated by the WMO, to integrate
measurements of meteorological variables important
to weather prediction and climate. Today a system of
non-satellite networks and satellites delivers
observations of physical variables such as temperature,
pressure, wind speed and humidity in real time
through a Global Telecommunications System (GTS) to
forecast modelling centres around the World. It also
delivers a more comprehensive data set consisting of
real-time and non-real-time observations to a data
archive, where it is accessible for re-analysis using
models at the NCEP, ECMWF and JMA. Using
sophisticated mathematical descriptions of
atmospheric processes and the most powerful
computers available, these models serve to integrate
the observations from a variety of sources into a
comprehensive picture of current and projected global
weather patterns.

Establishment of an integrated observational system
for chemical variables lags, by several decades, the
observational system for physical variables. Now, early
in the 21st century, all the components are either in
place or, with careful planning,will be readily available
at reasonable cost within the next 10 years. What is
missing is a well-focused strategic plan and an
internationally coordinated implementation effort,
which it is the purpose of this report to provide.
The authors and the reviewers of this document agree
that the establishment of IGACO to observe the
composition of the atmosphere and its changes is
necessary, timely, and feasible. It will be necessary to
start planning immediately for a necessary minimum
of satellites, ground stations and routine aircraft
programmes, with the required modelling support, to
produce IGACO.

'''''GR2 Continuity: the data products from satellite and non-satellite instruments, which are to be integrated into a global picture by IGACO,must have assured long-term continuity'''''.

In order to quantify the changing atmospheric
composition, an overarching and fundamental
requirement for IGACO is that continuity of
measurement is maintained for all the system
components.

'''''GR3 Management of IGACO. The responsibility for the co-ordination and implementation of IGACO should rest with a single international
body'''''.

International and national agencies
responsible for aspects of IGACO should be
committed partners and agree on their
appropriate responsibilities.

It is not yet clear where the responsibility for the
development of IGACO in its entirety should lie. The
current international meteorological, environmental
and space agencies all have different mandates with
respect to the generation of the data needed, and no
one body is responsible for observing the changing
atmosphere. This is a critical management issue to be
dealt with in the implementation phase of IGACO,
which is discussed further in Section 5.3.

==5.2 The Proposed Architecture for IGACO==

[[Image:IGACO Framework.png|400px|left]] Fig. 5.1: IGACO Framework. It should be emphasised that, although various components and elements of the IGACO system are presently available or projected, a complete system does not yet exist for any atmospheric constituent in the target list of variables (Tables 4.2 and 4.3). Essential to the proper functioning of IGACO is a system for data collection from various sources, a system for distribution of the data to users and of archiving these data for establishing long-term records, as well as an end-to-end quality-assurance and qualitycontrol system to quantify the uncertainties in the data.

===5.2.1 Measurements===
'''''GR4 Gaps in observational coverage: for each target species and variable, the present gaps in the current spatial and temporal coverage should be filled by extending the existing measurement systems.'''''

The proposed measurement system comprises an optimised network of ground-based stations, routine aircraft flights and satellites in a configuration suitable for addressing the temporal and spatial variability of a specified chemical variable. Since observations are made with a variety of instruments, which vary in their spatial and temporal resolutions, an efficient mathematical approach to assimilating them into a global picture is necessary to define an optimal mix of each instrument type.Currently, there are major gaps in the global coverage of observations for all the target chemicals.

===5.2.2 Quality-assurance and Data-handling protocols===

'''''GR5 Long-term validation of satellite observations: in order to ensure accuracy and consistency for satellite measurements, sustained qualityassurance measures, over the entire lifetime of the observations, are essential.'''''

There are presently large gaps in the quantitative information for each chemical parameter about the comparability of observations by different instruments, which limits our ability to merge observations into a global data set. Quality assurance is the only effective way to ensure the comparability of data. Three elements of quality assurance are essential in the measurement/analysis/assessment hierarchy: 
* Data Quality Objectives (DQO): defined by
trueness, precision, completeness, comparability
and representativeness.
* Quality Control (QC): achieved through
calibration/validation and traceability to globally
harmonised standards and reference methods,
instrument and method intercomparison, good
laboratory practice, etc.
* Quality Assessment (QA): achieved through
independent verification (e.g. system and
performance audits for ground-based stations).

For most ground-based and routine aircraft
observations of the gaseous species, the quality
assurance of measurements is already achievable using
internationally traceable standard reference materials
or reference methods, routine inter-laboratory
comparisons, instrument calibration programmes and
data analysis aimed at ensuring that the qualityassurance
criteria are being met. However, in practice,
the effectiveness of delivery of quality assurance in a
measurement programme is patchy at best. It is limited
by the resources needed for key quality-assurance
facilities and functions.

For satellite-borne sensors the quality-assurance
commonly referred to as calibration/validation is more
complicated. It involves the evaluation of the retrieval
algorithms used to extract geophysical quantities from
calibrated and well-characterised basic data products.
Calibration/validation is being addressed by the
Working Group on Calibration and Validation of the
CEOS subgroup on atmospheric composition. A policy
for validating data for satellite ozone and chemistry
missions was suggested in WMO/CEOS Report No. 140.
The components in the end-to-end calibration/
validation of satellite data are:

1. Pre-launch calibration. The purpose of
calibration is to assign the correct physical
quantities to instrument output. This
encompasses a precise characterisation of the
radiometric instrument response, including
conversion of the measured signal to atmospheric
radiances, as well as the accurate determination of
the spectral domain and resolution, the viewing
direction and, in some cases, the polarisation
sensitivity. The uncertainties of these parameters
and their sensitivity to environmental conditions
needs to be established. For a large class of
instruments, the need to carry out calibration
under vacuum conditions encountered in space
constitutes an important requirement. Calibration
requires standards and measurements provided
by the recognised national standards institutions
to be known and traceable.

2. In-flight calibration is needed to quantify
changes in the instrument induced by launch
vibrations, outgassing upon deployment in space,
thermal variations in orbit, ageing and radiation
damage to electrical and optical components. It
should include radiometric calibration using
internal sources, solar radiation or deep-space
measurements, spectral calibration with internal
sources or solar or terrestrial spectroscopic
measurements, pointing calibration, using welldefined
terrestrial or celestial references. Usually,
special calibration modes are required, which
form part of the instrument design and the
operational in-orbit scenario.
It must be recognised that both in-flight and onground
calibration will have inherent limitations.
For example, on-ground calibration in vacuum will
put limitations on the range and type of
measurements that can be carried out, and
terrestrial sources available will not always be able
to simulate the instrument input prevalent in
space. On the other hand, in space independent
radiometric and spectral sources are not always
available (for example, a source of known
polarisation state).Therefore, calibration is often a
compromise between what is needed ideally and
what is achievable in practice.

3. Retrieval algorithm consistency. Multiple
satellite systems will be in orbit measuring the
same constituent (e.g. O3 from NPOESS and
METOP). Every effort should be made to have a
common or well-compared radiative transfer
model and the same spectroscopic databases to
ensure that these fundamental parameters do not
cause biases. The respective algorithms for each
system have a different heritage and physical
basis. It is highly desirable to have a uniform data
set among the systems so as to avoid biases
between them. Comparisons of the algorithms
using a common radiance data set should be
conducted as part of the cal/val programme or
even sooner.

4. Geophysical validation. Trace gas or aerosol data
retrieved from spaceborne sensors are validated
by comparison to the most reliable correlative
data. Simultaneity and co-location of the
measurements must be optimised, and any
remaining sampling differences must be bridged
using, for example, CTMs. Ideally, validation
sources should be based on a different
measurement principle or different spectroscopic
features; they include ground-based networks
and airborne measurements, sometimes
complemented by chemical models to get access
to additional species. Profile data covering the
stratosphere and upper troposphere can often
only be validated by scientific instruments
operated on a campaign basis, e.g. on
stratospheric balloons and aircraft. Intense
geophysical validation campaigns are needed in
the early phase of a space mission, followed by
periodic short measurement campaigns and
continuous use of routine observation networks,
to monitor instrument performance throughout
its lifetime.

5. Inter-satellite comparison. Observations of old
and new satellite instruments require intercomparison
in order to establish consistent longterm
datasets, despite possible discrepancies
between absolute values of data.
Apart from their necessary role within IGACO, groundbased
networks are essential components in the
validation of satellite observations.

'''''GR6 Validation of vertical-profile data from satellite observations: a set of high performance scientific instruments using ground, aircraft and balloon platforms, possibly operated on campaign basis, must be maintained to provide the crucial validation data.'''''

Validation of satellite data also requires a component
outside the long-term continuous observation system
addressed in this report:

'''''GR7 Comparability: the ability to merge observations of different types must be ensured by insisting that appropriate routine calibration and comparison activities linking diverse measurements together are part of an individual instrument measurement programme.'''''

In order for the full power of diverse observations of
any parameter to be fully realised, the quality and
comparability of a measurement require that the data
quality objectives are routinely met, and that they be
available with the data.This is an essential requirement
of IGACO.

'''''GR8 Distribution of data: universally recognised distribution protocols for exchange of data on atmospheric chemical constituents should be established.'''''


The data and associated quality-assurance information
need to be in such a format that they are easily
accessible in real time (for assimilation in forecast
models) or on time scales of months to a year for
integration into a comprehensive data archive using
model integration techniques. Data distribution
protocols are needed that allow for effective
transmission of observations together with their
associated quality-assurance information to an
established archive and to forecast modellers. The
Global Telecommunications System (GTS) of WMO is an
example of such a system for real-time meteorological
data.

===5.2.3 World Integrated Data Archive Centres (WIDAC)===

'''''GR9 Multi-stake holder World Integrated Data Archive Centres (WIDAC) should be established for the targeted chemical variables.''' 


'''''GR10 Storage for raw data should be established so that they can be re-interpreted as models and understanding improves.'''''

In the IGACO system, observations should be made
available though a World Integrated Data Archive
Centre (WIDAC) for future applications and re-analysis
(see Fig. 5.1).The need for re-analysis will recur as more
sophisticated tools and improved observational data
become available. A WIDAC must be sufficiently robust
that it can last for many years.
The optimal configuration of the WIDAC will have to be
decided, but it may be centralised or distributed. The
meteorological community has such an archive for a
short list of meteorological parameters, but is facing
major challenges to extend it to other parameters such
as water vapour. Furthermore, despite the maturity of
meteorological data coordination, there is still no
WIDAC for meteorological data that has free access.
There is currently no potential WIDAC for any chemical
constituent on the target list. World Data Centres do
exist for individual network or satellite observations for
particular variables on the target list, but there is no
integration at present. For instance, GAW has a World
Ozone and UV Data Centre for non-satellite
observations and many organisations have surface
ozone observations. The satellite agencies have ozone
data centres for observations from specific satellite
instruments, and there have been releases of qualitycontrolled
data with suitable software tools to
manipulate them; these include updates as funding
allows. To bring all this information together in a
synergistic way is the challenge that IGACO intends to
meet.

Stakeholders in a WIDAC are: 
# providers of measurement data,
# data analysis and model groups,
# users of the integrated data set. 

A WIDAC should have 
periodic updates of observations and reanalysis built
into it, so as to cope with the continuous improvement
in satellite retrieval algorithms and revisions of nonsatellite
observations. Access to data should be free of
charge and expedited by software tools that are userfriendly.
5.2.4 Models and model inputs needed for the
IGACO system

'''''GR11 The development of comprehensive chemical modules in weather and climate models should be an integral part of IGACO.'''''

As the preceding sections illustrate, atmospheric
models that encompass the target constituents are
needed for use in forecasting air quality and for reanalysis
of the global comprehensive database in the
WIDAC to produce a 4-D composition distribution.
Data assimilation tools have started to provide
consistent distributions of stratospheric ozone and
water vapour on a routine basis. They now need to be
improved and extended to include other species on
the IGACO target list. The assimilation of all of these
species into the weather and climate models which
include a full chemical scheme will allow for more
confident predictions of the distributions of many
other compounds not on the target list.
GR12 Strong coordination with meteorological
services is essential so that the ancillary
meteorological data required by IGACO is
accessible.

The requirements for numerical weather prediction
have been mandated to and are addressed by the
national meteorological services represented by the
WMO. The essential ancillary data for IGACO listed in
Chapter 4 is, in principle, available from them.However,
some coordination is needed to ensure that the data is
in fact freely available.

==[http://www.eorc.jaxa.jp/Feb2004_Meetings/IGOS/report.pdf IGOS Workshop] IGOS Framework on Data==
B.7	Data Integration - Prof. Toshio Koike (Univ. of Tokyo)


Key issues identified for Theme implementation

===Data policies===
*	coordination/harmonization by relevant international organizations 
*	data accessibility
*	data timeliness
*	data provider's privilege for R&D 
*	acknowledgement
*	metadata (documentation on the data)

====Standards, practices and calibration====
*	standard format or conversion function based on data description
*	quality control 
*	real time or delayed mode

====Transformation of measurements into information====
*	combination with many other data sources: natural science - socio-economic 
*	too voluminous and heterogeneous
*	prediction needs:
*	Information sharing worldwide

===Key recommendations===

To IGOS-P or related entities:
* 	Establish a Data Integration Function, distributed/centerized, in each Theme and encourage collaboration among Themes;
* 	Promote co-operation between the IGOS Themes and Global Mapping for strengthening the Theme impact on socio-economical aspects;
* 	Promote collaboration among research communities and operational organizations for establishment of sustained Data Integration systems.

===To GEO, sponsoring bodies etc:===
* 	Develop a ten year implementation plan for Data Integration by considering perspectives on computing power and data amount;
* 	Emphasise the combination of in-situ data/satellite data /model output with socio-economic data for contributions to sound decision-making;
* 	Promote collaboration among research communities and operational organisations for establishment of sustained Data Integration systems.


===[[IGACO Data Assimilation]]===

[[Category:Interoperability]][[Category:GEOSS]]