Talk:Candidate Technical Topics

The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

General

in order to persuade nodes to use server package, we need reference implementation(s) demonstrating that it works and is performant. Good news: we are pretty far along this road!

CF-API

need to read and write CF-compliant files easily
- python interface for unidata libcf? http://www.unidata.ucar.edu/software/libcf/

Performance

non-compressed data preferred
many files vs. single file for queries
- mapping: many files -> single identifier
  - some concerns that this might be too slow, will have to try and find a sensible balance
  - queries might get very large (the only natural limit is the dataset/identifier)
  - need to limit query size on server side (datafed browser: currently client side management)

   --- discussion: purpose: protect server from unintelligent clients
   --- suggestion: link query restrictions to user connection; registered users could get additional benefits like larger query sizes; email information about tools (API) that allows them to do more intelligent queries

- - query size estimator on server side?
  - client library to chunk queries

Common NetCDF Python Interface, NetCDF4

Kari cloned PyNIO interface for Windows, so no problem right now for cross platform development
solve other problems first, keep an eye open
NetCDF4 might make things more complicated if you try to use all features, might not be easily mappable to WCS concept

Delivery of other data formats, other input formats

need to map other formats to WCS and/or CF concept
differentiate between format (NetCDF) and convention (CF)
chain with WMS server for default views/previews

revision tracking of Datasets

always try to get current data when dealing with real time data, always expect your data to be old
would be nice to have WCS field for "last updated" date, same for NetCDF/CF (global attribute?)
- we can make something up on our own for a start
- try to propose that for CF (and WCS)

--- related issue: intelligent harvesting of updates for catalogues (GI-cat); add "modification_time" to GetCapabilities metadata and allow for "updated_since <date>" request in DescribeCoverage. Note: this may be beyond current WCS standard specification.

Delivery of Point Station data

put logic/processing into SQL database as much as possible (views, stored procedures, etc)
- try to maintain unit tests for this
need to discuss in more detail how this can be served using WCS (Paul/NILU)

--- discussion: new OGC standard for netcdf/CF1.6 allowing for representation of point data. Different views on the same data depending on who (which client) wants to access data: geodynamical fluid view = WCS; detailed description of feature = WFS; sensor based view = SOS

--- really important workshop outcome will be to outline the architecture of a future AQ network: what are server capacities? what are the requirements for the clients? Do we need/want brokers that can translate requests (for example an SOS request into a WCS/WFS request) or should this become a part of the server?

--- netcdf output of station data: see CF-netCDF-ExtensionFor-netCDF Data Model

technical Access restrictions to WCS

HTTP Basic authentication
API key
does not have to be 100% secure, more about connecting with the users, knowing who they are and to establish an accepted way of accessing the data
firewall whitelisting might be an option for small user groups

--- discussion: this is about sharing and exposing data - hence, data that shall not be openly accessible should not be put on the network in the first place. Unfortunately, reality is more complex: some AQ data are restricted and we risk loosing a lot of (free) data if we can't also accomodate for at least some restricted data.

--- more important in the short-term is user tracking: might allow for more specific services (bandwidth etc.), alerts, guidance, etc. - also important to demonstrate service use and convince funding agencies.

Relationship with other Servers

write a wrapper to import data formats when needed

--- need to define which other servers we want to connect to.

WCS 2.0

more modular: core and extensions
potentially easier to use/implement?
proper CF-NetCDF extension coming

Processing Services

idea: community provides online processing service for their discipline
- not part of WCS, but separate service
- protocol: WPS (web processing service, http://www.opengeospatial.org/standards/wps)

extended (Time) filtering

day of week, hour of day, day of month,... (including ranges)
describe non-standard features in capabilities document?
might be difficult to get into official standard?
does not/should not interfere with standard if you don't use it

time zone support

server should be able to reprocess time axis time zone according to user request
- no option to do this on the user side as this will lead to too many mistakes made
- should be relatively easy to simply return time axis using the time zone from the request