Talk:Candidate Technical Topics

From Earth Science Information Partners (ESIP)

-- Michael Decker (MDecker) 10:03, 23 August 2011 (MDT)

CF-API

Performance/Virtual Datasets

  • non-compressed data preferred
  • many files vs. single file for queries
    • mapping: many files -> single identifier
      • some concerns that this might be too slow, will have to try and find a sensible balance
      • queries might get very large (the only natural limit is the dataset/identifier)
      • need to limit query size on server side (datafed browser: currently client side management)

Common NetCDF Python Interface, NetCDF4

  • Kari cloned PyNIO interface for Windows, so no problem right now for cross platform development
  • solve other problems first, keep an eye open
  • NetCDF4 might make things more complicated if you try to use all features, might not be easily mappable to WCS concept

Delivery of other data formats, other input formats

  • need to map other formats to WCS and/or CF concept
  • differentiate between format (NetCDF) and convention (CF)
  • chain with WMS server for default views/previews

Tracability and revision tracking of Datasets

  • always try to get current data when dealing with real time data, always expect your data to be old
  • would be nice to have WCS field for "last updated" date, same for NetCDF/CF (global attribute?)
    • we can make something up on our own for a start
    • try to propose that for CF (and WCS)

Delivery of Point Station data

  • put logic/processing into SQL database as much as possible (views, stored procedures, etc)
    • try to maintain unit tests for this
  • need to discuss in more detail how this can be served using WCS (Paul/NILU)

technical Access restrictions to WCS

  • HTTP Basic authentication
  • API key
  • does not have to be 100% secure, more about connecting with the users, knowing who they are and to establish an accepted way of accessing the data
  • firewall whitelisting might be an option for small user groups

Relationship with other Servers

  • write a wrapper to import data formats when needed

WCS 2.0

  • more modular: core and extensions
  • potentially easier to use/implement?
  • proper CF-NetCDF extension coming

Processing Services

extended (Time) filtering

  • day of week, hour of day, day of month,...
  • describe non-standard features in capabilities document?
  • might be difficult to get into official standard?
  • does not/should not interfere with standard if you don't use it