Difference between revisions of "Talk:Candidate Technical Topics"

From Earth Science Information Partners (ESIP)
Line 32: Line 32:
  
 
=Delivery of Point Station data=
 
=Delivery of Point Station data=
 +
(see also [[WCS_Server_Software#WCS_Server_for_Station-Point_Data_Type]])
 
* put logic/processing into SQL database as much as possible (views, stored procedures, etc)
 
* put logic/processing into SQL database as much as possible (views, stored procedures, etc)
 
** try to maintain unit tests for this
 
** try to maintain unit tests for this

Revision as of 01:26, August 24, 2011

CF-API

Performance

  • non-compressed data preferred
  • many files vs. single file for queries
    • mapping: many files -> single identifier
      • some concerns that this might be too slow, will have to try and find a sensible balance
      • queries might get very large (the only natural limit is the dataset/identifier)
      • need to limit query size on server side (datafed browser: currently client side management)
   --- discussion: purpose: protect server from unintelligent clients
   --- suggestion: link query restrictions to user connection; registered users could get additional benefits like larger query sizes; email information about tools (API) that allows them to do more intelligent queries
      • query size estimator on server side?
      • client library to chunk queries

Common NetCDF Python Interface, NetCDF4

  • Kari cloned PyNIO interface for Windows, so no problem right now for cross platform development
  • solve other problems first, keep an eye open
  • NetCDF4 might make things more complicated if you try to use all features, might not be easily mappable to WCS concept

Delivery of other data formats, other input formats

  • need to map other formats to WCS and/or CF concept
  • differentiate between format (NetCDF) and convention (CF)
  • chain with WMS server for default views/previews

revision tracking of Datasets

  • always try to get current data when dealing with real time data, always expect your data to be old
  • would be nice to have WCS field for "last updated" date, same for NetCDF/CF (global attribute?)
    • we can make something up on our own for a start
    • try to propose that for CF (and WCS)

Delivery of Point Station data

(see also WCS_Server_Software#WCS_Server_for_Station-Point_Data_Type)

  • put logic/processing into SQL database as much as possible (views, stored procedures, etc)
    • try to maintain unit tests for this
  • need to discuss in more detail how this can be served using WCS (Paul/NILU)

technical Access restrictions to WCS

  • HTTP Basic authentication
  • API key
  • does not have to be 100% secure, more about connecting with the users, knowing who they are and to establish an accepted way of accessing the data
  • firewall whitelisting might be an option for small user groups

Relationship with other Servers

  • write a wrapper to import data formats when needed

WCS 2.0

  • more modular: core and extensions
  • potentially easier to use/implement?
  • proper CF-NetCDF extension coming

Processing Services

extended (Time) filtering

  • day of week, hour of day, day of month,...
  • describe non-standard features in capabilities document?
  • might be difficult to get into official standard?
  • does not/should not interfere with standard if you don't use it