Difference between revisions of "WCS Wrapper Configuration"

Latest revision as of 11:29, August 27, 2010

Back to WCS Wrapper

Project on SourceForge

This page describes how to install the information for humans: Home pages, contact information etc.

For data Configuration go to following pages:

Configuring NetCDF based Cube Data

Configuring SQL based Point Data

Structure of OWS/web

OWS/web is for system developers only.

OWS/web/static contains static web content. You can put any documentation here and it will be served as a web page or download. The home page index.html is pretty much mandatory, and you should change favicon.ico to reflect your organization. We highly recommend, that you customize documents for your WCS service.

OWS/web/static/cache is a folder for temporary files. The service uses it for output files. Anything you put there will be deleted when space is needed.

The installation contains an example datasets OWS/web/static/testprovider and OWS/web/static/point. The testprovider is a demo NetCDF dataset, point is an example how to server point data from a SQL database. Every service will have a folder with the same name here. So create yourself a folder datasets OWS/web/static/myservice and your service URL will be at http://localhost:8080/myservice

The Human Interface: Create the index.html Front Pages for Visitors.

If no query is present, the server gives a default page index.html. You should create these pages for your server and for all the providers.

The server index.html is at OWS/web/static/index.html, which will be displayed from url http://128.252.202.19:8080/ at test server or at localhost if you have a server running.

Every provider folder should also have an index.html like OWS/web/static/testprovider/index.html which will be displayed from http://128.252.202.19:8080/HTAP at test server or at localhost.

Enter Provider Metadata

Every provider should have wcs_capabilities.conf that lists keywords and contact information. The format is simple, copy one from the testprovider and edit it.

   # this file provides some information about the provider
   # and is incorporated into the respective WCS responses.
   # all currently available field identifiers are listed below.
   # please define every identifier only once.
   # other identifiers will be ignored, input is case sensitive.
   # the format is always <identifier>: <value>.
   # whitespaces before and after <value> will be stripped.
   # KEYWORDS can take a comma separated list that will then be
   # included in the respective keyword tags
   # empty lines and lines starting with "#" will be ignored.
   PROVIDER_TITLE: National Climate Data Center
   PROVIDER_ABSTRACT: National Climate Data Center is the worlds largest archive of climate data.
   KEYWORDS: Domain:Aerosol, Platform:Network, Instrument:Unknown, DataType:Point, Distributor:DataFed, Originator:NCDC, TimeRes:Minute, Vertical:Surface, TopicCategory:climatologyMeteorologyAtmosphere
   FEES: NONE
   CONSTRAINTS: NONE
   PROVIDER_SITE: http://lwf.ncdc.noaa.gov/oa/ncdc.html
   CONTACT_INDIVIDUAL: Climate Contact, Climate Services Branch, National Climatic Data Center
   CONTACT_PHONE: 828-271-4800
   CONTACT_STREET: 151 Patton Avenue Room 468
   CONTACT_CITY: Asheville 
   CONTACT_ADM_AREA: North Carolina 
   CONTACT_POSTCODE: 28801-5001
   CONTACT_COUNTRY: USA
   CONTACT_EMAIL: ncdc.info@noaa.gov

Here is the real live NCDC wcs_capabilities.conf

Windows Implementation Issue

Important There is a bug deep in python core libraries that make serving text files a little special. The files need to be encoded with unix style line ending convention '\n', instead of windows style '\r\n'.

To fix this, issue command:

   python /OWS/web/owsadmin.py unix_nl filename.html

for every text file you serve. Binary files are unaffected.

@@ Line 3: / Line 3: @@
 [http://sourceforge.net/p/aq-ogc-services/home/ Project on SourceForge]
-Questions and comments should go to [http://sourceforge.net/p/aq-ogc-services/discussion/ sourceforge discussions], bug reports to [http://sourceforge.net/p/aq-ogc-services/tickets/ sourceforge tickets]. Urgent issues can be asked from Kari Hoijarvi 314-935-6099(w) or 314-843-6436(h)
+This page describes how to install the information for humans: Home pages, contact information etc.
-Last updated 2010-07-21
+For data Configuration go to following pages:
+* [[WCS_Wrapper_Configuration_for_Cubes| Configuring NetCDF based Cube Data]]
+* [[WCS_Wrapper_Configuration_for_Point_Data| Configuring SQL based Point Data]]
 == Structure of OWS/web ==
@@ Line 11: / Line 15: @@
 '''OWS/web''' is for system developers only.
-OWS/web/'''static''' contains static web content. You can put any documentation here and it will be served as a web page or download. The home page index.html is pretty much mandatory, and you shoud change favicon.ico to reflect your organization. We highly recommend, that you customize these to document your WCS service.
+OWS/web/'''static''' contains static web content. You can put any documentation here and it will be served as a web page or download. The home page '''index.html''' is pretty much mandatory, and you should change '''favicon.ico''' to reflect your organization. We highly recommend, that you customize documents for your WCS service.
 OWS/web/static/'''cache''' is a folder for temporary files. The service uses it for output files. Anything you put there will be deleted when space is needed.
-The installation contains an example datasets OWS/web/static/'''testprovider''' and OWS/web/static/'''point'''. The testprovider is a demo NetCDF dataset, point is an example how to server point data from a SQL database. Every service will have a folder with the same name here.
+The installation contains an example datasets OWS/web/static/'''testprovider''' and OWS/web/static/'''point'''. The testprovider is a demo NetCDF dataset, point is an example how to server point data from a SQL database. Every service will have a folder with the same name here. So create yourself a folder datasets OWS/web/static/'''myservice''' and your service URL will be at http://localhost:8080/myservice
-You may now check the provider page [http://localhost:8080/testprovider http://localhost:8080/testprovider] which is served as a static file. Any file under static becomes accessible
 == The Human Interface: Create the index.html Front Pages for Visitors. ==
-If no query is present, the server gives a default page '''index.html'''. You should provide pages for your server and for all the providers.
+If no query is present, the server gives a default page '''index.html'''. You should create these pages for your server and for all the providers.
-The server index.html is at '''OWS/web/static/index.html''', which will be displayed from url http://localhost:8080/, Index of an external server index.html is [http://128.252.202.19:8080/ here].
-Every provider folder should also have an index.html like '''OWS/web/static/testprovider\index.html''' which will be displayed from http://localhost:8080/testprovider, index of an external provider front page is [http://128.252.202.19:8080/HTAP here]
-Every provider should have wcs_capabilities.conf that lists keywords and contact information. The format is simple, copy one from the testprovider and edit it.
+The server index.html is at '''OWS/web/static/index.html''', which will be displayed from url http://128.252.202.19:8080/ at test server or at [http://localhost:8080/ localhost] if you have a server running.
-=== Windows Implementation Bug ===
+Every provider folder should also have an index.html like '''OWS/web/static/testprovider/index.html''' which will be displayed from  http://128.252.202.19:8080/HTAP at test server or at [http://localhost:8080/testprovider localhost].
-'''Important''' There is a bug deep in python core libraries that make serving text files tricky. The files need to be encoded with unix style line ending convention '\n', instead of windows style '\r\n'.
-To fix this, issue command:
+== Enter Provider Metadata ==
-    python /OWS/web/owsadmin.py unix_nl filename.html
+Every provider should have '''wcs_capabilities.conf''' that lists keywords and contact information. The format is simple, copy one from the testprovider and edit it.
-for every html file you serve.
+    # this file provides some information about the provider
+    # and is incorporated into the respective WCS responses.
+    # all currently available field identifiers are listed below.
+    # please define every identifier only once.
+    # other identifiers will be ignored, input is case sensitive.
+    # the format is always <identifier>: <value>.
+    # whitespaces before and after <value> will be stripped.
+    # KEYWORDS can take a comma separated list that will then be
+    # included in the respective keyword tags
+    # empty lines and lines starting with "#" will be ignored.
+    PROVIDER_TITLE: National Climate Data Center
+    PROVIDER_ABSTRACT: National Climate Data Center is the worlds largest archive of climate data.
+    KEYWORDS: Domain:Aerosol, Platform:Network, Instrument:Unknown, DataType:Point, Distributor:DataFed, Originator:NCDC, TimeRes:Minute, Vertical:Surface, TopicCategory:climatologyMeteorologyAtmosphere
+    FEES: NONE
+    CONSTRAINTS: NONE
+    PROVIDER_SITE: http://lwf.ncdc.noaa.gov/oa/ncdc.html
+    CONTACT_INDIVIDUAL: Climate Contact, Climate Services Branch, National Climatic Data Center
+    CONTACT_PHONE: 828-271-4800
+    CONTACT_STREET: 151 Patton Avenue Room 468
+    CONTACT_CITY: Asheville
+    CONTACT_ADM_AREA: North Carolina
+    CONTACT_POSTCODE: 28801-5001
+    CONTACT_COUNTRY: USA
+    CONTACT_EMAIL: ncdc.info@noaa.gov
-== Serving data from periodic collection of NetCDF files ==
+Here is the real live NCDC [http://128.252.202.19:8080/static/NCDC/wcs_capabilities.conf wcs_capabilities.conf]
-Sometimes you have accumulated a huge number of small NetCDF files, like daily slices from a model output. You could combine those into one big cube, but you for a terabyte of files, that may not be an option.
+== Windows Implementation Issue ==
+'''Important''' There is a bug deep in python core libraries that make serving text files a little special. The files need to be encoded with unix style line ending convention '\n', instead of windows style '\r\n'.
-Download our HTAP test package [http://sourceforge.net/downloads/aq-ogc-services/ows/custom-netcdf-1.2.0.zip/ custom-netcdf-1.2.0.zip]. It only has two days of data to make download small. Then read the [http://localhost:8080/HTAP custom provider page]
+To fix this, issue command:
-== Storing Point Data in a Relational Database ==
-Provider [http://localhost:8080/point point] is an example how to configure this service to use SQL database to serve point data.
-Point data is often stored in SQL databases. There's no standard schema like CF-1.0 convention for NetCDF files, so it is not possible to just connect and start serving. You have to create the configuration description.
-Therefore, the WCS query processor needs to know what to select and join. This information must be edited into the configuration script.
-=== Notes on SQL ===
-One of the most powerful ideas in relational database design is the concept of a view. You don't need to change the existing data tables, creating a view that makes your DB to look like the one needed is usually enough. This is by far the easiest way to configure your WCS.
-Views also often give the SQL optimizer extra advice for the optimal performance. In general, relational schema does not necessarily have an effect on performance, one DBMS may optimize queries when another does not, resulting in huge difference in performance.
-Denormalization does not necessary improve performance. In tested databases is filtering a small geographical area for big time range is to use location table to filter by latitude and longitude first. Common sense advice is to design a normalized schema and optimize with benchmarks.
-=== Location Table ===
-The common thing between them is that they have a location table.
-    table location
-    +----------+-------------------------+
-    | loc_code | lat   | lon     | elev  |
-    +----------+-------------------------+
-    | KMOD     | 37.63 | -120.95 | 30.0  |
-    | KSTL     | 38.75 |  -90.37 | 172.0 |
-    | KUGN     | 42.42 |  -87.87 | 222.0 |
-    |...       |       |         |       |
-    +----------+-------------------------+
-Here loc_code is the primary key and lat,lon is the location. Optional fields can be added. CIRA VIEWS database has a location table, but it's called '''Site''' and it spells full '''longitude'''. The datafed browser uses standard names loc_code, loc_name, lat and lon for browsing. For plug-and-play compatibility we recommend using these names. In the CIRA VIEWS database, the view creation would be:
-    create view location as
-    select
-        SiteCode as loc_code,
-        Latitude as lat,
-        Longitude as lon
-    from Site
-The primary key is loc_code, being unique for all the locations.
-== Some Different DB Schema types ==
-In this documentation three different schemas are presented. Each of them have good and bad points.
-=== One Big Data Table ===
-In this case, all the data is in the same table:
-    +----------+------------+------+------+-----+
-    | loc_code | datetime   | TEMP | DEWP | VIS |
-    +----------+------------+------+------+-----+
-    | KMOD     | 2009-06-01 | 87.8 | 51.4 | 10  |
-    | KMOD     | 2009-06-02 | 82.3 | 51.4 | 10  |
-    | KSTL     | 2009-06-01 | 78.6 | 34.9 | 18  |
-    | ...      |            |      |      |     |
-    +----------+------------+------+------+-----+
-The foreign key to location table is loc_code. The primary key is (loc_code, datetime)
-=== Long And Skinny Table ===
-In this case, all the data is in the same table:
-    +----------+------------+------+-------+
-    | loc_code | datetime   | data | param |
-    +----------+------------+------+-------+
-    | KMOD     | 2009-06-01 | 87.8 | TEMP  |
-    | KMOD     | 2009-06-02 | 82.3 | TEMP  |
-    | KSTL     | 2009-06-01 | 78.6 | TEMP  |
-    | KMOD     | 2009-06-01 | 51.4 | DEWP  |
-    | KMOD     | 2009-06-02 | 51.4 | DEWP  |
-    | KSTL     | 2009-06-01 | 34.9 | DEWP  |
-    | KMOD     | 2009-06-01 | 10   | VIS   |
-    | KMOD     | 2009-06-02 | 10   | VIS   |
-    | KSTL     | 2009-06-01 | 18   | VIS   |
-    | ...      |            |      |       |
-    +----------+------------+------+------+
-=== One Data Table For Each Param ===
-Each parameter has its own data table. In this case there's no need for nulls, and is the fastest for one parameter query.
-    +----------+------------+------+
-    | loc_code | datetime   | TEMP |
-    +----------+------------+------+
-    | KMOD     | 2009-06-01 | 87.8 |
-    | KMOD     | 2009-06-02 | 82.3 |
-    | KSTL     | 2009-06-01 | 78.6 |
-    | ...      |            |      |
-    +----------+------------+------+
-    +----------+------------+------+
-    | loc_code | datetime   | DEWP |
-    +----------+------------+------+
-    | KMOD     | 2009-06-01 | 51.4 |
-    | KMOD     | 2009-06-02 | 51.4 |
-    | KSTL     | 2009-06-01 | 34.9 |
-    | ...      |            |      |
-    +----------+------------+------+
+    python /OWS/web/owsadmin.py unix_nl filename.html
-    +----------+------------+-----+
+for every text file you serve. Binary files are unaffected.
-    | loc_code | datetime   | VIS |
-    +----------+------------+-----+
-    | KMOD     | 2009-06-01 | 10  |
-    | KMOD     | 2009-06-02 | 10  |
-    | KSTL     | 2009-06-01 | 18  |
-    | ...      |            |     |
-    +----------+------------+-----+