WCS Wrapper Installation WindowsOP

Back to WCS Wrapper

Datafed OWS package 1.2

Questions and Comments should be directed to Kari Hoijarvi hoijarvi@seas.wustl.edu Phone 314-935-6099(w) or 314-843-6436(h)

Binaries last updated July/15/10

Get Python 2.6.5

We recommend python 2.6.5.14 from ActiveState Python download page or ActiveState Direct Download or you can download from official site but activepython contains some goodies, like very nice language sensitive editor PythonWin.

Since the C and C++ dll's are 32 bit, get a 32 bit python even if you have 64 bit machine. If requested, we will support 64 bit binaries too.

Get the datafed WCS Wrapper

From datafed.net/ows/datafed_ows.zip. Unzip it into your installation folder C:\OWS. Other folders will do, this is just used in this example.

You will to have directories C:\OWS\datafed and C:\OWS\web

The folder datafed contains datafed package, which contains tools for ISO 8601 time parsing and python-NetCDF module. Web is the WCS framework.

Get 3rd Party Components

Download OWS_3rdparty.zip or hunt them down yourself.

Unzip the file. You'll see

• Folders:
  • msvc_redist
  • netcdf_dlls
  • utilites
  • web.py-0.33
• Files
  • activepython.txt
  • install_webpy.bat
  • lxml-2.2.4.win32-py2.6.exe
  • numpy-1.4.1-win32-superpack-python2.6.exe

Installing:

• activepython.txt is already done.

• Install webpy webpy.org Run batch file install_webpy.bat, it will print about two pages informational messages. There should be no warnings or errors.

• Install lxml (download page) (direct download link) by running the setup program lxml-2.2.4.win32-py2.6.exe, use default answer for everything. The lxml package is already in version 2.2.6, but the windows builds are not ready.

• Install numpy 1.4.1. Numpy is a great library for array manipulation, and this project is using it. (sourceforge download page) (direct download link)

• Install NetCDF The dynamis link libraries in folders msvc_redist and netcdf_dlls have no setup program. Just copy these dll's to C:\OWS\datafed. You should have
  • C:\OWS\datafed\Microsoft.VC90.CRT.manifest
  • C:\OWS\datafed\msvcm90.dll
  • C:\OWS\datafed\msvcp90.dll
  • C:\OWS\datafed\msvcr90.dll
  • C:\OWS\datafed\hdf5_hldll.dll
  • C:\OWS\datafed\hdf5dll.dll
  • C:\OWS\datafed\netcdf.dll
  • C:\OWS\datafed\szip.dll
  • C:\OWS\datafed\zlib1.dll

Disclaimer: The netcdf.dll is a custom compiled from ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4.1.2-beta1.tar.gz and all the documented library calls may not be present. It's heavily unit tested under datafed OWS framework, for other purposes use with caution.

Remember the Utilities for the Future

The utilities folder contains ncdump.exe and ncgen.exe Especially ncdump is a very useful utility, that you'll find useful when creating or inspecting netcdf files. The page UNIDATA NetCDF docs has more information about these utilities. Main installation page may be useful too.

You can copy them into C:\OWS\datafed folder as well.

Set PATH and PYTHONPATH variables and start the WCS server

Make sure that C:\OWS is in your PYTHONPATH variable. Create it via Control Panel / System / Advanced / Environment Variables

Run command prompt, start / run, "cmd.exe". Type "SET PYTHONPATH". If you see PYTHONPATH=C:\OWS you're OK

Change Directory to C:\OWS\web folder and type python ows.py. You have now your server running, listening port 8080.

You can pass the port as parameter. python ows.py 80 listens the standard port 80.

Congratulations if you got this far. Now we need to install some data to serve.

Putting your data into CF Compatible NetCDF Files

The installation contains an example dataset 'testprovider'.

C:\OWS\web\static\testprovider\ contains files CubeA.ncml, CubeA.py and CubeA.nc

• CubeA.ncml is a NetCDF Markup Language description of the test coverage.
• CubeA.py is a script that creates the empty CubeA.nc file and fills it with test data. It servers as an example code how to create your own CF 1.0 compatible files. It has no user interface, just run it.
• CubeA.nc is the output of two previous, and serves as our test coverage.

Now, when you have C:\OWS\web\static\testprovider\CubeA.nc you actually have a WCS Service.

More complete instructions on how to use the wrapper libraries can be fount in related page Creating NetCDF-CF Files.

You have now a WCS server

If you started the server, you can try this

http://localhost:8080/testprovider?service=WCS&version=1.1.0&Request=GetCoverage&identifier=CubeA&BoundingBox=-180,-90,180,90,urn:ogc:def:crs:OGC:2:84&TimeSequence=2007-01-01&RangeSubset=*&format=image/netcdf&store=true

or the same test query from an external server

Sections of the WCS url:

• http://your_domain_or_ip_address:8080 is the server url. It returns a static index.html page of providers using this server.
• /testprovider is the provider name. There must be a folder C:\OWS\web\static\testprovider\.
• &identifier=CubeA is the name of the NetCDF file. So in this example, C:\OWS\web\static\testprovider\ contains file CubeA.nc and it becomes a coverage.
• RangeSubset=* selects variables to be returned from the NetCFD, '*' gets all, &RangeSubset=Spam;Eggs gets variables Spam and Eggs.
• &format=image/netcdf&store=true specifies, that the result will be a NetCDF file that is stored in the server and can later be fetched via from an url. These are now the only supported options.

The wcs GetCoverage returns an Xml envelope

<?xml version="1.0" encoding="UTF-8"?>
<Coverages xmlns="http://www.opengis.net/wcs/1.1/ows" 
        xmlns:ows="http://www.opengis.net/ows" xmlns:xlink="http://www.w3.org/1999/xlink">
    <Coverage>
        <ows:Title>Select * From CubeA</ows:Title>
        <ows:Abstract>Where boundingbox=-180,-90,180,90,urn:ogc:def:crs:OGC:2:84 and timesequence=2007-01-01</ows:Abstract>
        <Identifier>CubeA</Identifier>
        <Reference xlink:href="http://localhost:8080/static/cache/CubeA_1.nc" xlink:role="urn:ogc:def:role:WCS:1.1:coverage"/>
    </Coverage>
</Coverages>

The request writes out a netcdf file C:\OWS\web\static\cache\CubeA_1.nc and so it can be retrieved later with url http://localhost:8080/static/cache/CubeA_1.nc and it will be deleted in ten minutes. The number after coverage is a generated sequence number.

In short:

• Every folder under static becomes a provider. Providers are unrelated. There can be any number of providers.
• Every NetCDF file in the provider folder becomes a coverage. The WCS Capabilities document is automatically compiled from all the NetCDF files.
• Every variable inside NetCDF becomes a field

Creating GetCapabilities and DescribeCoverage Documents

The metadata queries of WCS, GetCoverage and DescribeCoverage need to be precompiled.

Run the following commands:

c:

cd \OWS\web

python owsadmin.py wcs_cap -v 1.1.0 -d my.server.com -p 8080 testprovider

python owsadmin.py wcs_desc -v 1.1.0 testprovider

Replace my.server.com with your domain name or your raw ip address like 128.34.111.47, and port -p 8080 with the port you want to use. Default is the standard -p 80, but that is not always possible, since you might have another webserver in your machine. If you just want to test from your own machine, you can use localhost as the domain.

They will create files C:\OWS\web\static\testprovider\wcs_1.1.0_capabilities.xml and C:\OWS\web\static\testprovider\wcs_1.1.0_desc_CubeA.xml. The capabilities file contains all the coverages that this provider has, and each coverage gets its own desc file.

GetCapabilities query: http://localhost:8080/testprovider?service=WCS&version=1.1.0&Request=GetCapabilities or from the external server

DescribeCoverage query: http://localhost:8080/testprovider?service=WCS&version=1.1.0&Request=DescribeCoverage&identifier=CubeA or from the external server

Press ctrl+C to terminate the server.

Notice: WCS Wrapper core is written in C++, enabling large amount of data delivery. Python is used only for query parsing, making minimal impact in speed.

Installing a High Performance Server

Although the built-in webserver is intended mainly for development, you can use it to serve data from your own workstation just fine. For high load sites, look webpy.org/install documentation how to intergrate webpy with apache server etc.

Optional: Creating a service from periodic collection of NetCDF files.

Sometimes you have accumulated a huge number of small timeseries NetCDF files, like daily slices from a model output. You could combine those into one big cube, but you for a terabyte of files, that may not be an option.

The OWS web server can be configured to serve directly from time encoded files. We are serving some HTAP ozone model data with this method.

Live Query Examples:

Linux Server:

GetCapabilities

DescribeCoverage

GetCoverage

Windows Workstation:

GetCapabilities

DescribeCoverage

GetCoverage

How to set this up on your site:

Download our HTAP test package from " datafed.net/ows/datafed_ows_HTAP.zip. It only has two days of data to make download small.

Unzip it to the C:\OWS folder, the zip file contains directories C:\OWS\GEMAQ-v1p0, C:\OWS\GEOSChem-v45 and C:\OWS\web\static\HTAP\

The dataset name is HTAP. The coverages are GEMAQ-v1p0 and GEOSChem-v45.

If you used other folder than C:\OWS you need to edit C:\OWS\web\static\HTAP\HTAP_config.py and change the variable netcdf_root_folder to point the correct directory.

If you have the WCS running, you can now try this test query from your own machine.

The server returns the same short XML document, which contains a link into the query result.

The Human Interface: Create the index.html Front Pages for Visitors.

If no query is present, the server gives a default page index.html. You should provide pages for your server and for all the providers.

The server index.html is at C:\OWS\web\static\index.html, which will be displayed from url http://localhost:8080/, Index of an external server index.html here.

Every provider folder should also have an index.html like C:\OWS\web\static\testprovider\index.html which will be displayed from http://localhost:8080/testprovider, index of an external provider front page is here

If you modify the default index.html, make sure not to overwrite' it next time you upgrade the WCS Wrapper package. Keep the main copy elsewhere and copy it back.

Running in the background

This executable is meant to be a test server for developers. Therefore it does not support running as a service. Leaving your workstation on all the time is possible, but then you will have the server window open all the time, which maybe is nice for monitoring, but usually quite annoying.

But you can run any command line program in the background.

First, consider creating a new local user like WCS_RUNNER. This user requires with administrative privileges. It should not, but it seems that it is required in Windows XP and earlier. Those who know better, please inform the author. Go to Control panel / Administrative Tools / Computer management / Local Users and Groups. Add a new user, add administrators to groups and make password non-expiring, not changeable.

Go to control panel and open "Scheduled Tasks".

• Click on "add scheduled task."

• pick C:\OWS\web\ows.py as the executable.

• Check "When my computer starts" as the scheduling"

• Choose a proper user name. The user needs to be an Administrator.

• Check box "open advanced properties..."

Change the properties:

• The default port is 8080. On the task tab, add port number to the program C:\OWS\web\ows.py 8081

• on settings tab, check off the "Stop the task if ..." box

You can now start the task and end it from the "Scheduled tasks" form.

Notice, that if you choose your own username, you'll see the window when you log on. This is kind of nice, since the window will display incoming queries.

• Implementation of netCDF Server at Northrop Grumman

Significant changes since version 1.0

• Python version

python 2.5 will not work for you, use 2.6

• Unix and Windows style newlines.

With python 2.5, webpy development server required, that text files have Unix-style end of line marker, plain LF and not the Windows convention CR-LF. This was a bug deep in python libraries, and is fixed in 2.6.

The owsadmin.py script still supports turning files to unix and windows convention.

python C:\OWS\web\owsadmin.py unix_nl "C:\OWS\web\static\testprovider\index.html"

• errorneous "fieldsubset:" prefix

The WCS query used to have fieldsubset: prefix, this was incorrect and is fixed. It also means, that old queries that used fieldsubsets won't work anymore. Update your links.

• Bug fixes to queries

Some filters returned incorrect results. One good reason to upgrade.