WCS Wrapper Installation WindowsOP

Back to WCS Wrapper

Project on SourceForge

Questions and comments should go to sourceforge discussions, bug reports to sourceforge tickets. Urgent issues can be asked from Kari Hoijarvi 314-935-6099(w) or 314-843-6436(h)

Last updated 2010-12-13

Install as Administrator

This installation requires admin rights. After installation, you can switch to a user account.

Things to Consider About Security

The web.py framework and WCS service code has been written with security in mind, and there are no known security bugs. There is no code that writes on the disk, so running this service should not put your computer in danger.

To maximize the security of your computer, consider the following.

• Every file in every directory you put under C:\OWS\web\static becomes readable by anybody. This is by design, since the whole framework is meant to publish public data. Do not put confidential information under OWS folder!

• Windows Vista and later: Don't run this process with administrative rights. Create a low-rights account like WCS_RUNNER and use that.

• Under Windows XP, the WCS_RUNNER account requires administrative privileges. It should not, but it seems that it is required. Those who know better, please inform the author. Go to Control panel / Administrative Tools / Computer management / Local Users and Groups. Add a new user, add administrators to groups and make password non-expiring, not changeable.

Get Python 2.6.6

Attention. Activestate download is not available, so download from official site or wait that we support python 2.7

We recommend latest python 2.6.6 from [http://www.activestate.com/activepython/downloads ActiveState Python download pa ge] or ActiveState Direct Download or you can download from official site but activepython contains some goodies, like very nice language sensitive editor PythonWin.

Since the C and C++ dll's are 32 bit, get a 32 bit python even if you have 64 bit machine. If requested, we will support 64 bit binaries too.

Get the Datafed WCS Wrapper

If you have modified the default OWS/web/static/index.html, make sure not to overwrite' it. Keep the main copy elsewhere and copy it back.

Open the download page in another tab.

If you are serving cubes from NetCDF files, get win32-ows-1.3.1.zip or later. This distribution contains point data support also. If you are serving only point data, you can just download ows-point-1.3.1.zip.

   ows-point-1.3.1.zip = (win32-ows-1.3.1.zip) - (netcdf support) + (sample point provider)

Unzip it/both into your installation folder C:\OWS. You can install to other folders, this is just used as an example.

You will to have directories C:\OWS\datafed and C:\OWS\web

The folder datafed contains datafed package, which contains tools for ISO 8601 time parsing and python-NetCDF module nc3 (not in point). Web is the WCS framework.

Notice:

Linux is using PyNIO for NetCDF access from python. Since PyNIO is not supported under Windows, a subset datafed/minio, was developed for windows. Using C based libraries enables delivering large amounts of data efficiently. Python is used only for query parsing, making minimal impact in speed.

Although the built-in webserver is intended mainly for development, you can use it to serve data from your own workstation just fine. For high load sites, look webpy.org/install documentation how to integrate webpy with apache server etc.

Get 3rd Party Components

Download win32-3rdparty-1.2.0.zip.

Unzip the file. You'll see

• runtime
• web.py-0.33
• activepython.txt
• install_webpy.bat
• lxml-2.2.4.win32-py2.6.exe
• numpy-1.4.1-win32-superpack-python2.6.exe
• OWS version 1.3.x

Installing: all the files are in the 3rd party bundle, you don't need to go to web during installation.

• OWS version 1.3.x is just marking the version of this distribution.

• Copy all the files from runtime to C:\OWS\datafed. The NetCDF is not an unofficial build, so do not replace your old NetCDF dll's with it. (Not required in point)

• activepython.txt is already done.

• Install webpy Run batch file install_webpy.bat, it will print about two pages informational messages. There should be no warnings or errors. webpy home

• Install lxml by running the setup program lxml-2.2.4.win32-py2.6.exe, use default answer for everything. The lxml package is already in version 2.2.6, but the windows builds are not ready. lxml home download page direct download link

• Install numpy 1.4.1 by running numpy-1.4.1-win32-superpack-python2.6.exe. Numpy is a great library for array manipulation. numpy home sourceforge download page direct download link (Not required in point)

Disclaimer: (Not required in point) The C:\OWS\datafed\netcdf.dll is a custom compiled from netcdf-4.1.2-beta1 and all the documented library calls are not be present. The bundled ncgen.exe and ncdump.exe are compiled for this purpose, nctest.exe is not there. It's heavily unit tested under datafed OWS framework, for other purposes use with caution.

The datafed folder also contains utilities ncdump.exe and ncgen.exe Especially ncdump is very useful when creating or inspecting netcdf files. The page UNIDATA NetCDF docs has more information about these utilities. Main installation page may be useful too.

Configure Firewall PYTHONPATH

Since your computer is probably running a firewall, and if it isn't it should, it's necessary to allow python to act as a server for incoming connections. For Windows firewall on XP:

• Open Control panel/Windows Firewall
• Exceptions tab
• Add Program...
• "Python Interactive Shell" or browse for C:\python26\python.exe
• Hit OK

Make sure that C:\OWS and C:\OWS\web are in your PYTHONPATH variable. Create it via Control Panel / System / Advanced / Environment Variables. Pythonpath looks just like PATH.

Run command prompt, start / run, "cmd.exe". Type "SET PYTHONPATH". If you see PYTHONPATH=C:\OWS;C:\OWS\web you're OK.

Start the Server

To see your IP address, type

   C:\OWS\web>ipconfig

   Windows IP Configuration
   Ethernet adapter Local Area Connection:
       Connection-specific DNS Suffix  . : gateway.2wire.net
       IP Address. . . . . . . . . . . . : 192.168.1.65
       Subnet Mask . . . . . . . . . . . : 255.255.255.0
       Default Gateway . . . . . . . . . : 192.168.1.254

Your public IP address is 192.168.1.65

Change Directory to C:\OWS\web folder and type python ows.py. You have now your server running, listening port 8080.

You can pass the port as parameter. python ows.py 80 listens the standard port 80.

Open http://localhost:8080 and you should see the home page. Congratulations if you got this far. Now we need to install some data to serve. replace localhost with your IP address.

If the server won't start, try to set the path explicit (Not required in point):

   set path=C:\OWS\datafed;%PATH%

to make sure, that correct versions of NetCDF and HDF dynamic link libraries are used.

Putting your data into CF Compatible NetCDF Files

(Not required in point) This is for getting the demo run only, more detailed instructions on how to serve cube and point data is documented at WCS Wrapper Configuration How to use the wrapper libraries can be found in related page Creating NetCDF-CF Files.

Setting up the demo dataset:

C:\OWS\web\static\testprovider\ contains files:

• CubeA.ncml and CubeA.nc
• CubeB.ncml and CubeB.nc
• H5D.ncml and H5D.nc
• create_all.py

• index.html
• wcs_capabilities.conf

• CubeA.ncml is a NetCDF Markup Language description of the test coverage, same for teh rest of the .ncml files. The .nc files are files that are created from using the ncml as instructions.
• create_all.py is a script that creates the empty cube files and then fills it with test data. It servers as an example code how to create your own CF 1.0 compatible files. It has no user interface, just run it.

So do the following:

   cd C:\OWS\web\static\testprovider
   create_all

activepython registers python.exe as default program for .py files, so you don't need to say python CubeA.py

Now you have data to serve.

Extract metadata from the NetCDF Files

Change directory to and issue command

   cd C:\OWS\web
   owsadmin wcs_prepare -ao

This program extracts the metadata from the netcdf files. Each provider will get a 'metadata.dat' giving fast access to coverages, fields and dimensionality.

The WCS Service is now ready.

Test Your WCS server

GetCapabilities query for NetCDF: http://localhost:8080/testprovider?service=WCS&acceptversions=1.1.2&Request=GetCapabilities or from the external server

GetCapabilities query for point: http://localhost:8080/point?service=WCS&acceptversions=1.1.2&Request=GetCapabilities or from the external server

DescribeCoverage query for NetCDF: http://localhost:8080/testprovider?service=WCS&version=1.1.2&Request=DescribeCoverage&identifiers=CubeA or from the external server

DescribeCoverage query for point: http://localhost:8080/point?service=WCS&version=1.1.2&Request=DescribeCoverage&identifiers=SURF_MET or from the external server

GetCoverage query for NetCDF: http://localhost:8080/testprovider?service=WCS&version=1.1.2&Request=GetCoverage&identifier=CubeA&BoundingBox=-180,-90,180,90,urn:ogc:def:crs:OGC:2:84&TimeSequence=2007-01-01&RangeSubset=*&format=image/netcdf&store=true or from the external server

GetCoverage query for point http://localhost:8080/point?service=WCS&request=getCoverage&version=1.1.2&identifier=SURF_MET&BoundingBox=-130,24,-50,40,urn:ogc:def:crs:OGC:2:84&TimeSequence=2009-09-01T15:00:00&RangeSubset=TEMP&format=text/csv&store=true external server

Sections of the WCS url:

• http://your_domain_or_ip_address:8080 is the server url. It returns a static index.html page of providers using this server.
• testprovider is the provider name. There must be a folder C:\OWS\web\static\testprovider\.
• for NetCDF:
  • &identifier=CubeA is the name of the NetCDF file. So in this example, C:\OWS\web\static\testprovider\ contains file CubeA.nc and it becomes a coverage.
  • RangeSubset=* selects variables to be returned from the NetCFD, '*' gets all, &RangeSubset=Spam;Eggs gets variables Spam and Eggs.
  • &format=image/netcdf&store=true specifies, that the result will be a NetCDF file that is stored in the server and can later be fetched via from an url. These are now the only supported options.
• for point:
  • Coverage is often a database and RangeSubset selects a value column. This is not necessarily so, since configuring SQL is much more flexible than configuring NetCDF

The wcs GetCoverage returns an Xml envelope

<?xml version="1.0" encoding="UTF-8"?>
<Coverages xmlns="http://www.opengis.net/wcs/1.1" 
        xmlns:ows11="http://www.opengis.net/ows/1.1" xmlns:xlink="http://www.w3.org/1999/xlink">
    <Coverage>
        <ows11:Title>Select * From CubeA</ows11:Title>
        <ows11:Abstract>Where boundingbox=-180,-90,180,90,urn:ogc:def:crs:OGC:2:84 and timesequence=2007-01-01</ows11:Abstract>
        <ows11:Identifier>CubeA</ows11:Identifier>
        <ows11:Reference xlink:href="http://localhost:8080/static/cache/CubeA_1.nc" xlink:role="urn:ogc:def:role:WCS:1.1:coverage"/>
    </Coverage>
</Coverages>

The request writes out a NetCDF file C:\OWS\web\static\cache\CubeA_1.nc and so it can be retrieved later with url http://localhost:8080/static/cache/CubeA_1.nc and it will be deleted in ten minutes. The number after coverage is a generated sequence number.

In short:

• Every folder under static becomes a provider. Providers are unrelated. There can be any number of providers.
• Every NetCDF file in the provider folder becomes a coverage. The WCS Capabilities document is automatically compiled from all the NetCDF files.
• Every variable inside NetCDF becomes a field

GetCoverage query with store=false

for NetCDF: http://128.252.202.19:8080/testprovider?service=WCS&version=1.1.2&Request=GetCoverage&identifier=CubeA&BoundingBox=-180,-90,180,90,urn:ogc:def:crs:OGC:2:84&TimeSequence=2007-01-01&RangeSubset=*&format=image/netcdf&store=false

for point http://128.252.202.19:8080/point?service=WCS&request=getCoverage&version=1.1.2&identifier=SURF_MET&BoundingBox=-130,24,-50,40,urn:ogc:def:crs:OGC:2:84&TimeSequence=2009-09-01T15:00:00&RangeSubset=TEMP&format=text/csv&store=false

These queries return both the xml envelope and cube file bundled together in mime-multipart format.

All the browsers don't work with this. Firefox can open it, Internet Explorer can't.

Stopping the Server

Press ctrl+C to terminate the server, or if you are running as a scheduled task, end the task. This is a development web server, there is no shutdown command.

Running in the background

This executable is meant to be a test server for developers. Therefore it does not support running as a service. Leaving your workstation on all the time is possible, but then you will have the server window open all the time, which maybe is nice for monitoring, but usually quite annoying.

But you can run any command line program in the background.

Go to control panel and open "Scheduled Tasks".

• Click on "add scheduled task."

• Pick C:\OWS\web\ows.py as the executable.

• Check "When my computer starts" as the scheduling"

• Choose a proper user name. With Windows XP and earlier the user needs to be an Administrator.

• Check box "open advanced properties..."

Change the properties:

• The default port is 8080. On the task tab, add port number to the program C:\OWS\web\ows.py 8081

• On settings tab, check off the "Stop the task if ..." box

You can now start the task and end it from the "Scheduled tasks" form.

Notice, that if you choose your own username, you'll see the window when you log on. This is kind of nice, since the window will display incoming queries.

• Implementation of netCDF Server at Northrop Grumman

Put your own data online

• WCS Wrapper Configuration: Common information about the server
• WCS_Wrapper_Configuration_for_Point_Data: How to serve SQL-based point data.
• WCS_Wrapper_Configuration_for_Cubes: How to serve NetCDF-CF based cube data.

Running high traffic sites with IIS

These instructions are based on Deploying web.py on IIS7 via PyISAPIe

Hack around a bug

Unfortunately webpy has a slight problem. You need to edit C:\Python26\Lib\site-packages\web\wsgi.py and add return False to _is_dev_mode function. Make it look exactly like

   def _is_dev_mode():
       return False

With 4 space indentation as in the source code and case sensitive False

Create New Application Pool

Go to IIS management console, and create a new Application pool.

• In 64 bit machines allow 32 bit applications.
• Disable rapid-fail protection.
• In IIS7
  • Disable managed code, no .NET version
  • Classic pipeline
  • Create a new local user like OWS_RUNNER, and assign the applicationpool to run under that username. Do not make it power user or administrator, just a user is enough.

Give the user read/execute permissions under C:\OWS and additional write permissions under C:\OWS\web\static

Create OWS Website

• New website, name it OWS
• port 8080 or something unique, anyway not the standard 80, which will interfere with existing stuff.
• Site path something like C:\OWS\web so that static folder is directly under it.
• Permissions: Read, Run Scripts, Execute
• If the installation is to another directory than C:\OWS, edit the OWS\web\web.config file, change the PyISAPIe.dll path to yours.
• If a provider starts with /S, add it to the OWS\web\web.config file. Providers with other letters are configured automatically with wildcards.
• Edit OWS\web\config.py and change log_webpy_to_file = True for enabling logging via IIS

Significant changes since version 1.2

• Unix and Windows code base have been unified.

• filtering by any dimension works now. You can choose pressure or wavelength.

• Performance for querying large cubes as improved.

• Query Caching was implemented: Identical queries will return the same temporary file.

Significant changes since version 1.1

• Tested with activestate python 2.6.5.14 and 2.6.5.15, Earlier versions of python 2.6 may work.

• Unix and Windows style newlines bug is still present.

• Bug fixes to xml documents

• added optional wcs_capabilities.conf files for wcs metadata.

Finally we have xml schemas that actually work. 1.1.1 and 1.1.2 documents can be validated properly. This also exposed a huge number of wrong names ans namespaces in the capabilities and describecoverage documents. If you have code that used to read 1.1.0 xml, it needs to be updated.

Significant changes since version 1.0

• Python version

python 2.5 will not work for you, use 2.6

• Unix and Windows style newlines.

With python 2.5, webpy development server required, that text files have Unix-style end of line marker, plain LF and not the Windows convention CR-LF. This was a bug deep in python libraries, and is fixed in 2.6.

The owsadmin.py script still supports turning files to unix and windows convention.

python C:\OWS\web\owsadmin.py unix_nl "C:\OWS\web\static\testprovider\index.html"

• errorneous "fieldsubset:" prefix

The WCS query used to have fieldsubset: prefix, this was incorrect and is fixed. It also means, that old queries that used fieldsubsets won't work anymore. Update your links.

• Bug fixes to queries

Some filters returned incorrect results. One good reason to upgrade.

Moved from WCS Server for CF (GEO-AQ-CoP

• These two services are in the same computer: htap.icg.kfa-juelich.de. The two completely different services are created just putting the netcdf files to folders HTAP_FC_pressure and MACC_bnds. They are just ordinary folders: the folder name is used as service name. http://geo-aq-cop.org/node/1316/
• The HTAP_FC_pressure GetCapabilities contains plenty of coverages. For example GISS-PUCCINI-modelEaer_FC3SA_metm comes simply from the file GISS-PUCCINI-modelEaer_FC3SA_metm.nc which is in the HTAP_FC_pressure folder. One of the data variables is cnvflxup(time,lev,lat,lon) and that becomes automatically a field.