Difference between revisions of "WCS Wrapper Installation LinuxOP"

From Earth Science Information Partners (ESIP)
 
(33 intermediate revisions by 2 users not shown)
Line 7: Line 7:
 
Last updated 2010-12-13
 
Last updated 2010-12-13
  
== Know the basic installation tools. ==  
+
== Things to Consider About Security ==
  
=== Install as public or private? ===
+
The web.py framework and WCS service code has been written with security in mind, and there are no known security bugs. There is little code that writes on the disk, so running this service should not put your computer in danger.
 +
 
 +
To maximize the security of your computer, consider the following.
 +
 
 +
* '''Every file in every directory''' you put under OWS/web/static becomes '''readable by anybody'''. This is by design, since the whole framework is meant to publish public data. '''Do not put confidential information under OWS folder!'''
 +
 
 +
* Don't run the process as root. Create a low-rights account like WCS_RUNNER and use that.
 +
 
 +
'''Do you want to install as public or private?'''
  
 
The place to install is usually by default:
 
The place to install is usually by default:
Line 15: Line 23:
 
     /usr/local
 
     /usr/local
  
Installing to this directory, you will make the software packages available for everyone in that computer. It will also overwrite older versions, which is usually desirable but contains risks, or overwrite newer versions, which is usually not desirable. To install a shared package requires root privileges.
+
Installing to this directory, you will make the software packages available for everyone in that computer. It will also overwrite older versions, which is usually desirable but contains risks, or overwrite newer versions, which is usually not desirable. You need root privileges to install to /usr/local
  
 
Anyway, overwriting anything may break some other software, so in that case it is recommended to install somewhere else and not make the package automatically public. In that case, you don't need root privileges.
 
Anyway, overwriting anything may break some other software, so in that case it is recommended to install somewhere else and not make the package automatically public. In that case, you don't need root privileges.
  
=== AutoConf ===
+
== Get Python 2.6.6 ==
  
Different posix-like operating systems, like Linux, contain subtle differences which makes creating a portable C based library difficult. This problem is often addressed by [http://www.gnu.org/software/autoconf/ autoconf]. When you type command:
+
[[Image:Python-logo.gif]]
  
    # ./configure
+
We recommend latest '''python 2.6.6''' from [http://www.activestate.com/activepython/downloads ActiveState Python download page] or [http://downloads.activestate.com/ActivePython/releases/2.6.6.17/ActivePython-2.6.6.17-linux-x86.tar.gz ActiveState Direct Download] or you can download from [http://www.python.org/download/releases/2.6.6/ official site].
 +
 
 +
Since python is usually installed in posix-like systems, this step may be skipped. Other versions of python 2.x may work, but python 3.x won't yet.
 +
 
 +
32 bit as well as 64 bit installations have been done and work as expected. This is more of an issue for 3rd party software in any case, so make sure you install the correct versions of the required packages (as far as they contain compiled code) or python might not be able to load them.
 +
 
 +
Install with command
  
autoconf finds out important properties of your system, and creates the [http://www.gnu.org/software/make/manual/make.html makefile] that actually compiles your library.
+
    # ./install.sh -I /usr/local
  
=== Make ===
+
Warning: This will overwrite your existing python. It may or may not be OK to upgrade, but hardly good to downgrade. You can install several versions of python in the same machine and then edit your environment variables to use the desired version.
  
The traditional build tool is make, which contains rules for compiling and usually also for installing. By typing:
+
== Get easy_install for python ==
  
    # make; make install
+
Follow instructions at [http://pypi.python.org/pypi/setuptools#rpm-based-systems easy_install]. In this case download was [http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086 setuptools-0.6c11-py2.6.egg] and the next command installed it:
  
you will build the project and install it. See the individual project instructions to determine installation directory.
+
    # sh setuptools-0.6c11-py2.6.egg
  
=== easy_install ===
+
== Get the Datafed WCS Wrapper ==
  
Python contains a tool called [http://pypi.python.org/pypi/setuptools#rpm-based-systems easy_install] which you should have. It makes installing python packages a breeze.
+
A good place to install is /usr/local/OWS but any other is OK.
  
== Things to Consider About Security ==
+
If you have modified the default '''OWS/web/static/index.html'', make sure '''not to overwrite''' it. Keep the main copy elsewhere and copy it back.
 +
 
 +
Open the [http://sourceforge.net/projects/aq-ogc-services/files/ download page] in another tab.
 +
 
 +
If you are serving cubes from NetCDF files, get '''linux-ows-1.3.1.zip''' or later. This distribution contains point data support also. If you are serving only point data, you can just download '''ows-point-1.3.1.zip'''.
 +
 
 +
    ows-point-1.3.1.zip = (linux-ows-1.3.1.zip) - (netcdf support) + (sample point provider)
 +
 
 +
Issue command
 +
 
 +
    unzip linux-ows-1.3.1.zip
 +
 
 +
Move the contents of the directory linux-ows-1.3.1 to '''OWS'''. You can install to other directories, this is just used as an example.
 +
 
 +
You will to have directories '''OWS/datafed''' and '''OWS/web'''
 +
 
 +
The directory datafed contains ''datafed'' package, which contains tools for ISO 8601 time
 +
parsing and some other utilities. ''Web'' is the WCS framework.
 +
 
 +
Notice:
 +
 
 +
Using C based libraries enables delivering large amounts of data efficiently. Python is used only for query parsing, making minimal impact in speed.
 +
 
 +
Although the built-in webserver is intended mainly for development, you can use it to serve data from your own workstation just fine. For high load sites, look [http://webpy.org/install webpy.org/install] documentation how to integrate webpy with apache server etc.
 +
 
 +
== Get 3rd Party Components ==
 +
 
 +
=== WebPy ===
 +
 
 +
* [[Image:Webpy.png]]
  
The web.py framework and WCS service code has been written with security in mind, and there are no known security bugs. There is no code that writes on the disk, so running this service should not put your computer in danger.
+
Install webpy 0.34 or newer.
  
To maximize the security of your computer, consider the following.
+
    easy_install web.py
  
* '''Every file in every directory''' you put under OWS/web/static becomes '''readable by anybody'''. This is by design, since the whole framework is meant to publish public data. '''Do not put confidential information under OWS folder!'''
+
More detailed instructions are at [http://webpy.org/download webpy download].
  
* Don't run the process as root. Create a low-rights account like WCS_RUNNER and use that.
+
=== lxml ===
  
 +
* [[Image:Tagpython.png]]
  
== Get Python 2.6.6 ==
+
Install lxml 2.2.8 or 2.3.x
  
We recommend latest '''python 2.6.6''' from [http://www.activestate.com/activepython/downloads ActiveState Python download page] or [http://downloads.activestate.com/ActivePython/releases/2.6.6.17/ActivePython-2.6.6.17-linux-x86.tar.gz ActiveState Direct Download] or you can download from [http://www.python.org/download/releases/2.6.6/ official site].
 
  
Other versions of python 2.6.x or 2.7.x may work, but python 3.x won't yet.
+
    easy_install lxml
  
Since PyNIO is a 32 bit library, get a '''32 bit''' python even if you have 64 bit machine. If requested, we will support 64 bit binaries too.
+
In this case, Red Hat 4.1.1-52, the command installed 2.3.beta1
  
Install with command
+
More detailed instructions are at [http://codespeak.net/lxml/installation.html lxml installation instructions].
  
    $ ./install.sh -I /usr/local
+
=== NumPy ===
  
Warning: This will overwrite your existing python. It may or may not be OK to upgrade, but hardly good to downgrade. You can install several versions of python in the same machine and then edit your environment variables to use the desired version.
+
* [[Image:Numpy_logo.png]]
  
== Get the Datafed WCS Wrapper ==
+
Install numpy 1.5.1 from [http://sourceforge.net/projects/numpy/files/NumPy/ Numpy Files]. Currently there is no easy_install.
  
If you have modified the default '''OWS/web/static/index.html'', make sure '''not to overwrite''' it. Keep the main copy elsewhere and copy it back.
+
Linux testing is done with 1.5.1, but if you have 1.4.1 installed, it should work. The commands issued were:
  
Open the [http://sourceforge.net/projects/aq-ogc-services/files/ download page] in another tab.
+
    wget http://sourceforge.net/projects/numpy/files/NumPy/1.5.1/numpy-1.5.1.tar.gz/download
 +
    tar xvf  numpy-1.5.1.tar.gz
 +
    cd numpy-1.5.1
 +
    python setup.py install
  
If you are serving cubes from NetCDF files, get '''win32-ows-1.3.1.zip''' or later. This distribution contains point data support also. If you are serving only point data, you can just download '''ows-point-1.3.1.zip'''.
+
If you don't have a fortran compiler installed, for example ''gfortran'' on Debian, you will get warnings. You can ignore them, since fortran is not used.
  
    ows-point-1.3.1.zip = (win32-ows-1.3.1.zip) - (netcdf support) + (sample point provider)
+
=== PyNIO ===
  
Unzip it/both into your installation folder '''OWS'''. You can install to other folders, this is just used as an example.
+
[http://www.pyngl.ucar.edu/Nio.shtml PyNIO Home page]
  
You will to have directories '''OWS/datafed''' and '''OWS/web'''
+
Choose your operating system from [http://www.pyngl.ucar.edu/Download/list_pynio_binaries.shtml list of PyNIO binaries], write down the file name you want to download.
  
The folder datafed contains ''datafed'' package, which contains tools for ISO 8601 time
+
Start at [http://www.pyngl.ucar.edu/Download/ download] and follow the instructions. You need to request an account. Then you need to go to [http://www.earthsystemgrid.org/] and click on
parsing and some other utilities. ''Web'' is the WCS framework.
 
  
Notice:  
+
[[Image:NCL_download.png]]
  
Using C based libraries enables delivering large amounts of data efficiently. Python is used only for query parsing, making minimal impact in speed.
+
Then download '''PyNIO Version 1.4.0''', Choose '''PyNIO Version 1.4.0 precompiled binaries, OPeNDAP-enabled'''. Click on '''Download files for this collection''' and Accept license. You'll get a list of different supported systems,  
  
Although the built-in webserver is intended mainly for development, you can use it to serve data from your own workstation just fine. For high load sites, look [http://webpy.org/install webpy.org/install] documentation how to integrate webpy with apache server etc.
+
This version of PyNIO is compiled with 4.1.1 which has a known bug which occurs if all of the following conditions are true:
  
== Get 3rd Party Components ==
+
* writing a variable of any type with more than 2**32 values
 +
* the variable must have 2 or more dimensions
 +
* the platform on which the file is written must be 32-bit
 +
* the file can be either classic format or 64-bit offset format
 +
* the values to be written must begin beyond the first 2**32 values
  
* Install webpy 0.34 [[Image:webpy_favicon.png]] by following instructions at [http://webpy.org/download webpy download], or just type
+
In this case, a file offset is computed for writing using 32-bit arithmetic for a multiply, but it should be using 64-bit arithmetic. This results in an undetected integer overflow and can overwrite data earlier in the file, without an error or warning.
  
    easy_install web.py
+
This has been fixed in 4.1.2 beta 1. If you need to process very large files, you need to compile PyNIO yourself using [ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4.1.2-beta1.tar.gz netcdf-4.1.2-beta1] or later.
  
* Install lxml using [http://codespeak.net/lxml/installation.html lxml installation instructions.]. You need the utility libraries: the tested setup libxml was compiled from [ftp://xmlsoft.org/libxml2/libxml2-2.7.8.tar.gz libxml2-2.7.8.tar.gz] and libxslt from [ftp://ftp.gnome.org/pub/GNOME/sources/libxslt/1.1/libxslt-1.1.22.tar.gz libxslt-1.1.22.tar.gz] by simply unzipping them and performing
+
=== wsgilog ===
  
    ./configure ; make ; make install
+
[http://pypi.python.org/pypi/wsgilog/ WsgiLog] (0.1) is optional, only if you want to log webpy output to a file.
  
in both of the directories.
+
    easy_install wsgilog
  
After that,
+
== Install and Start the OWS Server ==
  
    easy_install lxml==2.2.8
+
=== Install ===
  
compiled and installed the lxml library.
+
* Unpack to your desired working directory. These instructions assume that your working directory is ''/usr/local/'', and these instructions assume you unzipped to ''/usr/local/OWS'' directory.
 +
* Add the path of the extracted ''OWS'' directory to your ''PYTHONPATH'' environment variable.
 +
** IIf you followed the above example, ''PYTHONPATH'' should contain ''/usr/local/OWS'' to find the OWS/datafed package.
 +
* Go to ''OWS/web'' and execute ''python ows.py'' and a server should be running on http://localhost:8080/ ready to be used.
 +
** A different port can be passed to the server as an argument, ''python ows.py 8601'' is used in the test code, don't use it.
 +
** You will have to create the log directory ''/var/log/ows'' (lowercase ows!) before running the server for the first time.
 +
** If the server is not being run as root, the user that runs it needs write access to the log directory.
 +
* You can stop the server by pressing ctrl-c in the console window.
 +
* To make the server available automatically at boot time, you can use an init script, an example script that starts the OWS server in a screen session as another user is available [http://repositories.icg.kfa-juelich.de/downloads/ows.sh here].
 +
* Typescript of installation on fresh and minimal [http://debian.org Debian] stable is [http://repositories.icg.kfa-juelich.de/downloads/debian_typescript.txt here].
  
* Install numpy 1.4.1 from [http://sourceforge.net/projects/numpy/files/NumPy/ Numpy Files]
+
=== Putting your data into CF Compatible NetCDF Files ===
  
* [[Image:Unidata_favicon.png]] Compile and install [ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4.1.2-beta1.tar.gz netcdf-4.1.2-beta1].
+
''(Not required in point)'' This is for getting the demo run only.
  
== notes/before you begin ==
+
How to use the wrapper libraries can be found in related page
* this package has been tested with Python 2.5 and should also run on version 2.6 without problems, 3.x is not supported yet
+
[http://wiki.esipfed.org/index.php/Creating_NetCDF_CF_Files Creating NetCDF-CF Files].
* system-wide installation of packages will require root access to the system
 
** it is theoretically possible to install all packages to a user directory, but it might be somewhat tricky and will not be covered here
 
* for installation of 3rd party python packages it is recommended to use [http://pypi.python.org/pypi/setuptools  easy_install]
 
** download the installer file matching your Python version and execute it on a shell as instructed
 
* 3rd party packages will receive updates over time. the last tested version of every package is noted below
 
* all required packages except for PyNIO can be installed via '''easy_install''', so don't bother to download them in advance
 
* unfortunately you will have to jump through some hoops for PyNIO. check out the [http://www.pyngl.ucar.edu/Download/ PyNIO/NGL download page] for instructions on how to obtain and install it
 
  
== required 3rd party software ==
+
Setting up the demo dataset:
You have to install the packages listed below first in order to run the OWS server. Use ''easy_install'' for all except PyNIO (see above)
 
* [http://webpy.org/ webpy] (0.32)
 
* [http://codespeak.net/lxml/ lxml] (2.2.2)
 
** might require additional development/header packages to be installed on your system (''python-dev, libxml2-dev, libxslt1-dev, zlib1g-dev'' on Debian)
 
* [http://numpy.scipy.org/ NumPy] (1.3.0)
 
** you should have a fortran compiler installed for that, for example ''gfortran'' on Debian
 
* [http://pypi.python.org/pypi/wsgilog/ WsgiLog] (0.1) (optional, only if you want to log webpy output to a file)
 
* [http://www.pyngl.ucar.edu/Nio.shtml PyNIO] (1.3.0b1)
 
** 1.3.0b1 depends on ''libgfortran1'' which is not part of debian stable/testing any more, you will have to pull it from oldstable (http://mirrors.kernel.org/debian/pool/main/g/gcc-4.1/libgfortran1_4.1.1-21_i386.deb for i386) and install it manually by copying the library file to ''/usr/lib'' and running ''ldconfig''
 
** newer releases of PyNIO will hopefully depend on a more recent libgfortran2/3
 
  
== install and start OWS server ==
+
'''OWS/web/static/testprovider/''' contains files:
* get the [http://htap.icg.fz-juelich.de/downloads/ows_linux.tar OWS code package for Linux]
+
* [http://localhost:8080/static/testprovider/CubeA.ncml CubeA.ncml] and [http://localhost:8080/static/testprovider/CubeA.nc CubeA.nc]
* unpack to your desired working directory, the contents of the archive will end up in ''./ows''
+
* [http://localhost:8080/static/testprovider/CubeB.ncml CubeB.ncml] and [http://localhost:8080/static/testprovider/CubeB.nc CubeB.nc]
** these instructions assume that your working directory is ''/usr/local/'', so the package contents would end up in ''/usr/local/ows''
+
* [http://localhost:8080/static/testprovider/CubeB.ncml H5D.ncml] and [http://localhost:8080/static/testprovider/CubeB.nc H5D.nc]
* add the path of the extracted ''ows'' directory to your ''PYTHONPATH'' environment variable
+
* [http://localhost:8080/static/testprovider/create_all.py create_all.py]
** if you followed the above example, ''PYTHONPATH'' should contain ''/usr/local/ows''
 
* cd to ''ows/web'' and execute ''python ows.py''
 
* a server should be running on http://localhost:8080/ now
 
** a different port can be passed to the server as an argument
 
** you will have to create the log directory ''/var/log/ows'' before running the server for the first time
 
**** if the server is not being run as root, the user that runs it needs write access to the log dir
 
* you can stop the server by pressing ctrl-c in the console window
 
* to make the server available automatically at boot time, you can use an init script
 
** an example script that starts the OWS server in a screen session as another user is available [http://htap.icg.fz-juelich.de/downloads/ows.sh here]
 
  
== example installations ==
+
* [http://localhost:8080/static/testprovider/index.html index.html]
=== Debian ===
+
* [http://localhost:8080/static/testprovider/wcs_capabilities.conf wcs_capabilities.conf]
* typescript of installation on fresh and minimal [http://debian.org Debian] stable: http://htap.icg.fz-juelich.de/downloads/debian_typescript.txt
 
  
= Creating Metadata =
+
*'''CubeA.ncml''' is a [http://www.unidata.ucar.edu/software/netcdf/ncml/v2.2/Tutorial.html NetCDF Markup Language] description of the test coverage, same for teh rest of the .ncml files. The .nc files are files that are created from using the ncml as instructions.
 +
*'''create_all.py''' is a script that creates the empty cube files and then fills it with test data. It servers as an example code how to create your own CF 1.0 compatible files. It has no user interface, just run it.
  
The metadata queries of WCS, '''GetCoverage''' and '''DescribeCoverage''' need to be precompiled.
+
So do the following:
  
Run the following commands:
+
    cd /usr/local/OWS/web/static/testprovider
:: cd /usr/local/OWS/web
+
    python create_all.py
:: python owsadmin.py wcs_prepare -o testprovider
 
:: or for all:
 
:: python owsadmin.py wcs_prepare -ao
 
  
-o is for optimize, -a prepares every provider
+
Now you have data to serve.
  
it will create file '''/usr/local/ows/web/static/testprovider/metadata.dat'''.
+
=== Creating Metadata ===
  
Try now test queries [http://localhost:8080/testprovider?service=WCS&version=1.1.0&Request=GetCapabilities GetCapabilities] and [http://localhost:8080/testprovider?service=WCS&version=1.1.0&Request=DescribeCoverage&identifier=CubeA DescribeCoverage CubeA]
+
The metadata queries of WCS, '''GetCoverage''' and '''DescribeCoverage''' need to be precompiled.
  
Press ctrl+C to terminate the server. Although this server is intended for development and debugging only, you can use it to server data from your own workstation just fine. For high load sites, look [http://webpy.org/install webpy.org/install] documentation how to set up a high load site.
+
Run the following commands:
  
Notice: WCS core functions are written in C/C++, enabling large amount of data delivery. Python is used only for query parsing, making minimal impact in speed.
+
    cd /usr/local/OWS/web
 +
    python owsadmin.py wcs_prepare -o testprovider
  
The server returns a short XML document, which contains a link into the query result,
+
or to extract metadata for all the providers:
you can download it by copying the url to your browser.
 
  
== index.html Pages ==
+
    python owsadmin.py wcs_prepare -ao
  
If no query is present, the server gives a default page '''index.html'''. You should provide pages for your server and for all the providers.
+
-o is for optimize, -a prepares every provider
  
The server is at '''/usr/local/ows/web/static/index.html''', which will be displayed from url [http://localhost:8080/ http://localhost:8080/]
+
it will create file '''/usr/local/OWS/web/static/testprovider/metadata.dat'''.
  
Every provider can have an own index.html like  '''/usr/local/ows/web/static/testprovider/index.html''' which will be displayed from [http://localhost:8080/testprovider http://localhost:8080/testprovider]
+
Try now test queries [http://localhost:8080/testprovider?service=WCS&acceptversions=1.1.2&Request=GetCapabilities GetCapabilities] and [http://localhost:8080/testprovider?service=WCS&version=1.1.2&Request=DescribeCoverage&identifier=CubeA DescribeCoverage CubeA]
  
If you modify the default index.html, make sure not to overwrite it next time you upgrade the WCS Wrapper package. Keep the main copy elsewhere and copy it back.
+
Press ctrl+C to terminate the server. Although this server is intended for development and debugging only, you can use it to server data from your own workstation just fine. For high load sites, look [http://webpy.org/install webpy.org/install] documentation how to set up a high load site.
  
'''Important!''' web.py development server requires, that text files have Unix-style end of line marker, plain '''LF''' and not the Windows convention '''CR-LF'''. The '''owsadmin.py''' script can be used to fix this:
+
== Where to go next ==
  
:: python /usr/local/OWS/web/owsadmin.py unix_nl "/usr/local/OWS/web/static/testprovider/index.html"
+
See [http://wiki.esipfed.org/index.php/WCS_Wrapper_Installation_WindowsOP#Put_your_own_data_online Windows Installation] for some platform-neutral information about installation.
  
Use your html editing tool support if possible.
+
How to create data for serving is documented at [[WCS_Wrapper_Configuration| WCS Wrapper Configuration]]

Latest revision as of 10:20, December 23, 2010

Back to WCS Wrapper

Project on SourceForge

Questions and comments should go to sourceforge discussions, bug reports to sourceforge tickets. Urgent issues can be asked from Kari Hoijarvi 314-935-6099(w) or 314-843-6436(h)

Last updated 2010-12-13

Things to Consider About Security

The web.py framework and WCS service code has been written with security in mind, and there are no known security bugs. There is little code that writes on the disk, so running this service should not put your computer in danger.

To maximize the security of your computer, consider the following.

  • Every file in every directory you put under OWS/web/static becomes readable by anybody. This is by design, since the whole framework is meant to publish public data. Do not put confidential information under OWS folder!
  • Don't run the process as root. Create a low-rights account like WCS_RUNNER and use that.

Do you want to install as public or private?

The place to install is usually by default:

   /usr/local

Installing to this directory, you will make the software packages available for everyone in that computer. It will also overwrite older versions, which is usually desirable but contains risks, or overwrite newer versions, which is usually not desirable. You need root privileges to install to /usr/local

Anyway, overwriting anything may break some other software, so in that case it is recommended to install somewhere else and not make the package automatically public. In that case, you don't need root privileges.

Get Python 2.6.6

Python-logo.gif

We recommend latest python 2.6.6 from ActiveState Python download page or ActiveState Direct Download or you can download from official site.

Since python is usually installed in posix-like systems, this step may be skipped. Other versions of python 2.x may work, but python 3.x won't yet.

32 bit as well as 64 bit installations have been done and work as expected. This is more of an issue for 3rd party software in any case, so make sure you install the correct versions of the required packages (as far as they contain compiled code) or python might not be able to load them.

Install with command

   # ./install.sh -I /usr/local

Warning: This will overwrite your existing python. It may or may not be OK to upgrade, but hardly good to downgrade. You can install several versions of python in the same machine and then edit your environment variables to use the desired version.

Get easy_install for python

Follow instructions at easy_install. In this case download was setuptools-0.6c11-py2.6.egg and the next command installed it:

   # sh setuptools-0.6c11-py2.6.egg

Get the Datafed WCS Wrapper

A good place to install is /usr/local/OWS but any other is OK.

If you have modified the default OWS/web/static/index.html, make sure not to overwrite' it. Keep the main copy elsewhere and copy it back.

Open the download page in another tab.

If you are serving cubes from NetCDF files, get linux-ows-1.3.1.zip or later. This distribution contains point data support also. If you are serving only point data, you can just download ows-point-1.3.1.zip.

   ows-point-1.3.1.zip = (linux-ows-1.3.1.zip) - (netcdf support) + (sample point provider)

Issue command

   unzip linux-ows-1.3.1.zip

Move the contents of the directory linux-ows-1.3.1 to OWS. You can install to other directories, this is just used as an example.

You will to have directories OWS/datafed and OWS/web

The directory datafed contains datafed package, which contains tools for ISO 8601 time parsing and some other utilities. Web is the WCS framework.

Notice:

Using C based libraries enables delivering large amounts of data efficiently. Python is used only for query parsing, making minimal impact in speed.

Although the built-in webserver is intended mainly for development, you can use it to serve data from your own workstation just fine. For high load sites, look webpy.org/install documentation how to integrate webpy with apache server etc.

Get 3rd Party Components

WebPy

  • Webpy.png

Install webpy 0.34 or newer.

   easy_install web.py

More detailed instructions are at webpy download.

lxml

  • Tagpython.png

Install lxml 2.2.8 or 2.3.x


   easy_install lxml

In this case, Red Hat 4.1.1-52, the command installed 2.3.beta1

More detailed instructions are at lxml installation instructions.

NumPy

  • Numpy logo.png

Install numpy 1.5.1 from Numpy Files. Currently there is no easy_install.

Linux testing is done with 1.5.1, but if you have 1.4.1 installed, it should work. The commands issued were:

   wget http://sourceforge.net/projects/numpy/files/NumPy/1.5.1/numpy-1.5.1.tar.gz/download
   tar xvf  numpy-1.5.1.tar.gz
   cd numpy-1.5.1
   python setup.py install

If you don't have a fortran compiler installed, for example gfortran on Debian, you will get warnings. You can ignore them, since fortran is not used.

PyNIO

PyNIO Home page

Choose your operating system from list of PyNIO binaries, write down the file name you want to download.

Start at download and follow the instructions. You need to request an account. Then you need to go to [1] and click on

NCL download.png

Then download PyNIO Version 1.4.0, Choose PyNIO Version 1.4.0 precompiled binaries, OPeNDAP-enabled. Click on Download files for this collection and Accept license. You'll get a list of different supported systems,

This version of PyNIO is compiled with 4.1.1 which has a known bug which occurs if all of the following conditions are true:

  • writing a variable of any type with more than 2**32 values
  • the variable must have 2 or more dimensions
  • the platform on which the file is written must be 32-bit
  • the file can be either classic format or 64-bit offset format
  • the values to be written must begin beyond the first 2**32 values

In this case, a file offset is computed for writing using 32-bit arithmetic for a multiply, but it should be using 64-bit arithmetic. This results in an undetected integer overflow and can overwrite data earlier in the file, without an error or warning.

This has been fixed in 4.1.2 beta 1. If you need to process very large files, you need to compile PyNIO yourself using netcdf-4.1.2-beta1 or later.

wsgilog

WsgiLog (0.1) is optional, only if you want to log webpy output to a file.

    easy_install wsgilog

Install and Start the OWS Server

Install

  • Unpack to your desired working directory. These instructions assume that your working directory is /usr/local/, and these instructions assume you unzipped to /usr/local/OWS directory.
  • Add the path of the extracted OWS directory to your PYTHONPATH environment variable.
    • IIf you followed the above example, PYTHONPATH should contain /usr/local/OWS to find the OWS/datafed package.
  • Go to OWS/web and execute python ows.py and a server should be running on http://localhost:8080/ ready to be used.
    • A different port can be passed to the server as an argument, python ows.py 8601 is used in the test code, don't use it.
    • You will have to create the log directory /var/log/ows (lowercase ows!) before running the server for the first time.
    • If the server is not being run as root, the user that runs it needs write access to the log directory.
  • You can stop the server by pressing ctrl-c in the console window.
  • To make the server available automatically at boot time, you can use an init script, an example script that starts the OWS server in a screen session as another user is available here.
  • Typescript of installation on fresh and minimal Debian stable is here.

Putting your data into CF Compatible NetCDF Files

(Not required in point) This is for getting the demo run only.

How to use the wrapper libraries can be found in related page Creating NetCDF-CF Files.

Setting up the demo dataset:

OWS/web/static/testprovider/ contains files:

  • CubeA.ncml is a NetCDF Markup Language description of the test coverage, same for teh rest of the .ncml files. The .nc files are files that are created from using the ncml as instructions.
  • create_all.py is a script that creates the empty cube files and then fills it with test data. It servers as an example code how to create your own CF 1.0 compatible files. It has no user interface, just run it.

So do the following:

   cd /usr/local/OWS/web/static/testprovider
   python create_all.py

Now you have data to serve.

Creating Metadata

The metadata queries of WCS, GetCoverage and DescribeCoverage need to be precompiled.

Run the following commands:

   cd /usr/local/OWS/web
   python owsadmin.py wcs_prepare -o testprovider

or to extract metadata for all the providers:

   python owsadmin.py wcs_prepare -ao

-o is for optimize, -a prepares every provider

it will create file /usr/local/OWS/web/static/testprovider/metadata.dat.

Try now test queries GetCapabilities and DescribeCoverage CubeA

Press ctrl+C to terminate the server. Although this server is intended for development and debugging only, you can use it to server data from your own workstation just fine. For high load sites, look webpy.org/install documentation how to set up a high load site.

Where to go next

See Windows Installation for some platform-neutral information about installation.

How to create data for serving is documented at WCS Wrapper Configuration