Talk:Web Services

This is a follow-up on today's telecon:

GMU WMS Service.

We have discussed the WCS-Modis data access from GMU to DataFed, Service_Interoperability_Tests As it is, the service is not directly usable. The problems include finding granules, encoding time query, very slow (7Kb/sec) data transfer, etc. All of these are stated in the above report and where discussed at the telecon. It is my understanding that a WCS service could be constructed by JPL/GMU, which uses BBOX and TIME to specify the location and time range of the requested data. We will be happy to help out with the implementation of the WCS service, but we cannot help with the granule finding. Rhusar 23:36, 9 May 2006 (EDT)

The granule finding problem can be solved through calls to GMU CSW(catalog service for web) service. In the CSW, user can specify the bbox, the time, and the data parameter they want, and then the CSW service will return the metadata for the matched granules. The returned metadata is detail enough for you to construct a valid WCS getCoverage call. CSW getRecord and then WCS getCoverage are the supposed OGC calling sequence for accessing large data archive. The getCapabilities in WCS is useless for large data archive. Peisheng can help you on CSW getRecord call and the construction of the WCS getCoverage call from the results. Please contact him on this. Liping

The fact that the GMU WMS/WCS service doesn't have all of the features you want is a symptom of the fact that the WCS standard does not *require* all of the features that we need. Obviously, your SOAP-ifying WCS effort and the GALEON project are heading in the right direction, but the fact that this isn't all ironed out, and isn't required as part of implementing a WCS server, currently reduces the usefulness of WCS as a universal standard for all Earth Science data needs. I'm sure the situation will improve but we aren't there yet. Thus, whether you are using WCS or SOAP services, you need some cooperation from your collaborators to agree on the *real* interface and *capabilities*.Brian D. Wilson (BrianWilson)

JPL Granule Finder Service.

Brian, it would be helpful if you would specify the URL pattern which is used to find data granules in a specific BBOX and TIME range. In other words, an HTTP Get version for your data query form. At this time it does not matter which particular dataset is queried. Hopefully, the above query for granules can have the same pattern regardless of the dataset. I would like to experiment with either the HTTP Get or the SOAP version of your granule finder service. Rhusar 23:36, 9 May 2006 (EDT)

Now, on to an explanation of SciFlo's data space/time query services. You can read all about our rationale, the detailed Use Case, and the service design in the SciFlo wiki at: http://sciflo.jpl.nasa.gov/ScifloWiki/SpaceTimeQuery Basically, a geoRegionQuery is this function call: GeoRegionQuery(datasetName, level, version, startDateTime, endDateTime, latMin, latMax, lonMin, lonMax, responseGroups) where datasetName can be 'AIRS', 'MOD04' (MODIS product), or any dataset that someone wants to "publish" via this interface. We will soon have AIRS, MODIS, MISR, GPS, AERONET, and a few others. Level means 'L2' or 'L3', or it can be blank,and version means a versionIdString, or blank for the latest version. ResponseGroups controls how much information is returned. Setting it to 'Medium' will get you object ID's, space/time metadata for the object (granule), and URL's pointing to on-line replicas of the granule files.

I claim that the interface of GeoRegionQuery is so simple and fundamental that it, or something like it, can serve as the agreed upon SOAP service interface for space/time data queries. If you look at the example SciFlo document I put on the ESIP

wiki, you'll see that I basically wrapped a WCS call to put this SOAP interface on top of it, and I accomplished that by simply authoring an XML document.

I've attached the WSDL file for the geoRegionQuery service to this email. Unfortunately, I have to resolve a firewall issue

before you will be able to call the services from this WSDL file. (Why do the sys admins rule the world?) If I wanted to call this SOAP service from python I could use the following 3 lines of code: from SOAPpy import WSDL proxy = WSDL('http://sciflo.jpl.nasa.gov/sciflo/wsdl/EOSServices.wsdl.xml') proxy.geoRegionQuery('AIRS', 'L2', None, '2003-01-03T00:00:00',

 '2003-01-03T18:00:00', -90., 90., -180., 180., 'Medium')

The service returns a ResultSet in XML format, from which one can extract space/time metadata or URL's to fetch the matched granules. In SciFlo, a "dataset" is a list of permanent object ID's (granule ID's), which at any moment may or may not be available on-line as a list of URL's. Brian D. Wilson (BrianWilson)

URL(REST) vs. SOAP Interface

JPL's GeoRegionQuery and FindDataById services are currently available via SOAP. We could create a one-line URL (HTTP GET)interface for these services, but I somewhat hesitate to do so because folks need to get beyond the barrier of making their first SOAP services call.

The business world has long ago made this jump, and all of the scientific IT folks need to make the leap. In most languages, it is two lines of code to create a proxy object from the WSDL file and then call a (service) method on that object. It takes a bit more code to publish a SOAP service, but it's also easy in most programming languages.

People should not be religious about either one-line URL's (REST) or SOAP. Not everything can be "shoehorned" into a one-line URL and the attempt to do so often leads to inconsistent, ad hoc, or not fully specified interfaces. OGC has spent a lot of effort in the last few years on cleaning up the WMS/WCS/WFS interfaces, which were unclean in the first place because they were informally specified as one-liners.

Rudy and others, I know I'm preaching to the choir here, but not everyone is fully aware of these issues. As I've said many times, an XML schema formalizing the interface should come first, and then one can describe the rules for "flattening" a service call into a one-liner. OGC has gone this route, finally. Brian D. Wilson (BrianWilson)

Interoperable Data Access.

Eric was asking if we are heading away from standard interfaces back into the dark ages of hand-crafted interfacing. Clearly, we are pursuing a path towards standard interfaces. The WCS interface has the promise to be the "universal" query language for most of the Earth Science data. So, in order to be interoperable we would have to adapt and homogenize our data access interfaces to WCS, either using HTTP Get or SOAP protocols. On this page you will find the description of the SOAP-wrapped version of WCS. SOAPifying_WCS Rhusar 23:36, 9 May 2006 (EDT)

Interoperability Experiments.

The pursuit of WCS as the "universal" data access interface is the topic of other interoperability experiments, GALEON, GSN, both coordinated by OGC. We have added a page to the ESIP wiki that lists and briefly describes these efforts. Service_Interoperability_Experiments Rhusar 23:36, 9 May 2006 (EDT)

Rationale for WCS.

We have been maintaining a page on ESIP wiki in which the rationale for WCS as well as its main features are described. WCS_for_Earth_Science_Data_Access. At this time the page only contains ppt slides. However, this page along with its "talk" discussion page can serve for collecting and exchanging our respective ideas on this matter. Rhusar 23:36, 9 May 2006 (EDT)