Difference between revisions of "ESIP FedSearchWorkshop 2010"

From Earth Science Information Partners (ESIP)
(Reverted edits by Clynnes (talk) to last revision by Hook)
 
(One intermediate revision by one other user not shown)
Line 2: Line 2:
 
For newbies and experts alike!
 
For newbies and experts alike!
 
===What is the ESIP Federated Search framework? (Lynnes - 10 min)===
 
===What is the ESIP Federated Search framework? (Lynnes - 10 min)===
*[[Media:ESIP_FedSearch.pdf]]
 
 
 
==Client / Server Demos and Implementations==
 
==Client / Server Demos and Implementations==
 
=== Mirador Client and Server (Lynnes - 5 min)===
 
=== Mirador Client and Server (Lynnes - 5 min)===
Line 9: Line 7:
  
 
===ECHO Server and Client (Newman - 10 min)===
 
===ECHO Server and Client (Newman - 10 min)===
ECHO provided an ESIP server following specifications from the GES-DISC and created a highly-used client almost as an after thought.
 
*[[Media:ECHO-ESIP.pptx]]
 
  
 
=== WSNEWS Reusable Solr-based Server + Python downloader client (Hua - 10 min)===
 
=== WSNEWS Reusable Solr-based Server + Python downloader client (Hua - 10 min)===
 
We will show our Federated OpenSearch server which was designed from the ground up to be simple, fast, lightweight, and portable. The talk will walk through our server implementation which extended Apache Solr (with its fast full-text search) with added space+time constraints. We will also show a simple and lightweight client used to automatically download all granules from a search result.
 
We will show our Federated OpenSearch server which was designed from the ground up to be simple, fast, lightweight, and portable. The talk will walk through our server implementation which extended Apache Solr (with its fast full-text search) with added space+time constraints. We will also show a simple and lightweight client used to automatically download all granules from a search result.
*[[Media:WSNEWS_Federated_OpenSearch_Server_and_Client.pdf]]
 
  
 
===OpenSearch Perl Client (Duggan - 10 min)===
 
===OpenSearch Perl Client (Duggan - 10 min)===
Line 24: Line 19:
 
to handle a lot of edge cases and has been heavily
 
to handle a lot of edge cases and has been heavily
 
optimized.
 
optimized.
*[[Media:data_downloader.pdf]]
 
  
 
= Work It Out (45 min) =
 
= Work It Out (45 min) =
Line 30: Line 24:
  
 
Actually, we will briefly discuss each item, then designate a book boss to lead a solution to the issue.
 
Actually, we will briefly discuss each item, then designate a book boss to lead a solution to the issue.
* Resolving duplicate entries
+
* Resolving duplicate entries from multiple servers
** Datasets from multiple servers
+
* Handling exceptions
** Granules from same or multiple servers
 
* Handling exceptions (Doug Newman)
 
* Extending search to other attributes (Ruth Duerr)
 
** Granule level
 
** Dataset level
 
* Recommendations/Best Practices for response contents
 
** Space and time extents
 
 
* Versioning the ESIP Framework
 
* Versioning the ESIP Framework
 
* Granule heterogeneity within datasets
 
* Granule heterogeneity within datasets
Line 45: Line 32:
 
** OPeNDAP (TBD)
 
** OPeNDAP (TBD)
 
** Integration with Service and Data casting?
 
** Integration with Service and Data casting?
* Addressing attribution and provenance and local data management issues
+
* Others?
* Addressing semantic heterogeneity problems in keyword search (now) an d attribute search (later)
 

Latest revision as of 22:20, July 18, 2012

Intro and Demos (45 min)

For newbies and experts alike!

What is the ESIP Federated Search framework? (Lynnes - 10 min)

Client / Server Demos and Implementations

Mirador Client and Server (Lynnes - 5 min)

We'll show the Mirador client accessing ECHO, GHRC and WSNEWS (see below).

ECHO Server and Client (Newman - 10 min)

WSNEWS Reusable Solr-based Server + Python downloader client (Hua - 10 min)

We will show our Federated OpenSearch server which was designed from the ground up to be simple, fast, lightweight, and portable. The talk will walk through our server implementation which extended Apache Solr (with its fast full-text search) with added space+time constraints. We will also show a simple and lightweight client used to automatically download all granules from a search result.

OpenSearch Perl Client (Duggan - 10 min)

I'd like to talk about and demo our implementation; it is an opensearch/federated search client written in Perl. We've been using it in production for almost a year where it's been mirroring data from our processing system to a shared computing environment. It has had to handle a lot of edge cases and has been heavily optimized.

Work It Out (45 min)

where we work on unresolved issues in the ESIP Federated Search framework. (Newbies are welcome to watch us make sausage...yum!)

Actually, we will briefly discuss each item, then designate a book boss to lead a solution to the issue.

  • Resolving duplicate entries from multiple servers
  • Handling exceptions
  • Versioning the ESIP Framework
  • Granule heterogeneity within datasets
  • Incorporating data services
    • Pomegranate and w10n-sci
    • OPeNDAP (TBD)
    • Integration with Service and Data casting?
  • Others?