ESIP Testbed

The ESIP Testbed - hosted by the Products & Services Committee

Objectives: Provide a test environment where prototype standards, services, protocols, and best practices can be explored and evaluated. The testbed serves as a forum for innovative collaborations across all sectors of the Federation to improve availability and access to our member products and services for mutual benefit. Comments about each testbed activity can be shared on this page. Also find information on Past Testbed Tasks here.

Current or Ongoing Testbed Activities

Amazon Web Services best practices

ESIP has opened an Amazon Web Services account that can be leveraged to provide virtual instances for accessing the testbed projects. In an effort to save pain and share experiences, please read and contribute to the ESIP Testbed Amazon Web Services Best Practices topic if you are going to work on the Amazon instances or have and want to contribute insight.

Testbed Portal

(Chen Xu, Jing Li, Chaowei Yang) The ESIP Testbed needs a more prominent web presence, both for visibility to the ESIP community, but also for easy access and eventual utility of community as well. The development team was tasked with developing a Testbed portal component in drupal that can be easily dropped into the latest ESIP portal. The current test version of the portal has a temporary home at... http://swp.gmu.edu/esiptestbed/ and is available to all ESIP members for testing.

Ultimate Benefit: Allow ESIP members to interact with hosted testbed activities and testbed task developers, and make ESIP testbed activities more visible to the public.

Comments:

Discovery Services and Clients

(Christine White) The Preservation and Stewardship Cluster and the NASA Technology Infusion Working Group have been considering permanent naming schemes for data products http://wiki.esipfed.org/index.php/Preservation_and_Stewardship. These identifiers can serve as references in journal articles and must include versioning representations. Many naming options have been promoted, but the best choices for Earth science data require careful examination. Two datasets may differ only in format, byte order, data type, access method, etc., creating facets (dimensions) not relevant to classification schemes for books (Library of Congress, Dewey Decimal). This testbed registers several Federation datasets to multiple naming conventions to evaluate the practical differences between them.

There is more information about this task provided here using a template for the "ESIP State of the Art" documentation

Ultimate Benefit:

Comments:

Data and information Quality

(Abdelmounaam Rezgui, Zhipeng Gui, Min Sun, Chaowei Yang) An automatic classification/annotation system that assesses, monitors, and accurately reports on the quality of ESIP data and services. The project will include: (1) a quality model and classification engine that will establish a set of quality metrics for data and services. The engine will automatically derive the quality of ESIP products and services, (2) the effort will also work on metadata quality which is not usually addressed, and (3) the activity will take into account feedback from users to help rate quality of data and services.

Ultimate Benefit:

Comments:

Re-usable Metadata Editor

(Jerry Yun Pan, Nigel Banks)
Develop a generic, reusable software system to facilitate the support for multiple metadata standards and their variations. The tool will be flexible and reusable for multiple metadata standards, and allows an administrator to design and tailor the metadata authoring tool/editor according to the targeted metadata schema without writing new code. The core of the tool suite consists of two parts: (1) a designing tool for "super" users who are responsible for designing the metadata editors, and (2) a rendering engine that makes use of a pre-made metadata editor definition. The designing tool defines a metadata editor based on user inputs and saves the definition for reuse. The rendering engine makes use of the definitions to facilitate metadata authoring and editing. The "editor-of-editors" is schema driven. The design tool allows for the selection of a subset of a whole schema (a "profile") to form an editor, or the selection of an extension of a schema. The editor definitions can be exported and shared among multiple installations. A cloud-based implementation of the editor is available at... http://ec2-50-19-146-114.compute-1.amazonaws.com/

Ultimate Benefit: : A general purpose metadata authoring and editing tool that is easily shareable across organizations. The code is open source for free use.

Comments:

Data Stewardship

(Greg Janee and Nancy Hoebelheinrich) The datasets to be addressed will include a relatively simple image collection and a second containing granule-‐level data objects such as a longtime series from multiple sensors / satellites. The project tasks include: (1) Preparing, transforming and performing quality control tasks on the metadata for each dataset in a storage environment that can be queried, and appended to add the identifiers from each scheme to each entity in the two datasets, (2) Map the existing metadata for each dataset into the metadata requirements for each identifier scheme for the purposes of identification and citation, (3) Track and discuss the implementation issues associated with each task per the questions previously identified by the Data Stewardship & Preservation cluster (see the initial list on the ESIP wiki at: http://wiki.esipfed.org/index.php/Implementation_Issues_to_be_addressed ), and others as they arise, (4) Bring implementation issues to the Data Stewardship cluster as needed for discussion and resolution/decision, (5) Develop list of practical considerations for each identifier scheme, and (6) develop draft set of best practices for discussion at future ESIP Federation meetings.

Ultimate Benefit: Provides a reference implementation/example of mapping dataset metadata requirements for the purpose of identification and citation.

Comments:

ESIP Ontologies in the Cloud

(Line Pouchard, Michael Huhns) This project will install BioPortal, a mature open source ontology repository developed at Stanford University. The BioPortal technology is currently hosting biological and omics ontologies for NIH. Once the technology portal is installed under the ESIP banner, we will populate it with ESIP ontologies and ontologies related to the Earth and Environmental Sciences relevant to ESIP members. The result of this task will be a full‐feature ontology portal, accessible from the Web, that allows ESIP members to upload, find, modify, and share their ontologies.

Ultimate Benefit: Provides a single site where the ESIP ontologies can be found and where ESIP members can share their ontologies with each other and build additional constructs on each other’s foundation

Comments:

Testbed FY 2011 Proposal Effort

The Products and Services committee submitted a proposal for new Testbed activities to the ESIP Finance and Appropriations Committee for suggested funding of the efforts. The proposal is available for review here (doc) and (pdf) .

The funding proposal was approved so below is the Request for Proposal solicitation to all ESIP members to encourage proposals for participation in building the new Testbed functionality. (pdf)

The Testbed Configuration Board decided Task 1 (Testbed Portal Development) needed to be kept more focused on the Drupal development of a site that would integrate with the ESIP portal, so it has been re-drafted and sent back out for new responses. The revised RFP for Task 1 is here: (pdf).