Difference between revisions of "NOAA Sessions on NIDIS, CLASS, the Data Centers, and much much more!"

From Earth Science Information Partners (ESIP)
m (1 revision imported)
 
(21 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 
NOAA is hosting a series of breakout sessions at the Summer ESIP Meeting and we hope you will participate.  In all of these sessions, we hope to inform you of some things going on within NOAA, and more importantly, be informed BY you.  Your input, feedback, and advice is critical to us!  Please join in the discussions.  Our initial list of topics is provided below, and of course we welcome your ideas as well... this is a wiki after all, so please feel free to add your ideas here as well.
 
NOAA is hosting a series of breakout sessions at the Summer ESIP Meeting and we hope you will participate.  In all of these sessions, we hope to inform you of some things going on within NOAA, and more importantly, be informed BY you.  Your input, feedback, and advice is critical to us!  Please join in the discussions.  Our initial list of topics is provided below, and of course we welcome your ideas as well... this is a wiki after all, so please feel free to add your ideas here as well.
  
 +
Overall Organizer:  Ken Casey (Kenneth.Casey@noaa.gov)
 +
 +
== National Integrated Drought Information System (NIDIS) ==
 +
* "how do we organize a larger activity based upon what we learn from smaller activities". 
 +
*Discussion led by Mike Brewer (Michael.J.Brewer@noaa.gov) 
 +
 +
== The NEW Comprehensive Large Array Data Stewardship System (CLASS) - NOAA Data Center Paradigm ==
 +
* "how the Open Archival Information System Reference Model is leading us into the light". 
 +
* Discussion led by Ken Casey (Kenneth.Casey@noaa.gov)
 +
 +
'''Report:'''
 +
 +
Over 25 people participated in this breakout discussion, which was lively and vigorous.  Our main goal in this breakout session was to begin breaking down the broad misunderstandings regarding the relationship between CLASS and the NOAA Data Centers.  Ken Casey led the discussion, which began with an explanation of how the OAIS Reference Model has helped  NOAA/NESDIS clarify the different roles and responsibilities.  The new paradigm clearly assigns CLASS solely an IT infrastructure role in support of the NOAA Data Centers and their archive missions. 
 +
 +
Three key conclusions were drawn:
 +
 +
1. The group did indeed develop a much better understanding of the new Data Center - CLASS relationships.  There were many questions along these lines and broad agreement regarding the new roles.  There was also consensus that those of us in NOAA do need to take this message out into the communities of interest, and that after doing that for some amount of time that we could eventually stop talking about CLASS as a a separate entity.
 +
 +
2. At future ESIP Federation meetings, there was agreement in the group that ESIP could help NOAA (the Data Centers and CLASS) develop and implement the connections that are needed for the Data Centers to effectively leverage CLASS capabilities to better serve the users.
 +
 +
3. As confusion regarding CLASS roles is eliminated, the ESIP meeting attendees wanted to begin talking more about the Access functions, and to an extent, the Ingest functions.
 +
 +
The following powerpoint presentation (not given, but a couple slides were referenced) and images were used to help clarify aspects of the discussion.
 +
 +
[[Image:OAIS_Overview_22Apr2008.ppt]]
 +
 +
[[Image:OAIS_FunctionalEntities.jpg]]
 +
 +
[[Image:OAIS_RM_Fig6-4_SharedStorage.jpg]]
 +
 +
== The CLASS Application Programming Interface - SNAAP! ==
 +
 
 +
* "could it really be that easy?".  (of course not!)
 +
* Discussion led by Ken Casey (Kenneth.Casey@noaa.gov)
 +
 +
'''Report:'''
 +
 +
Eric Kihn, the lead developer for SNAAP, could not be at the meeting
 +
but in his absence we attempted to have a meaningful conversation based on
 +
the presentation he provided below.
  
Overall Organizer:  Ken Casey (Kenneth.Casey@noaa.gov)
+
[[Image:SNAAP_ESIP08.ppt]]
 +
 
 +
About 21 people from across the Federation participated.  It was clear almost immediately that the questions being raised were beyond the other NOAA personnel present. It was also clear that there was a lot of technical interest in the approaches being pursued in this prototype to deliver machine-to-machine access to NOAA's archive holdings, both those physically residing in the CLASS IT infrastructure as well as in other NOAA Data Center IT systems. 
 +
 
 +
The group identified the topic of how NOAA should provide API access to its archive data holdings as a key item to take up at the January meeting, with a technical focus on the SNAAP prototype.
 +
 
 +
== The Evolution of Data Management at NOAA ==
 +
 
 +
* "We are data management - how NOAA is living up to this quote form the NOAA Administrator at a previous ESIP meeting". 
 +
* Discussion led by Ken McDonald (Kenneth.Mcdonald@noaa.gov)
 +
 
 +
'''Report:'''
 +
 
 +
The Evolution of Data Management at NOAA:
  
 +
* Reviewed the NOAA oversight of data management activities – NOSC, DMC, and DMIT
 +
* Introduced GEO-IDE initiative and concept
 +
* Gave overview of concept of operations, implementation plan, and sub-teams
 +
* Reviewed several NOAA projects that have self-identified as GEO-IDE pilot projects
  
* '''National Integrated Drought Information System (NIDIS)''' - "how do we organize a larger activity based upon what we learn from smaller activities"Discussion led by Mike Brewer (Michael.J.Brewer@noaa.gov) 
+
Issues/Questions Discussed:
 +
* SOA raises lots of questions:
 +
**How is the chaining of services being thought about?
 +
* The need to ensure reliability of individual components
 +
* There are trust and governance issues and
 +
* Standard notifications
 +
* Connection with other NOAA organizations (e.g. CIO, others) and projects is critical to moving forward successfully on GEO-IDE
 +
* Documentation and training for projects to implement and utilize standards is needed (funding?)
 +
* Effective mechanisms for people who are using standards to share information about this to other projects would help – infrastructure and some funding needed for thisFinding ways to take advantage of existing information.
 +
** How can we make “sharing this information” between and across projects part of how they are successful
 +
* Need to ensure that NOAA’s standards process is connected to applicable international standards bodies/efforts
 +
* Need to evaluate or document the cost of “not using standards”
 +
* What is role of data users in helping to define the processes/procedures/etc of GEO-IDE?
 +
** Answer – the teams that have been initiated are made up of users, producers, and people who interact with the external users
 +
** Need to involve archives in early phase of the process to ensure that long-term preservation is taken into consideration
  
* '''The NEW Comprehensive Large Array Data Stewardship System (CLASS) - NOAA Data Center Paradigm''' - "how the Open Archival Information System Reference Model is leading us into the light".  Discussion led by Ken Casey (Kenneth.Casey@noaa.gov)
+
== Metadata - One size does fit all!!==
 +
* Discussion led Ted Habermann (Ted.Habermann@noaa.gov)
  
* '''The CLASS Application Programming Interface - SNAAP!'''  - "could it really be that easy?". Discussion led by Bob Rank (Robert.Rank@noaa.gov)/Ken Casey (Kenneth.Casey@noaa.gov)
+
This discussion requires that you suspend your disbelief and imagine for a bit that the title is true. Imagine that there is a metadata standard that can be used successfully for many data types served, preserved, and used by NOAA and our ESIP partners. I will use the current ISO 19115 and 19115-2 standards as a straw-person for that imaginary "universal" metadata standard.
  
* '''The Evolution of Data Management at NOAA''' - "We are data management - how NOAA is living up to this quote form the NOAA Administrator at a previous ESIP meeting".  Discussion led by Ken McDonald (Kenneth.Mcdonald@noaa.gov)
+
This session was interesting and informative. The results are on the [[ISO Metadata Features]] page.
  
* '''Metadata''' - "One size does fit all!!".  Discussion led Ted Habermann (Ted.Habermann@noaa.gov)
+
[[Category: ISO Metadata]]

Latest revision as of 09:53, October 8, 2021

NOAA is hosting a series of breakout sessions at the Summer ESIP Meeting and we hope you will participate. In all of these sessions, we hope to inform you of some things going on within NOAA, and more importantly, be informed BY you. Your input, feedback, and advice is critical to us! Please join in the discussions. Our initial list of topics is provided below, and of course we welcome your ideas as well... this is a wiki after all, so please feel free to add your ideas here as well.

Overall Organizer: Ken Casey (Kenneth.Casey@noaa.gov)

National Integrated Drought Information System (NIDIS)

  • "how do we organize a larger activity based upon what we learn from smaller activities".
  • Discussion led by Mike Brewer (Michael.J.Brewer@noaa.gov)

The NEW Comprehensive Large Array Data Stewardship System (CLASS) - NOAA Data Center Paradigm

  • "how the Open Archival Information System Reference Model is leading us into the light".
  • Discussion led by Ken Casey (Kenneth.Casey@noaa.gov)

Report:

Over 25 people participated in this breakout discussion, which was lively and vigorous. Our main goal in this breakout session was to begin breaking down the broad misunderstandings regarding the relationship between CLASS and the NOAA Data Centers. Ken Casey led the discussion, which began with an explanation of how the OAIS Reference Model has helped NOAA/NESDIS clarify the different roles and responsibilities. The new paradigm clearly assigns CLASS solely an IT infrastructure role in support of the NOAA Data Centers and their archive missions.

Three key conclusions were drawn:

1. The group did indeed develop a much better understanding of the new Data Center - CLASS relationships. There were many questions along these lines and broad agreement regarding the new roles. There was also consensus that those of us in NOAA do need to take this message out into the communities of interest, and that after doing that for some amount of time that we could eventually stop talking about CLASS as a a separate entity.

2. At future ESIP Federation meetings, there was agreement in the group that ESIP could help NOAA (the Data Centers and CLASS) develop and implement the connections that are needed for the Data Centers to effectively leverage CLASS capabilities to better serve the users.

3. As confusion regarding CLASS roles is eliminated, the ESIP meeting attendees wanted to begin talking more about the Access functions, and to an extent, the Ingest functions.

The following powerpoint presentation (not given, but a couple slides were referenced) and images were used to help clarify aspects of the discussion.

File:OAIS Overview 22Apr2008.ppt

OAIS FunctionalEntities.jpg

OAIS RM Fig6-4 SharedStorage.jpg

The CLASS Application Programming Interface - SNAAP!

  • "could it really be that easy?". (of course not!)
  • Discussion led by Ken Casey (Kenneth.Casey@noaa.gov)

Report:

Eric Kihn, the lead developer for SNAAP, could not be at the meeting but in his absence we attempted to have a meaningful conversation based on the presentation he provided below.

File:SNAAP ESIP08.ppt

About 21 people from across the Federation participated. It was clear almost immediately that the questions being raised were beyond the other NOAA personnel present. It was also clear that there was a lot of technical interest in the approaches being pursued in this prototype to deliver machine-to-machine access to NOAA's archive holdings, both those physically residing in the CLASS IT infrastructure as well as in other NOAA Data Center IT systems.

The group identified the topic of how NOAA should provide API access to its archive data holdings as a key item to take up at the January meeting, with a technical focus on the SNAAP prototype.

The Evolution of Data Management at NOAA

  • "We are data management - how NOAA is living up to this quote form the NOAA Administrator at a previous ESIP meeting".
  • Discussion led by Ken McDonald (Kenneth.Mcdonald@noaa.gov)

Report:

The Evolution of Data Management at NOAA:

  • Reviewed the NOAA oversight of data management activities – NOSC, DMC, and DMIT
  • Introduced GEO-IDE initiative and concept
  • Gave overview of concept of operations, implementation plan, and sub-teams
  • Reviewed several NOAA projects that have self-identified as GEO-IDE pilot projects

Issues/Questions Discussed:

  • SOA raises lots of questions:
    • How is the chaining of services being thought about?
  • The need to ensure reliability of individual components
  • There are trust and governance issues and
  • Standard notifications
  • Connection with other NOAA organizations (e.g. CIO, others) and projects is critical to moving forward successfully on GEO-IDE
  • Documentation and training for projects to implement and utilize standards is needed (funding?)
  • Effective mechanisms for people who are using standards to share information about this to other projects would help – infrastructure and some funding needed for this. Finding ways to take advantage of existing information.
    • How can we make “sharing this information” between and across projects part of how they are successful
  • Need to ensure that NOAA’s standards process is connected to applicable international standards bodies/efforts
  • Need to evaluate or document the cost of “not using standards”
  • What is role of data users in helping to define the processes/procedures/etc of GEO-IDE?
    • Answer – the teams that have been initiated are made up of users, producers, and people who interact with the external users
    • Need to involve archives in early phase of the process to ensure that long-term preservation is taken into consideration

Metadata - One size does fit all!!

  • Discussion led Ted Habermann (Ted.Habermann@noaa.gov)

This discussion requires that you suspend your disbelief and imagine for a bit that the title is true. Imagine that there is a metadata standard that can be used successfully for many data types served, preserved, and used by NOAA and our ESIP partners. I will use the current ISO 19115 and 19115-2 standards as a straw-person for that imaginary "universal" metadata standard.

This session was interesting and informative. The results are on the ISO Metadata Features page.