Difference between revisions of "P&S Telecon - October 20, 2015"

From Earth Science Information Partners (ESIP)
(Created page with "== November RFP Q&A == Opportunity to ask questions about the [http://testbed.esipfed.org/node/8752 November 2015 Testbed RFP] == Updates on the Testbed Evaluation Framework ...")
 
 
Line 1: Line 1:
 
== November RFP Q&A ==
 
== November RFP Q&A ==
 
Opportunity to ask questions about the [http://testbed.esipfed.org/node/8752 November 2015 Testbed RFP]
 
Opportunity to ask questions about the [http://testbed.esipfed.org/node/8752 November 2015 Testbed RFP]
 +
* Thanks all for your help preparing the RFP!
  
 
== Updates on the Testbed Evaluation Framework ==
 
== Updates on the Testbed Evaluation Framework ==
 
+
Update provided by Annie B.
 +
* We're 3 weeks into the evaluation.
 +
* 3 Types of evaluation teams:
 +
** Domain scientists - federal sciences
 +
** ESIP-centric team
 +
** Private industry, academic, and a mix
 +
* All approaching it differently:
 +
** ARIA-MH, Hook's project
 +
*** Issue with software access and proprietary imagery
 +
*** So evaluation has to happen through Telecons
 +
*** Evaluators look at software and tech that is not at their fingertips
 +
*** Learning how to do that - group call yesterday. Team used the OSF wiki to brainstorm on how to approach this sort of online evaluation
 +
*** Will have two 2-hour long calls with the PI to come up with a Test Plan with the PI
 +
*** And then another call with the PI to execute the test plan, then evaluate if needs an additional call
 +
** Collaborative Workbench, Manil's project
 +
*** Using Slack to collaborate about evaluation
 +
*** Will meet with PI to go through
 +
*** Backend of evaluation hosted on AWS, for the 6 months of the evaluations
 +
** Fusion of Hurricane Models and Observations: Developing the Technology to Improve the Forecasts
 +
*** All evaluators have in-depth knowledge, hit the ground running
 
== Winter Meeting Planning Sessions ==
 
== Winter Meeting Planning Sessions ==
 
Continued planning for P&S and Testbed Winter Meeting activities - Last year's agenda here: https://docs.google.com/spreadsheets/d/1yLY2964NlwJY_S98vlc2hRe6Ar1cthVMG_dr78BdVtU/edit#gid=0
 
Continued planning for P&S and Testbed Winter Meeting activities - Last year's agenda here: https://docs.google.com/spreadsheets/d/1yLY2964NlwJY_S98vlc2hRe6Ar1cthVMG_dr78BdVtU/edit#gid=0
  
=== Testbed projects ===
+
Idea: A block of 3 sessions, maybe a full morn/afternoon
* Disaster Cluster, Fast Track
+
* Start with "What is the Testbed/P&S" session - Christine to Submit
* Other Projects from November RFP that will be awarded by then
+
* Then TEF/AIST session - Annie already Submitted
 +
* Other Testbeds/collaboration session - Emily and Ken to Submit
  
=== AIST Testbed Session ===
+
===What is the Testbed Session===
 +
* Fun, interactive overview of P&S, and Testbed/FUNding Friday
 +
* Introduce TEF, but not in-depth
 +
 
 +
=== Testbed Evaluation Framework/AIST Testbed Session ===
 
* Overview, Working Meeting
 
* Overview, Working Meeting
 
* Evaluators report out
 
* Evaluators report out
+
* Bring in potential other projects, and evaluators
=== FUNDing Friday Posters ===
+
* Talk about next round too - when, how?
* Need to send email to awardees to submit "session"? Or, encourage a session in which they should participate?
+
* Sit down with the group, talk through what worked, what didn't work, and how we can refine this process
 +
* Best Practices - for example:
 +
** Regardless of TRL, the PI & the Evaluator should meet the first week to go through the technologies
 +
*** Tech Overview Call
 +
*** Lack of documentation from PI - understand what they are trying to do, vs what evaluator thinks they are trying to do
 +
*** "What are the goals of this?"
 +
*** "If this is your goal, then this is what we see as best practice to accomplish the goal"
 +
* What evaluators get out of it:
 +
** 5K in funding and travel to the Winter Meeting to participate, and travel to AGU for the Independent Eval Session
 +
** In the future it might not be 5K b/c of fed limits on accepting salary -
 +
** Might be interesting to think about the Type relationship to the "what do you get" (i.e., Type IIIs, where the budget might be helpful)
 +
* Flexibility
 +
** Varying Access and TRL levels
 +
 
 +
=== Technology Evaluation Session: How we support evaluation, and build components to do so ===
 +
* Earth Science Community Testbeds - broader, diff. orgs that have Testbeds
 +
* Status, highlight capabilities
 +
* Conversation, see if there are collaborative opps
 +
* EarthCube Testbed & ESIP Testbed Collaboration discussion
 +
** Support '''integration''' of technologies, components and services that support EarthCube infrastructure; cloud platform, evaluation framework
 +
* RDA Testbed
 +
* OGC Testbed (?)
 +
* NDS
 +
=== Awarded Testbed Projects & FUNding Friday Projects===
 +
* Testbed:
 +
** Disaster Lifecycle - Expanding CCOP: Will be supported in Disaster Cluster sessin
 +
** John's Fast Track Project: Maybe highlight as part of AIST session?
 +
* Other Projects from November RFP that will be awarded by then - reminder to awardees that they can leverage the Winter Meeting to kick-off
 +
* FUNDing Friday Posters
 +
** Nancy to reach out to Brandon Whitehead and Sophie Hou regarding posters and session-related presentations
 +
** Soren and Christine to collaborate on reaching out to other projects
  
 
== Other logistics ==
 
== Other logistics ==
Line 23: Line 78:
 
** Budget template
 
** Budget template
 
** "Adoption" template (see nice example from Beta release of AppEEARS)
 
** "Adoption" template (see nice example from Beta release of AppEEARS)
* Next P&S Meetings: Nov. 24th (Tues before Thanksgiving) and Dec. 22nd (before Christmas). Do we need to adjust these?
+
* Next P&S Meetings: Nov. 17th (Tues before Thanksgiving) and Dec. 15th (before AGU). Nov. 17th OK; for December, we'll check in on Nov. call.
 
* Timeframe for TCB to evaluate proposals: First week of December
 
* Timeframe for TCB to evaluate proposals: First week of December

Latest revision as of 13:30, October 20, 2015

November RFP Q&A

Opportunity to ask questions about the November 2015 Testbed RFP

  • Thanks all for your help preparing the RFP!

Updates on the Testbed Evaluation Framework

Update provided by Annie B.

  • We're 3 weeks into the evaluation.
  • 3 Types of evaluation teams:
    • Domain scientists - federal sciences
    • ESIP-centric team
    • Private industry, academic, and a mix
  • All approaching it differently:
    • ARIA-MH, Hook's project
      • Issue with software access and proprietary imagery
      • So evaluation has to happen through Telecons
      • Evaluators look at software and tech that is not at their fingertips
      • Learning how to do that - group call yesterday. Team used the OSF wiki to brainstorm on how to approach this sort of online evaluation
      • Will have two 2-hour long calls with the PI to come up with a Test Plan with the PI
      • And then another call with the PI to execute the test plan, then evaluate if needs an additional call
    • Collaborative Workbench, Manil's project
      • Using Slack to collaborate about evaluation
      • Will meet with PI to go through
      • Backend of evaluation hosted on AWS, for the 6 months of the evaluations
    • Fusion of Hurricane Models and Observations: Developing the Technology to Improve the Forecasts
      • All evaluators have in-depth knowledge, hit the ground running

Winter Meeting Planning Sessions

Continued planning for P&S and Testbed Winter Meeting activities - Last year's agenda here: https://docs.google.com/spreadsheets/d/1yLY2964NlwJY_S98vlc2hRe6Ar1cthVMG_dr78BdVtU/edit#gid=0

Idea: A block of 3 sessions, maybe a full morn/afternoon

  • Start with "What is the Testbed/P&S" session - Christine to Submit
  • Then TEF/AIST session - Annie already Submitted
  • Other Testbeds/collaboration session - Emily and Ken to Submit

What is the Testbed Session

  • Fun, interactive overview of P&S, and Testbed/FUNding Friday
  • Introduce TEF, but not in-depth

Testbed Evaluation Framework/AIST Testbed Session

  • Overview, Working Meeting
  • Evaluators report out
  • Bring in potential other projects, and evaluators
  • Talk about next round too - when, how?
  • Sit down with the group, talk through what worked, what didn't work, and how we can refine this process
  • Best Practices - for example:
    • Regardless of TRL, the PI & the Evaluator should meet the first week to go through the technologies
      • Tech Overview Call
      • Lack of documentation from PI - understand what they are trying to do, vs what evaluator thinks they are trying to do
      • "What are the goals of this?"
      • "If this is your goal, then this is what we see as best practice to accomplish the goal"
  • What evaluators get out of it:
    • 5K in funding and travel to the Winter Meeting to participate, and travel to AGU for the Independent Eval Session
    • In the future it might not be 5K b/c of fed limits on accepting salary -
    • Might be interesting to think about the Type relationship to the "what do you get" (i.e., Type IIIs, where the budget might be helpful)
  • Flexibility
    • Varying Access and TRL levels

Technology Evaluation Session: How we support evaluation, and build components to do so

  • Earth Science Community Testbeds - broader, diff. orgs that have Testbeds
  • Status, highlight capabilities
  • Conversation, see if there are collaborative opps
  • EarthCube Testbed & ESIP Testbed Collaboration discussion
    • Support integration of technologies, components and services that support EarthCube infrastructure; cloud platform, evaluation framework
  • RDA Testbed
  • OGC Testbed (?)
  • NDS

Awarded Testbed Projects & FUNding Friday Projects

  • Testbed:
    • Disaster Lifecycle - Expanding CCOP: Will be supported in Disaster Cluster sessin
    • John's Fast Track Project: Maybe highlight as part of AIST session?
  • Other Projects from November RFP that will be awarded by then - reminder to awardees that they can leverage the Winter Meeting to kick-off
  • FUNDing Friday Posters
    • Nancy to reach out to Brandon Whitehead and Sophie Hou regarding posters and session-related presentations
    • Soren and Christine to collaborate on reaching out to other projects

Other logistics

  • "Templates" for projects
    • Scope of work template
    • Budget template
    • "Adoption" template (see nice example from Beta release of AppEEARS)
  • Next P&S Meetings: Nov. 17th (Tues before Thanksgiving) and Dec. 15th (before AGU). Nov. 17th OK; for December, we'll check in on Nov. call.
  • Timeframe for TCB to evaluate proposals: First week of December