P&S Telecon - October 20, 2015

From Earth Science Information Partners (ESIP)

November RFP Q&A

Opportunity to ask questions about the November 2015 Testbed RFP

  • Thanks all for your help preparing the RFP!

Updates on the Testbed Evaluation Framework

Update provided by Annie B.

  • We're 3 weeks into the evaluation.
  • 3 Types of evaluation teams:
    • Domain scientists - federal sciences
    • ESIP-centric team
    • Private industry, academic, and a mix
  • All approaching it differently:
    • ARIA-MH, Hook's project
      • Issue with software access and proprietary imagery
      • So evaluation has to happen through Telecons
      • Evaluators look at software and tech that is not at their fingertips
      • Learning how to do that - group call yesterday. Team used the OSF wiki to brainstorm on how to approach this sort of online evaluation
      • Will have two 2-hour long calls with the PI to come up with a Test Plan with the PI
      • And then another call with the PI to execute the test plan, then evaluate if needs an additional call
    • Collaborative Workbench, Manil's project
      • Using Slack to collaborate about evaluation
      • Will meet with PI to go through
      • Backend of evaluation hosted on AWS, for the 6 months of the evaluations
    • Fusion of Hurricane Models and Observations: Developing the Technology to Improve the Forecasts
      • All evaluators have in-depth knowledge, hit the ground running

Winter Meeting Planning Sessions

Continued planning for P&S and Testbed Winter Meeting activities - Last year's agenda here: https://docs.google.com/spreadsheets/d/1yLY2964NlwJY_S98vlc2hRe6Ar1cthVMG_dr78BdVtU/edit#gid=0

Idea: A block of 3 sessions, maybe a full morn/afternoon

  • Start with "What is the Testbed/P&S" session - Christine to Submit
  • Then TEF/AIST session - Annie already Submitted
  • Other Testbeds/collaboration session - Emily and Ken to Submit

What is the Testbed Session

  • Fun, interactive overview of P&S, and Testbed/FUNding Friday
  • Introduce TEF, but not in-depth

Testbed Evaluation Framework/AIST Testbed Session

  • Overview, Working Meeting
  • Evaluators report out
  • Bring in potential other projects, and evaluators
  • Talk about next round too - when, how?
  • Sit down with the group, talk through what worked, what didn't work, and how we can refine this process
  • Best Practices - for example:
    • Regardless of TRL, the PI & the Evaluator should meet the first week to go through the technologies
      • Tech Overview Call
      • Lack of documentation from PI - understand what they are trying to do, vs what evaluator thinks they are trying to do
      • "What are the goals of this?"
      • "If this is your goal, then this is what we see as best practice to accomplish the goal"
  • What evaluators get out of it:
    • 5K in funding and travel to the Winter Meeting to participate, and travel to AGU for the Independent Eval Session
    • In the future it might not be 5K b/c of fed limits on accepting salary -
    • Might be interesting to think about the Type relationship to the "what do you get" (i.e., Type IIIs, where the budget might be helpful)
  • Flexibility
    • Varying Access and TRL levels

Technology Evaluation Session: How we support evaluation, and build components to do so

  • Earth Science Community Testbeds - broader, diff. orgs that have Testbeds
  • Status, highlight capabilities
  • Conversation, see if there are collaborative opps
  • EarthCube Testbed & ESIP Testbed Collaboration discussion
    • Support integration of technologies, components and services that support EarthCube infrastructure; cloud platform, evaluation framework
  • RDA Testbed
  • OGC Testbed (?)
  • NDS

Awarded Testbed Projects & FUNding Friday Projects

  • Testbed:
    • Disaster Lifecycle - Expanding CCOP: Will be supported in Disaster Cluster sessin
    • John's Fast Track Project: Maybe highlight as part of AIST session?
  • Other Projects from November RFP that will be awarded by then - reminder to awardees that they can leverage the Winter Meeting to kick-off
  • FUNDing Friday Posters
    • Nancy to reach out to Brandon Whitehead and Sophie Hou regarding posters and session-related presentations
    • Soren and Christine to collaborate on reaching out to other projects

Other logistics

  • "Templates" for projects
    • Scope of work template
    • Budget template
    • "Adoption" template (see nice example from Beta release of AppEEARS)
  • Next P&S Meetings: Nov. 17th (Tues before Thanksgiving) and Dec. 15th (before AGU). Nov. 17th OK; for December, we'll check in on Nov. call.
  • Timeframe for TCB to evaluate proposals: First week of December