P&S Telecon - October 20, 2015

November RFP Q&A

Opportunity to ask questions about the November 2015 Testbed RFP

• Thanks all for your help preparing the RFP!

Updates on the Testbed Evaluation Framework

Update provided by Annie B.

• We're 3 weeks into the evaluation.
• 3 Types of evaluation teams:
  • Domain scientists - federal sciences
  • ESIP-centric team
  • Private industry, academic, and a mix
• All approaching it differently:
  • ARIA-MH, Hook's project
    • Issue with software access and proprietary imagery
    • So evaluation has to happen through Telecons
    • Evaluators look at software and tech that is not at their fingertips
    • Learning how to do that - group call yesterday. Team used the OSF wiki to brainstorm on how to approach this sort of online evaluation
    • Will have two 2-hour long calls with the PI to come up with a Test Plan with the PI
    • And then another call with the PI to execute the test plan, then evaluate if needs an additional call
  • Collaborative Workbench, Manil's project
    • Using Slack to collaborate about evaluation
    • Will meet with PI to go through
    • Backend of evaluation hosted on AWS, for the 6 months of the evaluations
  • Fusion of Hurricane Models and Observations: Developing the Technology to Improve the Forecasts
    • All evaluators have in-depth knowledge, hit the ground running

Winter Meeting Planning Sessions

Continued planning for P&S and Testbed Winter Meeting activities - Last year's agenda here: https://docs.google.com/spreadsheets/d/1yLY2964NlwJY_S98vlc2hRe6Ar1cthVMG_dr78BdVtU/edit#gid=0

Idea: A block of 3 sessions, maybe a full morn/afternoon

• Start with "What is the Testbed/P&S" session - Christine to Submit
• Then TEF/AIST session - Annie already Submitted
• Other Testbeds/collaboration session - Emily and Ken to Submit

What is the Testbed Session

• Fun, interactive overview of P&S, and Testbed/FUNding Friday
• Introduce TEF, but not in-depth

Testbed Evaluation Framework/AIST Testbed Session

• Overview, Working Meeting
• Evaluators report out
• Bring in potential other projects, and evaluators
• Talk about next round too - when, how?
• Sit down with the group, talk through what worked, what didn't work, and how we can refine this process
• Best Practices - for example:
  • Regardless of TRL, the PI & the Evaluator should meet the first week to go through the technologies
    • Tech Overview Call
    • Lack of documentation from PI - understand what they are trying to do, vs what evaluator thinks they are trying to do
    • "What are the goals of this?"
    • "If this is your goal, then this is what we see as best practice to accomplish the goal"
• What evaluators get out of it:
  • 5K in funding and travel to the Winter Meeting to participate, and travel to AGU for the Independent Eval Session
  • In the future it might not be 5K b/c of fed limits on accepting salary -
  • Might be interesting to think about the Type relationship to the "what do you get" (i.e., Type IIIs, where the budget might be helpful)
• Flexibility
  • Varying Access and TRL levels

Technology Evaluation Session: How we support evaluation, and build components to do so

• Earth Science Community Testbeds - broader, diff. orgs that have Testbeds
• Status, highlight capabilities
• Conversation, see if there are collaborative opps
• EarthCube Testbed & ESIP Testbed Collaboration discussion
  • Support integration of technologies, components and services that support EarthCube infrastructure; cloud platform, evaluation framework
• RDA Testbed
• OGC Testbed (?)
• NDS

Awarded Testbed Projects & FUNding Friday Projects

• Testbed:
  • Disaster Lifecycle - Expanding CCOP: Will be supported in Disaster Cluster sessin
  • John's Fast Track Project: Maybe highlight as part of AIST session?
• Other Projects from November RFP that will be awarded by then - reminder to awardees that they can leverage the Winter Meeting to kick-off
• FUNDing Friday Posters
  • Nancy to reach out to Brandon Whitehead and Sophie Hou regarding posters and session-related presentations
  • Soren and Christine to collaborate on reaching out to other projects

Other logistics

• "Templates" for projects
  • Scope of work template
  • Budget template
  • "Adoption" template (see nice example from Beta release of AppEEARS)
• Next P&S Meetings: Nov. 17th (Tues before Thanksgiving) and Dec. 15th (before AGU). Nov. 17th OK; for December, we'll check in on Nov. call.
• Timeframe for TCB to evaluate proposals: First week of December