Difference between revisions of "P&S Telecon - October 20, 2015"
From Earth Science Information Partners (ESIP)
(Created page with "== November RFP Q&A == Opportunity to ask questions about the [http://testbed.esipfed.org/node/8752 November 2015 Testbed RFP] == Updates on the Testbed Evaluation Framework ...") |
|||
Line 1: | Line 1: | ||
== November RFP Q&A == | == November RFP Q&A == | ||
Opportunity to ask questions about the [http://testbed.esipfed.org/node/8752 November 2015 Testbed RFP] | Opportunity to ask questions about the [http://testbed.esipfed.org/node/8752 November 2015 Testbed RFP] | ||
+ | * Thanks all for your help preparing the RFP! | ||
== Updates on the Testbed Evaluation Framework == | == Updates on the Testbed Evaluation Framework == | ||
− | + | Update provided by Annie B. | |
+ | * We're 3 weeks into the evaluation. | ||
+ | * 3 Types of evaluation teams: | ||
+ | ** Domain scientists - federal sciences | ||
+ | ** ESIP-centric team | ||
+ | ** Private industry, academic, and a mix | ||
+ | * All approaching it differently: | ||
+ | ** ARIA-MH, Hook's project | ||
+ | *** Issue with software access and proprietary imagery | ||
+ | *** So evaluation has to happen through Telecons | ||
+ | *** Evaluators look at software and tech that is not at their fingertips | ||
+ | *** Learning how to do that - group call yesterday. Team used the OSF wiki to brainstorm on how to approach this sort of online evaluation | ||
+ | *** Will have two 2-hour long calls with the PI to come up with a Test Plan with the PI | ||
+ | *** And then another call with the PI to execute the test plan, then evaluate if needs an additional call | ||
+ | ** Collaborative Workbench, Manil's project | ||
+ | *** Using Slack to collaborate about evaluation | ||
+ | *** Will meet with PI to go through | ||
+ | *** Backend of evaluation hosted on AWS, for the 6 months of the evaluations | ||
+ | ** Fusion of Hurricane Models and Observations: Developing the Technology to Improve the Forecasts | ||
+ | *** All evaluators have in-depth knowledge, hit the ground running | ||
== Winter Meeting Planning Sessions == | == Winter Meeting Planning Sessions == | ||
Continued planning for P&S and Testbed Winter Meeting activities - Last year's agenda here: https://docs.google.com/spreadsheets/d/1yLY2964NlwJY_S98vlc2hRe6Ar1cthVMG_dr78BdVtU/edit#gid=0 | Continued planning for P&S and Testbed Winter Meeting activities - Last year's agenda here: https://docs.google.com/spreadsheets/d/1yLY2964NlwJY_S98vlc2hRe6Ar1cthVMG_dr78BdVtU/edit#gid=0 | ||
− | + | Idea: A block of 3 sessions, maybe a full morn/afternoon | |
− | * | + | * Start with "What is the Testbed/P&S" session - Christine to Submit |
− | * Other | + | * Then TEF/AIST session - Annie already Submitted |
+ | * Other Testbeds/collaboration session - Emily and Ken to Submit | ||
− | === AIST Testbed Session === | + | ===What is the Testbed Session=== |
+ | * Fun, interactive overview of P&S, and Testbed/FUNding Friday | ||
+ | * Introduce TEF, but not in-depth | ||
+ | |||
+ | === Testbed Evaluation Framework/AIST Testbed Session === | ||
* Overview, Working Meeting | * Overview, Working Meeting | ||
* Evaluators report out | * Evaluators report out | ||
− | + | * Bring in potential other projects, and evaluators | |
− | === | + | * Talk about next round too - when, how? |
− | * | + | * Sit down with the group, talk through what worked, what didn't work, and how we can refine this process |
+ | * Best Practices - for example: | ||
+ | ** Regardless of TRL, the PI & the Evaluator should meet the first week to go through the technologies | ||
+ | *** Tech Overview Call | ||
+ | *** Lack of documentation from PI - understand what they are trying to do, vs what evaluator thinks they are trying to do | ||
+ | *** "What are the goals of this?" | ||
+ | *** "If this is your goal, then this is what we see as best practice to accomplish the goal" | ||
+ | * What evaluators get out of it: | ||
+ | ** 5K in funding and travel to the Winter Meeting to participate, and travel to AGU for the Independent Eval Session | ||
+ | ** In the future it might not be 5K b/c of fed limits on accepting salary - | ||
+ | ** Might be interesting to think about the Type relationship to the "what do you get" (i.e., Type IIIs, where the budget might be helpful) | ||
+ | * Flexibility | ||
+ | ** Varying Access and TRL levels | ||
+ | |||
+ | === Technology Evaluation Session: How we support evaluation, and build components to do so === | ||
+ | * Earth Science Community Testbeds - broader, diff. orgs that have Testbeds | ||
+ | * Status, highlight capabilities | ||
+ | * Conversation, see if there are collaborative opps | ||
+ | * EarthCube Testbed & ESIP Testbed Collaboration discussion | ||
+ | ** Support '''integration''' of technologies, components and services that support EarthCube infrastructure; cloud platform, evaluation framework | ||
+ | * RDA Testbed | ||
+ | * OGC Testbed (?) | ||
+ | * NDS | ||
+ | === Awarded Testbed Projects & FUNding Friday Projects=== | ||
+ | * Testbed: | ||
+ | ** Disaster Lifecycle - Expanding CCOP: Will be supported in Disaster Cluster sessin | ||
+ | ** John's Fast Track Project: Maybe highlight as part of AIST session? | ||
+ | * Other Projects from November RFP that will be awarded by then - reminder to awardees that they can leverage the Winter Meeting to kick-off | ||
+ | * FUNDing Friday Posters | ||
+ | ** Nancy to reach out to Brandon Whitehead and Sophie Hou regarding posters and session-related presentations | ||
+ | ** Soren and Christine to collaborate on reaching out to other projects | ||
== Other logistics == | == Other logistics == | ||
Line 23: | Line 78: | ||
** Budget template | ** Budget template | ||
** "Adoption" template (see nice example from Beta release of AppEEARS) | ** "Adoption" template (see nice example from Beta release of AppEEARS) | ||
− | * Next P&S Meetings: Nov. | + | * Next P&S Meetings: Nov. 17th (Tues before Thanksgiving) and Dec. 15th (before AGU). Nov. 17th OK; for December, we'll check in on Nov. call. |
* Timeframe for TCB to evaluate proposals: First week of December | * Timeframe for TCB to evaluate proposals: First week of December |
Latest revision as of 13:30, October 20, 2015
November RFP Q&A
Opportunity to ask questions about the November 2015 Testbed RFP
- Thanks all for your help preparing the RFP!
Updates on the Testbed Evaluation Framework
Update provided by Annie B.
- We're 3 weeks into the evaluation.
- 3 Types of evaluation teams:
- Domain scientists - federal sciences
- ESIP-centric team
- Private industry, academic, and a mix
- All approaching it differently:
- ARIA-MH, Hook's project
- Issue with software access and proprietary imagery
- So evaluation has to happen through Telecons
- Evaluators look at software and tech that is not at their fingertips
- Learning how to do that - group call yesterday. Team used the OSF wiki to brainstorm on how to approach this sort of online evaluation
- Will have two 2-hour long calls with the PI to come up with a Test Plan with the PI
- And then another call with the PI to execute the test plan, then evaluate if needs an additional call
- Collaborative Workbench, Manil's project
- Using Slack to collaborate about evaluation
- Will meet with PI to go through
- Backend of evaluation hosted on AWS, for the 6 months of the evaluations
- Fusion of Hurricane Models and Observations: Developing the Technology to Improve the Forecasts
- All evaluators have in-depth knowledge, hit the ground running
- ARIA-MH, Hook's project
Winter Meeting Planning Sessions
Continued planning for P&S and Testbed Winter Meeting activities - Last year's agenda here: https://docs.google.com/spreadsheets/d/1yLY2964NlwJY_S98vlc2hRe6Ar1cthVMG_dr78BdVtU/edit#gid=0
Idea: A block of 3 sessions, maybe a full morn/afternoon
- Start with "What is the Testbed/P&S" session - Christine to Submit
- Then TEF/AIST session - Annie already Submitted
- Other Testbeds/collaboration session - Emily and Ken to Submit
What is the Testbed Session
- Fun, interactive overview of P&S, and Testbed/FUNding Friday
- Introduce TEF, but not in-depth
Testbed Evaluation Framework/AIST Testbed Session
- Overview, Working Meeting
- Evaluators report out
- Bring in potential other projects, and evaluators
- Talk about next round too - when, how?
- Sit down with the group, talk through what worked, what didn't work, and how we can refine this process
- Best Practices - for example:
- Regardless of TRL, the PI & the Evaluator should meet the first week to go through the technologies
- Tech Overview Call
- Lack of documentation from PI - understand what they are trying to do, vs what evaluator thinks they are trying to do
- "What are the goals of this?"
- "If this is your goal, then this is what we see as best practice to accomplish the goal"
- Regardless of TRL, the PI & the Evaluator should meet the first week to go through the technologies
- What evaluators get out of it:
- 5K in funding and travel to the Winter Meeting to participate, and travel to AGU for the Independent Eval Session
- In the future it might not be 5K b/c of fed limits on accepting salary -
- Might be interesting to think about the Type relationship to the "what do you get" (i.e., Type IIIs, where the budget might be helpful)
- Flexibility
- Varying Access and TRL levels
Technology Evaluation Session: How we support evaluation, and build components to do so
- Earth Science Community Testbeds - broader, diff. orgs that have Testbeds
- Status, highlight capabilities
- Conversation, see if there are collaborative opps
- EarthCube Testbed & ESIP Testbed Collaboration discussion
- Support integration of technologies, components and services that support EarthCube infrastructure; cloud platform, evaluation framework
- RDA Testbed
- OGC Testbed (?)
- NDS
Awarded Testbed Projects & FUNding Friday Projects
- Testbed:
- Disaster Lifecycle - Expanding CCOP: Will be supported in Disaster Cluster sessin
- John's Fast Track Project: Maybe highlight as part of AIST session?
- Other Projects from November RFP that will be awarded by then - reminder to awardees that they can leverage the Winter Meeting to kick-off
- FUNDing Friday Posters
- Nancy to reach out to Brandon Whitehead and Sophie Hou regarding posters and session-related presentations
- Soren and Christine to collaborate on reaching out to other projects
Other logistics
- "Templates" for projects
- Scope of work template
- Budget template
- "Adoption" template (see nice example from Beta release of AppEEARS)
- Next P&S Meetings: Nov. 17th (Tues before Thanksgiving) and Dec. 15th (before AGU). Nov. 17th OK; for December, we'll check in on Nov. call.
- Timeframe for TCB to evaluate proposals: First week of December