Difference between revisions of "Cloud Telecons 3"

From Federation of Earth Science Information Partners
(Created page with "'''Dec 19, 2011 ESIP cloud computing cluster Telecon''' # Participants: Carl Meyer, Ken Keiser, Rich Martin, Thomas Huang, Phil Yang, Doug Nerbet, Qunying Huang Aigun Heo, Aij...")
 
Line 27: Line 27:
  
  
 +
ESIP Cloud Computing Testbed (Phil Yang)
  
Winter meeting cloud session plan
+
# Requirements for cloud comptuing testbed have been identified:
# Erin: Sessions introduction: two in am, two in pm, different number of speakers, total 1.5 hours
+
http://wiki.esipfed.org/index.php/Cloud_Computing_Testbed_Requirements
# P&S Tasks and cloud computing, it is not focused on cloud but could be setup requirements for cloud computing, e.g., "create on demand virtual machines for ESIP." where do we deploy the machines.
+
# Discussed in the 2011 summer meeting: objectives, activities
# Panel: NASA Cloud Computing, representatives from headquarter and centers
+
# More topics will be discussed in the winter meeting
## Phil will contact to invite Mike L. from HQ and Thomas will be inviting speakers from JPL, Ames, etc.
+
## The goal is to define the testbed, possible venders either to buy a testbed or contribute a testbed environment
# ESIP Cloud Computing Testbed
+
## Computing resources from Azure (Carl and Phil)
## Great interest from other clusters to have a testbed, several companies are contacted and we hope to have an outline, and presentation about what could this be and how we do.
+
## Nasa cloud resources for NASA users
## Defining goal, boundary, for commercial vendors, free would be great for ESIP, what vendors are providing?
 
## Emails discuss what should be included; assess use case for the testbed, what we want to get out of it?
 
## Evaluation: what the commercial services are provided for testbed
 
## Cloud as a test, whether it is suitable for supporting different tasks
 
## Using the cloud, where we will be going from cloud
 
## Compare the capabilities of different cloud vendors would be beneficial for the members
 
  
Webinar cloud presentations by companies.
+
Amazon cloud computing presentation (Rick Martin, Thomson Nguy )
  
# 30 minutes for presentations from vendors talking about their accomplishments; Introduce their technologies, products.
+
# Prensenter: Thomson Nguy (Amazon)
# Nov monthly meeting: Rick will set up a presentation
+
# Topic: Overall  of what Amazon doing, AWS clients doing, security
# Topics of the presentations based on audiences
+
# Presentation Notes:
## What do we think, what we want to hear, e.g., developing or applications, communities;
+
## Infrasturcture as a service
## Amazon three sessions structure: 1) Decision-making, 2) Architecture,3) Solution session
+
## Driving: on-demand easy to use, launch, turn off
## What the real/true costs for, processing I/O, storage?
+
## Customer choice: a variety of flexible choices for user, OS, programming tools, applications
## JPL : Big data in the cloud computing
+
## Flexible Pricing models: on-demand, elastic scale up and down, reserved instance or reserved fixed price, combination of price model , 50% saving price
 +
## More than 100 governments and institution client
 +
## Use cases: Mars exploration rovers, deep space network,carbon in the articic  reservoir vuluneratiblity experiment, lunar mapper
 +
## Mars Science Lab: fast motion field test- image processing in the cloud, massively parallel computations
 +
## European space agency: ESA centre for earth observations; data collected by satellites in s3; 50.000 users at peak 30TB at a time; scale up storage
 +
## Research and collaboration: Observable mdeical outcomes program(OMOP); Needed platform for cross-industry collaboration ; solution: OMOP use AWS for their research lab for providing a scientific computational platform for researchers; Benefits: avoided capital expense; flexibility to scale up and down based on computations demand and need; research tools can be shared with the extended community
 +
## Top 5 Pharma: dynamic molecular modeling;  large parallel problem; set up 30.000 cores –centos, 26.7 TB RAM, 2 PB disk space, HTTPs, ssh , 256-bit AES; supported software: condor(job management), chef (configuration management, Grill (cycle plugin for chef); speed and agility, $1.279 per hour, 3 hours batch job
 +
## Security: AWS certifications, sarbanes-oxley(SOX), SAS70 type II audit, pci data security standard company (Credit card) etc;
 +
## Security is a shared responsibility
 +
## AWS VPC Architecture
 +
## QA #1  Licence
 +
Thomson: Buy licensed virtual machines charged by hours
 +
Doug: If you buy the agreement in advance, e.g, ESRI, you can use and install the vm
  
A Cloud computing testbed for ESIP.
+
## QA #2  How to select optimal options for choosing the VMs
 +
Thomson: AWS provide the calculation model, but basically requires user to test and make decision
 +
## QA #3  Charge:  any tools to monitor usage and costs
 +
Thomson: a. create several accounts by IAM service,  credit card/GSA , CloudWatch service to monitor the usage
  
# Identify some use cases: Start from the current activities to form a wishlist for the cloud computing
+
## QA #4  Computing units for cloud computing
# After people signed up for the cloud cluster email list, we can send out the email to ESIP members to fill in the wish lists
+
Thomson: EC unit , ~ 1.0 -1.2 GHz
# Verify the requirements of the testbed activities for cloud computing, check how far we can go on the next telecon to decide a December telecon time
 
# Trial not for production, start out with free, looking at a number of providers to get free test resources
 
# Deploy your own, have many options so we won't be locked in, alert our potential users about the locked in problem, JPL is doing that with several vendors but having an internal one for itself.  
 
# Provide information about when using different platforms for different types of applications in terms of price, computing capacity, etc. Government data may need security so to alert potential users.
 
# Summarize the discussion for a potential December monthly telecon
 
  
 +
## QA #5  Storage indexing for massive data
 +
Thomson: S3,  users bulid index; Amazon has DB services, simple DB, could help mange massive data;
  
Activities outside biannual meeting.
+
## QA #6  Customer back-up storage
# could we have data.gov type of services for ESIP members (Brand).
+
Thomson: HSM, partners provide the tools for customizing the storage
 
 
Meeting Adjourned at 2pm
 

Revision as of 15:09, December 19, 2011

Dec 19, 2011 ESIP cloud computing cluster Telecon

  1. Participants: Carl Meyer, Ken Keiser, Rich Martin, Thomas Huang, Phil Yang, Doug Nerbet, Qunying Huang Aigun Heo, Aijun Chen, Brand Niemann, Liping Di, Long phan, Jason Simmons, Chris Webber, Peter Cornillon, Upendra Dadi, Thomson Nguy (Amazon)
  1. Co-Chair: Thomas Huang, Rick Martin, Phil Yang
  2. Supportor: Qunying Huang
  3. Call-in toll-free number (US/Canada): 1-877-668-4493
  4. Attendee access code: 23133897#


Agenda

  1. ESIP Winter Meeting Cloud Schedules (Thomas Huang)
  2. ESIP Cloud Computing Testbed (Phil Yang)
  3. Amazon cloud computing presentation starts around 2pm (Rick Martin)


Note:


ESIP Winter Meeting Cloud Schedules (Thomas Huang)

  1. Link for agenda: http://wiki.esipfed.org/index.php/Winter_2011_Agenda
  2. Talks from JPL : nebula, Amazon
  3. Talks from Nebula group: Nebula capabilities, architecture, open discussions for NASA cloud computing initiatives


ESIP Cloud Computing Testbed (Phil Yang)

  1. Requirements for cloud comptuing testbed have been identified:

http://wiki.esipfed.org/index.php/Cloud_Computing_Testbed_Requirements

  1. Discussed in the 2011 summer meeting: objectives, activities
  2. More topics will be discussed in the winter meeting
    1. The goal is to define the testbed, possible venders either to buy a testbed or contribute a testbed environment
    2. Computing resources from Azure (Carl and Phil)
    3. Nasa cloud resources for NASA users

Amazon cloud computing presentation (Rick Martin, Thomson Nguy )

  1. Prensenter: Thomson Nguy (Amazon)
  2. Topic: Overall of what Amazon doing, AWS clients doing, security
  3. Presentation Notes:
    1. Infrasturcture as a service
    2. Driving: on-demand easy to use, launch, turn off
    3. Customer choice: a variety of flexible choices for user, OS, programming tools, applications
    4. Flexible Pricing models: on-demand, elastic scale up and down, reserved instance or reserved fixed price, combination of price model , 50% saving price
    5. More than 100 governments and institution client
    6. Use cases: Mars exploration rovers, deep space network,carbon in the articic reservoir vuluneratiblity experiment, lunar mapper
    7. Mars Science Lab: fast motion field test- image processing in the cloud, massively parallel computations
    8. European space agency: ESA centre for earth observations; data collected by satellites in s3; 50.000 users at peak 30TB at a time; scale up storage
    9. Research and collaboration: Observable mdeical outcomes program(OMOP); Needed platform for cross-industry collaboration ; solution: OMOP use AWS for their research lab for providing a scientific computational platform for researchers; Benefits: avoided capital expense; flexibility to scale up and down based on computations demand and need; research tools can be shared with the extended community
    10. Top 5 Pharma: dynamic molecular modeling; large parallel problem; set up 30.000 cores –centos, 26.7 TB RAM, 2 PB disk space, HTTPs, ssh , 256-bit AES; supported software: condor(job management), chef (configuration management, Grill (cycle plugin for chef); speed and agility, $1.279 per hour, 3 hours batch job
    11. Security: AWS certifications, sarbanes-oxley(SOX), SAS70 type II audit, pci data security standard company (Credit card) etc;
    12. Security is a shared responsibility
    13. AWS VPC Architecture
    14. QA #1 Licence

Thomson: Buy licensed virtual machines charged by hours Doug: If you buy the agreement in advance, e.g, ESRI, you can use and install the vm

    1. QA #2 How to select optimal options for choosing the VMs

Thomson: AWS provide the calculation model, but basically requires user to test and make decision

    1. QA #3 Charge: any tools to monitor usage and costs

Thomson: a. create several accounts by IAM service, credit card/GSA , CloudWatch service to monitor the usage

    1. QA #4 Computing units for cloud computing

Thomson: EC unit , ~ 1.0 -1.2 GHz

    1. QA #5 Storage indexing for massive data

Thomson: S3, users bulid index; Amazon has DB services, simple DB, could help mange massive data;

    1. QA #6 Customer back-up storage

Thomson: HSM, partners provide the tools for customizing the storage