Cloud Telecons 03/28/2016
From Earth Science Information Partners (ESIP)
To start the online portion of the Personal Conference meeting
- Go to https://global.gotomeeting.com/join/445841573
- You can also dial in using your phone: United States +1 (571) 317-3112
- Access Code: 445-841-573
March 28, 2016 ESIP cloud computing cluster telecon recap
Participants:Michael R. Berganski, Christopher Lynnes, Frank Greguska, Hook Hua, Namrata Malarout, Nga T Quach, Phil Yang, Stephan Klene, Thomas Huang, Annie Burgess, Fei Hu
Presentation: Migrating the Earth Observing System Data and Information System into the Cloud (Presented By Chris Lynnes)
- Slides: File:Migrating the Earth Observing System Data and Information System into the Cloud.pdf
- EOSDIS: data downlink, capture, clean, archive, and distribute
- EOSDIS comprises discipline-focused Distributed Active Archive Centers (DAAC): browse images, metadata repository, user logs, etc.
- One-stop shopping: https://earthdata.nasa.gov/
- Science Driver: users need download data from archives → distribution volumes are very big in petabytes level → EOSDIS in the Big Data epoch will enable more analysis closer to data.
- More analysis: subset(data variables -> spatial area -> quality filter); transform: reprojection & mosaicking.
- Analyze: simple stats, complex stats, and end user’s algorithm
- EOSDIS Cloud prototypes:
- 1. archive management(cloud storage): cloud-based data distribution of Sentinel radar data by the Alaska Satellite Facility DAAC
- Pros: use bigger datasets; cost savings; Flexibility in assigning archiving and data servicing
- Cons: egress charge policy for the average scientists; user paradigm shift
- Any DAAC can add services to any product to serve their user community
- 2. Cloud-based analytics support: community open source tools; DAAC-developed tools;Cloud analytics examples and recipes
- Pros: analyze any subset slice of large datasets with long time series; avoid data management drudgery; reuse code from colleagues
- Cons: Long term paradigm shift
- 3. application hosting(EOSIDS Services): Common metadata repository; Global imagery browse system; Earthdata search client
- Compliance-as-a-Service: security controls, authorization to operate
- NGAP Services: monitoring, logging, security, autoscaling, biling, etc.
- Paradigm shift: IaaS, PaaS, SaaS
- Pros: Faster time to initial release; more effort on software; smaller custom code footprint; more code and service reuse
- How can we supply data to all users on a non-discriminatory basis?
- How can we avoid or mitigate vendor lock-in?
- How can we predict pricing 2-5 years out?
- How can we attract end users to the cloud?
- How can we migrate data-proximal services to Web Object Storage?
- What functionality or data should NOT go into the cloud?
- How do we handle provisioning and accounting of cycles and storage across the DAACs?
- Do we need new operations policies or procedures?