Cloud Telecons 03/28/2016

Telecon Info

To start the online portion of the Personal Conference meeting

Go to https://global.gotomeeting.com/join/445841573
You can also dial in using your phone: United States +1 (571) 317-3112
Access Code: 445-841-573

March 28, 2016 ESIP cloud computing cluster telecon recap

Participants:Michael R. Berganski, Christopher Lynnes, Frank Greguska, Hook Hua, Namrata Malarout, Nga T Quach, Phil Yang, Stephan Klene, Thomas Huang, Annie Burgess, Fei Hu
Presentation: Migrating the Earth Observing System Data and Information System into the Cloud (Presented By Chris Lynnes)

Slides: File:Migrating the Earth Observing System Data and Information System into the Cloud.pdf
EOSDIS: data downlink, capture, clean, archive, and distribute
EOSDIS comprises discipline-focused Distributed Active Archive Centers (DAAC): browse images, metadata repository, user logs, etc.

One-stop shopping: https://earthdata.nasa.gov/

Science Driver: users need download data from archives → distribution volumes are very big in petabytes level → EOSDIS in the Big Data epoch will enable more analysis closer to data.

More analysis: subset(data variables -> spatial area -> quality filter); transform: reprojection & mosaicking.

Analyze: simple stats, complex stats, and end user’s algorithm

EOSDIS Cloud prototypes:

1. archive management(cloud storage): cloud-based data distribution of Sentinel radar data by the Alaska Satellite Facility DAAC

Pros: use bigger datasets; cost savings; Flexibility in assigning archiving and data servicing
Cons: egress charge policy for the average scientists; user paradigm shift
Any DAAC can add services to any product to serve their user community

2. Cloud-based analytics support: community open source tools; DAAC-developed tools;Cloud analytics examples and recipes

Pros: analyze any subset slice of large datasets with long time series; avoid data management drudgery; reuse code from colleagues
Cons: Long term paradigm shift

3. application hosting(EOSIDS Services): Common metadata repository; Global imagery browse system; Earthdata search client

Compliance-as-a-Service: security controls, authorization to operate
NGAP Services: monitoring, logging, security, autoscaling, biling, etc.
Paradigm shift: IaaS, PaaS, SaaS
Pros: Faster time to initial release; more effort on software; smaller custom code footprint; more code and service reuse

Questions:

How can we supply data to all users on a non-discriminatory basis?
How can we avoid or mitigate vendor lock-in?
How can we predict pricing 2-5 years out?
How can we attract end users to the cloud?
How can we migrate data-proximal services to Web Object Storage?
What functionality or data should NOT go into the cloud?
How do we handle provisioning and accounting of cycles and storage across the DAACs?
Do we need new operations policies or procedures?