Difference between revisions of "Cloud Computing"

From Earth Science Information Partners (ESIP)
(add google calendar link)
(117 intermediate revisions by 11 users not shown)
Line 1: Line 1:
Cloud Computing for Earth Science
+
__NOTOC__
 +
<big>See also</big> [[Cloud Computing Cluster Plan 2021]]<big><center>'''Welcome to the Cloud Computing Cluster'''</center></big>
  
Tuesday Jan 4
+
==Description and Objectives==
:2:00-2:30pm GeoCloud, Doug Nebert, FGDC
+
<big>As geospatial computing continues to demand more volume, variety, and velocity, the core characteristics of big data, the Cloud Computing Cluster helps ESIP members tackle these challenges through cloud-based architectures, technologies, and software.  By bringing together users, developers, and researchers from industry, government, and academia, this Cluster provides a virtual platform for ESIP members and broader communities to share relevant knowledge and experience. The Cluster also aims to help foster collaborations among different organizations and individuals to work together to better apply cloud technologies to today’s data challenges.</big>
:3:00-3:00pm Cloud Enabled GEOSS clearinghouse, Qunying Huang, GMU
+
{| width="100%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;"
:3:00-3:30pm NASA Cloud Services, Phil Yang, NASA GSFC
+
|- valign="top" bgcolor="pink"
 +
| bgcolor="#FFFFBB" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="50%" |
  
Many Earth science problems cannot be explored by single computers and solved within a single science community, but through distributed computing paradigms and models interdisciplinary efforts, such problems can be tackled effectively. The emergence of cloud computing provides a potential solution to enable the addressing of the Earth science problems. This session provides the latest development on how cloud computing can help Earth sciences and how Earth sciences can help to shape cloud computing?
+
===Get Involved===
  
This session will include three talks:  
+
*'''Email List:''' [http://lists.esipfed.org/mailman/listinfo/esip-cloud ESIP-Cloud]
 +
*'''Email archive:''' [http://lists.esipfed.org/pipermail/esip-cloud/ ESIP-Cloud Web Email Archive]
 +
*Join the conversations on our '''[https://esip-slack-invite.herokuapp.com/ ESIPFed Slack Channel]''' and say hello on #general and #cloud-computing
 +
*'''Join the knowledge sharing at the last Monday of the month 1PM EDT via Zoom Meeting'''
 +
**<nowiki>https://us02web.zoom.us/j/86535177705?pwd=ay9yVDJ6UzNiSGRMWTFxbkNXdEJXUT09</nowiki>
 +
**Meeting ID: 865 3517 7705  Passcode: 354962
 +
**'''Add this Google calendar event to your calendar:''' https://calendar.google.com/event?action=TEMPLATE&tmeid=M2N2NWIxc2pjYmdhOWplc2xmbG8xYmI1ZjNfMjAyMjA2MjdUMTcwMDAwWiBhaW1lZUBkZXZlbG9wbWVudHNlZWQub3Jn&tmsrc=aimee%40developmentseed.org&scp=ALL
 +
*
 +
*'''Contact Cluster Chair:''' Aimee Barciauskas, Sudhir  Shrestha
 +
*'''Other leads:''' Rich Signell, Rob Casey, Jim Coll
  
1) Doug Nebert from FGDC will introduce GeoCloud, a cross-agency initiative led by FGDC. In early 2010, FGDC summoned government agencies such as Census, NOAA, USGS, and USDA to deploy their geospatial products and applications onto a cloud environment.  Objectives of this initiative are to define common operating system and software suites for geospatial applications, explore and document deployment and management strategies, monitor usage and costing of Cloud services in an operational environment, and pursue shared system security profiles for such solutions. The result of the project will serve as an guidance for governmental agencies in the future when considering Cloud service adoption for geospatial capabilities.
+
[[WS::{{PAGENAMEE}}]]
 +
[[category:CollabArea]]
 +
|-
 +
|
 +
|}
  
2) Qunying Huang from George Mason University Center for Intelligent Spatial Computing will introduce a joint project between the GeoCloud and GEO from their experiences on deploying the GEOSS clearinghouse to a cloud platform. The deployment of geospatial applications onto Amazon EC2 cloud computing platform will be introduced. Issues and Research will be reported about the leverage of cloud computing for the operational system.  
+
{| width="100%" cellpadding="0" cellspacing="0" style="zborder-top:1px solid #aaaaaa; border-collapse: collapse;"
 +
|- valign="top" bgcolor="#FFFFFF"
 +
| bgcolor="lightgreen" style="border: 1px solid gray;padding-left:0.5em;padding-right:0.5em;" width="50%" |
 +
We have a regularly scheduled cluster meeting on '''the last Monday of the month 1PM EDT'''. This is a knowledge sharing session with the schedule typically being:
  
3) Chaowei Phil Yang, the lead architect of NASA cloud services hosted by Goddard Space Flight Center, will introduce the NASA Cloud Services.
 
  
=== Notes from Session ===
+
'''Agenda'''
  
Geocloud Sandbox initiative was created as an Architecture and technology working group activity in December of 2009. Created to nominate geospatial applications for testing in the Cloud environment for 1 year prototype.
+
* 5-10 minutes - Announcements
* 2 deployment environments were abstracted from the projects, 1 open source service stack on linux64 other is windows 2008 stack.
+
* 20-30 minutes presentation
* Cloud computing could create a ceiling payment for use similar to that of a utility bill.
+
* Questions and discussion
* Platform is the main idea of going for cloud computing. Platform is a service that delivers solution stack as a service generally consuming cloud infrastructure and supporting cloud applications. It facilitates deployment of applications without the cost and complexity of buying and managing the underlying hardware and software layers.
 
* Geocloud is piloting the deployment of infrastructure as a service
 
* Hope to save money in hardware operations and scalability, reduce maintenance, cost effective testing
 
* Platform consists of application servers platform enablers app frameworks and runtime systems.
 
* Basic image :Windows 2008 hardened open source linex centos then harden and build base platforms to open source additions such as java, php, etc and open souce core like apache, postgres, java, ruby on rails, specialize for target apps * * * ArcGIS server Geospatial platform semantic drupal glassfish, open geo, geoserver network, thredds, geospatial hhs and semantic apps
 
* Cost evaluation for each of the initial project was performed based on data transfer story
 
* Most projects could be hosted inAWS at 350-500$ a month
 
* Amazon Web Services was selected as the primary public cloud computing environement for various sizes and numbers of virtual machines.
 
* Dell vmwarevcloud environment was selected for government hosted cloud infrastructure.
 
* USA Arc GIS Geoss LINUx64 NOAA Linux64
 
* Working on this January- march
 
Doug Nebert
 
  
  
* Geocloud deployed ten geospatial application projects in the vloud network
+
Meeting Information
* Cloud computing a model that enables shareing on demand network access to share pooled computing resources that can be rapidly provisioned and released
 
* On demand self service, multitenancy measured services device and location independence rapid elasticity
 
* SaaS software as service gmail skype facebook
 
* PaaS platform as a service windows Azure google app engine
 
* Infrastructure as a service IAAS AWS
 
* Elastic Compute Cloud EC2 IAAS
 
* Simple Storage service S3 Iaas
 
* Elastic Block storage EBS IAAS
 
* EC2 a web service that provides resizable compute capacity in the cloud
 
* AMI amazon machine image a bootable vm image which can be launched as a
 
* EC2 instance
 
* Scalability load balancer
 
* Reliability network disaster recovery
 
* Reducing duplicated efforts infrastructure and development
 
* We are at a prescient time
 
* Technologies cloud architecture platform independent languages
 
  
Open data standards
+
'''Join Zoom Meeting'''
* Challenges?: network bottlenecks data transfer, performance unpredictability, data and personal privacy, scalable storage computing power Amazon EBS, Bugs in  large distributed systems
 
  
Earth Science collaboration Discussion
+
<nowiki>https://us02web.zoom.us/j/86535177705?pwd=ay9yVDJ6UzNiSGRMWTFxbkNXdEJXUT09</nowiki>
* 2011 Suggestions continue webinar
 
* Invite external speakers, need to create a wish list of speakers, mix In more hands on tutorials, capture the presentations as videos
 
* ESIP info commons; esip uses drupal site poorly put together and needs to be redone
 
* Esip tried Google Knol but it was buggy, hard to create a uniform look, not intuitive to create one, not sure how to handle multiple authors. Links only to your gmail identity, linking esip collection is not straightforward
 
* Decide to go with drupal extension and have a prototype ready by summer
 
* Summer meeting; blue sky session, future of science data and info systems
 
Rahul
 
  
Suggestion for construction of an Earth Science Collaboratory
+
Meeting ID: 865 3517 7705
* Convergent evolution to ESC
 
* ESC= rich data analysis environment that provides access across a wide spectrum of Earth science data, provides a diverse set of science analysis services and tools, supports the application of services and tools , supports collaboration on data analysis, supports sharing of data tools etc.
 
* Several graphics were shown showcasing the applications of using the ESC
 
* Mediator Mediates combinations tool with data data with data tool with tool tool with workflow data with workflow
 
* Based on data access standards common data model
 
* Cyber infrastructure services used by all the other components: security, social, cloud, discovery, information mgmt, semantic web, evaluation
 
* Advantages of ESC tool availability will be a force multiplier, more tools will be usable
 
* Knowledge sharing will evolve from test on paper to a mixture of data tools workflows and articles
 
* A wikihow for earth science data will emerge
 
* ESC will maintain a record of the analysis process
 
* Why now? Cause its possible and  the need is growing, cloud helps with provisioning resources tools workflows etc multiple data sets
 
* 7 year plan of the ideas of what to do
 
Chris Lynnes
 
  
KEVIN
+
Passcode: 354962
* Context nasa earth science data systems is a large continuing investment, websites are the front door to data and service for users but are discordant
+
 
 +
 
 +
'''Notes:''' https://docs.google.com/document/d/1vuzQVqIcsKpbOiWEoxhrmcQGYg73uSQtL8M5l7YS65Y/edit#
 +
 
 +
 
 +
YouTube channel with presentations from 2021 and 2022: https://www.youtube.com/playlist?list=PL8X9E6I5_i8i3ZEl7mIrt2meHI3z8KzqB
 +
----------Previous years of the cluster had a different format, more discussion-based:
 +
 
 +
*2019-05-06: Cloud Computing Lightning Talks: [https://youtu.be/BAH3Eo39I8E View Recording]
 +
*2016-10-24: [[Cloud_Telecons_10/24/2016|Cloud Computing Telecon]]
 +
*2016-09-26: [[Cloud_Telecons_09/26/2016|Cloud Computing Telecon]]
 +
*2016-06-28: [[Cloud_Telecons_06/28/2016|Cloud Computing Telecon]]
 +
*2016-05-23: [[Cloud_Telecons_05/23/2016|Cloud Computing Telecon]]
 +
*2016-04-25: [[Cloud_Telecons_04/25/2016|Cloud Computing Telecon]]
 +
*2016-03-28: [[Cloud_Telecons_03/28/2016|Cloud Computing Telecon]]
 +
*2016-02-22: [[Cloud_Telecons_02/22/2016|Cloud Computing Telecon]]
 +
*2016-01-25: [[Cloud_Telecons_01/25/2016|Cloud Computing Telecon]]
 +
*2015-01-07: ESIP Winter Meeting 2015
 +
**Zhenlong Li: [http://wiki.esipfed.org/index.php/File:Enabling_Big_Geoscience_Data_Analytics_with_a_Cloud-based,_MapReduce-enabled_and_Service-oriented_Workflow_Framework.pdf - Enabling Big Geoscience Data Analytics with a Cloud-based, MapReduce-enabled and Service-oriented Workflow Framework]
 +
**George Chang: [http://wiki.esipfed.org/index.php/File:CLoudburstingDebriefV1ESIP2.pdf - Cloudbursting Debrief]
 +
*2014-07-10: ESIP Summer Meeting 2014
 +
**Thomas Huang and Phil Yang: [http://wiki.esipfed.org/images/f/f6/2014_esip_summer_c3.pdf Agenda - Science Data Analysis on the Cloud]
 +
**Petr Votava [http://wiki.esipfed.org/images/b/bb/Votava-OpenNEX-v2.pdf NASA Earth Exchange (NEX): Community Engagement in the Cloud through OpenNEX]
 +
**Wes Lloy: [http://wiki.esipfed.org/images/e/ef/Lloy_vm_scaler.pdf The Virtual Machine (VM) Scaler: Supporting Environmental Modeling for the Cloud Serivces Innovation Platform (CSIP)]
 +
*2014-04-28: [http://wiki.esipfed.org/images/6/66/Euca_overview_-_2014.pdf Eucalyptus Overview by Rich Lee, Eucalyptus Systems, Inc {lee.rich@eucalyptus.com}]
 +
*2014-03-31: [http://wiki.esipfed.org/images/f/f7/ESIPC3-3312014-MonthlyTeleconMinutes.pdf Cloud Computing Telecon]
 +
*2014-01-27: [[Cloud_Telecons_12|Cloud Computing Telecon]]
 +
*2014-01-08: ESIP Winter Meeting 2014
 +
**Cloud Governance
 +
***Hook Hua: [http://wiki.esipfed.org/index.php/File:Esip2013winter-ariamh-20140109.pdf Leveraging On‐premise and Public Cloud Computing to Enable Advanced Rapid Imaging & Analysis for Monitoring Hazards]
 +
***Jonathan Chiang: [http://wiki.esipfed.org/index.php/File:ESIP-Governance-jchiang.pdf Overcoming “Cloud Creep”]
 +
***Matthew Derenski: [http://wiki.esipfed.org/index.php/File:ESIP_Cloud_Security.pdf Cloud Security]
 +
*2013-08-26: [http://wiki.esipfed.org/images/e/e5/Cloud_101_ESIP.pdf Webinar: Introduction to Cloud Computing]
 +
*2013-07-22: [http://wiki.esipfed.org/images/1/15/C3-minutes-20130722.pdf Cloud Computing Telecon]
 +
*2013-1-09: [[2013Winter_Cloud_Talks|2013 ESIP Winter Slides]]
 +
*2012-9-24: [[Cloud_Telecons_11|Cloud Computing Telecon]]
 +
*2012-8-27: [[Cloud_Telecons_10|Cloud Computing Telecon]]
 +
*2012-6-25: [[Cloud_Telecons_8|Cloud Computing Telecon]]
 +
*2012-6-4: [[Cloud_Telecons_7|Cloud Computing Telecon]]
 +
*2012-4-23: [[Cloud_Telecons_6|Cloud Computing Telecon]]
 +
*2012-3-26: [[Cloud_Telecons_5|Cloud Computing Telecon]]
 +
*2012-2-27: [[Cloud_Telecons_4|Cloud Computing Telecon]]
 +
*2012-1-23: [[Cloud Computing Telecon]]
 +
*2012-1-5: [[ESIP Cloud Computing Testbed Initiative]]
 +
*2012-1-5: [[NASA Cloud Computing Panel]]
 +
*2012-1-5: [http://wiki.esipfed.org/index.php/File:Cloud_Computing_Talk_by_Adrian_Gartner.pdf Cloud Computing Talk by Adrian Gartner, CIO, NASA Goddard]
 +
*2011-12-19: [[Cloud_Telecons_3|Cloud Computing Telecon]]
 +
*2011-10-25: [[Cloud_Telecons_2|Cloud Computing Telecon]]
 +
*2011-09-16: [[Cloud_Telecons|Cloud Computing Telecon]]
 +
*July, 2011 Summer Meeting Cluster Formed
 +
|}
 +
[[category:CollabArea]]

Revision as of 09:51, June 10, 2022

See also Cloud Computing Cluster Plan 2021

Welcome to the Cloud Computing Cluster

Description and Objectives

As geospatial computing continues to demand more volume, variety, and velocity, the core characteristics of big data, the Cloud Computing Cluster helps ESIP members tackle these challenges through cloud-based architectures, technologies, and software. By bringing together users, developers, and researchers from industry, government, and academia, this Cluster provides a virtual platform for ESIP members and broader communities to share relevant knowledge and experience. The Cluster also aims to help foster collaborations among different organizations and individuals to work together to better apply cloud technologies to today’s data challenges.

Get Involved

Cloud_Computing

We have a regularly scheduled cluster meeting on the last Monday of the month 1PM EDT. This is a knowledge sharing session with the schedule typically being:


Agenda

  • 5-10 minutes - Announcements
  • 20-30 minutes presentation
  • Questions and discussion


Meeting Information

Join Zoom Meeting

https://us02web.zoom.us/j/86535177705?pwd=ay9yVDJ6UzNiSGRMWTFxbkNXdEJXUT09

Meeting ID: 865 3517 7705

Passcode: 354962


Notes: https://docs.google.com/document/d/1vuzQVqIcsKpbOiWEoxhrmcQGYg73uSQtL8M5l7YS65Y/edit#


YouTube channel with presentations from 2021 and 2022: https://www.youtube.com/playlist?list=PL8X9E6I5_i8i3ZEl7mIrt2meHI3z8KzqB


Previous years of the cluster had a different format, more discussion-based: