ESIP Testbed Amazon Web Services Best Practices

This page outlines best practices, lessons learned, insights, and anything else relevant to deploying testbed instances on Amazon Web Services. This initial content describes the experience of the Discovery Cluster in deploying the Esri Geoportal Server on an Amazon instance as part of the Discovery Testbed Work Plan.

Steps to deploy an instance

Step 1: Create an AWS account

You'll need to go to the Amazon Web Services website and create an account. Note that you will need to associate your account with a payment method - this personal and independent account needs to be on file with Amazon as a backup, and Amazon will also use this account info to verify your identify. In Step 2, you will be added to the ESIP Consolidated billing account, and that is where the instance charging will be applied. So as long as you don't sign up for services that are outside the scope of the consolidated billing, your account on file should not be charged.

After creating an account, Amazon will call you to verify your identity. For this to go most smoothly, recommendation is to use a phone number that does not have an extension associated with it.

Be aware that all of the account activity will be subsumed under the ESIP framework, so if you have an existing Amazon account that you use for other purposes, you should create a new account for ESIP use.

Step 2: Consolidated Billing

Contact Erin Robinson and request that she add you to the ESIP AWS Consolidated billing account. This account should cover all your activity on AWS except Amazon DevPay.

Step 3: Set up an EC2 instance account

Once you have an AWS account, you will need to create an EC2 account. When you click on the EC2 tab from the AWS Management Console, you will be prompted to sign in or sign up. Proceed with signing up.

Step 4: Get started with the EC2 instances

Brief notes: US East instances are typically less expensive than West instances. Linux instances are less expensive than Windows, but hearsay is that support is better for Windows.

There is very good step-by-step documentation on how to create and launch an instance - see the Amazon Getting Started Guide.

As you review instance costs, you will see that the short-term or spot instances tend to have higher costs per compute cycle than long-term agreements. Also, some common resource combinations are only available in a long-term agreement. It is also the case that spot instance pricing can change, sometimes drastically, as demand for them varies. As this is being written, the shortest long-term agreement is 1 year, but cost savings can be more than 50% over spot instances. Depending on the schedules in your project, pre-purchasing a long-term instance may be the most cost-effective strategy, even if some of those cycles are unused at the end of the contract.

Step 5: Create and manage images (optional)

After you've deployed an instance and done some work on your instance (e.g. deploy software, configure users, etc.) you may want to preserve that work by creating an image. Conceptually, creating the image works the same as creating an image using VM-based snapshot tools. Documentation for how to do this is available at Creating an Image from a Running Instance. IMPORTANT: Delete unused images, as the consolidated billing account will be charged for space stored on the Volumes associated with images).

Step 6: How to Open Ports so you can access web apps deployed on instances (optional)

If you want external users to be able to access web (or other) applications that are hosted on your instance, you will need to configure the port access. There are two places where the ports will need to be configured: first in your amazon instance through the AWS Management Console, and also on the instance itself.

For the configuration in the AWS Management Console, click on your EC2 instance. In the left navigation window, there is a section called “Security Groups” - click on this. The rules associated with your security group are displayed in the window at the bottom of the screen. You can add ports and access rules here; for example, a web application that requires port 8080 to be open would be added as shown in the following:

click to enlarge

For the configuration on the Amazon instance, you will likely also need to configure ports. This section describes opening ports on a Windows Server 2008 R2 SP1 instance; for other instances, there are different methods.

• Open the Windows Firewall with Advance Security menu:

click to enlarge

• In the resulting window, select “Inbound Rules” from the left menu, and then click “New Rule” from the far right window. In the resulting dialog, select a new rule for “ports” - see below. Then click Next.

click to enlarge

• In the next window, enter the "Specific Ports" for which you want to apply the rule. Click Next.
• Now determine what should happen – e.g., “Allow the connection” and click Next.
• You can accept the defaults for the rest of the settings (unless you have a reason not to), except where you are prompted to enter a name for the rule. Enter whatever name adequately describes your rule.

You can following these same instructions for opening outbound ports, using the "Outbound Rules" option from the left menu.

Notes specific to the Discovery Cluster Testbed deployment

Instance details

In this deployment, Christine White worked to install the base software for the Esri Geoportal Server 1.2. It was deployed on a Windows Server 2008 R2 SP1 instance, configured with a 'Small' size, based on East Coast hosting resources. Estimated pricing for such an instance is reportedly ~ $0.115/hour, with one elastic IP address - see EC2 Pricing.

You can download a .zip file mp4 video of the deployment (warning: not so exciting, but at least its short), here: File:Discovery AWS instance.zip.

The Geoportal Server deployment is accessible at http://23.23.211.222:8080/geoportal. As an anonymous user, you can search for resources using the Search tab (although not many are registered yet, just some ArcGIS Server services for test purposes). User logins will be added for testbed users interested in contributing/testing service endpoints - to become a user of the geoportal, contact Christine and she will add you.

Software Deployed

The following software - all open source - was downloaded and installed:

Software	Function	Version	Download Link
PostgreSQL	RDBMS	9.1.3	http://www.enterprisedb.com/products-services-training/pgbindownload
JDBC4 Postgresql Driver	JDBC driver	9.1	http://jdbc.postgresql.org/download.html
Java Development Kit (JDK)	Java controller	SE 6 Update 31	http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u31-download-1501634.html
Apache Tomcat	Web container	6.0.35	http://tomcat.apache.org/download-60.cgi#6.0.35
Apache DS	LDAP server	1.5.7	http://directory.apache.org/apacheds/1.5/
Apache Directory Studio	LDAP browser	2.0 M3	http://directory.apache.org/studio/2.0/download/
Esri Geoportal Server	Geoportal	1.2	http://sourceforge.net/projects/geoportal/files/Distribution/v1.2/geoportal-1.2.zip/download

Moving to EC2 Micro Instance

At some point, you may want to move your Amazon instance to a smaller, more cost-effective instance. For notes on this, please see Documentation on moving to EC2 Micro Instance - Christine White.

---

Back to the Testbed