Showing posts with label Cloud computing. Show all posts
Showing posts with label Cloud computing. Show all posts

Tuesday, October 11, 2011

History of Virtualization



When you think of the beginning of Server Virtualization, companies like VMWare may come to mind. The thing you may not realize is Server Virtualization actually started back in the early 1960’s and was pioneered by companies like General Electric (GE), Bell Labs, and International Business Machines (IBM).

The Invention of the Virtual Machine

In the Early 1960’s IBM had a wide range of systems; each generation of which was substantially different from the previous. This made it difficult for customers to keep up with the changes and requirements of each new system. Also, computers could only do one thing at a time. If you had two tasks to accomplish, you had to run the processes in batches. This Batch processing requirement wasn’t too big of a deal to IBM since most of their users were in the Scientific Community and up until this time Batch processing seemed to have met the customers needs.

Because of the wide range of hardware requirements, IBM began work on the S/360 mainframe system designed as a broad replacement for many of their other systems; and designed to maintain backwards compatibility. When the system was first designed, it was meant to be a single user system to run Batch Jobs.

However, this focus began to change in July 1, 1963 when Massachusetts Institute of Technology (MIT) announced Project MAC. Project MAC stood for Mathematics and Computation, but was later renamed to Multiple Access Computer. Project MAC was funded by a $2 Million grant from DARPA to fund research into computers, specifically in the areas of Operating Systems, Artificial Intelligence, and Computational Theory.

As part of this research grant, MIT needed new computer hardware capable of more than one simultaneous user and sought proposals from various computer vendors including GE and IBM. At this time, IBM was not willing to make a commitment towards a time sharing computer because they did not feel there was a big enough demand, and MIT did not want to have to use a specially modified system. GE on the other hand, was willing to make a commitment towards a time-sharing computer. For this reason MIT chose GE as their vendor of choice.

The loss of this opportunity was a bit of a wake-up call for IBM who then started to take notice as to the demand for such a system. Especially when IBM heard of Bell Labs’ need for a similar system.

In response to the need from MIT and Bell Labs, IBM designed the CP-40 main frame. The CP-40 was never sold to customers, and was only used in labs. However, it is still importatnt since the CP-40 later evolved into the CP-67 system; which is the first commercial Main Frame to support Virtualization. The Operating system which ran on the CP-67 was referred to as CP/CMS. CP Stands for Control Program, CMS stands for Console Monitor System. CMS was a small single-user operating system designed to be interactive. CP was the program which created Virtual Machines. The idea was the CP ran on the Mainframe, and created Virtual Machines which ran the CMS; which the user would then interact with.

The User interaction portion is important. Before this system, IBM focused on systems where there was no user interaction. You would feed your program into the computer, it would do it’s thing; then spit out the output to a printer or a screen. An Interactive Operating System meant you actually had a way of interacting with the programs while they ran.

The first version of the CP/CMS operating system was known as CP-40, but was only used in the lab. The Initial release of CP/CMS to the public was in 1968, the first stable release wasn’t until 1972.

The traditional approach for a time sharing computer was to divide up the memory and other system resources between users. An Example of a time sharing operating system from the era is MultiCS. MultiCS was created as part of Project MAC at MIT. Additional research and development was performed on MultiCS at Bell Labs, where it later evolved into Unix.

The CP approach to time sharing allowed each user to have their own complete operating system which effectively gave each user their own computer, and the operating was much more simple.

The main advantages of using virtual machines vs a time sharing operating system was more efficient use of the system since virtual machines were able to share the overall resources of the mainframe, instead of having the resources split equally between all users. There was better security since each users was running in a completely separate operating system. And it was more reliable since no one user could crash the entire system; only their own operating system.

Portability of Software

In the previous section, I mentioned MultiCS and how it evolved into Unix. While UNIX is not running virtualized operating systems, it is still a good example of application from another perspective. Unix is not the first multi-user operating system, but it is a very good example of one, and is one of the most widely used ever.

Unix is an example of Virtualization at the User or Workspace Level. Multiple users share the same CPU, Memory, Hard Disk, etc... pool of resources, but each have their own profile, separate from the other users on the system. Depending on the way the system is configured, the user may be able to install their own set of applications, and security is handled on a per user basis. Not only was Unix the first step towards multi-user operating systems, but it was also the first step towards application virtualization.

Unix is not an example of application virtualization, but it did allow users much greater portability of their applications. Prior to Unix, almost all operating systems were coded in assembly language. Alternatively, Unix was created using the C programming language. Since Unix was written in C, only small parts of the operating system had to be customized for a given hardware platform, the rest of the operating system could easily be re-compiled for each hardware platform with little or no changes.

Application Virtualization

Through the use of Unix, and C compilers, and adept user could run just about any program on any platform, but it still required users to compile all the software on the platform they wished to run on. For true portability of software, you needed some sort of software virtualization.

In 1990, Sun Microsystems began a project known as “Stealth”. Stealth was a project run by Engineers who had become frustrated with Sun’s use of C/C++ API’s and felt there was a better way to write and run applications. Over the next several years the project was renamed several times, including names such as Oak, Web Runner, and finally in 1995, the project was renamed to Java.

In 1994 Java was targeted towards the Worldwide web since Sun saw this as a major growth opportunity. The Internet is a large network of computers running on different operating systems and at the time had no way of running rich applications universally, Java was the answer to this problem. In January 1996. the Java Development Kit (JDK) was released, allowing developers to write applications for the Java Platform.

At the time, there was no other language like Java. Java allowed you to write an application once, then run the application on any computer with the Java Run-time Environment (JRE) installed. The JRE was and still is a free application you can download from then Sun Micro-systems website, now Oracle's website.

Java works by compiling the application into something known as Java Byte Code. Java Byte Code is an intermediate language that can only be read by the JRE. Java uses a concept known as Just in Time compilation (JIT). At the time you write your program, your Java code is not compiled. Instead, it is converted into Java Byte Code, until just before the program is executed. This is similar to the way Unix revolutionized Operating systems through it’s use of the C programming language. Since the JRE compiles the software just before running, the developer does not need to worry about what operating system or hardware platform the end user will run the application on; and the user does not need to know how to compile a program, that is handled by the JRE..
The JRE is composed of many components, most important of which is the Java Virtual Machine. . Whenever a java application is run, it is run inside of the Java Virtual Machine. You can think of the Java Virtual Machine is a very small operating system, created with the sole purpose of running your Java application. Since Sun/Oracle goes through the trouble of porting the Java Virtual Machine to run on various systems from your cellular phone to the servers in your Data-center, you don’t have to. You can write the application once, and run anywhere. At least that is the idea; there are some limitations.

Mainstream Adoption of Hardware Virtualization


As was covered in the Invention of the Virtual Machine section, IBM was the first to bring the concept of Virtual Machines to the commercial environment. Virtual Machines as they were on IBM’s Mainframes are still in use today, however most companies don’t use mainframes.
In January of 1987, Insignia Solutions demonstrated a software emulator called SoftPC. SoftPC allowed users to run Dos applications on their Unix workstations. This is a feat that had never been possible before. At the time, a PC capable of running MS DOS cost around $1,500. SoftPC gave users with a Unix workstation the ability to run DOS applications for a mere $500.
By 1989, Insignia Solutions had released a Mac version of SoftPC, giving Mac users the same capabilities; and had added the ability to run Windows applications, not Just DOS applications. By 1994, Insignia Solutions began selling their software packaged with operating systems pre-loaded, including: SoftWindows, and SoftOS/2.
Inspired by the success of SoftPC, other companies began to spring up. In 1997, Apple created a program called Virtual PC and sold it through a company called Connectix. Virtual PC, like SoftPC allowed users to run a copy of windows on the Mac computer, in order to work around software incompatibilities. In 1998, a company called VMWare was established, and in 1999 began selling a product similar to Virtual PC called VMWare workstation. Initial versions of VMWare workstation only ran on windows; but later added support for other operating systems.
I mention VMWare because they are really the market leader in Virtualization in today's market. In 2001, VMWare released two new products as they branched into the enterprise market, ESX Server and GSX Server. GSX Server allowed users to run virtual machines on top of an existing operating system, such as Microsoft Windows, this is known as a Type-2 Hypervisor. ESX Server is known as a Type-1 Hypervisor, and does not require a host operating system to run Virtual Machines.
A Type-1 Hypervisor is much more efficient than a Type-2 hypervisor since it can be better optimized for virtualization, and does not require all the resources it takes to run a traditional operating system.
Since releasing ESX Server in 2001, VMWare has seen exponential growth in the enterprise market; and has added many complimentary products to enhance ESX Server. Other vendors have since entered the market. Microsoft acquired Connectix in 2003, after which they re-released Virtual PC as Microsoft Virtual PC 2004, then Microsoft Virtual Server 2005, both of which were un-released products from Connectix at the time Microsoft acquired them.
Citrix Inc, entered the Virtualization market in 2007 when they acquired Xensource, an open source virtualization platform which started in 2003. Citrix soon thereafter renamed the product to Xenserver.

Published Applications

In the early days of UNIX, you could access published applications via a Telnet Interface; and later SSH. Telnet is a small program allowing you to remotely access another computer. SSH is a version of telnet including various features such as encryption.
Telnet/SSH allows you to access either a text interface, or a Graphical interface, although it is not really optimized for graphics. Using telnet, you can access much of the functionality of the given server, from almost anywhere.
Windows and OS/2 had no manner of remotely accessing applications without third party tools. And the third party tools available only allowed one user at a time.
Some Engineers at IBM had an idea to create a multi-user interface for OS/2, however IBM did not share the same vision. So in 1989 Ed Lacobucci left IBM and started his own company called Citrus. Due to an existing trademark, the company was quickly re-branded as Citrix, a combination of Citrus and Unix.
Citrix licenced the source code to OS/2 through Microsoft and began working on creating their extension to OS/2. The company operated for two years and created a Multi-User interface for OS/2 called MULTIUSER. However Citrix was forced to abandon the project in 1991 after Microsoft announced it was no longer going to support OS/2. At that point, Citrix licensed source code from Microsoft and began working on a similar product focused on Windows.
In 1993 Citrix Acquired Netware Access Server from Novell. This product was similar to what Citrix had accomplished for OS/2 in that it gave multiple users access to a single system. Citrix Licensed the Windows NT source code in from Microsoft, then in 1995 began selling a product called WinFrame. WinFrame was a version of Windows NT 3.5 with remote access capabilities; allowing multiple users to access the system at the same time in order to remotely run applications.
While developing WinFrame for Windows NT 4.0, Microsoft decided to no longer grant the necessary licenses to Citrix. At this point Citrix licensed WinFrame to Microsoft, and it was included with Windows NT 4.0 as Terminal Services. As part of this agreement, Citrix agreed not to create a competing product, but was allowed to extend the functionality of Terminal Services.

Virtual Desktops

Virtual Desktop Infrastructures (VDI) is the practice of running a users Desktop Operating system, such as Windows XP within a virtual machine on a centralized infrastructure. Virtual Desktop Computers as we think of them today are a fairly new topic of conversation. But are very similar to the idea IBM had back in the 1960’s with the virtual machines on their mainframe computers. You give each user on the system their own operating system, then each user can then do as the please without disrupting anyother users on the system. Each user has their own computer, it is centralized, and it is a very efficient use of resources.
If you compare MultiCS from back in the 1960’s to the IBM Mainframes, it would be similar to comparing a Microsoft Terminal Server to a Virtual Desktop infrastructure today.
The jump from Virtual Desktops on Mainframes to Virtual Desktops as we know them today didn’t really happen until 2007 when VMWare introduced their VDI product. Prior to this release, it was possible for users in a company to use virtual desktops as their primary computers. However, it wasn’t really a viable solution due to management headaches. The introduction of Virtual Machine Manager from VMWare, and similar products from companies like Microsoft and Citrix has allowed this area to grow very rapidly.

Summary

Computer Virtualization has a long history, spanning nearly half a century. It can be used for making your applications easier to access remotely, allowing your applications to run on more systems than originally intended, improving stability, and more efficient use of resources.

Some technologies can be traced back to the 60’s such as Virtual Desktops, others can only be traced back a few years, such as virtualized applications.
 

Ubuntu Enterprise Cloud (UEC) : How to

Grow Your Own Cloud Servers With Ubuntu




Have you been wanting to fly to the cloud, to experiment with cloud computing? Now is your chance. With this article, we will step through the process of setting up a private cloud system using Ubuntu Enterprise Cloud (UEC), which is powered by the Eucalyptus platform.
The system is made up of one cloud controller (also called a front-end server) and one or more node controllers. The cloud controller manages the cloud environment. You can install the default Ubuntu OS images or create your own to be virtualized. The node controllers are where you can run the virtual machine (VM) instances of the images.

System Requirements

At least two computers must be dedicated to this cloud for it to work:
  • One for the front-end server (cloud or cluster controller) with a minimum 1GHz CPU, 512MB of memory, CD-ROM, 40GB of disk space, and an Ethernet network adapter
  • One or more for the node controller(s) with a CPU that supports Virtualization Technology (VT) extensions, 1GB of memory, CD-ROM, 40GB of disk space and an Ethernet network adapter
You might want to reference a list of Intel processors that include VT extensions. Optionally, you can run a utility, called SecurAble, in Windows. You can also check in Linux if a computer supports VT by seeing if "vmx" or "svm" is listed in the /proc/cpuinfo file. Run the command: egrep '(vmx|svm)' /proc/cpuinfo. Bear in mind, however, this tells you only if it's supported; the BIOS could still be set to disable it.

Preparing for the Installation

First, download the CD image for the Ubuntu Server remix — we're using version 9.10 — on any PC with a CD or DVD burner. Then burn the ISO image to a CD or DVD. If you want to use a DVD, make sure the computers that will be in the cloud read DVDs. If you're using Windows 7, you can open the ISO file and use the native burning utility. If you're using Windows Vista or later, you can download a third-party application like DoISO.
Before starting the installation, make sure the computers involved are setup with the peripherals they need (i.e., monitor, keyboard and mouse). Plus, make sure they're plugged into the network so they'll automatically configure their network connections.

Installing the Front-End Server

The installation of the front-end server is straightforward. To begin, simply insert the install CD, and on the boot menu select "Install Ubuntu Enterprise Cloud", and hit Enter. Configure the language and keyboard settings as needed. When prompted, configure the network settings.
When prompted for the Cloud Installation Mode, hit Enter to choose the default option, "Cluster". Then you'll have to configure the Time Zone and Partition settings. After partitioning, the installation will finally start. At the end, you'll be prompted to create a user account.
Next, you'll configure settings for proxy, automatic updates and email. Plus, you'll define a Eucalyptus Cluster name. You'll also set the IP addressing information, so users will receive dynamically assigned addresses.

Installing and Registering the Node Controller(s)

The Node installation is even easier. Again, insert the install disc, select "Install Ubuntu Enterprise Cloud" from the boot menu, and hit Enter. Configure the general settings as needed.
When prompted for the Cloud Installation Mode, the installer should automatically detect the existing cluster and preselect "Node." Just hit Enter to continue. The partitioning settings should be the last configuration needed.

Registering the Node Controller(s)

Before you can proceed, you must know the IP address of the node(s). To check from the command line:
/sbin/ifconfig
Then, you must install the front-end server's public ssh key onto the node controller:
  1. On the node controller, set a temporary password for the eucalyptus user using the command:
    sudo passwd eucalyptus
  2. On the front-end server, enter the following command to copy the SSH key:
    sudo -u eucalyptus ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub eucalyptus@
  3. Then you can remove the eucalyptus account password from the node with the command:
    sudo passwd -d eucalyptus
  4. After the nodes are up and the key copied, run this command from the front-end server to discover and add the nodes:
    sudo euca_conf --no-rsync --discover-nodes

Getting and Installing User Credentials

Enter these commands on the front-end server to create a new folder, export the zipped user credentials to it, and then to unpack the files:
mkdir -p ~/.euca
chmod 700 ~/.euca
cd ~/.euca
sudo euca_conf --get-credentials mycreds.zip (It takes a while for this to complete; just wait)
unzip mycreds.zip
cd -
The user credentials are also available via the web-based configuration utility; however, it would take more work to download the credentials there and move them to the server.

Setting Up the EC2 API and AMI Tools

Now you must setup the EC2 API and AMI tools on your front-end server. First, source the eucarc file to set up your Eucalyptus environment by entering:
~/.euca/eucarc
For this to be done automatically when you login, enter the following command to add that command to your ~/.bashrc file:
echo "[ -r ~/.euca/eucarc ] && . ~/.euca/eucarc" >> ~/.bashrc
Now to install the cloud user tools, enter:
sudo apt-get install ^31vmx32^4
To make sure it's all working, enter the following to display the cluster availability details:
. ~/.euca/eucarc
euca-describe-availability-zones verbose

Accessing the Web-Based Control Panel

Now you can access the web-based configuration utility. From any PC on the same network, go to the URL, https://:8443. The IP address of the cloud controller is displayed just after logging onto the front-end server. Note that that is a secure connection using HTTPS instead of just HTTP. You'll probably receive a security warning from the web browser since the server uses a self-signed certificate instead of one handled out by a known Certificate Authority (CA). Ignore the alert by adding an exception. The connection will still be secure.
The default login credentials are "admin" for both the Username and Password. The first time logging in you'll be prompted to setup a new password and email.

Installing images

Now that you have the basic cloud set up, you can install images. Bring up the web-based control panel, click the Store tab, and click the Install button for the desired image. It will start downloading, and then it will automatically install, which takes a long time to complete.

Running images

Before running an image on a node for the first time, run these commands to create a keypair for SSH:
touch ~/.euca/mykey.priv
chmod 0600 ~/.euca/mykey.priv
euca-add-keypair mykey > ~/.euca/mykey.priv
You also need to open port 22 up on the node, using the following commands:
euca-describe-groups
euca-authorize default -P tcp -p 22 -s 0.0.0.0/0
Finally, you can run your registered image. The command to run it is available via the web interface. Login to the web interface, click the Store tab, and select the How to Run link for the desired image. It will display a popup with the exact command.
The first time you run an instance, it will likely take a while for the image to be cached. You can get the status of your instance by running the command:
watch -n5 euca-describe-instances
Once it moves from "pending" to "running", reference the assigned IP address and connect to it:
IPADDR=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk '{print $4}')
ssh -i ~/.euca/mykey.priv ubuntu@$IPADDR
To terminate the SSH connection for the instance:
INSTANCEID=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk '{print $2}')
euca-terminate-instances $INSTANCEID

Maintaining the cloud

Now you should have a working cloud on your network. If you run into problems, you might have to reference the official documentation or hit the message boards. Before I leave, here are a few final tips:
  • To restart the front-end server run: sudo service eucalyptus [start|stop|restart]
  • To fresh a node run: sudo service eucalyptus-nc [start|stop|restart]
  • Here are some key file locations:
    • Log files
      /var/log/eucalyptus
    • Configuration files
      /etc/eucalyptus
    • Database
      /var/lib/eucalyptus/db
    • Keys
      /var/lib/eucalyptus
      /var/lib/eucalyptus/.ssh
Eric Geier is the Founder and CEO of NoWiresSecurity, which helps businesses easily protect their Wi-Fi with enterprise-level encryption by offering an outsourced RADIUS/802.1X authentication service. He is also the author of many networking and computing books for brands like For Dummies and Cisco Press.

Wednesday, July 29, 2009

Scalr core settings

= Introduction =

Guide for installing the application. Please add any relevant information.


= Details =

Add your content here. Format your content with:
* Text in *bold* or _italic_
* Headings, paragraphs, and lists
* Automatic links to other wiki pages

1. Install the PHP app on your host

2. change the permissions on the following:
chmod -R 0777 app/cache/smarty
chmod -R 0777 app/cache/smarty_bin
chmod -R 0777 app/logs
chmod -R 0777 app/etc/client_keys
chmod -R 0777 app/cron/cron.pid

3. Add your AWS cert and private key in etc/

4. Install and configure BIND. The following examples use ns1.scalr.net and ns2.scalr.net

5. Login to the app as admin and setup the nameservers (settings > nameservers)

6. Change core settings (example):


Dynamic A record TTL: 90 seconds
Default SOA owner: admin.scalr.net
Default SOA parent: ns1.scalr.net
Named.conf Zone template:
zone "{zone}" {
type master;
file "{db_filename}";
};


Account ID: 1234567890
Key name: UIBXXXXXXTMN3
Security groups prefix: scalr.
S3cfg template:

[default]
access_key = [access_key]
acl_public = False
force = False
host = s3.amazonaws.com
human_readable_sizes = False
recv_chunk = 4096
secret_key = [secret_key]
send_chunk = 4096
verbosity = WARNING

Instances limit: 20

Path to snmpinform: /usr/bin/snmptrap
Event handler URL: 67.19.29.18 (ip of the app)
Terminate instance if it will not send 'rebootFinish' event after reboot in seconds. 160
Terminate instance if it will not send 'hostUp' or 'hostInit' event after launch in seconds. 160




Privacy Policy - About - Control Panel


Add new


View all


View all


System log


Event log


View all


Nameservers


Manage shared roles



Core settings



Manage templates


Add new


View all

Scalr Links

Group discussion
http://groups.google.com/group/scalr-discuss?lnk=

Scalr Project site: (wiki)
http://code.google.com/p/scalr/

Monday, July 27, 2009

instance in scalr farm rebooting continuously

Answer:
Most likely, SNMP monitoring does not work or event handler configured
improperly.

Steps:
# apt-get install snmpd snmp
# vim /etc/default/snmpd
----------------------
#SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid 127.0.0.1'
## remove the localhost parameter from the end of the line
SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid '

----------------------
# vim /etc/snmp/snmpd.conf
----------------------
com2sec readonly default cc54b971b0bd1a
group MyROGroup v1 readonly
group MyROGroup v2c readonly
group MyROGroup usm readonly
view all included .1 80
view system included .iso.org.dod.internet.mgmt.mib-2.system
# context sec.model sec.level match read write notif
access MyROSystem "" any noauth exact system none none
access MyROGroup "" any noauth exact all none none
access MyRWGroup "" any noauth exact all all none
#
syslocation Amazon EC2
syscontact Nativetung Inc.
----------------------
# vim /etc/snmp/snmptrapd.conf
----------------------
authCommunity log,execute,net cc54b971b0bd1a
----------------------
# /etc/init.d/snmpd restart

Still have the same issue... ?
Have you configure the event handler
?

Login to scalr --> Settings --> Core settings --> add the event handler url and IP address as same as the scalr machine

Now pull up ur sleeves and lets explore more now..

Sunday, July 26, 2009

I am getting the following error while I try to create a new application or a new farm.

"Unexpected exception in javascript:Badly formed JSON string: '
\nFatal error: Class \'HttpRequest\' not found in /var/scalr/app/src/
LibWebta/library/NET/API/AWS/class.AmazonS3.php on line 454
\n'"

Solution:
pecl_http php extension is missing on your system.

So start here to resolve the issue...
# apt-get install libcurl4-openssl-dev
Make sure that curl is installed then procees again
# pecl install pecl_http

add extension=http.so in /etc/php5/apache2/php.ini & /etc/php5/cli/php.ini & "copy the http.so file to the php5 lib folder "

Friday, July 17, 2009

Scalr errors after install

After installing Scalr and adding a client I've tried to add an application to test out my setup. But at the second step I've got an alert saying:
-----------------------------
Error Type: LoadXML
Description: Incorrect XML
-----------------------------

A quick look at the apache log revealed the problem:
-----------------------------
File does not exist: /var/scalr/app/www/farm_amis.xml
-----------------------------

I thought that maybe I've missed a file so I did a svn checkout of scalr repository and tried to find the specified file:

#apt-get install subversion
#svn checkout http://scalr.googlecode.com/svn/trunk/ scalr-read-only
#find scalr-read-only -name farm_amis.xml -type f


Nothing came up. Weird. After a bit of looking around I've found a file called farm_amis_xml.php. So in fact farm_amis.xml was only a mod_rewrite directive.

Time to fix it, enable mod_rewrite and .htaccess files for apache2:

Edit /etc/apache2/sites-available/000-default and change the lines containing AllowOverride None to AllowOverride All

Go to /etc/apache2/mods-enabled and execute the following command:
# ln -s ../mods-available/rewrite.load

Restart apache2 server and everything should be ok:
#/etc/init.d/apache2 restart

Saturday, June 6, 2009

How to install Scalr on Ubuntu 8.10 EC2 Instance

Presentation:


If Amazon EC2 doesn't ring a bell to you, chances are that you are looking at the wrong page to find solutions for your problems. EC2 stands for Elastic Compute Cloud and it's a service offered by Amazon. I will not enter in details about the advantages of using it, since this is not the scope of this post. You can read more about it here: http://aws.amazon.com/ec2/

Scalr is a fully redundant, self-curing and self-scaling hosting environment utilizing Amazon's EC2. You basically can build farms of Amazon's instances that can do load balancing using nginx, serve web pages using Apache 2, use MySQL master-slave servers or maybe you may want to define your own roles.

The beauty of this is that you don't have to monitor the health of your server infrastructure, scalr will do it for you. If a node type gets overloaded scalr will launch another instance to spread the load and the cluster will be reconfigured.

Preparing:

Generate a new key for scalr instance:
ec2-add-keypair scalr-keypair > id_rsa-scalr-keypair
Edit id_rsa-scalr-keypair so it begins with
-----BEGIN RSA PRIVATE KEY-----
and is terminated with
-----END RSA PRIVATE KEY-----
Make sure you have the correct permissions for this key:

#chmod 600 id_rsa-scalr-keypair

If everything went ok you should see your new key when executing

>ec2-describe-keypairs


Choosing the right AMI:

For the instance we will be using ami-7806e211 which is an AMI containing a base install of Ubuntu 8.10 Intrepid Ibex Release. More details here.

Start the instance:

ec2-run-instances -z us-east-1a -k scalr-keypair ami-7806e211
You will get some output, look for the line that begins with INSTANCE and write down somewhere the id of the instance (i-XXXXXXXX) and the address of the instance(ec2-XX-XX-XX-XX.compute-1.amazonaws.com). The status of your instance should be pending.

Check in a couple of minutes the status of your instance:
>ec2-describe-instances i-XXXXXXXX

When the status is running it means that your instance is ready for work. You should have at least ssh and web ports open(22 and 80). If you are not sure execute the following commands:

ec2-authorize default -p 22
ec2-authorize default -p 80



Now connect to your instance using ssh:
ssh -i id_rsa-scalr-keypair -v ec2-XX-XX-XX-XX.compute-1.amazonaws.com
First time you connect you will be asked if
Are you sure you want to continue connecting (yes/no)?
Type yes and you should be the happy owner of a fresh Ubuntu Intrepid Ibex instance.

Update your system now:

#apt-get update
#apt-get upgrade


After the update is completed, logout and reboot your instance:
>ec2-reboot-instances i-XXXXXXXX

Installing required software:

Reconnect to your instance and install MySQL server and php extensions:
#apt-get install bind9 mysql-server mysql-client apache2 php5-cli libapache2-mod-php5 php5-mysql php5-mcrypt php5-mhash

When you install MySQL server you will be prompted to setup a password for the root account. Don't forget it, you will need it. Also you will have to restart Apache2 server after you finish installing everything, like this:

#/etc/init.d/apache2 restart

You could also download their php script that checks if your system has all the prerequisites:
#wget http://scalr.googlecode.com/files/testenvironment.php
#mv testenvironment.php /var/www/
#chmod a+r /var/www/testenvironment.php

Now point your browser to http://ec2-XX-XX-XX-XX.compute-1.amazonaws.com/checkenvironment.php and see if everything it's ok.

Most likely you will get only these 2 errors:
Errors:
• Cannot find SSH2 functions. Make sure that SSH2 Functions enabled.
• Cannot find SNMP functions. Make sure that SNMP Functions enabled.
Here is how to quick fix it:

Adding SSH2 support to PHP5, better known as: why don't we have php5-ssh2?
#apt-get install php5-dev php-pear libssh2-1 libssh2-1-dev
Thought it will be easy? Not so quick. Try to install it with:
#pecl install ssh2 "channel://pecl.php.net/ssh2-0.10"
I got an error saying:
ERROR: `make' failed
Great! Let's fix that stupid error. Edit the file /tmp/pear/download/ssh2-0.10/ssh2.c and replace the line containing:

#if LIBSSH2_APINO <>
with
#if false


(Delete the if statement stating with if LIBSSH2_APINO <>


Go to directory /tmp/pear/download/ssh2-0.10/ and compile the stuff manually:
#make && make install
#echo extension=ssh2.so >> /etc/php5/apache2/php.ini


I don't get it why they don't fix this thing. A lot of people are having this problem and are complaining!

Luckily for you and me snmp is a breeze, it is already in repositories:
#apt-get install php5-snmp

Restart apache server and check now if you have all the required extensions for scalr. You should have them.

Getting the latest version of Scalr:

At the time of writing this article latest version was 1.0 RC2
Go to Scalr download page and copy the link to the latest release. Download it using wget:
#wget http://scalr.googlecode.com/files/scalr-1.0RC2.tar.gz
Extract it:
#tar zxvf scalr-1.0RC2.tar.gz

Create database for scalr and import the sql:
#mysqladmin -p create scalr
#mysql -p scalr <>


Put the scalr application in /var/scalr and change permissions as suggested in the spartan documentation of scalr:
#mkdir /var/scalr && cp -R scalr/* /var/scalr/
#chmod 777 -R /var/scalr/app/cache /var/scalr/app/cron/cron.pid /var/scalr/app/etc/.passwd

ATTENTION: I'm not planing on using this instance for anything else except scalr. Also this is a clean install so I don't have anything of interest in /var/www. Read carefully the following first line:

#rm -rf /var/www/
#ln -sf /var/scalr/app/www /var/
#chmod a+rX -R /var/www


Edit the file /var/scalr/app/etc/config.ini and update it to your values:

[db]
driver=mysql ;Actually mysql is the only option here - mysqli doesn't support nconnect(), which is essential for PCNTL (which is essential for crobjobs)
host = "localhost"
name = "scalr"
user = "root"
pass = "*YOUR MYSQL PASS HERE*"


TO DO: make a mysql user for scalr.

Put your EC2 access certificate into /var/scalr/app/etc/cert-XXXXXXXXXXXX.pem
Put your EC2 private key into /var/scalr/app/etc/pk-XXXXXXXXXXXX.pem
ATTENTION: This part is a bit tricky. If you don't put the right settings you will not be able to start instances. I warned you!

Login to Amazon AWS and go to Home->Your Account->Access Identifiers
Go to http://ec2-XX-XX-XX-XX.compute-1.amazonaws.com/ and login with admin/admin
Go to Settings->Core Settings. Modify the following fields:
Password: duh!!! change it!
Email: your email address here
Account ID: AWS Account Number, it's called Account Number in AWS and it's top right. Remove the '-' from the number
Key Name:Scroll down in AWS untill you see Your X.509 Certificate: Copy everything between 'cert-' and '.pem'. Also XXXXXXXXXXXXXX is the string in the filenames of cert-XXXXXXXXXXXXX.pem and pk-XXXXXXXXXXX.pem. If they don't match you will have problems.
Access Key:Look for Your Secret Access Key: in AWS and click on Show. Copy the string.
Access key ID: is Your Access Key ID: in AWS.

Hit save.

setting crontab:


Type crontab -e and add the following lines to cron:
* * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --Poller
1 1 * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --RotateLogs
*/15 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --MySQLMaintenance
*/6 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --DNSMaintenance
*/3 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --DNSZoneListUpdate
*/2 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --DBQueueEvent
*/11 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --Cleaner

You are done. I hope.

How to terminate the instance:

I thought to write down instructions on how to terminate an instance. You should know how, but just in case, here it is how to stop the instance forever and not pay for it anymore. ATTENTION: terminate will really delete the instance so there is no way you can reconnect to it or recover it. Double check what instance you terminate!
ec2-terminate-instances i-XXXXXXXX

Wednesday, May 13, 2009

Amazone Cloud computing : How to

Prerequisites:


Topics

This document assumes that the reader is comfortable working in a Linux/UNIX enviornment

An installation of a Java 5 compatible Java Runtime Environment (JRE) is required. Additionally, accessing Linux and UNIX instances requires access to an SSH client and accessing Windows instances requires access to a Remote Desktop client. For more information, refer to the two following sections.

As a convention, all command line text is prefixed with a generic PROMPT> command line prompt. The actual command line prompt on your machine is likely to be different. We also use $ to indicate a Linux/UNIX specific command and C:\> for a Windows specific command. While we don't currently provide explicit instructions, the tools also work correctly on Mac OS X (which resemble the Linux and UNIX commands). The example output resulting from the command is shown immediately thereafter without any prefix.

[Note]Note

If you are using Cygwin, EC2_HOME, EC2_PRIVATE_KEY, and EC2_CERT must use Linux/UNIX paths (e.g,, /usr/bin instead of C:\usr\bin). However, JAVA_HOME should have a Windows path. Additionally, the value of EC2_HOME cannot contain any spaces, even if the value is quoted or the spaces are escaped.

The Java Runtime Environment

The command line tools used in this guide require Java version 5 or later to run. Either a JRE or JDK installation is acceptable. To view and download JREs for a range of platforms, including Linux/UNIX and Windows, go to http://java.sun.com/j2se/1.5.0/.

The command line tools depend on an environment variable (JAVA_HOME) to locate the Java runtime. This environment variable should be set to the full path of the directory that contains a sub-directory named bin which in turn contains the java (on Linux and UNIX) or the java.exe (on Windows) executable. You might want to simplify things by adding this directory to your path before other versions of Java.

Following is an example of how to set this environment variable in Linux and UNIX.

$ export JAVA_HOME=

You can confirm this by running $JAVA_HOME/bin/java -version and checking the output.

$ $JAVA_HOME/bin/java -version
java version "1.5.0_09"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)

Accessing Linux and UNIX Instances through SSH Clients

For some of the examples illustrated in this guide you'll need access to an SSH client. Most Linux and UNIX installations include an SSH client by default. If yours does not, the OpenSSH project provides a free implementation of the full suite of SSH tools. For more information, go to the OpenSSH home page.

Windows users can download and install PuTTY, a free SSH client. To download the client and installation instructions, go to the PuTTY home page. For information on how to use PuTTY with Amazon EC2, see Appendix: PuTTY.

Setting up an Account

To use Amazon EC2, you must sign up for an AWS Account, sign up for Amazon Simple Storage Service (Amazon S3), and sign up for Amazon EC2. These are three different actions that must be performed separately. For information on obtaining an AWS Account, go to the Amazon AWS Home Page. For information on signing up for Amazon S3, see Signing up for Amazon S3. For information on signing up for Amazon EC2, see Signing up for Amazon EC2.

Signing up for Amazon S3

Amazon EC2 AMIs are stored in and retrieved from Amazon S3. This means you need to sign up for Amazon S3. If you already have an Amazon S3 account, you can skip this step.

Amazon S3 Signup Process

1

Go to the Amazon S3 home page.

2

Click the Sign up for this service button.

If you don't already have an AWS account, you are prompted to create one as part of the sign up process.


After signing up for Amazon S3, point to the button labeled Your Web Services Account and select the AWS Access Key Identifiers link on the menu that appears. Be sure to note down your AWS account's Access Key ID and Secret Access Key. You need these to bundle your own image (see Making Your New Amazon Machine Image).

Signing up for Amazon EC2

After you sign up for Amazon S3, you'll need to sign up for Amazon EC2.

Amazon EC2 Signup Process

1

Log into your AWS account and follow the link to Amazon EC2 under the Browse Web Services section on the left.

2

Click Sign Up For Web Service in the top right of the screen and follow the on-screen instructions.


After you sign up for Amazon EC2, you'll want to get your Access Key identifiers.

Access Key Identifiers Process

1

Point to the button labeled Your Web Services Account and select the View Access Key Identifiers link on the menu that appears.

2

Click Create New in the Your X.509 Certificate section to create a new X.509 certificate.

3

Save the certificate and private key. You'll need this when you set up our command line tools (see Setting up the Tools).

4

Create a .ec2 directory in your home directory, and save these files to it with the filenames offered by your browser. You should end up with a PEM-encoded X509 certificate and a private key file named as shown in the following examples.

The following is an example of a PEM encoded signed X.509 certificate.

cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem

The following is an example of an unencrypted, PEM encoded RSA private key that corresponds to the preceding X.509 certificate.

pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem

This X.509 certificate is associated with your account until you generate or upload a new certificate. If you have an existing certificate that you prefer to use, you can return to the Access Key Identifiers upload it later.

5

Finally, you'll need to look up your AWS account ID. You should use this value whenever you need to provide an Amazon EC2 user ID. From the AWS portal page, point to Your Web Services Account and select the Account Activity link on the menu that appears. At the top of this page, locate your the Account Number which is a hyphenated number that looks similar to 4952-1993-3132. This number, with the hyphens removed, is your AWS account ID. In this example, it is 495219933132.


That's it. You're all set to start setting up your tools.


Setting up the Tools

One step remains before you'll be able to use Amazon EC2. You need to get our command line tools and set them up to use your AWS account.

Getting the Command Line Tools

The command line tools are available as a ZIP file in the Amazon EC2 Resource Center. These tools are written in Java and include shell scripts for both Windows 2000/XP and Linux/UNIX/Mac OSX. The ZIP file is self-contained; no installation is required. You just download it and unzip it.

Some additional setup is required in order for the tools to use your AWS account credentials. These are discussed next.

Telling the Tools Where They Live

The command line tools depend on an environment variable (EC2_HOME) to locate supporting libraries. You'll need to set this environment variable before you can use the tools. This should be set to the path of the directory into which the command line tools were unzipped. This directory is named ec2-api-tools-A.B-nnnn (A, B and n are version/release numbers), and contains sub-directories named bin and lib.

On Linux and UNIX, you can set this environment variable as follows.

$ export EC2_HOME=  

On Windows the syntax is slightly different.

C:\> set EC2_HOME=  

In addition, to make your life a little easier, you probably want to add the tools' bin directory to your system PATH. The rest of this guide assumes is done.

On Linux and UNIX, you can update your PATH as follows.

$ export PATH=$PATH:$EC2_HOME/bin 

On Windows the syntax is slightly different.

C:\> set PATH=%PATH%;%EC2_HOME%\bin 
[Note]Note

The Windows environment variables are reset when you close the command window. You might want to set them permanently.

Telling the Tools Who You Are

The command line tools need access to the private key and X.509 certificate you generated after signing up for the Amazon EC2 service (see Setting up an Account).

Since there's nothing stopping you from having more than one AWS account, you need to identify yourself to the command line API tools so they know which credentials to use for requests. It's possible, but tedious, to provide this information on the command line every time you invoke the tools. But it's far simpler to set up some environment variables and be done with it.

Two environment variables are supported to make this possible. They can be set to point at your private key and certificate. If these environment variables are set, the tools use their values to find the relevant credentials. The environment variable EC2_PRIVATE_KEY should reference your private key file, and EC2_CERT should reference your X509 certificate.

On Linux and UNIX, you can set these environment variables as follows.

$ export EC2_PRIVATE_KEY=~/.ec2/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem 
$ export EC2_CERT=~/.ec2/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem

On Windows the syntax is slightly different.

C:\> set EC2_PRIVATE_KEY=c:\ec2\pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem 
C:\> set EC2_CERT=c:\ec2\cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem

Changing the Region

By default, the Amazon EC2 tools use the Eastern United States region (us-east-1) with the us-east-1.ec2.amazonaws.com service endpoint. This section describes how to specify a different region.

To specify a different region

  1. View available regions by entering the following:

    PROMPT> ec2-describe-regions

    REGION us-east-1 us-east-1.ec2.amazonaws.com
    REGION eu-west-1 eu-west-1.ec2.amazonaws.com

  2. If you want to change the service endpoint on Linux and UNIX, set the EC2_URL environment variable as follows:

    $ export EC2_URL=https://  
  3. If you want to change the service endpoint on Windows, set the EC2_URL environment variable as follows:

    C:\> set EC2_URL=https://  

You're ready to start using Amazon EC2.

Running an Instance

This section describes how to run an instance that uses Linux or UNIX.

[Note]Note

For information about creating your own AMI, refer to the Amazon Elastic Compute Cloud Developer Guide.

Before We Begin

Before running an instance, verify the requirements in the following table.

Verification Steps

1

Ensure you have a version 1.5.0 compatible Java Runtime installation, and that the JAVA_HOME environment variable has been correctly set. If not, see The Java Runtime Environment.

2

Ensure you have an active Amazon Web Services Account, and that you've signed up for both Amazon S3 and Amazon EC2. If not, see Setting up an Account.

3

Ensure that you have created a directory called .ec2 in your home directory for Linux/UNIX or a directory without spaces in Windows (e.g., C:\EC2), that contains your X.509 certificate and private key, and that they're named correctly. If not, see Signing up for Amazon EC2.

4

Ensure that the EC2_HOME environment variable has been correctly set. If not, see Telling the Tools Where They Live.

5

Ensure that the EC2_CERT and EC2_PRIVATE_KEY environment variables have been correctly set. If not, see Telling the Tools Who You Are.


Once these are correct, you are ready to launch your first instance.

Finding a Suitable AMI

To find a suitable AMI

  1. Use the ec2-describe-images command.

    PROMPT> ec2-describe-images -o self -o amazon | grep machine

    IMAGE ami-2c5fba45 ec2-public-images/demo-paid-AMI-v1.07.manifest.xml amazon available public A79EC0DB i386 machine
    IMAGE ami-bd9d78d4 ec2-public-images/demo-paid-AMI.manifest.xml amazon available public A79EC0DB i386 machine
    IMAGE ami-2f5fba46 ec2-public-images/developer-image-i386-v1.07.manifest.xml amazon available public i386 machine
    IMAGE ami-26b6534f ec2-public-images/developer-image.manifest.xml amazon available public i386 machine
    IMAGE ami-f51aff9c ec2-public-images/fedora-8-i386-base-v1.06.manifest.xml amazon available public i386 machine aki-a71cf9ce ari-a51cf9cc
    IMAGE ami-2b5fba42 ec2-public-images/fedora-8-i386-base-v1.07.manifest.xml amazon available public i386 machine aki-a71cf9ce ari-a51cf9cc
    IMAGE ami-f21aff9b ec2-public-images/fedora-8-x86_64-base-v1.06.manifest.xml amazon available public x86_64 machine aki-b51cf9dcari-b31cf9da
    IMAGE ami-2a5fba43 ec2-public-images/fedora-8-x86_64-base-v1.07.manifest.xml amazon available public x86_64 machine aki-b51cf9dcari-b31cf9da
    IMAGE ami-a21affcb ec2-public-images/fedora-core-6-x86_64-base-v1.06.manifest.xml amazon available public x86_64 machine aki-a53adfccari-a23adfcb
    IMAGE ami-2d5fba44 ec2-public-images/fedora-core-6-x86_64-base-v1.07.manifest.xml amazon available public x86_64 machine aki-a53adfccari-a23adfcb
    IMAGE ami-225fba4b ec2-public-images/fedora-core4-apache-mysql-v1.07.manifest.xml amazon available public i386 machine
    IMAGE ami-25b6534c ec2-public-images/fedora-core4-apache-mysql.manifest.xml amazon available public i386 machine
    IMAGE ami-2e5fba47 ec2-public-images/fedora-core4-apache-v1.07.manifest.xml amazon available public i386 machine
    IMAGE ami-23b6534a ec2-public-images/fedora-core4-apache.manifest.xml amazon available public i386 machine
    IMAGE ami-215fba48 ec2-public-images/fedora-core4-base-v1.07.manifest.xml amazon available public i386 machine
    IMAGE ami-20b65349 ec2-public-images/fedora-core4-base.manifest.xml amazon available public i386 machine
    IMAGE ami-205fba49 ec2-public-images/fedora-core4-i386-base-v1.07.manifest.xml amazon available public i386 machine aki-9b00e5f2
    IMAGE ami-255fba4c ec2-public-images/fedora-core4-mysql-v1.07.manifest.xml amazon available public i386 machine
    IMAGE ami-22b6534b ec2-public-images/fedora-core4-mysql.manifest.xml amazon available public i386 machine
    IMAGE ami-36ff1a5f ec2-public-images/fedora-core6-base-x86_64.manifest.xml amazon available public x86_64 machine
    IMAGE ami-235fba4a ec2-public-images/getting-started-v1.07.manifest.xml amazon available public i386 machine
    IMAGE ami-2bb65342 ec2-public-images/getting-started.manifest.xml amazon available public i386 machine

    The command lists your AMIs and Amazon's public AMIs. The output might not exactly match the preceding example.

  2. Look for the line containing the public image identified by the ec2-public-images/getting-started.manifest.xml value in the third column and note the corresponding value in the second column.

    This is the AMI ID you need. In this example, it is ami-2bb65342.

Generating an SSH Key Pair

You will run an instance of a public AMI. Since it has no password, you need a public/private key pair to login to the instance. One half of this key pair is embedded in your instance, allowing you to login securely without a password using the other half of the key pair. After learning to create your own images, you can choose other mechanisms to allow you to securely login to your new instances. Every key pair you generate requires a name. Be sure to choose a name that is easy to remember.

[Note]Note

If you lose the key pair file, you will not be able to access the instances launched with that key pair. However, you can always generate a new key pair to launch new instances.

To generate a key pair using gsg-keypair

  1. Enter the following information.

    PROMPT>  ec2-add-keypair gsg-keypair 

    Amazon EC2 returns a key pair, similar to the key pair in the following example.

    KEYPAIR gsg-keypair  1f:51:ae:28:bf:89:e9:d8:1f:25:5d:37:2d:7d:b8:ca:9f:f5:f1:6f
    -----BEGIN RSA PRIVATE KEY-----
    MIIEoQIBAAKCAQBuLFg5ujHrtm1jnutSuoO8Xe56LlT+HM8v/xkaa39EstM3/aFxTHgElQiJLChp
    HungXQ29VTc8rc1bW0lkdi23OH5eqkMHGhvEwqa0HWASUMll4o3o/IX+0f2UcPoKCOVUR+jx71Sg
    5AU52EQfanIn3ZQ8lFW7Edp5a3q4DhjGlUKToHVbicL5E+g45zfB95wIyywWZfeW/UUF3LpGZyq/
    ebIUlq1qTbHkLbCC2r7RTn8vpQWp47BGVYGtGSBMpTRP5hnbzzuqj3itkiLHjU39S2sJCJ0TrJx5
    i8BygR4s3mHKBj8l+ePQxG1kGbF6R4yg6sECmXn17MRQVXODNHZbAgMBAAECggEAY1tsiUsIwDl5
    91CXirkYGuVfLyLflXenxfI50mDFms/mumTqloHO7tr0oriHDR5K7wMcY/YY5YkcXNo7mvUVD1pM
    ZNUJs7rw9gZRTrf7LylaJ58kOcyajw8TsC4e4LPbFaHwS1d6K8rXh64o6WgW4SrsB6ICmr1kGQI7
    3wcfgt5ecIu4TZf0OE9IHjn+2eRlsrjBdeORi7KiUNC/pAG23I6MdDOFEQRcCSigCj+4/mciFUSA
    SWS4dMbrpb9FNSIcf9dcLxVM7/6KxgJNfZc9XWzUw77Jg8x92Zd0fVhHOux5IZC+UvSKWB4dyfcI
    tE8C3p9bbU9VGyY5vLCAiIb4qQKBgQDLiO24GXrIkswF32YtBBMuVgLGCwU9h9HlO9mKAc2m8Cm1
    jUE5IpzRjTedc9I2qiIMUTwtgnw42auSCzbUeYMURPtDqyQ7p6AjMujp9EPemcSVOK9vXYL0Ptco
    xW9MC0dtV6iPkCN7gOqiZXPRKaFbWADp16p8UAIvS/a5XXk5jwKBgQCKkpHi2EISh1uRkhxljyWC
    iDCiK6JBRsMvpLbc0v5dKwP5alo1fmdR5PJaV2qvZSj5CYNpMAy1/EDNTY5OSIJU+0KFmQbyhsbm
    rdLNLDL4+TcnT7c62/aH01ohYaf/VCbRhtLlBfqGoQc7+sAc8vmKkesnF7CqCEKDyF/dhrxYdQKB
    gC0iZzzNAapayz1+JcVTwwEid6j9JqNXbBc+Z2YwMi+T0Fv/P/hwkX/ypeOXnIUcw0Ih/YtGBVAC
    DQbsz7LcY1HqXiHKYNWNvXgwwO+oiChjxvEkSdsTTIfnK4VSCvU9BxDbQHjdiNDJbL6oar92UN7V
    rBYvChJZF7LvUH4YmVpHAoGAbZ2X7XvoeEO+uZ58/BGKOIGHByHBDiXtzMhdJr15HTYjxK7OgTZm
    gK+8zp4L9IbvLGDMJO8vft32XPEWuvI8twCzFH+CsWLQADZMZKSsBasOZ/h1FwhdMgCMcY+Qlzd4
    JZKjTSu3i7vhvx6RzdSedXEMNTZWN4qlIx3kR5aHcukCgYA9T+Zrvm1F0seQPbLknn7EqhXIjBaT
    P8TTvW/6bdPi23ExzxZn7KOdrfclYRph1LHMpAONv/x2xALIf91UB+v5ohy1oDoasL0gij1houRe
    2ERKKdwz0ZL9SWq6VTdhr/5G994CK72fy5WhyERbDjUIdHaK3M849JJuf8cSrvSb4g==
    -----END RSA PRIVATE KEY-----

    The private key returned must be saved to a local file so that you can use it later.

  2. Create a file named id_rsa-gsg-keypair and paste the entire key generated in step 1, including the following lines.

    "-----BEGIN RSA PRIVATE KEY-----"
    "-----END RSA PRIVATE KEY-----"
  3. Confirm that the file contents looks similar to the following and save the file.

    You can save the file in any directory, but if you do not put it in your current directory, you should specify the full path when using commands that require the key pair.

     -----BEGIN RSA PRIVATE KEY-----
    MIIEoQIBAAKCAQBuLFg5ujHrtm1jnutSuoO8Xe56LlT+HM8v/xkaa39EstM3/aFxTHgElQiJLChp
    HungXQ29VTc8rc1bW0lkdi23OH5eqkMHGhvEwqa0HWASUMll4o3o/IX+0f2UcPoKCOVUR+jx71Sg
    5AU52EQfanIn3ZQ8lFW7Edp5a3q4DhjGlUKToHVbicL5E+g45zfB95wIyywWZfeW/UUF3LpGZyq/
    ebIUlq1qTbHkLbCC2r7RTn8vpQWp47BGVYGtGSBMpTRP5hnbzzuqj3itkiLHjU39S2sJCJ0TrJx5
    i8BygR4s3mHKBj8l+ePQxG1kGbF6R4yg6sECmXn17MRQVXODNHZbAgMBAAECggEAY1tsiUsIwDl5
    91CXirkYGuVfLyLflXenxfI50mDFms/mumTqloHO7tr0oriHDR5K7wMcY/YY5YkcXNo7mvUVD1pM
    ZNUJs7rw9gZRTrf7LylaJ58kOcyajw8TsC4e4LPbFaHwS1d6K8rXh64o6WgW4SrsB6ICmr1kGQI7
    3wcfgt5ecIu4TZf0OE9IHjn+2eRlsrjBdeORi7KiUNC/pAG23I6MdDOFEQRcCSigCj+4/mciFUSA
    SWS4dMbrpb9FNSIcf9dcLxVM7/6KxgJNfZc9XWzUw77Jg8x92Zd0fVhHOux5IZC+UvSKWB4dyfcI
    tE8C3p9bbU9VGyY5vLCAiIb4qQKBgQDLiO24GXrIkswF32YtBBMuVgLGCwU9h9HlO9mKAc2m8Cm1
    jUE5IpzRjTedc9I2qiIMUTwtgnw42auSCzbUeYMURPtDqyQ7p6AjMujp9EPemcSVOK9vXYL0Ptco
    xW9MC0dtV6iPkCN7gOqiZXPRKaFbWADp16p8UAIvS/a5XXk5jwKBgQCKkpHi2EISh1uRkhxljyWC
    iDCiK6JBRsMvpLbc0v5dKwP5alo1fmdR5PJaV2qvZSj5CYNpMAy1/EDNTY5OSIJU+0KFmQbyhsbm
    rdLNLDL4+TcnT7c62/aH01ohYaf/VCbRhtLlBfqGoQc7+sAc8vmKkesnF7CqCEKDyF/dhrxYdQKB
    gC0iZzzNAapayz1+JcVTwwEid6j9JqNXbBc+Z2YwMi+T0Fv/P/hwkX/ypeOXnIUcw0Ih/YtGBVAC
    DQbsz7LcY1HqXiHKYNWNvXgwwO+oiChjxvEkSdsTTIfnK4VSCvU9BxDbQHjdiNDJbL6oar92UN7V
    rBYvChJZF7LvUH4YmVpHAoGAbZ2X7XvoeEO+uZ58/BGKOIGHByHBDiXtzMhdJr15HTYjxK7OgTZm
    gK+8zp4L9IbvLGDMJO8vft32XPEWuvI8twCzFH+CsWLQADZMZKSsBasOZ/h1FwhdMgCMcY+Qlzd4
    JZKjTSu3i7vhvx6RzdSedXEMNTZWN4qlIx3kR5aHcukCgYA9T+Zrvm1F0seQPbLknn7EqhXIjBaT
    P8TTvW/6bdPi23ExzxZn7KOdrfclYRph1LHMpAONv/x2xALIf91UB+v5ohy1oDoasL0gij1houRe
    2ERKKdwz0ZL9SWq6VTdhr/5G994CK72fy5WhyERbDjUIdHaK3M849JJuf8cSrvSb4g==
    -----END RSA PRIVATE KEY-----
  4. If you're using OpenSSH (or any reasonably paranoid SSH client), you should set the permissions of this file so it is only readable by you.

    On Linux and UNIX, enter the information in the following example.

    $  chmod 700 id_rsa-gsg-keypair ; ls -l id_rsa-gsg-keypair  

    You receive output similar to the following example.

    -rw-------  1 fred flintstones 1701 Jun 19 17:57 id_rsa-gsg-keypair 
[Note]Note

If you are using PuTTY in Windows, convert the private key to PuTTY's format. For more information on using PuTTy with Amazon EC2, see Appendix: PuTTY .

Running an Instance

To launch an instance of your AMI

  1. Use the ec2-run-instances command.

    PROMPT>  ec2-run-instances ami-235fba4a -k gsg-keypair

    Amazon EC2 returns output similar to the following example.


    RESERVATION r-7430c31d 924417782495 default
    INSTANCE i-ae0bf0c7 ami-2bb65342 pending gsg-keypair 0 m1.small 2008-03-21T16:19:25+0000 us-east-1a
  2. Look for the instance ID in the second field and write it down.

    You use it to manipulate this instance (including terminating it when you are finished).

    It takes a few minutes for the instance to launch.

  3. The following command displays the launch status of the instance.

    PROMPT>  ec2-describe-instances i-ae0bf0c7 
    RESERVATION r-7430c31d 924417782495 default
    INSTANCE i-ae0bf0c7 ami-2bb65342 ec2-67-202-7-236.compute-1.amazonaws.com ip-10-251-31-162.ec2.internal running gsg-keypair 0 m1.small 2008-03-21T16:19:25+0000us-east-1a
[Important]Important

After launching an instance, you are billed hourly for running time. If you leave this tutorial at any time, make sure you terminate any instances you have started as described in Terminating Your Instances.

When the instance state in the field just before the key pair name reads "running" the instance started booting. There might be a short time before it is accessible over the network, however. The first DNS name is your instance's external DNS name, i.e. the one that can be used to contact it from the Internet. The second DNS name is your instance's local DNS name, and is only contactable by other instances within the Amazon EC2 network. The DNS names of your instances are different than those shown in the preceding example and you should use yours instead. The examples in this guide use the public DNS name.

Authorizing Network Access to Your Instances

To authorize access to your instance

  • Enter the ec2-authorize command.

    PROMPT>  ec2-authorize default -p 22
    PERMISSION default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0
    PROMPT> ec2-authorize default -p 80
    PERMISSION default ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0

    Since we didn't specify otherwise, your instance was launched in your default group. The first command authorizes network access to instances in your default group on the standard SSH port (22). Similarly, the second command opens up the standard HTTP port (80). For more details on controlling network security groups, see the Amazon EC2 Developer Guide.

[Caution]Caution

In this example, you enable any IP address to access port 22 of the instance. Although this might be acceptable for testing purposes, it is extremely unsafe for production environments. For production systems, you must obtain your public IP address ranges and grant access to those ranges only. For example, if your IP address is 123.123.123.123, you specify 123.123.123.123/32.

Connecting to your Instance

To connect to your instance

  1. Open a web browser and go to http:///, where is your instance's public hostname as returned by ec2-describe-instances (ec2-67-202-51-223.compute-1.amazonaws.com in the example).

    A webpage welcoming you to your instance displays.

    [Note]Note

    If the website times out, your instance might not have finished starting up. Wait a couple of minutes and try again.

  2. Whenever you launch a public AMI that you have not rebundled, run the ec2-get-console-output command and locate the SSH HOST KEY FINGERPRINTS section.

    PROMPT>  ec2-get-console-output instance_id

    ...
    ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
    ec2: 2048 bc:89:29:c6:45:4b:b3:e2:c1:41:81:22:cb:3c:77:54
    /etc/ssh/ssh_host_key.pub
    ec2: 2048 fc:8d:0c:eb:0e:a6:4a:6a:61:50:00:c4:d2:51:78:66
    /etc/ssh/ssh_host_rsa_key.pub
    ec2: 1024 b5:cd:88:6a:18:7f:83:9d:1f:3b:80:03:10:17:7b:f5
    /etc/ssh/ssh_host_dsa_key.pub
    ec2: -----END SSH HOST KEY FINGERPRINTS-----

    ...

    Note the fingerprints. You will compare them in the next step.

  3. Use the following command to login as root and exercise full control over this instance as you would any host.

    PROMPT>  ssh -i id_rsa-gsg-keypair root@ec2-67-202-51-223.compute-1.amazonaws.com 
    The authenticity of host 'ec2-67-202-51-223.compute-1.amazonaws.com (216.182.225.42)' can't be established.
    RSA key fingerprint is fc:8d:0c:eb:0e:a6:4a:6a:61:50:00:c4:d2:51:78:66.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'ec2-67-202-51-223.compute-1.amazonaws.com' (RSA) to the list of known hosts.
    Last login: Wed Jun 21 08:02:08 2006
    root@ec2-67-202-51-223 #

    If you are launching a public AMI, verify the fingerprint matches one of the fingerprints from the output of the ec2-get-console-output command. If it doesn't, someone might be attempting a "man-in-the-middle" attack.

[Note]Note

Your machine might have a different name for the preceding ssh command or even use different command line options. Consult the documentation for your machine or download one of the clients described in Accessing Linux and UNIX Instances through SSH Clients if you are unsure whether you have such a client installed. For more information about using PuTTY on Windows, see Appendix: PuTTY.

Congratulations

You've set up the tools and used them to run an instance based on a public AMI. You have learned enough to successfully use to run as many standard Linux/UNIX instances as you wish. You can run instances based on any of the public AMIs by following this process.

The next section builds on this success by having you connect to the running instance and customize it to create your own image. If you wish to save this for a later date, be sure to terminate any instances you have started as described in the section called “Terminating Your Instances”.

Cleaning Up

Terminating Your Instances

As soon as your instance starts to boot, you are billed for the resources it consumes. Once you have decided that you no longer require its services, you can terminate an instance using the instance identifier you received in Running an Instance.

[Note]Note

You cannot recover a terminated instance. However, you can launch additional instances of an AMI.

To terminate an instance

  • Execute the following command:

    PROMPT>  ec2-terminate-instances i-10a64379 
    INSTANCE i-10a64379 running shutting-down

    It takes a few minutes for the instance to terminate because Amazon EC2 needs to clean-up your data. For information on checking the status of your instance, see Running an Instance

[Tip]Tip

For Linux and UNIX, you can also terminate your instances by logging onto the instances with your ssh tool and running the "shutdown -h" command. Don't forget the "-h", otherwise you put your instance into single user mode, which is quite useless.

# /sbin/shutdown -h now  

For Windows, you can also terminate your instances by selecting Shutdown from the Start menu or by entering the following from the command line:

C:\> shutdown /s /t 0  

Deregistering Your AMI

If you no longer need your AMI you should deregister it from Amazon EC2.

For this task you'll need your AWS Access Key ID () and AWS Secret Access Key (). For more information on how to find these keys, see Signing up for Amazon S3.

To deregister an AMI

  • Execute the following command:

    PROMPT>  ec2-deregister ami-2bb65342 
    IMAGE ami-2bb65342

Removing Your AMI from Amazon S3

You can delete your deregistered AMIs from Amazon S3. If you don't, Amazon S3 continues to charge you for the space you are using.

The AMI tools contain a command to delete your image. You can get these from the resource center or they are installed on the instance from which you created the image.

To delete your AMI

  • Execute the following command:

    PROMPT> ec2-delete-bundle -b  -p sampleimage -a  -s  
    Deleting AMI bundle parts from https://s3.amazonaws.com/...
    Deleted sampleimage.part.00.
    Deleted sampleimage.part.01.
    Deleted sampleimage.part.02.
    Deleted sampleimage.part.03.
    ...
    Deleted sampleimage.part.23.
    Deleted sampleimage.part.24.
    Delete Bundle complete.
[Note]Note

If you are unable to run the AMI tools, you can use any Amazon S3 utility to delete AMIs.


//Lin u x u niL