Ubuntu Enterprise Cloud (UEC) : How to

Grow Your Own Cloud Servers With Ubuntu

Have you been wanting to fly to the cloud, to experiment with cloud computing? Now is your chance. With this article, we will step through the process of setting up a private cloud system using Ubuntu Enterprise Cloud (UEC), which is powered by the Eucalyptus platform.

The system is made up of one cloud controller (also called a front-end server) and one or more node controllers. The cloud controller manages the cloud environment. You can install the default Ubuntu OS images or create your own to be virtualized. The node controllers are where you can run the virtual machine (VM) instances of the images.

System Requirements

At least two computers must be dedicated to this cloud for it to work:

• One for the front-end server (cloud or cluster controller) with a minimum 1GHz CPU, 512MB of memory, CD-ROM, 40GB of disk space, and an Ethernet network adapter
• One or more for the node controller(s) with a CPU that supports Virtualization Technology (VT) extensions, 1GB of memory, CD-ROM, 40GB of disk space and an Ethernet network adapter

You might want to reference a list of Intel processors that include VT extensions. Optionally, you can run a utility, called SecurAble, in Windows. You can also check in Linux if a computer supports VT by seeing if "vmx" or "svm" is listed in the /proc/cpuinfo file. Run the command: egrep '(vmx|svm)' /proc/cpuinfo. Bear in mind, however, this tells you only if it's supported; the BIOS could still be set to disable it.

Preparing for the Installation

First, download the CD image for the Ubuntu Server remix — we're using version 9.10 — on any PC with a CD or DVD burner. Then burn the ISO image to a CD or DVD. If you want to use a DVD, make sure the computers that will be in the cloud read DVDs. If you're using Windows 7, you can open the ISO file and use the native burning utility. If you're using Windows Vista or later, you can download a third-party application like DoISO.
Before starting the installation, make sure the computers involved are setup with the peripherals they need (i.e., monitor, keyboard and mouse). Plus, make sure they're plugged into the network so they'll automatically configure their network connections.

Installing the Front-End Server

The installation of the front-end server is straightforward. To begin, simply insert the install CD, and on the boot menu select "Install Ubuntu Enterprise Cloud", and hit Enter. Configure the language and keyboard settings as needed. When prompted, configure the network settings.
When prompted for the Cloud Installation Mode, hit Enter to choose the default option, "Cluster". Then you'll have to configure the Time Zone and Partition settings. After partitioning, the installation will finally start. At the end, you'll be prompted to create a user account.
Next, you'll configure settings for proxy, automatic updates and email. Plus, you'll define a Eucalyptus Cluster name. You'll also set the IP addressing information, so users will receive dynamically assigned addresses.

Installing and Registering the Node Controller(s)

The Node installation is even easier. Again, insert the install disc, select "Install Ubuntu Enterprise Cloud" from the boot menu, and hit Enter. Configure the general settings as needed.
When prompted for the Cloud Installation Mode, the installer should automatically detect the existing cluster and preselect "Node." Just hit Enter to continue. The partitioning settings should be the last configuration needed.

Registering the Node Controller(s)

Before you can proceed, you must know the IP address of the node(s). To check from the command line:

/sbin/ifconfig

Then, you must install the front-end server's public ssh key onto the node controller:

1. On the node controller, set a temporary password for the eucalyptus user using the command:

   sudo passwd eucalyptus

2. On the front-end server, enter the following command to copy the SSH key:

   sudo -u eucalyptus ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub eucalyptus@

3. Then you can remove the eucalyptus account password from the node with the command:

   sudo passwd -d eucalyptus

4. After the nodes are up and the key copied, run this command from the front-end server to discover and add the nodes:

   sudo euca_conf --no-rsync --discover-nodes

Getting and Installing User Credentials

Enter these commands on the front-end server to create a new folder, export the zipped user credentials to it, and then to unpack the files:

mkdir -p ~/.euca chmod 700 ~/.euca cd ~/.euca sudo euca_conf --get-credentials mycreds.zip (It takes a while for this to complete; just wait) unzip mycreds.zip cd -

The user credentials are also available via the web-based configuration utility; however, it would take more work to download the credentials there and move them to the server.

Setting Up the EC2 API and AMI Tools

Now you must setup the EC2 API and AMI tools on your front-end server. First, source the eucarc file to set up your Eucalyptus environment by entering:

~/.euca/eucarc

For this to be done automatically when you login, enter the following command to add that command to your ~/.bashrc file:

echo "[ -r ~/.euca/eucarc ] && . ~/.euca/eucarc" >> ~/.bashrc

Now to install the cloud user tools, enter:

sudo apt-get install ^31vmx32^4

To make sure it's all working, enter the following to display the cluster availability details:

. ~/.euca/eucarc euca-describe-availability-zones verbose

Accessing the Web-Based Control Panel

Now you can access the web-based configuration utility. From any PC on the same network, go to the URL, https://:8443. The IP address of the cloud controller is displayed just after logging onto the front-end server. Note that that is a secure connection using HTTPS instead of just HTTP. You'll probably receive a security warning from the web browser since the server uses a self-signed certificate instead of one handled out by a known Certificate Authority (CA). Ignore the alert by adding an exception. The connection will still be secure.
The default login credentials are "admin" for both the Username and Password. The first time logging in you'll be prompted to setup a new password and email.

Installing images

Now that you have the basic cloud set up, you can install images. Bring up the web-based control panel, click the Store tab, and click the Install button for the desired image. It will start downloading, and then it will automatically install, which takes a long time to complete.

Running images

Before running an image on a node for the first time, run these commands to create a keypair for SSH:

touch ~/.euca/mykey.priv chmod 0600 ~/.euca/mykey.priv euca-add-keypair mykey > ~/.euca/mykey.priv

You also need to open port 22 up on the node, using the following commands:

euca-describe-groups euca-authorize default -P tcp -p 22 -s 0.0.0.0/0

Finally, you can run your registered image. The command to run it is available via the web interface. Login to the web interface, click the Store tab, and select the How to Run link for the desired image. It will display a popup with the exact command.
The first time you run an instance, it will likely take a while for the image to be cached. You can get the status of your instance by running the command:

watch -n5 euca-describe-instances

Once it moves from "pending" to "running", reference the assigned IP address and connect to it:

IPADDR=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk '{print $4}') ssh -i ~/.euca/mykey.priv ubuntu@$IPADDR

To terminate the SSH connection for the instance:

INSTANCEID=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk '{print $2}') euca-terminate-instances $INSTANCEID

Maintaining the cloud

Now you should have a working cloud on your network. If you run into problems, you might have to reference the official documentation or hit the message boards. Before I leave, here are a few final tips:

• To restart the front-end server run: sudo service eucalyptus [start|stop|restart]
• To fresh a node run: sudo service eucalyptus-nc [start|stop|restart]
• Here are some key file locations:
• Log files
  /var/log/eucalyptus
• Configuration files
  /etc/eucalyptus
• Database
  /var/lib/eucalyptus/db
• Keys
  /var/lib/eucalyptus
  /var/lib/eucalyptus/.ssh

Eric Geier is the Founder and CEO of NoWiresSecurity, which helps businesses easily protect their Wi-Fi with enterprise-level encryption by offering an outsourced RADIUS/802.1X authentication service. He is also the author of many networking and computing books for brands like For Dummies and Cisco Press.

Setup of VSFTPD virtual users

If you are hosting several web sites, for security reason, you may want the webmasters to access their own files only. One of the good way is to give them FTP access by setup of VSFTPD virtual users and directories. This article describes how you can do that easily.
(See also: Setup of VSFTPD virtual users – another approach)
1. Installation of VSFTPD
For Red Hat, CentOS and Fedora, you may install VSFTPD by the command

# yum install vsftpd

For Debian and Ubuntu,

# apt-get install vsftpd

2. Virtual users and authentication
We are going to use pam_userdb to authenticate the virtual users. This needs a username / password file in `db’ format – a common database format. We need `db_load’ program. For CentOS, Fedora, you may install the package `db4-utils’:

# yum install db4-utils

For Ubuntu,

# apt-get install db4.2-util

To create a `db’ format file, first create a plain text file `virtual-users.txt’ with the usernames and passwords on alternating lines:

mary
123456
jack
654321

Then execute the following command to create the actual database:

# db_load -T -t hash -f virtual-users.txt /etc/vsftpd/virtual-users.db

Now, create a PAM file /etc/pam.d/vsftpd-virtual which uses your database:

auth required pam_userdb.so db=/etc/vsftpd/virtual-users
account required pam_userdb.so db=/etc/vsftpd/virtual-users

3. Configuration of VSFTPD
Create a configuration file /etc/vsftpd/vsftpd-virtual.conf,

# disables anonymous FTP
anonymous_enable=NO
# enables non-anonymous FTP
local_enable=YES
# activates virtual users
guest_enable=YES
# virtual users to use local privs, not anon privs
virtual_use_local_privs=YES
# enables uploads and new directories
write_enable=YES
# the PAM file used by authentication of virtual uses
pam_service_name=vsftpd-virtual
# in conjunction with 'local_root',
# specifies a home directory for each virtual user
user_sub_token=$USER
local_root=/var/www/virtual/$USER
# the virtual user is restricted to the virtual FTP area
chroot_local_user=YES
# hides the FTP server user IDs and just display "ftp" in directory listings
hide_ids=YES
# runs vsftpd in standalone mode
listen=YES
# listens on this port for incoming FTP connections
listen_port=60021
# the minimum port to allocate for PASV style data connections
pasv_min_port=62222
# the maximum port to allocate for PASV style data connections
pasv_max_port=63333
# controls whether PORT style data connections use port 20 (ftp-data)
connect_from_port_20=YES
# the umask for file creation
local_umask=022

4. Creation of home directories
Create each user’s home directory in /var/www/virtual, and change the owner of the directory to the user `ftp’:

# mkdir /var/www/virtual/mary
# chown ftp:ftp /var/www/virtual/mary

5. Startup of VSFTPD and test
Now we can start VSFTPD by the command:

# /usr/sbin/vsftpd /etc/vsftpd/vsftpd-virtual.conf

and test the FTP access of a virtual user:

# lftp -u mary -p 60021 192.168.1.101

The virtual user should have full access to his directory.

Hints on how to check your machine for intrusion

The compromise of kernel.org and related machines has made it clear that
some developers, at least, have had their systems penetrated.  As we
seek to secure our infrastructure, it is imperative that nobody falls
victim to the belief that it cannot happen to them.  We all need to
check our systems for intrusions.  Here are some helpful hints as
proposed by a number of developers on how to check to see if your Linux
machine might be infected with something:


0. One way to be sure that your system is not compromised is to simply
   do a clean install; we can all benefit from a new start sometimes.
   Before reinstalling any systems, though, consider following the steps
   below to learn if your system has been hit or not.

1. Install the chkrootkit package from your distro repository and see if it
   reports anything.  If your distro doesn't have the chkroot package,
   download it from:
 http://www.chkrootkit.org/

   Another tool is the ossec-rootcheck tool which can be found at:
 http://www.ossec.net/main/rootcheck

   And another one is the rkhunter program:
    http://www.rootkit.nl/projects/rootkit_hunter.html
   [Note, this tool has the tendancy to give false-positives on some
   Debian boxes, please read /usr/share/doc/rkhunter/README.Debian.gz if
   you run this on a Debian machine]

2. Verify that your package signatures match what your package manager thinks
   they are.

   To do this on a rpm-based system, run the following command:
    rpm --verify --all
   Please read the rpm man page for information on how to interpret the
   output of this command.

   To do this on a Debian based system, run the following bash snippet:
 dpkg -l \*|while read s n rest; do if [ "$s" == "ii" ]; then echo $n;
 fi; done > ~/tmp.txt
 for f in `cat ~/tmp.txt`; do debsums -s -a $f; done
   If you have a source-based system (Gentoo, LFS, etc.) you presumably
   know what you are doing already.

3. Verify that your packages are really signed with the distro's keys.

   Here's a bash snippet that can do this on a rpm based system to
   verify that the packages are signed with any key, not necessarily
   your distro's key.  That exercise is left for the reader:

 for package in `rpm -qa`; do
  sig=`rpm -q --qf '%{SIGPGP:pgpsig}\n' $package`
  if [ -z "$sig" ] ; then
   # check if there is a GPG key, not a PGP one
   sig=`rpm -q --qf '%{SIGGPG:pgpsig}\n' $package`
   if [ -z "$sig" ] ; then
    echo "$package does not have a signature!!!"
   fi
  fi
 done
   Unfortunately there is no known way of verifying this on Debian-based
   systems.

4. To replace a package that you find suspect, uninstall it and install
   it anew from your distro.  For example, if you want to reinstall the
   ssh daemon, you would do:
 $ /etc/init.d/sshd stop
 rpm -e openssh
 zypper install openssh # for openSUSE based systems
 yum install openssh # for Fedora based systems

   Ideally do this from a live cdrom boot, using the 'rpm --root' option
   to point rpm at the correct location.


5. From a liveCD environment, look for traces such as:
   a. Rogue startup scripts in /etc/rc*.d and equivalent directories.
   b. Strange directories in /usr/share that do not belong to a package.
      This can be checked on an rpm system with the following bash snippet:
 for file in `find /usr/share/`; do
  package=`rpm -qf -- ${file} | grep "is not owned"`
  if [ -n "$package" ] ; then
   echo "weird file ${file}, please check this out"
  fi
 done
6. Look for mysterious log messages, such as:
   a. Unexpected logins in wtmp and /var/log/secure*, quite possibly
      from legitimate users from unexpected hosts.
   b. Any program trying to touch /dev/mem.
   c. References to strange (non-text) ssh version strings in
      /var/log/secure*.  These do not necessarily indicate *successful*
      breakins, but they indicate *attempted* breakins which means your
      system or IP address has been targeted.

7. If any of the above steps show possible signs of compromise, you
   should investigate further and identify the actual cause.  If it
   becomes clear that the system has indeed been compromised, you should
   certainly reinstall the system from the beginning, and change your
   credentials on all machines that this machine would have had access
   to, or which you connected to through this machine.  You will need
   to check your other systems carefully, and you should almost
   certainly notify the administrators of other systems to which you
   have access.

Finally, please note that these hints are not guaranteed to turn up
signs of a compromised systems.  There are a lot of attackers out there;
some of them are rather more sophisticated than others.  You should
always be on the alert for any sort of unexpected behavior from the
systems you work with.

----------------------------------------------------------------------------

----------------------------------------------------------------------------

I would like to add here a few controls I ran on firewall and system logs,
that are easy to perform and which report few false positives :

  - check that communications between your local machines are expected ;
    for instance if you have an SSH bouncing machine, it probably receives
    tens of thousands of SSH connection attempts from outside every day,
    but it should never ever attempt to connect to another machine unless
    it's you who are doing it. So checking the firewall logs for SSH
    connections on port 22 from local machines should only report your
    activity (and nothing should happen when you sleep).

  - no SSH log should report failed connection attempts between your
    local machines (you do have your keys and remember your password).
    And if it happens from time to time (eg: user mismatch between
    machines), it should look normal to you. You should never observe
    a connection attempt for a user you're not familiar with (eg: admin).

     $ grep sshd /var/log/messages
     $ grep sshd /var/log/messages | grep 'Invalid user'
  - outgoing connections from your laptop, desktop or anything should
    never happen when you're not there, unless there is a well known
    reason (package updates, browser left open and refreshing ads). All
    unexpected activity should be analysed (eg: connections to port 80
    not coming from a browser should only match one distro mirror).
    This is particularly true for cheap appliances which become more
    and more common and are rarely secured. A NAS or media server, a
    switch, a WiFi router, etc... has no reason to ever connect anywhere
    without you being aware of it (eg: download a firmware update).

  - check for suspicious DNS requests from machines that are normally
    not accessed. A number of services perform DNS requests when
    connected to, in order to log a resolved address. If the machine
    was penetrated and the logs wiped, the DNS requests will probably
    still lie in the firewall logs. While there's nothing suspect from
    a machine that does tens of thousands DNS requests a day, one that
    does 10 might be suspect.

  - check for outgoing SMTP connections. Most machines probably never
    send any mail outside or route them through a specific relay. If
    one machine suddenly tries to send mails directly to the outside,
    it might be someone trying to steal some data (eg: mail ssh keys).

  - check for long holes in logs various service logs. The idea is that
    if a system was penetrated and the guy notices he left a number of
    traces, he will probably have wiped some logs. A simple way to check
    for this is to count the number of events per hour and observe huge
    variations. Eg:

       $ cut -c1-9 < /var/log/syslog |uniq -c
       8490 Oct  1 00
       7712 Oct  1 01
       8316 Oct  1 02
       6743 Oct  1 03
       7428 Oct  1 04
       7041 Oct  1 05
       7762 Oct  1 06
       6562 Oct  1 07
       7137 Oct  1 08
        160 Oct  1 09
    Activity looks normal here. Something like this however would be
    extremely suspect :

       8490 Oct  1 00
        712 Oct  1 01
       6743 Oct  1 03

  - check that you never observe in logs a local address that you
    don't know. For instance, if your reverse proxy is on a DMZ which
    is provided by the same physical switch as your LAN and your switch
    becomes ill and loses all its VLAN configuration, it them becomes
    easy to add an alias to the reverse-proxy to connect directly to
    LAN machines and bypass a firewall (and its logs).

  - it's always a good exercise to check for setuids on all your machines.
    You'll generally discover a number of things you did not even suspect
    existed and will likely want to remove them. For instance, my file
    server had dbus-daemon-launch-helper setuid root. I removed this crap
    as dbus has nothing to do on such a machine. Similarly I don't need
    fdmount to mount floppies. I might not use floppies often, and if I do,
    I know how to use sudo.

       $ find / -user root -perm -4000 -ls

  - last considerations to keep in mind is that machines which receive
    incoming connections from outside should never be able to go out, and
    should be isolated in their own LAN. It's not hard to do at all, and
    it massively limits the ability to bounce between systems and to steal
    information. It also makes firewall logs much more meaningful, provided
    they are stored on a support with limited access, of course :-)
 

Also refer: 

http://www.ossec.net/main/rootcheck

http://www.rootkit.nl/projects/rootkit_hunter.html

http://www.chkrootkit.org/