Friday, July 24, 2009

Secure Websites Using SSL And Certificates

This how-to will guide you through the entire process of setting up a secure website using SSL and digital certificates. This guide assumes that you have already a fully functional (and configured) server running Apache, BIND, and OpenSSL. Just as a side note, this guide was written based on a Fedora Core 6 distribution, but should be the same for most other distros out there.

Introduction

Today it is possible to create a secure website with relative ease by requiring a client to present a digitally signed certificate. A digitally signed certificate is simply a piece of information that contains data about the subject, public key, dates of validity, identification of the Certificate Authority (CA), and the digital signature. There are typically two ways to go about creating a secure website. First is by the use of a self-signed certificate. The second way is by using a Trusted Certificate signed by a CA. The choice is up to you, and this tutorial will show you how to do both. Go ahead and su into root and let's begin!

Step1. Preconfigure SSL


We can preconfigure OpenSSL to fill in the fields of the certificate by modifying its configuration file. Doing this will save us a few steps during the certificate creation process. First, navigate to the SSL directory and do a listing of files

# cd /etc/pki/tls
# ls -al

In the listing you should see the file openssl.cnf. Go ahead and open up this file in your favorite editor and find the beginning of the [ req_distinguished_name ] section. Modify the countryName_default, stateOrProvinceName_default, 0.organizationName_default, 1.organizationName_default values with the values that suit your needs. The section of the file you are looking for looks like this

---------------------------------------------------------------------------

[ req_distinguished_name ]
countryName = Country Name (2 letter code)
countryName_default = US
countryName_min = 2
countryName_max = 2
stateOrProvinceName = State or Province Name (full name)
stateOrProvinceName_default = Berkshire
localityName = Locality Name (eg, city)
localityName_default = Berkshire Hills
0.organizationName = Organization Name (eg, company)
0.organizationName_default = My Secure Website Company
# we can do this but it is not needed normally :-)
#1.organizationName = Second Organization Name (eg, company)
#1.organizationName_default = MSWC
organizationalUnitName = Organizational Unit Name (eg, section)
#organizationalUnitName_default = Your IT Department
commonName = Common Name (eg, your name or your server\'s hostname)
commonName_max = 64
emailAddress = Email Address
emailAddress_max = 64

---------------------------------------------------------------------------

Once you have made the changes, save and quit the editor.

Step2: Creating the Server CA

Now we must create the server's CA. You should still be in the /etc/pki/tls directory, if you are not sure just run a quick pwd to find out. We now need to make sure that we have no other certificates in the system:

# rm -rf ../../CA

Once that is done, we will use the shell scripts OpenSSL comes with to create a new CA:

# misc/CA -newca

A dialog will appear saying:

---------------------------------------------------------------------------

CA certificate filename (or enter to create)

---------------------------------------------------------------------------

Go ahead and press enter. At this point the program will begin generating the CA. During this process you will be asked to enter a PEM pass phrase, create one and write it down! Once you have entered the PEM and verified it you will be asked some questions about the Distinguished Name (DN). A lot of the questions will have the correct values in them by default since we edited the openssl.cnf file. So press enter until it asks for the Common Name. When asked for the common name you should put down the server's hostname. It should look like this:

---------------------------------------------------------------------------

#misc/CA -newca
CA certificate filename (or enter to create)
Making CA certificate ...
Generating a 1024 bit RSA private key
.............++++++
.......................++++++
writing new private key to '../../CA/private/./cakey.pem'
Enter PEM pass phrase: XXXXXXXXX
Verifying - Enter PEM pass phrase: XXXXXXXXX
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [US]:
State or Province Name (full name) [Berkshire]:
Locality Name (eg, city) [Berkshire Hills]:
Organization Name (eg, company) [My Secure Website Company]:
Organizational Unit Name (eg, section) [Your IT Department]:
Common Name (eg, your name or your server's hostname) []:hostname
Email Address []:admin-email@yourdomain.com

---------------------------------------------------------------------------

The CA for the server has been created and placed in the /etc/CA/private/cakey.pem file.

Step3: Creating A Server Certificate Request

Now that we have the CA established, we need to make a request to create the certificate. Once again, using the shell scripts included with OpenSSL we want to run this command:

# misc/CA -newreq

Once the command has been entered, the program will being generating the request. When asked for the PEM, go ahead and enter it and press enter. Once again you will be asked the same questions about the Distinguished Name (DN). Everything should be the same, except this time when asked for the common name you should enter the complete address of the site you wish to secure (not just the hostname as this will cause internet browsers to see a conflict!). Once you have entered the Common Name you will be asked to create a Challenge Password, create one and write it down. This process should look like this:

---------------------------------------------------------------------------

#misc/CA -newreq
CA certificate filename (or enter to create)
Making CA certificate ...
Generating a 1024 bit RSA private key
.............++++++
.......................++++++
writing new private key to 'newkey.pem'
Enter PEM pass phrase: XXXXXXXXX
Verifying - Enter PEM pass phrase: XXXXXXXXX
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [US]:
State or Province Name (full name) [Berkshire]:
Locality Name (eg, city) [Berkshire Hills]:
Organization Name (eg, company) [My Secure Website Company]:
Organizational Unit Name (eg, section) [Your IT Department]:
Common Name (eg, your name or your server's hostname) []:hostname.yourdomain.com
Email Address []:admin-email@yourdomain.com
A challenge password []:
An optional company name []:
Request is in newreq.pem, private key is in newkey.pem

The certificate request has now been created and placed in the file /etc/pki/tls/newreq.pem and the private key is in /etc/pki/tls/newkey.pem.

---------------------------------------------------------------------------

Step4: Signing the Certificate

Before you move on to the next step you need to consider which of the two routes you are going to take - the self-signed certificate or trusted third-party signed certificate. The choice is entirely up to you. If your website is not for commercial use, you can probably get away with using a self-signed certificate, however, most web browsers will not recognize it as a "trusted website" right away since it does not have the signature of a trusted CA. Alternatively to both the self-signed or purchased methods, I recommend looking into using CACert.org. CACert.org is a completely public, community driven, and FREE CA that is growing.

If you want to create a Self-Signed certificate go to Step 4A.

If you want to create a Trusted certificate thru a third party go to Step 4B.

Step 4A: Self-Signed certificate

This is a fairly simple process. Using the shell scripts we can sign our own certificate with ease by running the following command

# misc/CA -sign

Once executed the program loads the configuration file from the openssl.cnf file. Once it opens the CA key information you will be prompted for the PEM pass phrase, go ahead and enter it. Once the pass phrase is entered the program then verifies the integrity and validity of the data. Once it is done checking, it will ask if you want to sign the certificate, enter "y" and continue. The whole result should look something like this:

---------------------------------------------------------------------------

# misc/CA -sign
Using configuration from /etc/pki/tls/openssl.cnf
Enter pass phrase for ../../CA/private/cakey.pem: XXXXXXXXX
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 1 (0x1)
Validity
Not Before: Dec 22 18:52:38 2006 GMT
Not After : Dec 22 18:52:38 2007 GMT
Subject:
countryName = US
stateOrProvinceName = Berkshire
localityName = Berkshire Hills
organizationName = My Secure Website Company
organizationalUnitName = Your IT Department
commonName = hostname.yourdomain.com
emailAddress = admin-email@yourdomain.com
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
CA:D9:9B:01:D2:9C:15:39:96:62:53:29:D6:6E:D8:B8:62:9D:A0:BD
X509v3 Authority Key Identifier:
keyid:DB:B6:B7:15:40:C4:7B:14:AE:F6:CB:A9:DF:44:C3:1E:39:AE:E0:DD

Certificate is to be certified until Dec 22 18:52:38 2007 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 1 (0x1)
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, ST=Berkshire, O=My Secure Website Company, OU=Your IT Department,
CN=hostname/emailAddress=admin-email@yourdomain.com
Validity
Not Before: Dec 22 18:52:38 2006 GMT
Not After : Dec 22 18:52:38 2007 GMT
Subject: C=US, ST=Berkshire, L=Berkshire Hills, O=My Secure Website Company,
OU=Your IT Department, CN=hostname.yourdomain.com/emailAddress=admin-email@yourdomain.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (1024 bit)
Modulus (1024 bit):
00:cc:91:db:ea:95:c6:d3:03:75:cd:74:b5:58:28:
b7:df:e5:33:4b:82:53:90:b0:98:5f:14:0b:d1:1a:
44:e4:41:0b:e8:59:f6:f9:d1:26:6a:d9:25:a5:ac:
8e:9d:f0:cd:65:f2:3a:13:b7:37:e2:82:3f:02:e4:
fd:f6:0b:eb:4d:27:97:2b:ab:74:07:8d:fa:0c:4b:
27:ea:c8:78:f6:1e:60:b9:fc:5f:30:0f:8f:02:4b:
d4:d1:b4:bc:a1:bb:d6:e9:dd:78:2e:76:28:21:b5:
5b:76:88:f7:cd:d3:40:26:07:33:2e:95:71:09:e1:
b4:5f:b2:95:99:fd:30:17:49
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
CA:D9:9B:01:D2:9C:15:39:96:62:53:29:D6:6E:D8:B8:62:9D:A0:BD
X509v3 Authority Key Identifier:
keyid:DB:B6:B7:15:40:C4:7B:14:AE:F6:CB:A9:DF:44:C3:1E:39:AE:E0:DD
Signature Algorithm: sha1WithRSAEncryption
69:a4:15:98:77:7e:fb:38:88:c3:a0:2e:d8:2b:24:51:50:2c:
cd:dc:a7:56:e0:f5:f3:01:73:47:b2:67:69:4b:94:28:e9:86:
51:fd:e0:2f:a0:30:8a:0d:0c:10:3d:5c:d5:4a:53:df:ea:0c:
38:49:61:1b:92:61:fe:7f:96:bb:65:f4:f9:14:40:52:7d:aa:
8c:0d:07:e1:06:aa:fb:bf:72:6f:7c:d1:68:6e:78:d6:30:7c:
6b:f0:b9:ce:57:5b:f9:d2:cc:df:2c:42:f3:e0:42:cf:0b:5e:
57:ff:72:ac:4a:2b:b9:ac:df:b7:23:80:ea:e4:37:01:b3:13:
9f:73

-----BEGIN CERTIFICATE-----
MIIDIjCCAougAwIBAgIBATANBgkqhkiG9w0BAQUFADCBgDELMAkGA1UEBhMCVVMx
ETAPBgNVBAgTCENvbG9yYWRvMQ0wCwYDVQQKEwRVQ0NTMRkwFwYDVQQLExBDb21w
dXRlciBTY2llbmNlMQ4wDAYDVQQDEwVDQWZjNjEkMCIGCSqGSIb3DQEJARYVY2FA
ZmM2LmNzbmV0LnVjY3MuZWR1MB4XDTA2MTIyMjE4NTIzOFoXDTA3MTIyMjE4NTIz
OFowga8xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhDb2xvcmFkbzEZMBcGA1UEBxMQ
Q29sb3JhZG8gU3ByaW5nczENMAsGA1UEChMEVUNDUzEZMBcGA1UECxMQQ29tcHV0
ZXIgU2NpZW5jZTEbMBkGA1UEAxMSZmM2LmNzbmV0LnVjY3MuZWR1MSswKQYJKoZI
hvcNAQkBFhx3ZWJtYXN0ZXJAZmM2LmNzbmV0LnVjY3MuZWR1MIGfMA0GCSqGSIb3
DQEBAQUAA4GNADCBiQKBgQDMkdvqlcbTA3XNdLVYKLff5TNLglOQsJhfFAvRGkTk
QQvoWfb50SZq2SWlrI6d8M1l8joTtzfigj8C5P32C+tNJ5crq3QHjfoMSyfqyHj2
HmC5/F8wD48CS9TRtLyhu9bp3XgudightVt2iPfN00AmBzMulXEJ4bRfspWZ/TAX
SQIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NMIEdl
bmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUytmbAdKcFTmWYlMp1m7YuGKd
oL0wHwYDVR0jBBgwFoAU27a3FUDEexSu9sup30TDHjmu4N0wDQYJKoZIhvcNAQEF
BQADgYEAaaQVmHd++ziIw6Au2CskUVAszdynVuD18wFzR7JnaUuUKOmGUf3gL6Aw
ig0MED1c1UpT3+oMOElhG5Jh/n+Wu2X0+RRAUn2qjA0H4Qaq+79yb3zRaG541jB8
a/C5zldb+dLM3yxC8+BCzwteV/9yrEoruazftyOA6uQ3AbMTn3M=
-----END CERTIFICATE-----
Signed certificate is in newcert.pem

---------------------------------------------------------------------------

The certificate has now been self-signed and placed in the /etc/pki/tls directory. Both the private key and the certificate need to be renamed to better identify what server they represent so do the following:

# mv newcert.pem hostnameCert.prm
# mv newkey.pem hostnameKey.pem

That is it! Continue on to step 5 to configure Apache to use the newly created information.

Step 4B: The Trusted Certificate

To get a server certificate created by some third-party company like Thawte, VeriSign, GeoTrust, or CACert.org, look for the certification request file that was created in step 3 (This file can be found in /etc/pki/tls/newreq.pem). Since each CA operates differently, you will have to research which way they want you to submit the data (also called the CSR by some websites)held within this file; however, most places have a simple on-line copy-and-paste procedure, or a file upload system. Once you have submitted the data held in the newreq.pem file you will be given back another batch of code similar to what you submitted. This snippet of code is your new signed/trusted certificate.

Once you have that code you will need to save it back to the /etc/pki/tls directory with a filename which makes it easily identifiable such as hostnameCert.pem

Once you have completed this step you are ready to move on to Step 5.

Step 5: Configuring Apache To Use Your Certificates

We must now configure the Apache server to utilize SSL. The first thing we will want to do is copy our newly made certificates to the proper location. Do the following:

# cd /etc/httpd/conf
# ls -al

In the listing, look for directories called ssl.key and ssl.crt. If they do not exist go ahead and make them:

# mkdir ssl.key ssl.crt

Once you have found or created the directories, it is time to copy our certificates over:

# cp /etc/pki/tls/hostnameCert.pem ssl.crt/server.crt
# cp /etc/pki/tls/hostnameKey.pem ssl.key/server.key

Once you have copied over the files, it is time to configure the Apache server to use them. Do this:

# cd /etc/httpd/conf.d
# ls -al

You should see a file called ssl.conf. Go ahead and open this file with your favorite editor. There are two thing that need to be changed in this file. They are around lines 112 and 119, it looks like this:

---------------------------------------------------------------------------

#   Server Certificate:
# Point SSLCertificateFile at a PEM encoded certificate. If
# the certificate is encrypted, then you will be prompted for a
# pass phrase. Note that a kill -HUP will prompt again. A new
# certificate can be generated using the genkey(1) command.
SSLCertificateFile /etc/httpd/conf/ssl.crt/localhost.crt


# Server Private Key:
# If the key is not combined with the certificate, use this
# directive to point at the key file. Keep in mind that if
# you've both a RSA and a DSA private key you can configure
# both in parallel (to also allow the use of DSA ciphers, etc.)
SSLCertificateKeyFile /etc/httpd/conf/ssl.key/localhost.key

---------------------------------------------------------------------------

Change the SSLCertificateFile path to the new path of /etc/httpd/conf/ssl.crt/server.crt. Then, we need to change the SSLCertificateKeyFile to the new path of /etc/httpd/conf/ssl.key/server.key. Once these changes are made save and quit the editor.

If we leave the configuration just as it is now, the SSL will work with a restart of the apache server. However, there is bit of a bump - each time we restart the httpd server it will ask us for the PEM passphrase. This is ok if you are constantly able to monitor the system, but for the sake of saving you that headache let's fix that problem. To fix this issue do the following:

# cd /etc/httpd/conf/ssl.key
# cp server.key server.key.orig
# openssl rsa -in server.key.orig -out server.key

You will be asked to enter the PEM pass phrase, so go ahead and input it and press enter. The program will finish and will have created the key file that will not require you to enter the PEM pass phrase on starting Apache. For security, we need to put security permissions on this file and restart the Apache server:

# chmod 700 server.key
# service httpd restart

If everything was done correctly, Apache will start with no problems. Now it is time to try out your new secure website. Open your browser and navigate to https://hostname.yourdomain.com.

If you decided to do a self-signed certificate, or one through CACert.org you may get a message about the certificate not being trusted. This is normal and you should just tell the browser to accept the certificate. Once your on the website you should be able to view your certificate and the details on it.

Congratulations - you have just set up up an SSL secure website!

Thursday, July 23, 2009

CentOS / Redhat (RHEL) / Fedora Linux enable/disable an apache module

Under Redhat based Linux distributions you need to modify *.conf file stored in /etc/httpd/conf.d/ directory. Apache scans for files with the .conf suffix at start up.

So if the system does not need to use mod_python, rename 'python.conf' to 'python.bak' and restart Apache with the command 'service httpd restart in order to disable that particular module and save memory.


# cd /etc/httpd/conf.d/
# mv perl.conf no.perl.bak
# /etc/init.d/httpd

Enable a Module

To re-enable modules, simply rename them to their original names and restart Apache to get back module functionality:


# cd /etc/httpd/conf.d/
# mv no.perl.bak perl.conf
# /etc/init.d/httpd


More about /etc/httpd/httpd.conf directory

This directory holds Apache 2.0 module-specific configuration files; any files in this directory which have the ".conf" extension will be processed as Apache configuration files. Files are processed in alphabetical order, so if using configuration directives which depend on, say, mod_perl being loaded, ensure that
these are placed in a filename later in the sort order than "perl.conf".

  • manual.conf : This configuration file allows the manual to be accessed at http://localhost/manual/
  • perl.conf : mod_perl incorporates a Perl interpreter into the Apache web server, so that the Apache web server can directly execute Perl code.
  • php.conf : php5 module for php
  • proxy_ajp.conf : When loaded, the mod_proxy_ajp module adds support for proxying to an AJP/1.3 backend server such as Tomcat.
  • python.conf : mod_python is a module that embeds the Python language interpreter within the server, allowing Apache handlers to be written in Python.
  • squid.conf : Access to squid cache manager
  • ssl.conf : Apache SSL server configuration
  • webalizer.conf : Webalizer stats configuration
  • welcome.conf : This configuration file enables the default "Welcome" page if there is no default index page present for
    the root URL.

Debian / Ubuntu Linux enable/disable an apache module (simple way)

Debian / Ubuntu Linux comes with two scripts:

(a) a2enmod is a script that enables the specified module within the apache2 configuration. It does this by creating symlinks within /etc/apache2/mods-enabled.

(b) a2dismod disables a module by removing those symlinks. It is not an error to enable a module which is already enabled, or to disable one which is already disabled.

Syntax:
a2dismod {module-name}

For example disable a module called foo or mime_magic, enter:


# a2dismod foo
# a2dismod mime_magic

If you ever need to re-enable disabled modules, use a2enmod command as follows:

Syntax:
a2enmod {module-name}

For example, enable imagemap module, enter:
# a2enmod imagemap

Note if don't know module name, it will show a list of all available modules:
# a2enmod

----------------------------
Which module would you like to disable?
Your choices are: alias auth_basic authn_file authz_default authz_groupfile authz_host authz_user autoindex cgi dir env fastcgi mime negotiation php5 setenvif status userdir
Module name? authz_user
----------------------------

Enable apache modules under Debian based system

Apache is usually suitable out of the box for most common used. The apache structure under debian based systems is actually really well made as it is really easy to activate or deactivate module.

This how-to will show how to activate or deactivate available modules under a debian system running apache2.

1. How it works:

There is 2 kinds of modules used by apache:

  • Modules compiled in
  • Modules that are loaded when you launch apache

In order to check which modules were compiled in with apache, you can type the following command:

$apache2 -l

---------------
Compiled in modules:
core.c
mod_access.c
mod_auth.c
mod_log_config.c
mod_logio.c
mod_env.c
mod_setenvif.c
prefork.c
http_core.c
mod_mime.c
mod_status.c
mod_autoindex.c
mod_negotiation.c
mod_dir.c
mod_alias.c
mod_so.c

---------------

This list correspond to the modules compile with apache on an Ubuntu Dapper system. As you can see, there is no php, rewrite.... modules compiled in. Those modules are meant to be included when running apache.

Now, let check the main apache configuration file, namely /etc/apache2/apache2.conf, around line 115, you can see those 2 lines:

---------------
# Include module configuration:
Include /etc/apache2/mods-enabled/*.load
Include /etc/apache2/mods-enabled/*.conf
---------------

As you can see, apache load any files ending with .load first and .conf after, in /etc/apache2/mods-enabled/.

Now, let's have a look in that directory:

$ ls /etc/apache2/mods-enabled/

---------------
actions.load php5.conf rewrite.load userdir.load
cgi.load php5.load userdir.conf

---------------

As you can see, I have cgi, actions, php5, userdir and rewrite modules enabled. This allow me to run an php5 scripts in /home/user/public_html using rewriting rules.

Going further up into the investigation, we can see that files in mods-enabled are not actually files, but links to files contained in mods-available:

$ ls -l /etc/apache2/mods-enabled/userdir.load

---------------
lrwxrwxrwx 1 root root 30 2006-05-15 03:00 /etc/apache2/mods-enabled/userdir.load -> ../mods-available/userdir.load

---------------

Now, let's have a look at /etc/apache2/mods-available:

$ ls /etc/apache2/mods-available/

---------------
actions.load dav_fs.conf info.load proxy.load
asis.load dav_fs.load ldap.load rewrite.load
auth_anon.load dav.load mem_cache.load speling.load
auth_dbm.load deflate.load mime_magic.conf ssl.conf
auth_digest.load disk_cache.load mime_magic.load ssl.load
auth_ldap.load expires.load php5.conf suexec.load
cache.load ext_filter.load php5.load unique_id.load
cern_meta.load file_cache.load proxy.conf userdir.conf
cgid.conf headers.load proxy_connect.load userdir.load
cgid.load imap.load proxy_ftp.load usertrack.load
cgi.load include.load proxy_http.load vhost_alias.load

---------------

This basically contains all the files linked by mods-enabled plus a whole load of available modules.

2. Adding modules:

Now, taking into account the strucutre of apache, it is pretty easy to add modules to be loaded by apache. Let's assume that you want to add mime_magic module. To do so, you can either:

  • add it by hand:

    $cd /etc/apache2/mods-enabled
    $ sudo ln -s ../mods-available/mime_magic.conf mime_magic.conf
    $sudo ln -s ../mods-available/mime_magic.load mime_magic.load

    OR

  • add it the debian way with a2enmod:

    $sudo a2enmod

  • Which module would you like to enable?
    Your choices are: actions asis auth_anon auth_dbm auth_digest auth_ldap cache cern_meta cgid cgi dav_fs dav deflate disk_cache expires ext_filter file_cache headers imap include info ldap mem_cache mime_magic php5 proxy_connect proxy_ftp proxy_http proxy rewrite speling ssl suexec unique_id userdir usertrack vhost_alias
    Module name? mime_magic
    Module mime_magic installed; run /etc/init.d/apache2 force-reload to enable.


That's it, your module will now be loaded next time you start apache. You can actually avoid restarting apache, by asking it to simply reload its configuration:

$ sudo /etc/init.d/apache2 reload

And here you go, your new added module is included in apache.

Tuesday, July 21, 2009

Active FTP vs. Passive FTP

Active FTP
  • FTP server's port 21 from anywhere (Client initiates connection)
  • FTP server's port 21 to ports > 1023 (Server responds to client's control port)
  • FTP server's port 20 to ports > 1023 (Server initiates data connection to client's data port)
  • FTP server's port 20 from ports > 1023 (Client sends ACKs to server's data port)

When drawn out, the connection appears as follows:

In step 1, the client's command port contacts the server's command port and sends the command PORT 1027. The server then sends an ACK back to the client's command port in step 2. In step 3 the server initiates a connection on its local data port to the data port the client specified earlier. Finally, the client sends an ACK back as shown in step 4.

The main problem with active mode FTP actually falls on the client side. The FTP client doesn't make the actual connection to the data port of the server--it simply tells the server what port it is listening on and the server connects back to the specified port on the client. From the client side firewall this appears to be an outside system initiating a connection to an internal client--something that is usually blocked.


Passive FTP
  • FTP server's port 21 from anywhere (Client initiates connection)
  • FTP server's port 21 to ports > 1023 (Server responds to client's control port)
  • FTP server's ports > 1023 from anywhere (Client initiates data connection to random port specified by server)
  • FTP server's ports > 1023 to remote ports > 1023 (Server sends ACKs (and data) to client's data port)

When drawn, a passive mode FTP connection looks like this:

In step 1, the client contacts the server on the command port and issues the PASV command. The server then replies in step 2 with PORT 2024, telling the client which port it is listening to for the data connection. In step 3 the client then initiates the data connection from its data port to the specified server data port. Finally, the server sends back an ACK in step 4 to the client's data port.

While passive mode FTP solves many of the problems from the client side, it opens up a whole range of problems on the server side. The biggest issue is the need to allow any remote connection to high numbered ports on the server. Fortunately, many FTP daemons, including the popular WU-FTPD allow the administrator to specify a range of ports which the FTP server will use. See Appendix 1 for more information.

The second issue involves supporting and troubleshooting clients which do (or do not) support passive mode. As an example, the command line FTP utility provided with Solaris does not support passive mode, necessitating a third-party FTP client, such as ncftp.

With the massive popularity of the World Wide Web, many people prefer to use their web browser as an FTP client. Most browsers only support passive mode when accessing ftp:// URLs. This can either be good or bad depending on what the servers and firewalls are configured to support.

Monday, July 20, 2009

The 15-Minute Backup Solution

Secure Network Backups in a Heterogeneous Environment (All Using Open Source Software!)

This setup was performed using Amanda 2.5.1p2. To learn how to set up:
    - the latest version of Amanda 2.6.0p2 with new configuration tools
    - the new Volume Shadow Copy Service (VSS) based Zmanda Windows Client Community Edition
Please register on Zmanda Network and read Setting-Up an Open Source Backup Software Amanda Community in About 15 Minutes white paper available in the Resources section.

Today’s businesses rarely run on just one operating system. Linux users and administrators often have strong preferences for one distribution over another; web designers might lean towards the Mac; legacy software and hardware can include various UNIX operating systems. Despite the complexity of modern business computing environments, a system administrator is expected to find a reliable backup solution.

Even in the case where users are expected to keep important files on networked resources, for true intellectual data security, desktop machines and laptops will also be backed up. The price of hard disk storage is continuously falling, bringing terabytes of storage within reach, and increasing the amount of data that can potentially be lost. (The amount of data that you have will always expand to fit the storage available; as the golden rule states.) We live in a global and e-commerce economy, where businesses run around the clock and crucial business data changes commensurately.

The Challenge

For our 15-minute challenge, you will backup two Linux systems (each running a different Linux distribution) and one Windows system, using freely downloadable open source software.

Our scenario is as follows:

The user "pavel" works with sensitive information. We need to make an encrypted backup of his home directory, /home/pavel, which resides on a Fedora Core Linux system called Iron. Our webmaster needs the webserver's document home backed up, the /var/www/html directory on a SUSE Enterprise Linux system called Copper. Our manager works solely on a Windows XP system called Uranium, and keeps all of his work in the MyDocuments folder, so we will need to add //Uranium/MyDocuments to our backup configuration.

The Solution: Amanda

Amanda is open source backup software that is flexible, secure and scalable to dynamic computing environments. Amanda can save you from expensive proprietary backup software and those custom backup scripts that have a propensity to break at the worst times. Dating back to 1991, Amanda has been used successfuly in environments from one standalone machine to hundreds of clients. Amanda is so thoroughly documented, from community wikis to published system administration texts, that it might be hard to discern just how easy an Amanda backup can be.

This article will show you how, in about 15 minutes, you can:

1. Install and configure the Amanda backup server.

2. Prepare three different clients for backup.

3. Set backup parameters.

4. Verify the configuration.

5. Verify the backup.


We will install and configure Amanda backup server software on Quartz, which is running Red Hat Enterprise Linux. We will install and configure Amanda backup client software on Copper and on Iron. The Windows XP client, Uranium, will be backed up with Amanda server software running in conjunction with Samba on the backup server, Quartz.

Client

Filesystem

OS

Compression

Encryption

Copper

/var/www/html

SLES9

Yes

No

Iron

/home/pavel

FC4

Yes

Yes

Uranium

//uranium/MyDocuments*

WINXP

Yes

No

* using Samba (i.e. without installing any software on the Windows system)

Open Source backup software

Amanda gives you the capability to use disk storage as backup media. Configuring, initiating and verifying a backup will complete the backup cycle, all in less than the time it takes for a pizza to be delivered!

Prerequisites

The basic Amanda setup consists of an Amanda server, the Amanda client or clients that are to be backed up, and the backup storage media such as a tape or hard disk device. An intermediate holding area for caching data is not absolutely necessary, but will improve performance significantly and is considered part of a basic setup.

Before we begin, please review the introduction to Amanda. Then, note the following prerequisites:

  • tar 1.15 or later and xinetd are installed on Quartz, Iron and Copper.

  • Quartz is able to send mail to the root user.

  • The systems are all on the same network and available.

  • You have root access, and root access through SSH is enabled and working.

  • The directories to be backed up exist.

  • The Amanda 2.5.1p2 backup_server RPM should be available on Quartz, and the backup_client RPM should be available on Iron and Copper. Amanda binary and source RPM packages and source tarballs are freely available from Zmanda.

  • Quartz, the backup server, is running Samba client software. Samba is also freely available open source software.

To support the encrypted backup of /home/pavel on Iron, the following packages should be installed and available on Iron:

Also note that this article assumes a fresh install of Amanda. If you have an existing Amanda installation, additional steps are needed to ensure the proper upgrade to the latest Amanda release, (2.5.1p2 and later).

TIP: You can copy and paste all of the examples here, making appropriate modifications for your environment.

Order Pizza

Call your favorite pizza delivery place, set your stopwatch and...

Install and Configure the Amanda Backup Server

1. Log in as root on Quartz, the Red Hat Enterprise Linux 4 server.

2. Install the Amanda 2.5.1p2 amanda-backup_server RPM. Installing the package also creates a user named amandabackup who belongs to the group disk.

[root@quartz server]# rpm -ivh amanda-backup_server-2.5.1p2-1.rhel4.i386.rpm

warning: amanda-backup_server-2.5.1p2-1.rhel4.i386.rpm: V3 DSA signature: NOKEY, key ID 3c5d1c92
Preparing... ########################################### [100%]
Jan 5 2007 12:12:55: Preparing to install: Amanda Community Edition - version 2.5.1p2
Jan 5 2007 12:12:55: Checking for 'amandabackup' user...
Jan 5 2007 12:12:55:
Jan 5 2007 12:12:55: The Amanda backup software is configured to operate as the
Jan 5 2007 12:12:55: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 12:12:55: been modified. To ensure that Amanda functions properly,
Jan 5 2007 12:12:56: please see that the following parameters are set for that
Jan 5 2007 12:12:56: user.:
Jan 5 2007 12:12:56:
Jan 5 2007 12:12:56: SHELL: /bin/sh
Jan 5 2007 12:12:56: HOME: /var/lib/amanda
Jan 5 2007 12:12:56: Default group: disk
Jan 5 2007 12:12:56:
Jan 5 2007 12:12:56: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 12:12:57:
Jan 5 2007 12:12:57: === Amanda backup server installation started. ===

1:amanda-backup_server ########################################### [100%]

Jan 5 2007 12:13:05: Updating system library cache...done.
Jan 5 2007 12:13:21: Installing '/etc/amandates'.
Jan 5 2007 12:13:21: The file '/etc/amandates' has been created.
Jan 5 2007 12:13:21: Ensuring correct permissions for '/etc/amandates'.
Jan 5 2007 12:13:21: '/etc/amandates' Installation successful.
Jan 5 2007 12:13:22: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 12:13:22: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 12:13:23: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 12:13:23: === Amanda backup server installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.

3. The Amanda services are started by the extended internet daemon, xinetd, which is why you must have xinetd installed on every Amanda server and client. In any text editor, create one xinetd startup file, /etc/xinetd.d/amandaserver , with content as follows.

For the /etc/xinetd.d/amandaserver file, on Quartz:

# default: on
#
# description: Amanda services for Amanda server and client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump amindexd amidxtaped
}

4. Restart xinetd on Quartz.

[root@quartz xinetd.d]# service xinetd reload
Reloading configuration: [ OK ]

5. Note the time. Only about five minutes should have passed!

Install and Configure Three Different Amanda Clients

Installation of Amanda Client RPM on Iron (FC4)

1. Log in as root on Iron, your Fedora Core 4 client.

2. Install the Amanda 2.5.1p2 backup_client RPM. Installing the package also creates a user named amandabackup who belongs to the group disk.

[root@iron client]# rpm -ivh amanda-backup_client-2.5.1p2-1.fc4.i386.rpm
warning: amanda-backup_client-2.5.1p2-1.fc4.i386.rpm: Header V3 DSA signature: NOKEY, key ID 3c5d1c92
Preparing... ########################################### [100%]
Jan 5 2007 10:17:16: Preparing to install: Amanda Community Edition - version 2.5.1p2
Jan 5 2007 10:17:16: Checking for 'amandabackup' user...
Jan 5 2007 10:17:16:
Jan 5 2007 10:17:16: The Amanda backup software is configured to operate as the
Jan 5 2007 10:17:17: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 10:17:17: been modified. To ensure that Amanda functions properly,
Jan 5 2007 10:17:17: please see that the following parameters are set for that
Jan 5 2007 10:17:17: user.:
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: SHELL: /bin/sh
Jan 5 2007 10:17:17: HOME: /var/lib/amanda
Jan 5 2007 10:17:17: Default group: disk
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: === Amanda backup client installation started. ===

1:amanda-backup_client ########################################### [100%]

Jan 5 2007 10:17:21: Updating system library cache...done.
Jan 5 2007 10:17:30: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 10:17:31: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 10:17:31: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 10:17:31: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 10:17:31: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 10:17:31: === Amanda backup client installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.

3. In any text editor, create an xinetd startup file, /etc/xinetd.d/amandaclient, with content as follows.

# default: on
#
# description: Amanda services for Amanda client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump
}

4. Restart xinetd on Iron.

[root@ironxinetd.d]# service xinetd reload
Reloading configuration: [ OK ]

5. Become the amandabackup user and append the line "quartz.zmanda.com amandabackup amdump" to the /var/lib/amanda/.amandahosts file on Iron. This allows Quartz, the Amanda backup server, to connect to Iron, the Amanda client.

Note that you should use fully qualified domain names when configuring Amanda.

-bash-3.00$ echo quartz.zmanda.com amandabackup amdump >> /var/lib/amanda/.amandahosts
-bash-3.00$ chmod 700 /var/lib/amanda/.amandahosts

6. Save the passphrase as a hidden file in the home directory of the amandabackup user. Protect the file with the proper permissions.

As the user amandabackup:

-sh-3.00$ chown amandabackup:disk ~amandabackup/.am_passphrase
-sh-3.00$ chmod 700 ~amandabackup/.am_passphrase

7. Create a script that enables encryption on the client Iron.

As root create a file /usr/sbin/amcryptsimple:

#!/usr/bin/perl -w
use Time::Local;
my $AMANDA='amandabackup';
$AMANDA_HOME = (getpwnam($AMANDA) )[7] || die "Cannot find $AMANDA home directory\n" ;
$AM_PASS = "$AMANDA_HOME/.am_passphrase";
$ENV{'PATH'} = '/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin';
$ENV{'GNUPGHOME'} = "$AMANDA_HOME/.gnupg";
sub encrypt() {

system "gpg --batch --disable-mdc --symmetric --cipher-algo AES256 --passphrase-fd 3 3<$AM_PASS";} sub decrypt() {
system "gpg --batch --quiet --no-mdc-warning --decrypt --passphrase-fd 3 3<$AM_PASS"; } if ( $#ARGV > 0 ) {
die "Usage: $0 [-d]\n";
}
if ( $#ARGV==0 && $ARGV[0] eq "-d" ) {

decrypt();
}
else {

encrypt();
}

7. Change the owership and the permissions on the file /usr/sbin/amcryptsimple you just created:

[root@iron sbin]# chown amandabackup:disk /usr/sbin/amcryptsimple
[root@iron sbin]# chmod 750 /usr/sbin/amcryptsimple

9. This completes configuration of the Amanda client on Iron.

Installation of Amanda Client RPM on Copper (SLES9)

1. Log in as the root user on Copper, your SUSE Linux Enterprise Server 9 client.

2. Install the Amanda 2.5.1p2 backup_client RPM. Installing the package also creates a user named amandabackup who belongs to the group disk.

copper:/ # rpm -ivh amanda-backup_client-2.5.1p2-1.sles9.i586.rpm
warning: amanda-backup_client-2.5.1p2-1.sles9.i586.rpm: V3 DSA signature: NOKEY, key ID 3c5d1c92

Preparing... ########################################### [100%]

Jan 5 2007 07:20:21: Preparing to install: Amanda Community Edition - version 2.5.1p2
Jan 5 2007 07:20:21: Checking for 'amandabackup' user...
Jan 5 2007 07:20:21:
Jan 5 2007 07:20:21: The Amanda backup software is configured to operate as the
Jan 5 2007 07:20:21: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 07:20:21: been modified. To ensure that Amanda functions properly,
Jan 5 2007 07:20:21: please see that the following parameters are set for that
Jan 5 2007 07:20:22: user.:
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: SHELL: /bin/sh
Jan 5 2007 07:20:22: HOME: /var/lib/amanda
Jan 5 2007 07:20:22: Default group: disk
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: === Amanda backup client installation started. ===

1:amanda-backup_client ########################################### [100%]

Jan 5 2007 07:20:26: Updating system library cache...done.
Jan 5 2007 07:20:26: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 07:20:27: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 07:20:27: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 07:20:27: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 07:20:27: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 07:20:27: === Amanda backup client installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.

3. In any text editor, create an xinetd startup file, /etc/xinetd.d/amandaclient, with content as follows.

# default: on
#
# description: Amanda services for Amanda client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump
}

5. Restart xinetd on Copper.

copper:/ # /etc/rc.d/xinetd restart
Reload INET services (xinetd). done

6. Become the amandabackup user and append the line "quartz.zmanda.com amandabackup amdump" to the /var/lib/amanda/.amandahosts file on Copper. This allows Quartz, the Amanda backup server, to connect to Copper, the Amanda client.
Note that you should use fully qualified domain names when configuring Amanda.

-bash-3.00$ echo quartz.zmanda.com amandabackup amdump >> /var/lib/amanda/.amandahosts
-bash-3.00$ chmod 700 /var/lib/amanda/.amandahosts

7. This completes configuration of the Amanda client on Copper. If you check your watch, you should find that only about ten minutes have passed!


Configurations Required to Backup Windows Client Uranium

· Configuration done on backup server Quartz:

1. The file /etc/amandapass must be created manually, owned by the amandabackup user and have permissions of 700. The amandapass file contains share name to user name, password and workgroup mapping.

As the root user:

[root@quartz /]# echo //uranium/MyDocuments zmanda%amanda Workgroup >> /etc/amandapass

2. Change the ownership and permissions on this file:

[root@quartz etc]# chown amandabackup:disk /etc/amandapass
[root@quartz etc]# chmod 700 /etc/amandapass

· Configuration done on Windows client Uranium:

The directory getting backed up must be shared from Windows and must be
accessible by the Windows user zmanda with the password amanda.

Set Backup Parameters

1. On Quartz, as the amandabackup user, create the Amanda configuration directory.

[root@quartz etc]# su - amandabackup
-bash-3.00$ mkdir /etc/amanda/DailySet1

2. Copy the /var/lib/amanda/example/amanda.conf file to the /etc/amanda/DailySet1 directory. The amanda.conf file is the most important file for configuring your Amanda setup.

-bash-3.00$ cp /var/lib/amanda/example/amanda.conf /etc/amanda/DailySet1

3. The sample amanda.conf distributed with Amanda is over 700 lines long and is extensively commented. For more information, search for amanda.conf on the Amanda wiki. We will focus on just a few lines and make minimal modifications.

Open /etc/amanda/DailySet1/amanda.conf with any text editor and edit it to suit your environment.

· The following lines control some details specific to your organization and to your tape configuration.

org "YourCompanyName" # your organization name for reports
mailto "root@localhost" # space separated list of operators at your site
tpchanger "chg-disk" # the tape-changer glue script
tapedev "file://space/vtapes/DailySet1/slots" # the no-rewind tape device to be used
tapetype HARDDISK # use hard disk intead of tapes (vtape config)

· We add the following lines to specify the size of the virtual tapes:

define tapetype HARDDISK {
length 100000 mbytes
}

· We add the following lines to support the encrypted backup of /home/pavel on Iron:

define dumptype encrypt-simple {
root-tar
comment "client simple symmetric encryption, dumped with tar"

encrypt client
compress fast
client_encrypt "/usr/sbin/amcryptsimple"
client_decrypt_option "-d"
}

. Go to the “define dumptype global” section in the amanda.conf file and add the “auth "bsdtcp"” line right before the last “}” bracket. This is done to enable “BSDTCP” authentication.

# index yes
# record no
# split_diskbuffer "/raid/amanda"
# fallback_splitsize 64m
auth "bsdtcp"

4. As the root user, create a cache directory to use as a holding disk.

[root@quartz ~]# mkdir -p /dumps/amanda
[root@quartz ~]# chown amandabackup:disk /dumps/amanda
[root@quartz ~]# chmod 750 /dumps/amanda

5. Create the virtual tapes. Dedicated directories are used as “virtual tapes” called vtapes. You work with vtapes in the same way that you work with physical tapes. Vtapes can even simulate tape changers, as you will see in our example.

For security reasons, limit access to the vtapes directory to the amandabackup user.

As the root user:

[root@quartz ~]# mkdir -p /space/vtapes
[root@quartz ~]# chown amandabackup:disk /space/vtapes
[root@quartz ~]# chmod 750 /space/vtapes

As the amandabackup user:

-bash-3.00$ touch /etc/amanda/DailySet1/tapelist
-bash-3.00$ mkdir -p /space/vtapes/DailySet1/slots
-bash-3.00$ cd /space/vtapes/DailySet1/slots
-bash-3.00$ for ((i=1; $i<=25; i++)); do mkdir slot$i;done -bash-3.00$ ln -s slot1 data

6. Test the virtual tape setup.

-bash-3.00$ ammt -f file:/space/vtapes/DailySet1/slots status
file:/space/vtapes/DailySet1/slots
status: ONLINE

7. Just as with physical tapes, the virtual tapes now need to be labeled. (Please note that the output below has been truncated.)

bash-3.00$ for ((i=1; $i<=9;i++)); do amlabel DailySet1 DailySet1-0$i slot $i; done changer: got exit: 0 str: 1 file://space/vtapes/DailySet1/slots labeling tape in slot 1 (file://space/vtapes/DailySet1/slots): rewinding, reading label, not an amanda tape (Read 0 bytes) rewinding, writing label DailySet1-01, checking label, done. ... changer: got exit: 0 str: 9 file://space/vtapes/DailySet1/slots labeling tape in slot 9 (file://space/vtapes/DailySet1/slots): rewinding, reading label, not an amanda tape (Read 0 bytes) rewinding, writing label DailySet1-09, checking label, done.

-bash-3.00$ for ((i=10; $i<=25;i++)); do amlabel DailySet1 DailySet1-$i slot $i; done changer: got exit: 0 str: 10 file://space/vtapes/DailySet1/slots labeling tape in slot 10 (file://space/vtapes/DailySet1/slots): rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-10, checking label, done.
...
changer: got exit: 0 str: 25 file://space/vtapes/DailySet1/slots
labeling tape in slot 25 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-25, checking label, done.

8. Now we need to reset the virtual tape changer back to the first slot.

-bash-3.00$ amtape DailySet1 reset
changer: got exit: 0 str: 1
amtape: changer is reset, slot 1 is loaded.

9. Create an /etc/amanda/DailySet1/disklist file in the Amanda configuration directory. The disklist contains the fully qualified backup client names, the directory or directories to be backed up and the dumptype.

copper.zmanda.com /var/www/html comp-user-tar
iron.zmanda.com /home/pavel encrypt-simple
quartz.zmanda.com //uranium/MyDocuments comp-user-tar

10. As the user amandabackup, append the following lines to the /var/lib/amanda/.amandahosts file to allow the backup clients to connect back to the server when doing restores. Specify fully qualified domain names.

iron.zmanda.com root amindexd amidxtaped
copper.zmanda.com root amindexd amidxtaped
quartz.zmanda.com root amindexd amidxtaped
quartz.zmanda.com amandabackup admump

11. Create a cron job that will execute amdump and initiate your backups automatically. As the amandabackup user, run crontab -e,and add the following line to run backups Monday through Friday at 1am.

0 1 * * 1-5 /usr/sbin/amdump DailySet1

Verify Your Configuration

1. On Quartz, as amandabackup, run the amcheck tool to verify that you can successfully perform a backup.

-bash-3.00$ amcheck DailySet1
Amanda Tape Server Host Check
-----------------------------
Holding disk /dumps/amanda: 16714488 KB disk space available, using 16612088 KB
slot 1: read label `DailySet1-01', date `X'
NOTE: skipping tape-writable test
Tape DailySet1-01 label ok
NOTE: conf info dir /etc/amanda/DailySet1/curinfo does not exist
NOTE: it will be created on the next run.
NOTE: index dir /etc/amanda/DailySet1/index does not exist
NOTE: it will be created on the next run.
Server check took 4.259 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 3 hosts checked in 27.097 seconds, 0 problems found
(brought to you by Amanda 2.5.1p2)


Run a Backup

1. On Quartz, as amandabackup, run amdump to start the DailySet1 backup.

-bash-3.00$ amdump DailySet1

2. Amanda will email a detailed status report from the amandabackup user to you, the root user on Quartz.

From amandabackup@quartz.zmanda.com Fri Jan 5 13:04:20 2007
Date: Fri, 5 Jan 2007 13:04:19 -0800
From: Amanda user
To: root@quartz.zmanda.com
Subject: YourCompanyName AMANDA MAIL REPORT FOR January 5, 2007

These dumps were to tape DailySet1-02.
The next tape Amanda expects to use is: a new tape.
The next new tape already labelled is: DailySet1-02.

STATISTICS:
Total Full Incr.
-------- -------- --------

Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 3.5 3.5 0.0
Original Size (meg) 11.8 11.8 0.0
Avg Compressed Size (%) 29.7 29.7 --
Filesystems Dumped 3 3 0
Avg Dump Rate (k/s) 292.8 292.8 --
Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 3.7 3.7 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 3 3 0
Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) 8509.1 8509.1 --

USAGE BY TAPE:

Label Time Size % Nb Nc
DailySet1-02 0:00 3744K 0.0 3 0

NOTES:
planner: Forcing full dump of copper.zmanda.com:/var/www/html as directed.
planner: Forcing full dump of iron.zmanda.com:/home/pavel as directed.
planner: Forcing full dump of quartz.zmanda.com://uranium/MyDocuments as directed.
taper: tape DailySet1-02 kb 3744 fm 3 [OK]

DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s
-------------------------- ------------------------------------- -------------
copper.zmand -r/www/html 0 7640 2336 30.6 0:03 910.6 0:00 8680.7
iron.zmanda. /home/pavel 0 3530 1024 29.0 0:07 149.1 0:00 12486.1
quartz.zmand -yDocuments 0 960 384 40.0 0:03 101.0 0:00 4295.3
(brought to you by Amanda version 2.5.1p2)

3. You can also run the tool amadmin with a find argument for a quick summary of what has been backed up.

-bash-3.00$ amadmin DailySet1 find
Scanning /dumps/amanda...

date host disk lv tape or file file part status
2007-01-05 13:04:03 copper.zmanda.com /var/www/html 0 DailySet1-02 2 -- OK
2007-01-05 13:04:03 iron.zmanda.com /home/pavel 0 DailySet1-02 3 -- OK
2007-01-05 13:04:03 quartz.zmanda.com //uranium/MyDocuments 0 DailySet1-02 1 -- OK

Success!

In just about 15 minutes, we installed and configured a secure, heterogeneous network backup, verified our configurations and ran a backup. We did it with freely downloadable open source software that you can install from binaries or compile for your unique needs. The pizza, which should be getting delivered right about now, will be that much more enjoyable with the clear conscience and peace of mind that comes with knowing that your data is secure.

Recovery

Based on feedback received on our forums we are adding a section that shows the ability to do a restore.

1. On Copper, as root, create the "/etc/amanda" directory.

copper:~ # mkdir /etc/amanda

copper:~ # chown amandabackup:disk /etc/amanda


2. As amandabackup, create a file "/etc/amanda/amanda-client.conf" and insert the lines below in to the file.

# amanda.conf - sample Amanda client configuration file.
#
# This file normally goes in /etc/amanda/amanda-client.conf.
#
conf "DailySet1" # your config name

index_server "quartz.zmanda.com" # your amindexd server

tape_server "quartz.zmanda.com" # your amidxtaped server

#tapedev "/dev/null" # your tape device
# auth - authentication scheme to use between server and client.
# Valid values are "bsd", "bsdudp", "bsdtcp" and "ssh".
# Default: [auth "bsdtcp"]

auth "bsdtcp"

# your ssh keys file if you use ssh auth

ssh_keys "/var/lib/amanda/.ssh/id_rsa_amrecover"


3. As root run "amrecover" to initiate the data recovery process.

copper:/etc/amanda # amrecover
AMRECOVER Version 2.5.1p2. Contacting server on quartz.zmanda.com ...
220 quartz AMANDA index server (2.5.1p2) ready.
Setting restore date to today (2007-01-08)
200 Working date set to 2007-01-08.
200 Config set to DailySet1.
501 Host copper is not in your disklist.
Trying host copper.zmanda.com ...
200 Dump host set to copper.zmanda.com.
Use the setdisk command to choose dump disk to recover
amrecover>


4. The list of commands below will demonstrate a recovery of a set of different files and directories to the "/tmp" directory.

amrecover> listdisk
200- List of disk for host copper.zmanda.com
201- /var/www/html
200 List of disk for host copper.zmanda.com
amrecover> setdisk /var/www/html
200 Disk set to /var/www/html.
amrecover> ls
2007-01-05-13-04-03 tar-1.15/
2007-01-05-13-04-03 .
amrecover> cd tar-1.15
/var/www/html/tar-1.15
amrecover> ls
2007-01-05-13-04-03 scripts/
2007-01-05-13-04-03 doc/
2007-01-05-13-04-03 configure
2007-01-05-13-04-03 config/
2007-01-05-13-04-03 COPYING
2007-01-05-13-04-03 AUTHORS
2007-01-05-13-04-03 ABOUT-NLS
amrecover> add scripts/
Added dir /tar-1.15/scripts/ at date 2007-01-05-13-04-03
amrecover> add configure
Added file /tar-1.15/configure
amrecover> add doc/
Added dir /tar-1.15/doc/ at date 2007-01-05-13-04-03
amrecover> lcd /tmp
amrecover> extract
Extracting files using tape drive chg-disk on host quartz.zmanda.com.
The following tapes are needed: DailySet1-02
Restoring files into directory /tmp
Continue [?/Y/n]? y
Extracting files using tape drive chg-disk on host quartz.zmanda.com.
Load tape DailySet1-02 now
Continue [?/Y/n/s/t]? y
./tar-1.15/doc/
./tar-1.15/scripts/
./tar-1.15/configure
./tar-1.15/doc/Makefile.am
./tar-1.15/doc/Makefile.in
./tar-1.15/doc/convtexi.pl
./tar-1.15/doc/fdl.texi
./tar-1.15/doc/freemanuals.texi
./tar-1.15/doc/getdate.texi
./tar-1.15/doc/header.texi
./tar-1.15/doc/stamp-vti
./tar-1.15/doc/tar.info
./tar-1.15/doc/tar.info-1
./tar-1.15/doc/tar.info-2
./tar-1.15/doc/tar.texi
./tar-1.15/doc/version.texi
./tar-1.15/scripts/Makefile.am
./tar-1.15/scripts/Makefile.in
./tar-1.15/scripts/backup-specs
./tar-1.15/scripts/backup.in
./tar-1.15/scripts/backup.sh.in
./tar-1.15/scripts/dump-remind.in
./tar-1.15/scripts/restore.in
amrecover> quit
200 Good bye.


5. We can now verify that the files have been recovered successfully by running run the following command.

copper:/ # tree /tmp/tar-1.15
/tmp/tar-1.15
|-- configure
|-- doc
| |-- Makefile.am
| |-- Makefile.in
| |-- convtexi.pl
| |-- fdl.texi
| |-- freemanuals.texi
| |-- getdate.texi
| |-- header.texi
| |-- stamp-vti
| |-- tar.info
| |-- tar.info-1
| |-- tar.info-2
| |-- tar.texi
| `-- version.texi
`-- scripts
|-- Makefile.am
|-- Makefile.in
|-- backup-specs
|-- backup.in
|-- backup.sh.in
|-- dump-remind.in
`-- restore.in


2 directories, 21 files

For more information about Amanda, please visit http://amanda.zmanda.com