Tuesday, February 8, 2011

Linux Process - Tips


What is a Process ?
When a program is read from disk into memory and its execution begins, the currently executing image is called a process

PID
The process ID is the number between 1 - 32767 by default (Certainly customizable). To set the limit, as root run the following,You can set the value higher (up to 2^22 on 32-bit machines: 4,194,304)
with:
# echo 4194303 > /proc/sys/kernel/pid_max
you can instead add a line to your /etc/sysctl.conf. You would do this instead of the above  commands. This will be the more natural solution for such systems, but you'll need to reboot the  system or use the sysctl program for it to take effect. You need to append the following to your  /etc/sysctl.conf:
#Allow for more PIDs (to reduce rollover problems); may break some programs
kernel.pid_max = 4194303
PPID
Each process in linux has a parent. Once the system starts a single process is created, called INIT,  whose PID is 1. The INIT process then begins to start the system up, creating processes as needed. These newly created processes may start other processes but the ultimate parent is always INIT.

PS command

"ps" command with no option shows: 
  • Process ID (PID)
  • The terminal (TTY)
  • The amount of CPU time that the process has accumulated (TIME)
  • The command used (CMD)

"ps -f" give more info(full option). It displays the below options in addition to above
  • The Parent PID (PPID)
  • The process start time (STIME)
  • The user ID (UID)
Process ither than your own cab be also checked with "ps" command using -e option. This displays all the process in the system
Also:
  • -u to ps command limit display to users
  • -g limits display to groups
  • -p limits display to PID
  • -t limit display to terminal

Managing process In Linux
Usually 2 ways used to manage the process
1. Using a signaling system - (sending signals to process using commands kill,skill and pkill)
     Signals are the software interupts used to communicate status and information amongst processes. The TERM signal can be caught or ignored. The KILL signal "9" is not able to be caught or ignored,  and causes immediate termination of the process. Ctrl C sends the INT (2)(Interrupt) signal to the  process- this is the reason of Process termination. "Ctrl \" sends quit (3) signal to running process TERM (Terminate)signal (15) is the default signal send to the process while running the kill  command.
HUP signal is generated by a modem hangup. It often tell a daemon to reconfigure (restart) itself. 
kill -1 (kills the shell and logs out).


2. Using /proc interface
     Much of the processess information is available to a user through a special interface known as  /proc file system. Every process running in Linux system has an correspondence directory in proc file system. The /proc file system does not exist on disk. It is an interface to the running system and  present kernel. It gives kernel and process information in an easy to access manner. Every process  that runs on a Linux system has a corresponding directory in /proc named with the PID of the process.

Managing process using /proc:
There is wealth of information about a running proces in its /proc entry. Most of this information is meant for use by programs like ps, So we need to do some pre-processing before we can view it. You can use "tr" command to do it. By translating ASCII NUL characters to LF (Line feed) characters, we  can get a meaningful display.
Eg:-
# tr '\0' '\n' < /proc/1223/environ
This example shows the details about the process 1233.

"environ" is the file which contains the environment details of the process.
"cwd" folder shows the current working directory
"fd" contains the links to every file that a process may have opened. This directory called fd (File  Descriptor). File Descriptor is a number used by a program to identify an open file. Each process in /proc file system will have a "fd". This is a vital information for a system administrator trying to  manage a large and complex system. For instance, a file system may not be unmounted if any process  has a file opened in that file system. By checking /proc, and administrator can determine and  resolve the problem

Eg:
# umount /home
Umount: /home: device us busy
#ls -al /proc/*/fd | grep home
This will show what all process opened the file /home
Killing a Job
# kill -9 %1
The % should be added before the job number. This will make shell to replace job number with the process ID

Log Files, Errors and Status

Syslog facilities:
Authpriv
Corn
Daemon
Kern
lpr
Mail
News
Uucp

User
Local0 - Local7


Syslog Priority:
emerg -  Emergency condition, such as an imminent system crash, usually broadcast to all users
alert -    Condition that should be corrected immediately, such as a corrupted system database
crit -     Critical condition, such as a hardware error
err -      Ordinary error
warning - Warning
notice -  Condition that is not an error, but possibly should be handled in a special way
info -    Informational message
debug -  Messages that are used when debugging programs
none -   Do not send messages from the indicated facility to the selected file. For example, specifying
*.debug;mail.none sends all messages except mail messages to the selected file.

Note:-
Logrotate keeps 4 weeks of logs before the oldest log is rottated out or deleted. Syslog entries all  share a common format. The entry starts with the date and time, followd by the name of the system  which logged message.

CORE Error handling:
When unexpected errors occur, the system may create a core file. A core file contains a copy of the  memory image of the process at the time that the error occurred. It is named "core" because the mail  system emory was originally called core memory, as it was made up of ferrite donuts that were wired  together through their holes, or cores.

A core file can be used to autospy a dead pricess. Even if you are not a programmer, and do not have  the access to core analysis tools, core files can still be used to find information that may help  you to identify the cause if the program's death.

The first thing to do with a "core" file is use the "file" command to determine what program caused  the core and what (if any) signal initiated the dumping of core. Core files are normally called core or  core.xxxx where "xxxx" is the PID of the process before it died. Using "man 7 signal" will bring up a  list of signals. By this mean we can determine the issue and also the author can be notified if there is  any kind og bugs (If any Invalid memory reference error occurs).

strings Command:
     Strings program displays printable strings from a binary file. Using strings on a core file, you can  display all of the strings included in the core image. At the end of the core file will be the process  environment. This includes the command used to start program.
This information can give vital clues to the case of death. Looking through the core file for pathnames  can also give information about the configuration files and shared libraries required to run the program.

Customizing the Shell
     In Bash, there are 4 prompt strings used. All of them are able to customize. These strings are  represented by the environment variables PS1, PS2, PS3 and PS4. The normal command prompt,  which is displayed to indicate the shell is ready for a new command, is found in the PS1 variable. Should a command require more than a single line of input, the secondary prompt string PS2 is  displayed. This can be seen when typing in flow control statements interactvely.
The select statement uses PS3 to display the prompt for the generated menu. The default is "#?"
Finally, the PS4 prompt is used when debugging shell scripts. The shell allows an exection trace,  showing each command as it is executed. This is enabled by using -x option to the shell, or using "set  -x" at the start of the script.


PS3 and PS4 can be set to any text. The text is displayed with no change. There is no way to place  variable text within these strings. However, PS1 and PS2 can have test that is evaluated each time the  prompt is displayed. This can be done with the $(command) syntax, or with a special set of  characters used specifically for the purpose.

Some notes about Linux File System
     The structure of a file system determines its use and the manner in which commands and utilities  interact with it. This is especially true of management commands that change or effect the file system. Beacause of this we need to explore the structure of a Linux file system before we can look at the file  system management commands.

All Linux filesystem have a similar logical structure as far as the user or system commands are  concerned. This is achieved by the file system driver logic in the Linux Kernel, regardless of the  underlying data layout. "A file system usually consists of a master information table called the  superblock, a list of file summary information blocks, called inodes, and the data blocks assosiated  with your data.

Every filesystem has its own root directory, which is always identified by inode number 2. This is the  first usable inode in a Linux filesystem. This directory is special, in that it can be used to attach the  filesystem to the main, or root filesystem. The directory on the root or parent filesystem at which the  new filesystem is attached is called the mount point.

/dev files:
     A /dev entry looks like any other file except that it does not have a size. Instead it has a major and  minor device number, and a block or character designation. The major number identifies which  device driver is being used. There are two kind of device drivers: Block and Character, each with  their own set of major numbers.

The minor number identifies the sub-device or operation for the device driver. For example, a tape  drive may have different minor numbers for operation in compressed and uncompressed mode. There are a number of general-purpose devices as well. The /dev/bull file is also known as the "bit bucket"  because it will take anything that is written to it and discard it. It is often used to discard unwanted  error messages or to test commands. A similar file is /dev/zero which does the same for writes, but  when read will return as many NUL (Hex 00) characters as you ask to read. This is often used to  create zero-filled files for testing or for database initialization.

lost+found Directory:
     Every file system requires a directory called lost+found in the root directory of the filesystem. This is used by the system when checking and rebuilding a corrupted file system. Files that have  inodes, but no directory entry, are moved to the lost+found directory. If there are files in this  directory, they will be named with the inode number, as an indication that the file system has suffered  some damage.

Saturday, January 15, 2011

Install Packages Via yum Command Using DVD / CD as Repo - CentOS (RHEL Based)



CentOS Linux comes with CentOS-Media.repo which is used to mount the default locations for a CDROM / DVD on CentOS-5.*. You can use this repo and yum to install items directly off the DVD ISO that we release.
Open /etc/yum.repos.d/CentOS-Media.repo file, enter:
# vi /etc/yum.repos.d/CentOS-Media.repo
Make sure enabled is set to 1:
enabled=1
Save and close the file. To use repo put your DVD and along with the other repos, enter:
# yum --enablerepo=c5-media install pacakge-name
To only use the DVDmedia repo, do this:
# yum --disablerepo=\* --enablerepo=c5-media install pacakge-name
OR use groupinstall command
# yum --disablerepo=\* --enablerepo=c5-media groupinstall 'Virtualization'

Sunday, December 26, 2010

Installing and configuring mod_security-Ubuntu 9.04


This how-to is reported to work in Ubuntu 8.04-10.10 as well.

What is mod_security you ask ?


Mod Security can significantly increase the security of your Apache installation.
 
What Is ModSecurity?

ModSecurity is a web application firewall that can work either embedded or as a reverse proxy. It provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis.

It is also an open source project that aims to make the web application firewall technology available to everyone.

Do not think you need this ? Follow along with the examples and decide for yourself (This tutorial assumes you already have Apache and php5 installed).
First, let us look at the default Apache behavior. I will use “ubuntuVPS” as the server of interest.

“Insecure” Example 1 – curl

Use curl to obtain information on the server (bodhi@home is a remote machine connecting to “ubutnuVPS”. You can test all this with any browser if you wish, simply use your server’s home page).
bodhi@home# curl -i ubuntuVPS
HTTP/1.1 200 OK
Date: Tue, 28 Apr 2009 22:06:21 GMT
Server: Apache/2.2.11 (Ubuntu) PHP/5.2.6-3ubuntu4.1 with Suhosin-Patch
Last-Modified: Tue, 28 Apr 2009 21:39:54 GMT
ETag: "50d4a-2d-468a44dadbe80"
Accept-Ranges: bytes
Content-Length: 45
Vary: Accept-Encoding
Content-Type: text/html
< html>< body>< h1>It works!< /h1>< /body>< /html>

Looks like this in your browser (the famous It works! page)

See how with a single command we already know the server is Ubuntu running Apache 2.2.11 and PHP 5.2.6 ?

“Insecure” Example 2 – bad .php

For this I will ask you to create a file “/var/www/insecure.php”
Put the following code in the file :
# vim /var/www/insecure.php
< ? $secret_file = $_GET['secret_file'];
include ( $secret_file); ? >;

Note: I had to put a space at the front of the php tag “<; ?”, remove it.

Now what ? Open a browser and enter http://ubuntuVPS/insecure.php?secret_file=/etc/passwd

I shall use curl in this example:
bodhi@home# curl -i "http://ubuntuVPS/insecure.php?secret_file=/etc/passwd"
HTTP/1.1 200 OK
Date: Tue, 28 Apr 2009 22:24:11 GMT
Server: Apache/2.2.11 (Ubuntu) PHP/5.2.6-3ubuntu4.1 with Suhosin-Patch
X-Powered-By: PHP/5.2.6-3ubuntu4.1
Vary: Accept-Encoding
Content-Length: 860
Content-Type: text/html
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
lp:x:7:7:lp:/var/spool/lpd:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
uucp:x:10:10:uucp:/var/spool/uucp:/bin/sh
proxy:x:13:13:proxy:/bin:/bin/sh
www-data:x:33:33:www-data:/var/www:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/sh
list:x:38:38:Mailing List Manager:/var/list:/bin/sh
irc:x:39:39:ircd:/var/run/ircd:/bin/sh
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/bin/sh
nobody:x:65534:65534:nobody:/nonexistent:/bin/sh
libuuid:x:100:101::/var/lib/libuuid:/bin/sh
sshd:x:101:65534::/var/run/sshd:/usr/sbin/nologin
postfix:x:104:107::/var/spool/postfix:/bin/false

YIKES !!!

Install and configure mod_secure

There was a time when installing mod_security was a bit difficult, now it is as easy as :
sudo apt-get -y install libapache-mod-security
The “hard part” is that we need to configure mod_security and obtain a few rules.

Configure mod_security

Using any editor, make a file “/etc/apache2/conf.d/modsecurity2.conf” and put the following contents in the file.
#vim /etc/apache2/conf.d/modsecurity2.conf
< ifmodule mod_security2.c>
Include conf.d/modsecurity/*.conf
< /ifmodule>

Note: I had to add a space at the front of the tag “< ifmodule mod_security2.c>” and “< /ifmodule>”, remove them.

By default, mod_security logs to /etc/apache2/logs, the following commands will put the log in /var/log/apache2/mod_security and create a symbolic link back to /etc/apache2/logs

sudo mkdir /var/log/apache2/mod_security
sudo ln -s /var/log/apache2/mod_security/ /etc/apache2/logs
Download and install rules
Download rules from here

As of this writing, the rule set was “modsecurity-core-rules_2.5-1.6.1.tar.gz”, you may need to adjust accordingly as new rules are released.
sudo mkdir /etc/apache2/conf.d/modsecurity
cd /etc/apache2/conf.d/modsecurity
sudo wget http://www.modsecurity.org/download/modsecurity-core-rules_2.5-1.6.1.tar.gz
sudo tar xzvf modsecurity-core-rules_2.5-1.6.1.tar.gz
sudo rm CHANGELOG LICENSE README modsecurity-core-rules_2.5-1.6.1.tar.gz


Enable mod_security:

sudo a2enmod mod-security
Now restart Apache
That’s it :)

Testing mod_security

“Secure” Example 1 – curl
bodhi@home# curl -i http://ubuntuVPS
HTTP/1.1 200 OK
Date: Tue, 28 Apr 2009 22:44:42 GMT
Server: Apache/2.2.0 (Fedora)
Last-Modified: Tue, 28 Apr 2009 21:39:54 GMT
ETag: "50d4a-2d-468a44dadbe80"
Accept-Ranges: bytes
Content-Length: 45
Vary: Accept-Encoding
Content-Type: text/html
< html>< body>< h1>It works!< /h1>< /body>< /html>

Look no more server or php information (Fedora apache 2.2.0 , LOL !!! )

“Secure” Example 2 – bad .php
bodhi@home# curl -i "http://ubuntuVPS/insecure.php?secret_file=/etc/passwd"
HTTP/1.1 501 Method Not Implemented
Date: Tue, 28 Apr 2009 22:47:38 GMT
Server: Apache/2.2.0 (Fedora)
Allow: TRACE
Vary: Accept-Encoding
Content-Length: 291
Connection: close
Content-Type: text/html; charset=iso-8859-1
< !DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
< html>< head>
< title>501 Method Not Implemented< /title>
< /head>< body>
< h1>Method Not Implemented< /h1>
< p>GET to /insecure.php not supported.< br />
< /p>
< hr>
< address>Apache/2.2.0 (Fedora) Server at ubuntuvps Port 80< /address>
< /body>< /html>

Looks like this in your browser:
"501 Method Not Implemented
Method Not Implemented"
GET to /insecure.php not supported.Apache/2.2.0 (Fedora) Server at ubuntuvps Port 80
Ah 501 Error looks much better then the contents of /etc/passwd :)
Where to go from here ?

1. Monitor your logs :
tail /var/log/apache2/mod_security/modsec_audit.log
 
2. Learn / edit your mod_security rules : ModSecurity Reference Manual

3. Delete bad.php, LOL
sudo rm -rf /var/www/insecure.php

I hope you enjoyed and learned from this tutorial :)

Reference:
"This is just a copy cat of the post from http://blog.bodhizazen.net/linux/how-to-mod_security-ubuntu-904/
All credit should go to the respective author. I tried the method in Ubuntu 10.10 and it works fine."

Note:-
Some of the rules may deny the access to you applications (eg: phpmyadmin/drupal etc). Test the rules well before you implement.