Postfix Mail Queue Management

Like any other MTA , one of the important activity of Postfix is to handle Mail Queue. In Postfix architecture there and many process involve to handle mail delivery task . Cleanup is one of the postfix  process and it perform task of putting mail into Incoming Queue. 

Once the message enters into Incoming Queue , Queue Manager process become active and handle it . Queue manager process called as heart of Postfix mail system and it handle task of mail delivery . Queue manager process contact delivery agent . Delivery agents such as smtp,lmtp,local etc are those process who deliver mail to right address .

 Postfix manages following types of Mail Queue

Incoming Queue
Once the message enters the system , no matter whether it entered via smtpd or qmqpd or any other means the message will placed in the Incoming Queue. The Cleanup process is the Postfix process which put s missing headers before placing the message in the queue .

Active Queue
Once the message placed into Incoming Queue , the queue manager process picks the message from Incoming Queue and place that message into Active Queue .This movement depend upon the resource limit of the Active Queue.
The concept of active queue has been taken to keep queue light  for MTA processes. The Active Queue holds only those messages which are ready to be delivered . The most important point from performance  point of view is that the Active Queue is kept in memory not on disk.

 Postdrop
This mail queue holds the message which are sent via the local sendmail program . The mail recieved via smtpd or any other network interface never stored inside this queue.

Deferred 
From my personal experiences of Sendmail MTA which doesn't manage any separate queque for deferred mail , I can say keeping a separate queue for rejected mail is really a very good approach adopted by Postfix .  The messages which are failed in delivery by Postfix are stored inside this Queue .

Hold
We can set a policy to keep the mail on hold for time being and then releases it . Any message that enters into this queue , remains here unless released by Administrator. This queue is helpful in the content inspection and header checking .

Corrupt
Damaged queue files kept inside this queue .







 

MySQL MyISAM tables Recovery

Sorry Readers i was busy from last few days. Now I am here to discuss few MySQL related issues .
One of the common problem MySQL MyISAM tables faces are Crashing. Once MyISAM tables crash it take long time to recover. To improve recovery time following parameters can be used
myisam_sort_buffer_size and
myisam_max_sort_file_size
If above given parameters not set correctly then,  recovery done using key_cache which may take longer time.

The first we must know about myisam_sort_buffer_size  is that , this is a per session buffer. This buffer memory assigned per connection/thread. myisam_sort_buffer_size variable used when no index present in table to help the sorting . 

LINUX BOOTING SEQUENCE TROUBLE SHOOTINGS

Friends last few days were so busy days for most of my seniors.

Actually one of our client has deleted (accidently) some important configuration files from server. We did lot’s of troubleshooting to make server up.

On those trouble shootings I got opportunity to learn some precious things. For me it was like practical review of LINUX BOOTING SEQUENCE TROUBLE SHOOTINGS.

Let me share some boot troubleshooting topics.

Linux boot process

a) Bios initialization

b) Boot loader

c) Kernel initialization

d) init

a) BIOS is the interface between hardware & software in very basic level. It runs POST, and then it looks for peripherals & a device to boot from. at the end of POST , a boot device is selected from list of detected boot device.(it may be floopy,hddcd-rom,NIC)

b) BIOS reads & executes the 1st physical sector of chosen boot media on system. Usually this is contained in the 1st 446 byte of hard disk.(in case of GRUB)

1) So BIOS 1st pass control to IPL within MBR.(This stage is called 1st stage of booting of Linux)

Note: - If in system more than one OS is installed then boot loader must be configured to pass control to other desired OS.

2) If 1st stage of booting pass control to Linux then it seeks /boot partition & by finding grob.conf,initrd & other files it completed 2nd stage of booting.

c) Kernel initialization: - lots of process generated for compiling device driver & to attempt to locate their corresponding drivers.

You can see /var/log/dmesg that contains snapshoot of kernel messages taken just after control pass to init.

Note: - if essential (need to boot) drivers have been compiled as modules instead of into the kernel, and then they must be include in initrd.img. This is then temporarily mounted by kernel on RAM disk to make module available for initialization process.

After loading of all essential drivers, kernel mounts root file system (/) read-only & pass the control to 1st process (init).

d) INIT:-init reads /etc/inittab (because now / is read-only) & execute all that written in this file as follows

1) Selection of desired run level

2) Execute /etc/rc.d/rc.sysinit

A) Activate udev & selinux

B) Set kernel parameter from /etc/sysctl.conf

C) Set system clock

D) Enable swap partition

E) Set hostname

F) Checking root file system & remount (in read write mode)

G) Activate RAID & LVM

H) Enable disk Quota

I) Check & mount other file system read write mode

J) Clean up stale lock & PID files

3) Runs /etc/rc.d/rc?.d/

(/etc/rc.d/init.d has soft link for corresponding run level)

4) It runs /etc/rc.d/rc.local

5) It starts xinetd service.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

This is the complete booting sequence of Linux. While troubleshooting Linux booting problem it helps to find actual cause for booting failure

Linux booting troubleshooting is as follows

LINUX BOOTING SEQUENCE TROUBLESHOOTING

Case 1: No boot loader splash screen or prompt appears

Cause:

1) Grub.conf miss configures

2) Initrd misplaced or deleted

3) MBR cruuppet

4) /boot partition miss

Grub.conf miss configures

Remedy: try to pass boot location initrd location & kernel module location as follows

Step a) Grub > root (hd0,0)

Note: here hd0,0 means boot partition is in 1st partition of 1st hdd.(please use hd for SATA & SCSI hdd also )

Step b) grub > kernel /vmlinuz-$(uname -r) root=LABEL=/ rhgb quiet

Step c) grub > initrd /initrd-$(uname -r).img

Step d) grub > boot

Then after rebooting try to recreate grub.conf file

Initrd misplaced or deleted

Case a) system is up

This is the most fortunate situation for system admins who are managing the server remotely because now once the system will down it will not be up without rescue mode

Remedy:

[root@dh-localhost ~]# mkinitrd /boot/ initrd-$(uname -r).img $(uname -r)

Case b) system is down

Then boot from system via DVD & start it by rescue mode

Remedy:

Step 1) Boot: linux recue

After some process & ittrective prompt you will be on sh prompt

Step 2) chroot /mnt/sysimage

cd /boot

mkinitrd /boot/ initrd-$(uname -r).img $(uname -r)

Then reboot the system by HDD

Note: - in case of fstab is also misconfigured (ie / & other partition will not mount after boot) then chroot command will not work. So in this case your 1st step would be to correct fstab then perform above steps as follows

Step a) boot from DVD or other bootable media

Step b) chroot /mnt/sysimage (you will find chroot error here )

Step c) mkdir /test

e2label /dev/sda1 (if it will show boot them mount it otherwise try to search by e2label /dev/sda2 ownwards)

mount /dev/sda1 /test

cd /test

cd grub

vi grub.conf (please make it correct)

mkdir /data

mount /dev/sda2 /data (mounting / partition on /data )

cd /data

cd /etc

vi fstab (please correct it )

Then reboot the system & follow previous step

Mbr corrupt

Case a) system is up

Method 1

[root@dh-localhost ~]# /sbin/grub-install /dev/sda

Method 2

[root@dh-localhost ~]# grub

grub> root (hd0,0)

grub > setup (hd0)

grub > quit

Case b) system is down

Boot system by rescue mode

Sh #Chroot /mnt/sysimage

Sh # /sbin/grub-install /dev/sda

Note : A smart system admin always take backup of MBR as follows

[root@dh-localhost ~]# dd if=/home/mbr of=/home/mbr bs=1 count 500

&

For restoration for MBR

[root@dh-localhost ~]# dd if=/home/mbr of=/home/mbr bs=1 count 500

/boot partition miss

In this case if you have backup of /boot then you can recover it otherwise you need to rebuild the server again.

Redirect http to https

Suppose you have a requirement to divert your http traffic to https. There are many ways to achieve this , but as a Apache web administrator i prefer to apply following Rewrite rule for my site

RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://abc.com/$1 [R,L]

Here in above example example site is abc.com

Caching Configuration in Apache

The goad of caching in Apache could be

-Reduce Number of request in man cases
-Eliminate need to send full response in many cases

For former we use Expiration mechanism and for later we use validation mechanism.

For Apache mod_expires and mod_headers handles cache control through http headers.
mod_expires module control setting of expires and cache-control http header in server response.

For example to enable mod_expires for all .html files, we can use following syntax

     ExpiresActive On
     ExpiresDefault A300
   
     Expires A86400
     

In Above example, default caching is for 5 minutes(A300) but for html files caching is 1Day(A86400).

Now suppose we want to disable caching for dynamic pages located in folder /var/www/phpfiles

  Header Set Cache-Control "max-age=0,no-store"

Prevent DDOS attack on web site

There can a number of idea to prevent Distributed Denial of Service attack on website . Here i am sharing a basic idea that i use now a days to mitigate DDOS from malicious web crawler .As I noticed when a malicious crawler access web site we found  log entry  by a  particular User Agent in access-log. What i do is rewrite request from such User Agent to Forbidden messages using Rewrite module 
RewriteCond  %{HTTP_USER_AGENT} Malicious_User_Agent [NC]  
RewriteRule .*  - [F,L]

http traffic load

On some occasion , we need to calculate http traffic of our system. Although there are a number of tools and utilities but i avoid to install and use them in small size set-up.
 I decided to use tcpdump command ,  tcpdump command is very useful command that help to decode network traffic. For example for above given requirement i applied tcpdump command in following format.
tcpdump -s 1500 -Svni eth0 tcp and port 80
Above given command will display http packets (tcp packet of port number 80). In output of command we can get total length which is equal to IP Header Length + TCP Header Length + Payload Length . As we know IP Header + TCP Header length will be 52 , so we can calculate payload length which will be Total Length -52.

    

Hack Attempt

Today i noticed ../../../../../../../../. entries in my access log file of one of my web site.
I wonder is it a hack attempt ? If some hacker trying to doing scanning to find vulnerability of my system?
  Yes my doubt was genuine , It was a hack attempt. It was Local File Intrusion vulnerability. The attempt was to check if /proc/self/environ is accessible, if anyway it access that then do attempt to inject code and get access to shell.


Still i am working to way to mitigate such attack, immediate solution for me was to block ip which was doing such attempt.

log-rotate in Linux

Today my client claimed to reset the log rotation of access_log to 1 day and 1 log of all his 532 websites hosted in a server .

Also he claimed that all rotated log will be mailed to him .

Then I did :->

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Step 1:-> create a file

# vi logger.conf

/var/www/vhosts/*/statistics/logs/access_log

{daily

rotate 1

mail amit@abc.com

compress

}

:wq!

Then I simply type

Step2:-> logrotate logger.conf

++++++++++++++++++++++++++++++++++++++++++++

There are other options here :->

compress

This is used to compress the rotated log file with gzip.

nocompress

This is used when you do not want to compress rotated log files.

copytruncate

This is used when processes are still writing information to open log files. This option copies the active log file to a backup and truncates the active log file.

nocopytruncate

This copies the log files to backup, but the open log file is not truncated.

create mode owner group

This rotates the log file and creates a new log file with the specified permissions, owner, and group. The default is to use the same mode, owner, and group as the original file.

nocreate

This prevents the creation of a new log file.

delaycompress

When used with the compress option, the rotated log file is not compressed until the next time it is cycled.

nodelaycompress

This overrides delaycompress. The log file is compressed when it is cycled.

errors address

This mails logrotate errors to an address.

ifempty

With this, the log file is rotated even if it is empty. This is the default forlogrotate.

notifempty

This does not rotate the log file if it is empty.

mail address

This mails log files that are cycled to an address. When mail log files are cycled, they are effectively removed from the system.

nomail

When mail log files are cycled, a copy is not mailed.

olddir directory

With this, cycled log files are kept in the specified directory. This directory must be on the same filesystem as the current log files.

noolddir

Cycled log files are kept in the same directory as the current log files.

prerotate/endscript

These are statements that enclose commands to be executed prior to a log file being rotated. The prerotate and endscript keywords must appear on a line by themselves.

postrotate/endscript

These are statements that enclose commands to be executed after a log file has been rotated. The postrotate and endscript keywords must appear on a line by themselves.

daily

This is used to rotate log files daily.

weekly

This is used to rotate log files weekly.

monthly

This is used to rotate log files monthly.

rotate count

This specifies the number of times to rotate a file before it is deleted. A count of 0 (zero) means no copies are retained. A count of 5 means five copies are retained.

tabootext [+] list

This directs logrotate to not rotate files with the specified extension. The default list of extensions is .rpm-orig, .rpmsave, v, and ~.

size size

With this, the log file is rotated when the specified size is reached. Size may be specified in bytes (default), kilobytes (sizek), or megabytes (sizem)

For more detail you can read man page of logrotate .