Postfix Mail Queue Management

Like any other MTA , one of the important activity of Postfix is to handle Mail Queue. In Postfix architecture there and many process involve to handle mail delivery task . Cleanup is one of the postfix  process and it perform task of putting mail into Incoming Queue. 

Once the message enters into Incoming Queue , Queue Manager process become active and handle it . Queue manager process called as heart of Postfix mail system and it handle task of mail delivery . Queue manager process contact delivery agent . Delivery agents such as smtp,lmtp,local etc are those process who deliver mail to right address .

 Postfix manages following types of Mail Queue

Incoming Queue

Once the message enters the system , no matter whether it entered via smtpd or qmqpd or any other means the message will placed in the Incoming Queue. The Cleanup process is the Postfix process which put s missing headers before placing the message in the queue .

Active Queue

Once the message placed into Incoming Queue , the queue manager process picks the message from Incoming Queue and place that message into Active Queue .This movement depend upon the resource limit of the Active Queue.

The concept of active queue has been taken to keep queue light  for MTA processes. The Active Queue holds only those messages which are ready to be delivered . The most important point from performance  point of view is that the Active Queue is kept in memory not on disk.

 Postdrop

This mail queue holds the message which are sent via the local sendmail program . The mail recieved via smtpd or any other network interface never stored inside this queue.

Deferred 

From my personal experiences of Sendmail MTA which doesn't manage any separate queque for deferred mail , I can say keeping a separate queue for rejected mail is really a very good approach adopted by Postfix .  The messages which are failed in delivery by Postfix are stored inside this Queue .

Hold

We can set a policy to keep the mail on hold for time being and then releases it . Any message that enters into this queue , remains here unless released by Administrator. This queue is helpful in the content inspection and header checking .

Corrupt
Damaged queue files kept inside this queue .







 

MySQL MyISAM tables Recovery

Sorry Readers i was busy from last few days. Now I am here to discuss few MySQL related issues .

One of the common problem MySQL MyISAM tables faces are Crashing. Once MyISAM tables crash it take long time to recover. To improve recovery time following parameters can be used

myisam_sort_buffer_size and
myisam_max_sort_file_size
If above given parameters not set correctly then,  recovery done using key_cache which may take longer time.

The first we must know about myisam_sort_buffer_size  is that , this is a per session buffer. This buffer memory assigned per connection/thread. myisam_sort_buffer_size variable used when no index present in table to help the sorting . 

LINUX BOOTING SEQUENCE TROUBLE SHOOTINGS

Friends last few days were so busy days for most of my seniors.

Actually one of our client has deleted (accidently) some important configuration files from server. We did lot’s of troubleshooting to make server up.

On those trouble shootings I got opportunity to learn some precious things. For me it was like practical review of LINUX BOOTING SEQUENCE TROUBLE SHOOTINGS.

Let me share some boot troubleshooting topics.

Linux boot process

a) Bios initialization

b) Boot loader

c) Kernel initialization

d) init

a) BIOS is the interface between hardware & software in very basic level. It runs POST, and then it looks for peripherals & a device to boot from. at the end of POST , a boot device is selected from list of detected boot device.(it may be floopy,hddcd-rom,NIC)

b) BIOS reads & executes the 1^st physical sector of chosen boot media on system. Usually this is contained in the 1^st 446 byte of hard disk.(in case of GRUB)

1) So BIOS 1^st pass control to IPL within MBR.(This stage is called 1^st stage of booting of Linux)

Note: - If in system more than one OS is installed then boot loader must be configured to pass control to other desired OS.

2) If 1^st stage of booting pass control to Linux then it seeks /boot partition & by finding grob.conf,initrd & other files it completed 2^nd stage of booting.

c) Kernel initialization: - lots of process generated for compiling device driver & to attempt to locate their corresponding drivers.

You can see /var/log/dmesg that contains snapshoot of kernel messages taken just after control pass to init.

Note: - if essential (need to boot) drivers have been compiled as modules instead of into the kernel, and then they must be include in initrd.img. This is then temporarily mounted by kernel on RAM disk to make module available for initialization process.

After loading of all essential drivers, kernel mounts root file system (/) read-only & pass the control to 1^st process (init).

d) INIT:-init reads /etc/inittab (because now / is read-only) & execute all that written in this file as follows

1) Selection of desired run level

2) Execute /etc/rc.d/rc.sysinit

A) Activate udev & selinux

B) Set kernel parameter from /etc/sysctl.conf

C) Set system clock

D) Enable swap partition

E) Set hostname

F) Checking root file system & remount (in read write mode)

G) Activate RAID & LVM

H) Enable disk Quota

I) Check & mount other file system read write mode

J) Clean up stale lock & PID files

3) Runs /etc/rc.d/rc?.d/

(/etc/rc.d/init.d has soft link for corresponding run level)

4) It runs /etc/rc.d/rc.local

5) It starts xinetd service.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

This is the complete booting sequence of Linux. While troubleshooting Linux booting problem it helps to find actual cause for booting failure

Linux booting troubleshooting is as follows

LINUX BOOTING SEQUENCE TROUBLESHOOTING

Case 1: No boot loader splash screen or prompt appears

Cause:

1) Grub.conf miss configures

2) Initrd misplaced or deleted

3) MBR cruuppet

4) /boot partition miss

Grub.conf miss configures

Remedy: try to pass boot location initrd location & kernel module location as follows

Step a) Grub > root (hd0,0)

Note: here hd0,0 means boot partition is in 1^st partition of 1^st hdd.(please use hd for SATA & SCSI hdd also )

Step b) grub > kernel /vmlinuz-$(uname -r) root=LABEL=/ rhgb quiet

Step c) grub > initrd /initrd-$(uname -r).img

Step d) grub > boot

Then after rebooting try to recreate grub.conf file

Initrd misplaced or deleted

Case a) system is up

This is the most fortunate situation for system admins who are managing the server remotely because now once the system will down it will not be up without rescue mode

Remedy:

[root@dh-localhost ~]# mkinitrd /boot/ initrd-$(uname -r).img $(uname -r)

Case b) system is down

Then boot from system via DVD & start it by rescue mode

Remedy:

Step 1) Boot: linux recue

After some process & ittrective prompt you will be on sh prompt

Step 2) chroot /mnt/sysimage

cd /boot

mkinitrd /boot/ initrd-$(uname -r).img $(uname -r)

Then reboot the system by HDD

Note: - in case of fstab is also misconfigured (ie / & other partition will not mount after boot) then chroot command will not work. So in this case your 1^st step would be to correct fstab then perform above steps as follows

Step a) boot from DVD or other bootable media

Step b) chroot /mnt/sysimage (you will find chroot error here )

Step c) mkdir /test

e2label /dev/sda1 (if it will show boot them mount it otherwise try to search by e2label /dev/sda2 ownwards)

mount /dev/sda1 /test

cd /test

cd grub

vi grub.conf (please make it correct)

mkdir /data

mount /dev/sda2 /data (mounting / partition on /data )

cd /data

cd /etc

vi fstab (please correct it )

Then reboot the system & follow previous step

Mbr corrupt

Case a) system is up

Method 1

[root@dh-localhost ~]# /sbin/grub-install /dev/sda

Method 2

[root@dh-localhost ~]# grub

grub> root (hd0,0)

grub > setup (hd0)

grub > quit

Case b) system is down

Boot system by rescue mode

Sh #Chroot /mnt/sysimage

Sh # /sbin/grub-install /dev/sda

Note : A smart system admin always take backup of MBR as follows

[root@dh-localhost ~]# dd if=/home/mbr of=/home/mbr bs=1 count 500

&

For restoration for MBR

[root@dh-localhost ~]# dd if=/home/mbr of=/home/mbr bs=1 count 500

/boot partition miss

In this case if you have backup of /boot then you can recover it otherwise you need to rebuild the server again.

What in NDR Spam ?

Redirect http to https

Suppose you have a requirement to divert your http traffic to https. There are many ways to achieve this , but as a Apache web administrator i prefer to apply following Rewrite rule for my site

RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://abc.com/$1 [R,L]

Here in above example example site is abc.com

Caching Configuration in Apache

The goad of caching in Apache could be

-Reduce Number of request in man cases

-Eliminate need to send full response in many cases

For former we use Expiration mechanism and for later we use validation mechanism.

For Apache mod_expires and mod_headers handles cache control through http headers.

mod_expires module control setting of expires and cache-control http header in server response.

For example to enable mod_expires for all .html files, we can use following syntax

     ExpiresActive On

     ExpiresDefault A300

   

     Expires A86400

     

In Above example, default caching is for 5 minutes(A300) but for html files caching is 1Day(A86400).

Now suppose we want to disable caching for dynamic pages located in folder /var/www/phpfiles

  Header Set Cache-Control "max-age=0,no-store"

Prevent DDOS attack on web site

There can a number of idea to prevent Distributed Denial of Service attack on website . Here i am sharing a basic idea that i use now a days to mitigate DDOS from malicious web crawler .As I noticed when a malicious crawler access web site we found  log entry  by a  particular User Agent in access-log. What i do is rewrite request from such User Agent to Forbidden messages using Rewrite module 

RewriteCond  %{HTTP_USER_AGENT} Malicious_User_Agent [NC]  

RewriteRule .*  - [F,L]

http traffic load

On some occasion , we need to calculate http traffic of our system. Although there are a number of tools and utilities but i avoid to install and use them in small size set-up.

 I decided to use tcpdump command ,  tcpdump command is very useful command that help to decode network traffic. For example for above given requirement i applied tcpdump command in following format.

tcpdump -s 1500 -Svni eth0 tcp and port 80

Above given command will display http packets (tcp packet of port number 80). In output of command we can get total length which is equal to IP Header Length + TCP Header Length + Payload Length . As we know IP Header + TCP Header length will be 52 , so we can calculate payload length which will be Total Length -52.

    

Hack Attempt

Today i noticed ../../../../../../../../. entries in my access log file of one of my web site.

I wonder is it a hack attempt ? If some hacker trying to doing scanning to find vulnerability of my system?

  Yes my doubt was genuine , It was a hack attempt. It was Local File Intrusion vulnerability. The attempt was to check if /proc/self/environ is accessible, if anyway it access that then do attempt to inject code and get access to shell.

Still i am working to way to mitigate such attack, immediate solution for me was to block ip which was doing such attempt.

log-rotate in Linux

Today my client claimed to reset the log rotation of access_log to 1 day and 1 log of all his 532 websites hosted in a server .

Also he claimed that all rotated log will be mailed to him .

Then I did :->

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Step 1:-> create a file

# vi logger.conf

/var/www/vhosts/*/statistics/logs/access_log

{daily

rotate 1

mail amit@abc.com

compress

}

:wq!

Then I simply type

Step2:-> logrotate logger.conf

++++++++++++++++++++++++++++++++++++++++++++

There are other options here :->

compress	This is used to compress the rotated log file with gzip.
nocompress	This is used when you do not want to compress rotated log files.
copytruncate	This is used when processes are still writing information to open log files. This option copies the active log file to a backup and truncates the active log file.
nocopytruncate	This copies the log files to backup, but the open log file is not truncated.
create mode owner group	This rotates the log file and creates a new log file with the specified permissions, owner, and group. The default is to use the same mode, owner, and group as the original file.
nocreate	This prevents the creation of a new log file.
delaycompress	When used with the compress option, the rotated log file is not compressed until the next time it is cycled.
nodelaycompress	This overrides delaycompress. The log file is compressed when it is cycled.
errors address	This mails logrotate errors to an address.
ifempty	With this, the log file is rotated even if it is empty. This is the default forlogrotate.
notifempty	This does not rotate the log file if it is empty.
mail address	This mails log files that are cycled to an address. When mail log files are cycled, they are effectively removed from the system.
nomail	When mail log files are cycled, a copy is not mailed.
olddir directory	With this, cycled log files are kept in the specified directory. This directory must be on the same filesystem as the current log files.
noolddir	Cycled log files are kept in the same directory as the current log files.
prerotate/endscript	These are statements that enclose commands to be executed prior to a log file being rotated. The prerotate and endscript keywords must appear on a line by themselves.
postrotate/endscript	These are statements that enclose commands to be executed after a log file has been rotated. The postrotate and endscript keywords must appear on a line by themselves.
daily	This is used to rotate log files daily.
weekly	This is used to rotate log files weekly.
monthly	This is used to rotate log files monthly.
rotate count	This specifies the number of times to rotate a file before it is deleted. A count of 0 (zero) means no copies are retained. A count of 5 means five copies are retained.
tabootext [+] list	This directs logrotate to not rotate files with the specified extension. The default list of extensions is .rpm-orig, .rpmsave, v, and ~.
size size	With this, the log file is rotated when the specified size is reached. Size may be specified in bytes (default), kilobytes (sizek), or megabytes (sizem)

For more detail you can read man page of logrotate .