Dovecot -> MySQL SElinux issue on CentOS6

I set up a postfix, dovecot, mysql, postfixadmin combo on SElinux enabled CentOS6, but I was keep getting following error in mail.log:

dovecot: dict: Error: mysql: Connect failed to localhost (mail): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13) - waiting for 1 seconds before retry

The problem is that Selinux prevents dovecot connecting to mysql socket. I was unable to find simple solution by setting some setsebol parameter, so I went through the audit.log with audit2allow.

Solution:

Create dovecot2mysql.te file under /etc/selinux:

cat > /etc/selinux/dovecot2mysql.te
module dovecot2mysqldb 1.0.0;

require {
    type dovecot_t;
    type dovecot_deliver_t;
    type var_t;
    type mysqld_db_t;
    type mysqld_t;
    type mysqld_var_run_t;
    type usr_t;
    class file { rename read create write getattr link unlink open };
    class dir search;
    class unix_stream_socket connectto;
    class sock_file write;
    class file { read getattr open };
}


allow dovecot_deliver_t var_t:file { rename read create write getattr link unlink open };
allow dovecot_t mysqld_db_t:dir search;
allow dovecot_t mysqld_t:unix_stream_socket connectto;
allow dovecot_t mysqld_var_run_t:sock_file write;
allow dovecot_t usr_t:file { read getattr open };


cat >/etc/selinux/sel.sh <<EOF
name=\${1%%.*}
echo "\$name"
checkmodule -M -m -o \$name.mod \$name.te \
  && semodule_package -o \$name.pp -m \$name.mod \
  && /usr/sbin/semodule -i \$name.pp
EOF

chmod 700 /etc/selinux/sel.sh
/etc/selinux/sel.sh /etc/selinux/dovecot2mysql.te

djndns on CentOS6

I use DJB's dns package for a long time and I am satisfied with it. Recently I started migrating it to a CentOS6 machine.
My new requirements were following:

• run services from init script
• collect log messages by syslog-ng
• have config files under /etc

There are a few guides on the net but they use DJB's svscan service to start-stop djb services and they use multilog as log collector.
There is my guide to achieve my requirements:
Install prerequisites (build environment):

yum install gcc make

Build daemontools

cd /usr/local/src
wget http://cr.yp.to/daemontools/daemontools-0.76.tar.gz
tar xvfz daemontools-0.76.tar.gz
cd admin/daemontools-0.76
sed -i '/gcc/ s|$| -include /usr/include/errno.h|' src/conf-cc
package/compile
for i in $(cat package/commands); do cp command/$i /usr/local/bin/$i; done

Build ucspi-tcp

cd /usr/local/src
wget http://cr.yp.to/ucspi-tcp/ucspi-tcp-0.88.tar.gz
tar xvfz ucspi-tcp-0.88.tar.gz
cd ucspi-tcp-0.88
sed -i '/gcc/ s|$| -include /usr/include/errno.h|' conf-cc
make
make setup check

Build djbdns

cd /usr/local/src
wget http://cr.yp.to/djbdns/djbdns-1.05.tar.gz
tar xvfz djbdns-1.05.tar.gz
cd djbdns-1.05
sed -i '/gcc/ s|$| -include /usr/include/errno.h|' conf-cc
make
make setup check

UPDATE:

Add /usr/local/bin to your PATH:

sed -i '/PATH=/ s|$|:/usr/local/bin|' /root/.bash_profile

Re-login or simply add to bash session:

export PATH=$PATH:/usr/local/bin

dnscache

Create dnscache users

groupadd -g 200 djbdns
useradd --no-create-home -s /bin/false -u 201 -g 200 dnscache
useradd --no-create-home -s /bin/false -u 202 -g 200 dnslog

Setup dnscache

dnscache-conf dnscache dnslog /etc/dnscache 192.168.233.254

Specify client's address range

touch /etc/dnscache/root/ip/192.168.233

Create init script

cat >/etc/init.d/dnscache <<EOF
#!/bin/bash
#
# dnscache      DNS cache server
#
# chkconfig: 345 11 89
# Provides: dnscache
# Required-Start: \$network
# Required-Stop: \$network
# Short-Description: start and stop dnscache
# Description: Dan Bernstein (DJB) dns cache server
# ZsZs
 
# Source function library.
. /etc/rc.d/init.d/functions
 
prog=dnscache
progdir=/usr/local/bin
 
# pull in sysconfig settings
[ -f /etc/sysconfig/dnscache ] && . /etc/sysconfig/dnscache
 
start(){
    echo -n \$"Starting \$prog: "
    pid=\$(pidof \$progdir/\$prog)
    if [[ "\$pid" == "" ]]; then
        # Read up dnscache environment files and export them
        for env in \$(cd \$dnscachedir/env/; ls -1); do envs="\$envs \$env=\$(cat \$dnscachedir/env/\$env)"; done
        export \$envs
        \$progdir/envuidgid dnscache \$progdir/softlimit -o250 -d "\$DATALIMIT" \
          \$progdir/\$prog 2>&1 | (\$progdir/setuidgid dnslog /usr/bin/logger -p local4.info -t \$prog) &
        if [[ "\$?" == "0" ]]
        then
            success
            echo
        else
            failure \$"\$prog start"
            echo
            exit 1
        fi
    else
        failure \$"\$prog start"
        echo
        exit 1
    fi
}
 
stop(){
    pid=\$(pidof \$progdir/\$prog)
    if [[ "\$pid" == "" ]]
    then
        echo -n "Stopping  \$prog: "
        failure \$"\$prog not running."
        echo
    else
        echo -n "Stopping \$prog: "
        kill \$pid && success || failure \$"\$prog stop"
        echo
    fi
}
 
# See how we were called.
case "\$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        sleep 1
        start
        ;;
    *)
        echo "Usage: \$prog {start|stop|restart}"
        exit 1
esac
 
exit 0
EOF

Create dnscache config file

echo "dnscachedir=/etc/dnscache" >/etc/sysconfig/dnscache

Start dnscache service

chmod 755 /etc/init.d/dnscache
service dnscache start
chkconfig dnscache on

tinydns

Create tinydns users

useradd --no-create-home -s /bin/false -u 203 -g 200 tinydns
useradd --no-create-home -s /bin/false -u 204 -g 200 tinylog

Install tinydns

tinydns-conf tinydns tinylog /etc/tinydns your.pub.lic.ip
cd /etc/tinydns/root

Create example domain Add SOA record

./add-ns example.hu 123.123.123.123

Add host

./add-host garfield.examle.hu 123.123.123.124

Add MX record

./add-mx examle.hu 123.123.123.123

Add alias

./add-alias www.example.hu 123.123.123.123

Make data.cdb

make

Create init script

cat >/etc/init.d/tinydns <<EOF
#!/bin/bash
#
# tinydns      Authoritative DNS server
#
# chkconfig: 345 11 89
# Provides: tinydns
# Required-Start: \$network
# Required-Stop: \$network
# Short-Description: start and stop tinydns
# Description: Dan Bernstein (DJB) authoritative dns server
# ZsZs
 
# Source function library.
. /etc/rc.d/init.d/functions
 
prog=tinydns
progdir=/usr/local/bin
 
# pull in sysconfig settings
[ -f /etc/sysconfig/tinydns ] && . /etc/sysconfig/tinydns
 
start(){
    echo -n \$"Starting \$prog: "
    pid=\$(pidof \$progdir/\$prog)
    if [[ "\$pid" == "" ]]; then
        # Read up tinydns environment files and export them
        for env in \$(cd \$tinydnsdir/env/; ls -1); do envs="\$envs \$env=\$(cat \$tinydnsdir/env/\$env)"; done
        export \$envs
        \$progdir/envuidgid tinydns \$progdir/softlimit -d300000 \$progdir/\$prog 2>&1 | \
          (\$progdir/setuidgid tinylog /usr/bin/logger -p local4.info -t \$prog) &
        if [[ "\$?" == "0" ]]
        then
            success
            echo
        else
            failure \$"\$prog start"
            echo
            exit 1
        fi
    else
        failure \$"\$prog start"
        echo
        exit 1
    fi
}
 
stop(){
    pid=\$(pidof \$progdir/\$prog)
    if [[ "\$pid" == "" ]]
    then
        echo -n "Stopping  \$prog: "
        failure \$"\$prog not running."
        echo
    else
        echo -n "Stopping \$prog: "
        kill \$pid && success || failure \$"\$prog stop"
        echo
    fi
}
 
# See how we were called.
case "\$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        sleep 1
        start
        ;;
    *)
        echo "Usage: \$prog {start|stop|restart}"
        exit 1
esac
 
exit 0
EOF

Create tinydns config file

echo "tinydnsdir=/etc/tinydns" >/etc/sysconfig/tinydns

Start tinydns service

chmod 755 /etc/sysconfig/tinydns
service tinydns start
chkconfig tinydns on

Test dns server: ask it for some type of records

dnsq soa example.hu 123.123.123.123
dnsq a example.hu 123.123.123.123

axfrdns

Create axfrdns user

useradd --no-create-home -s /bin/false -u 205 -g 200 axfrlog

Install axfrdns

axfrdns-conf axfrdns axfrlog /etc/axfrdns /etc/tinydns 123.123.123.123

Allow a cleint to tranfser zone

cd /etc/axfrdns
cat >>tcp <<EOF
123.123.123.123:allow,AXFR="example.hu"
EOF
make

Create init script

cat >/etc/init.d/axfrdns <<-EOF
#!/bin/bash
#
# axfrdns      DNS zone transfer server
#
# chkconfig: 345 11 89
# Provides: axfrdns
# Required-Start: \$network
# Required-Stop: \$network
# Short-Description: start and stop axfrdns
# Description: Dan Bernstein (DJB) dns zone transfer server
# ZsZs
 
# Source function library.
. /etc/rc.d/init.d/functions
 
prog=axfrdns
progdir=/usr/local/bin
 
# pull in sysconfig settings
[ -f /etc/sysconfig/axfrdns ] && . /etc/sysconfig/axfrdns
 
start(){
    echo -n \$"Starting \$prog: "
    pid=\$(pidof \$progdir/tcpserver \$progdir/\$prog)
    if [[ "\$pid" == "" ]]; then
        # Read up axfrdns environment files and export them
        for env in \$(cd \$axfrdnsdir/env/; ls -1); do envs="\$envs \$env=\$(cat \$axfrdnsdir/env/\$env)"; done
        export \$envs
        \$progdir/envuidgid axfrlog \$progdir/softlimit -d300000 \$progdir/tcpserver -vDRHl0 -x \$axfrdnsdir/tcp.cdb -- "\$IP" 53 \
          \$progdir/\$prog 2>&1 | (\$progdir/setuidgid axfrlog /usr/bin/logger -p local4.info -t \$prog) &
        if [[ "\$?" == "0" ]]
        then
            success
            echo
        else
            failure \$"\$prog start"
            echo
            exit 1
        fi
    else
        failure \$"\$prog start"
        echo
        exit 1
    fi
}
 
stop(){
    pid=\$(pidof \$progdir/tcpserver \$progdir/\$prog)
    if [[ "\$pid" == "" ]]
    then
        echo -n "Stopping  \$prog: "
        failure \$"\$prog not running."
        echo
    else
        echo -n "Stopping \$prog: "
        kill \$pid && success || failure \$"\$prog stop"
        echo
    fi
}
 
# See how we were called.
case "\$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        sleep 1
        start
        ;;
    *)
        echo "Usage: \$prog {start|stop|restart}"
        exit 1
esac
 
exit 0
EOF

Create axfrdns config file

echo "axfrdnsdir=/etc/axfrdns" >/etc/sysconfig/axfrdns

Start axfrdns service

chmod 755 /etc/init.d/axfrdns
service axfrdns start
chkconfig axfrdns on

Test axfrdns server

tcpclient 123.123.123.123 53 axfr-get example.hu aa aa.tmp
cat aa
#1352995218 auto axfr-get
Zexample.hu:0.ns.example.hu.:hostmaster.example.hu.:1352995218:16384:2048:1048576:2560:2560
&example.hu::0.ns.example.hu.:259200
+0.ns.example.hu:123.123.123.123:259200
+garfield.example.hu:123.123.123.123:86400
@example.hu::a.mx.example.hu.:0:86400
+a.mx.example.hu:123.123.123.123:86400
+www.example.hu:123.123.123.123:86400

syslog-ng

Relevant lines of syslog-ng.conf to collect logs in sepatare log files:

destination df_djb { file("/var/log/$YEAR/$MONTH/$DAY/djb_$PROGRAM.$YEAR$MONTH$DAY.log" ); };
filter f_djb { facility(local4); };
log { source(s_sys); filter(f_djb); destination(df_djb); };

Thanks to http://www.koelzer.us/~thomas/postfix-CentOS6.html

Comments are welcome!

Ratelimit callbacks suppressed

Recently I hardened some RHEL6 based machines.
During this hardening process - among the others - I did the following:
- disdabled ipv6 by "options ipv6 disable=1" in /etc/modprobe.d/hardening.conf
- added some more audit rules according to NSA guide
- stopped auditd therefore audit log events are redirected to kernel log
- filtered audit logs by following filter to separate audilt.log:
filter f_audit { match(' audit\(' value("MESSAGE")); };

After this hardening there were staerted appearing "__ratelimit: XX callbacks suppressed" messages regularly in kernel log, like following:

Oct 18 01:00:01 test1 kernel: __ratelimit: 4 callbacks suppressed
Oct 18 01:01:01 test1 kernel: __ratelimit: 192 callbacks suppressed
Oct 18 01:05:07 test1 kernel: __ratelimit: 188 callbacks suppressed

It didn't cause any problem but after a while I've started investigating (thx Cipo) what can cause this strange behaviour.

The root cause of this problem is a bit complex. Let's see them:
- disabling ipv6 module caused that some programs would like to insert it
- I've set an audit rule which logs any module instertion attempts: "-w /sbin/modprobe -p x -k modules"
- there are kernel.printk_ratelimit* kernel parameters
# cat /proc/sys/kernel/printk_ratelimit
5
# cat /proc/sys/kernel/printk_ratelimit_burst
10
This means that there is a 10 msgs/5 seconds limit in kernel logging. Exceeding this limit messages will be dropped AND "__ratelimit NumberOfDroppedMessages: callbacks suppressed" messages will be written to the kernel log.

To put the pieces of the picture together:
- my script runs curl 20 times
- curl tried to insert ipv6 module by modprobe
- modprobe tries were logged by audit (5 lines/ modprobe)
- too much audit log in short time exceeded the ratelimit

My solution:
- re-enable ipv6 by commenting  "options ipv6 disable=1" line in /etc/modprobe.d/hardening.conf
- prevent use of ipv6 calls by appending following line to /etc/sysctl.conf:
net.ipv6.conf.all.disable_ipv6 = 1 
(Thanks to Daniel Walsh)

These resulted that:
- ipv6 module is already inserted, so programs do not want to insert it
- ipv6 remained pseudo-disabled

After finishing this I've continued getting ratelimit messaged but not regularly.
I've  find out that Midnight Commander deletes a few temporary files and changes permissions of its config files after exiting. This process resulted more than 150 lines within one second in audit log...
My solution is that I've appended following line to the /etc/sysctl.conf file:
kernel.printk_ratelimit = 30
kernel.printk_ratelimit_burst = 200
This means that kernel accepts 200 messages/30 sec

Comments are welcome.

CentOS6 KVM with Shorewall

I wanted to install a CentOS6 host and some vitrual machines in it but I have only one public ip address so I cannot use the most common method to make a bridge as public interface.
There are a lot of guides describing how to install KVM with bridged public interface but just a few of them mentioned my situation.


By default there is just a simple firewall configuration tool on CentOS. I am not an iptables guru that is why I started using shorewall as firewall configuration tool long time ago and I am very satisfied with it..

According to libvirt's documentation when a libvirt manageded network bridge starts, some iptables rules applies and it can confuse administrators when there are two iptables configuration tool working on the same time.
It look like it is the time to set up the virtual network manually...

In shorewall there will be four defined zones:
- net: Internet on host's eth0 interface
- fw: the host computer itself
- dmz: virtual machines with configured portfowards from public ip (public services), interface: virbr0
- virt: virtual machines without portforwarding (testing purpose), interface: virbr1

1. First of all after installing libvirt stop and disable autostarting default network:

# virsh net-destroy default
Network default destroyed

# virsh net-autostart default --disable
Network default unmarked as autostarted

# virsh net-list --all
Name                 State      Autostart
-----------------------------------------
default              inactive   no

2. Create bridge manually, aka let the os network configuration scripts make them

# cat >/etc/sysconfig/network-scripts/ifcfg-virbr0 <<EOF
DEVICE=virbr0
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.233.254
NETMASK=255.255.255.0
DELAY=0
STP=off
EOF
# cat >/etc/sysconfig/network-scripts/ifcfg-virbr1 <<EOF
DEVICE=virbr1
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.244.254
NETMASK=255.255.255.0
DELAY=0
STP=off
EOF
# service network restart

3. If you would like to have a DHCP server on these subnets, configure and start dnsmasq as well:

# cat >/etc/dnsmasq.conf <<EOF
domain=example.com
domain=dmz.example.com,192.68.233.90,192.168.233.99
dhcp-range=dmz,192.168.233.90,192.168.233.99, 12h
dhcp-option=dmz,15,dmz.example.com
domain=virt.example.com,192.68.244.90,192.168.244.99
dhcp-range=virt,192.168.244.90,192.168.244.99, 12h
dhcp-option=virt,15,virt.example.com

bind-interfaces
domain-needed
bogus-priv
resolv-file=/etc/dnsmasq.d/resolv.conf
strict-order
listen-address=192.168.233.254
log-facility=/var/log/dnsmasq.log
#log-queries
dns-forward-max=150
cache-size=10000
EOF
# cat >/etc/dnsmasq.d/resolv.conf <<EOF
nameserver 8.8.8.8
nameserver 8.8.4.4
EOF
# chkconfig dnsmasq on
# service dnsmasq start

4. Ensure that you have one-interface config installed in /etc/shorewall.
Update: one interface config is the default, so you can skip this step.

# cd /usr/share/doc/shorewall-4.*/Samples/one-interface
# cp interfaces policy rules shorewall.conf zones /etc/shorewall

5. Append virtual zones definitions to zones file:

# TAB="$(printf '\t')"
# cat >>/etc/shorewall/zones <<EOF
virt${TAB}ipv4
dmz${TAB}ipv4
EOF

6. Append interfaces definitions to interfaces file:

# TAB="$(printf '\t')"
# cat >>/etc/shorewall/interfaces <<EOF
net${TAB}${TAB}eth0${TAB}${TAB}${TAB}dhcp,tcpflags,logmartians,nosmurfs
dmz${TAB}${TAB}virbr0${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback,bridge
dmz${TAB}${TAB}vnet0${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback
virt${TAB}${TAB}virbr1${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback,bridge
virt${TAB}${TAB}vnet1${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback
EOF

7. Rewrite policy definitions according to your needs. For example:

# TAB="$(printf '\t')"
# cat >>/etc/shorewall/policy <<EOF
# Connecto from firewall
\$FW${TAB}ALL${TAB}ACCEPT

# Connect from DMZ machines

dmz${TAB}dmz${TAB}ACCEPT
dmz${TAB}net${TAB}ACCEPT
dmz${TAB}\$FW${TAB}DROP${TAB}${TAB}info

# Connect from VIRT machines
virt${TAB}virt${TAB}ACCEPT
virt${TAB}all${TAB}DROP${TAB}${TAB}info

# Connect from net
net${TAB}all${TAB}DROP${TAB}${TAB}info

# The FOLLOWING POLICY MUST BE LAST
all${TAB}all${TAB}REJECT${TAB}${TAB}info
EOF

8. You should re-configure your sshd on the host to listen on different port and define an alternate SSH macro in shorewall: Warning! After ths step your host's sshd will listen on non default port! Be sure, that you can reach it on this different ports!

# sed -i "s/#Port 22/Port 39022/" /etc/ssh/sshd_config
# service sshd restart
# TAB="$(printf '\t')"
# cat >/etc/shorewall/macro.SSHAlt <<EOF
###############################################################################
#ACTION SOURCE  DEST    PROTO   DEST    SOURCE  RATE    USER
#                               PORT(S) PORT(S) LIMIT   GROUP
PARAM${TAB}-${TAB}-${TAB}tcp${TAB}39022

EOF

9. Define an admin host in params file

# cat >>/etc/shorewall/params <<EOF
ADMIN_HOSTS=11.22.33.44,11.22.66.44
DMZ_GUEST1=192.168.233.1
EOF

10. Enabe masquerading from dmz zone.

echo -e "eth0\t\t\t192.168.233.0/24" >>/etc/shorewall/masq

11. Append or rewrite rules file according to your configuration. For example:

# TAB="$(printf '\t')"
# cat >>/etc/shorewall/rules <<EOF
VNC(ACCEPT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}\$FW
SSHAlt(ACCEPT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}\$FW
SSH(DNAT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}dmz:\$DMZ_GUEST1
EOF

This configuration let the admin connect from ADMIN_HOSTS to the:

• host on 39022 port (ssh) 
• host on 5900-5999 (vnc) 
• DMZ_GUEST1 virtual machine on port 22 (ssh)

12. Enable shorewall startup:

# sed -i "s/startup=0/startup=1/" /etc/sysconfig/shorewall

13. Start shorewall

# shorewall start

14. Create your virtual machine using

• --network bridge:virbr0 if you want to connect it to dmz network
• --network bridge:virbr1 to connect it to virt network.

15. Enjoy!

Create local CentOS repository

If you have to install a lot if machines with CentOS or you just play with kickstart script you should create a local repository to speed the process up. All you need as prerequisite is a configured web server and a script which copies the changes only from the public repository.
This script handles the CentOS and EPEL repository as well.

Here comes the script:

# cat > /usr/local/bin/repo-sync.sh
#!/bin/bash

rsync=/usr/bin/rsync
rsyncopts='--progress -avHSP --bwlimit=512 --delete'
rsyncexcl='--exclude=openoffice*.rpm --exclude=fonts-chinese*.rpm 
--exclude=fonts-japanese*.rpm --exclude=fonts-korean*.rpm 
--exclude=libreoffice*.rpm --exclude=firefox*.rpm 
--exclude=thunderbird*.rpm --exclude=ImageMagick*.rpm --exclude=/SRPMS/ 
--exclude=/headers/ --exclude=*.src.rpm --exclude=*.drpm 
--exclude=*-debuginfo-*.rpm --exclude=/debug/ --exclude=/repoview/'

if [ -f /var/lock/subsys/repo-sync ]; then
    echo "Updates via rsync already running."
    exit 0
fi

mirror=ftp.fsn.hu::linux/centos
verlist="6 5"
archlist="i386 x86_64"
baselist="updates os"
local=/var/www/html/mirror/centos/

for ver in $verlist; do
  for arch in $archlist; do
    for base in $baselist; do
      echo "==================================="
      echo "===== CentOS $ver/$base/$arch ====="
      echo "==================================="
      lrepo=$local/$ver/$base/$arch
      /bin/mkdir $lrepo 2> /dev/null
      /bin/touch /var/lock/subsys/rsync_sync
      remote=$mirror/$ver/$base/$arch/
      $rsync $rsyncopts $rsyncexcl $remote $lrepo
      newpkgs=`/usr/bin/find $lrepo -ctime -1 | wc -l`
      if [ $newpkgs -gt 0 ]; then
       /usr/bin/createrepo $lrepo #>> /dev/null
      fi
    done
  done
done

mirror=ftp.linux.cz::pub/linux/fedora/epel
verlist="6 5"
archlist="i386 x86_64"
baselist="."
local=/var/www/html/mirror/epel/

for ver in $verlist; do
  for arch in $archlist; do
    for base in $baselist; do
      echo "==================================="
      echo "===== EPEL $ver/$base/$arch ====="
      echo "==================================="
      lrepo=$local/$ver/$base/$arch
      /bin/mkdir $lrepo 2> /dev/null
      /bin/touch /var/lock/subsys/rsync_sync
      remote=$mirror/$ver/$base/$arch/
      $rsync $rsyncopts $rsyncexcl $remote $lrepo
      newpkgs=`/usr/bin/find $lrepo -ctime -1 | wc -l`
      if [ $newpkgs -gt 0 ]; then
       /usr/bin/createrepo $lrepo #>> /dev/null
      fi
    done
  done
done

Ctrl-D

Some explanation:
- there are some exclusions at the beginning of the script because I do not need any graphical programs for servers
- you should replace mirror hosts and path with one which is closer to you
- be sure that you choose a mirror which supports rsync protocol. http and/or ftp is not necessary for this script. - you can select which version (verlist) and which architecture (archlist) to sync

Extract filaname from path with regex

Extracting filname from the full path when the path is OPTIONAL is not so easy as it looks at first look.
I would like to extract cc.cc from
/aa/bb/cc.cc 
either from
cc.cc
as well.

Solution:

$ echo "/aa/bb/cc.cc" | perl -pe "s|(?:.*/)?([^/]*)|\1|"
cc.cc
$ echo "/aa/cc.cc" | perl -pe "s|(?:.*/)?([^/]*)|\1|"
cc.cc
$ echo "cc.cc" | perl -pe "s|(?:.*/)?([^/]*)|\1|"
cc.cc

Some explanation: 

• ?: start non capturing group
• [^/] any character except slash

Amazon ec2 dynamic ip updater

How to setup ddclient on Centos machine running in Amazon EC2 to update it's public ip address to dyndns-like services.

1. Install ddclient from EPEL repository
$ sudo yum install ddclient
2. modify/add following lines in /etc/ddclient.conf:

use=cmd, cmd='curl http://169.254.169.254/2007-08-29//meta-data/public-ipv4'
server=no-ip.org
login=noipuser
password=noippassword
yourhostname=no-ip.org

Some explanation on "use" line: this is the recommended way to get the public ip address of the running machine.

3. Start ddclient service:

$ sudo /etc/init.d/ddclient start

4. Do not forget to set ddclient's default runlevels to start after reboot

$ sudo chkconfig ddclient on

That's all.

UPDATE:
It seems that dynupdate.no-ip.com does not accept SSL traffic other than TLSv1.
T workaround this put following line to /usr/sbin/ddclient after 1864th line:
            SSL_version => 'TLSv1',

The diff between orig and new file will lokk like this:

# diff -u ddclient.orig ddclient
--- ddclient.orig       2012-10-27 10:06:10.000000000 +0200
+++ ddclient    2012-10-27 09:49:56.000000000 +0200
@@ -1861,6 +1861,7 @@
             Proto => 'tcp',
             MultiHomed => 1,
             Timeout => opt('timeout'),
+            SSL_version => 'TLSv1',
         );
            defined $sd or warning("cannot connect to $peer:$port socket: $@ " . IO::Socket::SSL::errstr());
     } else {

Thanks to https://bugs.archlinux.org/task/30280

Do not forget to re-apply this patch after updating ddclient!

Edit crontab from shell script

Editing crontabs from shell script is not so easy as it sounds first.
You can't simply edit /var/spool/cron/crontabs/user file because you have to install new crontab by running crontab command.

So the right method is:
1. list the current crontabs to a temporary file:
$ crontab -l > crontab.lst


2. Edit/append the created file
$ echo "*/10 * * * * /path/to/new/job" >>crontab.lst

3. Install edited crontab file:
$ crontab crontab.lst

Nagios/Icinga plugin for amanda

I have a simple amanda server which contains a single tape drive only (no changer).
Need I say that I always forget changing the tape and backups are filling the holding disk?
I've written a simple check, which reminds me changing the tape or warns that the holding disk is almust full.
This script is very simple and it assumes that there is only one slot.
Comments/improvements are welcome!

#!/bin/bash
# Nagios plugin to check amanda server by amcheck
# (c) ZsZs 2012

# put following lines to sudoers file:
# Defaults:icinga  !requiretty
# icinga ALL=(amandabackup) NOPASSWD: /usr/sbin/amcheck

STATE_OK=0
STATE_WARNING=1
STATE_CRITICAL=2
STATE_UNKNOWN=3
STATE_DEPENDENT=4
scr=${0##*/}
# Default alarm levels
warn=50
crit=25

usage() {
  cat <<EOF
  Usage: ${scr} [-w xx] [-c yy] -C amConf

  Check if the correct tape inserted into the tape device and there is enough space on the holding disk
  Options:
    -C  Amanda config name
    -w  Holding disk warning threshhold (GB)
    -c  Holding disk critical threshhold (GB)
    -?  Show this help
EOF
exit $STATE_UNKNOWN
}

[ -z "$1" ] && usage
until [ -z "$1" ]  # Until all command line parameters read up
do
  case $1 in
    -w) warn=$2
      shift 2;;
    -c) crit=$2
      shift 2;;
    -C) conf=$2
      shift 2;;
     *) usage
  esac
done

[ $crit -gt $warn ] && { echo "Critical treshold can not be greater than warning treshold!"; exit $STATE_UNKNOWN; }
tmpfile=$(mktemp)
sudo -u amandabackup /usr/sbin/amcheck $conf >$tmpfile
slot=$(grep '^slot' $tmpfile)
holdtmp=$(grep '^Holding' $tmpfile)
holdtmp2=${holdtmp##*: }
holding=${holdtmp2%%GB*}
rm $tmpfile >dev/null

if [[ "$slot" =~ 'is not' ]]; then
  echo "CRITICAL - $slot"
  exit $STATE_CRITICAL
elif [ $holding -lt $crit ]; then
  echo "CRITICAL - Holding disk space: $holding GB free"
  exit $STATE_CRITICAL
elif [ $holding -lt $warn ]; then
  echo "WARNING - Holding disk space: $holding GB free"
  exit $STATE_WARNING
elif [[ "$slot" =~ 'still active' ]]; then
  echo "WARNING - $slot"
  exit $STATE_WARNING
else
  echo "OK - $slot"
  exit $STATE_OK
fi

Leap seconds on CentOS6/RHEL6 masterclock

Setting up handling leap seconds on Centos6/RHEL6

1. Download actual leapseconds file
# cd /etc/ntp
# wget ftp://tycho.usno.navy.mil/pub/ntp/leap-seconds.*
2. Make a symlink to the downloaded file
# ln -s leap-seconds.3535142400 ntpkey_leap
3. Generate host keys for ntpd
# ntp-keygen -H -p password
Using OpenSSL version 10000003
Generating RSA keys (512 bits)...
RSA 0 0 2       1 11 24 3 1 2
Generating new host file and link
ntpkey_host_ntptest->ntpkey_RSAkey_ntptest.3550552917
Using host key as sign key
Generating certificate RSA-MD5
X509v3 Basic Constraints: critical,CA:TRUE
X509v3 Key Usage: digitalSignature,keyCertSign
Generating new cert file and link
ntpkey_cert_ntptest->ntpkey_RSA-MD5cert_ntptest.3550552917
4. Add following lines to the ntpd.conf:
# echo >>/etc/ntpd.conf
keysdir /etc/ntp
crypto pw password
5. Restart ntpd
# /etc/init.d/ntpd restart
6. Check results
# ntpq -c "rv 0 leap,tai,leapsec,expire,update"
assID=0 status=4944 leap_add_sec, sync_telephone, 4 events, event_peer/strat_chg, leap=01, update=201206302154, leapsec=201201100000, tai=34, expire=201207061434


# ntpq -c "rv 0" | egrep "(update|leapsec|expire)"
assID=0 status=4944 leap_add_sec, sync_telephone, 4 events, event_peer/strat_chg, leap=01, update=201206302154, leapsec=201201100000, tai=34, expire=201307061434 
Note that there can be a bug because two different commands give two different values for expire.


After leap second event ntpq output looks like this. Main differences to previous state are shown in green.
assID=0 status=0944 leap_none, sync_telephone, 4 events, event_peer/strat_chg, leap=00, update=201206302154, leapsec=201201100000, tai=35, expire=201307061434 

Kernel log reports leap second insertion as follows:
Jul  1 01:59:59 ntptest kernel: Clock: inserting leap second 23:59:60 UTC
Note that the leap second insertion is at UTC midnight.

Updating leapseconds file.

Last but not least: DO NOT FORGET to check newly published leap seconds file regularly!
Here is the latest announcement about leap second insertion.
If it refers to a newer leap second than your configured leapsecond file you sholud repeat 1-2 steps above.
ntpd doesn not need to be restarted because it will re-read the ntp_leap file on daily basis.

Serial Terminal for linux

There are several ways to connect to a device with serial port console.
Example settings are for Sun HW, and Cisco routers.

Minicom:

Create settings file

# cat >/etc/minirc.sunterm
pu baudrate         9600
pu bits             8
pu parity           N
pu stopbits         1
pu minit            ~^M~
Ctrl-D
# minicom sunterm

Screen

$ screen /dev/ttyS0 9600,cs8,-parenb,-cstopb,-hupcl

Parameters from man stty:
9600: baud setting
cs8: character size
-parenb: disable parity detection
-cstopb: one stop bit per character
-hupcl: do not hangup on last connection

Jumpstart on CentOS

Jumpstart on CentOS5 howto

We have an old Sun Netra machine and we would like to install Solaris 10 on it. Unfortunately this machine is built with cdrom and we was'nt able to find any Soalris 10 cdrom ISO. I tried to change the drive to a dvd drive from an Sun Fire V490 but it was not not detected: probe-ide-all did not show any drive.
There is no chance to avoid building JumpStart server but we do not have another Sun hw to do this.
Let's try Linux. My first though was CentOS6 but it lacks of old rarpd, bootparamsd. I did some research to find out that CentOS5 contains these daemons as well.
In the following example there are two machines:
JumpStart server hostname: jumpstart, ip address: 192.168.10.100
JumpStart client hostname: netra, ip address: 192.168.10.200

Prerequisites

1. Install CentOS5 minimum environment: I have kickstarted: %packages --nobase and @core group
2. Install necessary services:
# yum install gdb nfs-utils rarpd tftp-server bootparamd tcpdump
# for services in "nfs portmap rarpd bootparamd tftp"
do 
  chkconfig $ service on
done
3. Disable unnecessary services:
# for services in "iscsi iscsid kudzu rawdevices restorecond mcstrans lvm2-monitor"
do 
  chkconfig $ service off
done

Setup JumpStart Server

1. Add server entry to hosts file
# echo '192.168.10.100 jumpstart' >> /etc/hosts
2. To successfully run setup_install_server script make some soft links and create mach "utility":
# ln -s /bin/tar /bin/bar
# ln -s /bin/sed /usr/bin/sed
# ln -s /usr/bin/gdb /usr/bin/adb
# echo '#!/bin/bash' > /bin/mach
# echo 'uname -p' >>/bin/mach
# chmod 755 /bin/mach
3. Copy install files to install directory
# mkdir /media/cdrom
# mount /dev/cdrom /media/cdrom
# cd /media/cdrom/Solaris_10/Tools
# mkdir -p /var/js/install
# ./setup_install_server /var/js/install
4. Configure and start NFS service
Disable NFSv4
# sed -i 's/#RPCNFSDARGS=\"-N 4\″/RPCNFSDARGS=\"-N 4\″/' /etc/sysconfig/nfs
Export /var/js/install
# echo '/var/js/install 192.168.10.0/24(ro,no_root_squash,async,no_subtree_check)' >> /etc/exports
Start NFS service
# service portmap start
# service nfs start
Test NFS service
# mount jumpstart:/var/js/install /mnt
You shoud see the mount point in the 'df -h' output.
5. Enable tftpd in xinetd
# sed -i 's/disable\t\t\t= yes/disable\t\t\t= no/' /etc/xinetd.d/tftp
# service xinetd restart
6. Copy inetboot images to tftpd's directory
# cp /var/js/install/Solaris_10/Tools/Boot/platform/sun4u/inetboot /tftpboot/inetboot.sun4u
# cp /var/js/install/Solaris_10/Tools/Boot/platform/sun4us/inetboot /tftpboot/inetboot.sun4us
# cp /var/js/install/Solaris_10/Tools/Boot/platform/sun4v/inetboot /tftpboot/inetboot.sun4v

Add client related config

1. Add an entry to hosts file
# echo '192.168.10.200 netra' >> /etc/hosts
2. Add an entry to rarpd's config file /etc/ethers
# echo '8:0:20:aa:bb:cc netra' >> /etc/ethers
Make sure that you modify the MAC address according to your machine's address
3. Configure bootparamd
# echo 'netra root=jumpstart:/var/js/install/Solaris_10/Tools/Boot \
install=jumpstart:/var/js/install boottype=:in' >> /etc/bootparams
4. Configure TFTP service
Calculate client's ip address in hex format:
# printf %02x 192 168 10 200|tr a-z A-Z
C0A810C8
# cd /tftpboot
# ln -s inetboot.sun4u C0A810C8
Make the link to the appropriate inetboot platform file (sun4u in this example)

Boot the client

ok boot net -v - install

Troubleshooting

If you get something like following, you should disable NFSv4 in nfs config:
root directory: /var/js/install/Solaris_10/Tools/Boot

boot failed
Enter filename [boot_archive]:

Thanks to following sites:

http://linax.wordpress.com/2009/06/09/linux-as-a-solaris-jumpstart-server/
http://hintshop.ludvig.co.nz/show/solaris-jumpstart-linux-server/