Monday, September 17, 2012

CentOS6 KVM with Shorewall


I wanted to install a CentOS6 host and some vitrual machines in it but I have only one public ip address so I cannot use the most common method to make a bridge as public interface.
There are a lot of guides describing how to install KVM with bridged public interface but just a few of them mentioned my situation.


By default there is just a simple firewall configuration tool on CentOS. I am not an iptables guru that is why I started using shorewall as firewall configuration tool long time ago and I am very satisfied with it..

According to libvirt's documentation when a libvirt manageded network bridge starts, some iptables rules applies and it can confuse administrators when there are two iptables configuration tool working on the same time.
It look like it is the time to set up the virtual network manually...

In shorewall there will be four defined zones:
- net: Internet on host's eth0 interface
- fw: the host computer itself
- dmz: virtual machines with configured portfowards from public ip (public services), interface: virbr0
- virt: virtual machines without portforwarding (testing purpose), interface: virbr1

1. First of all after installing libvirt stop and disable autostarting default network:
# virsh net-destroy default
Network default destroyed

# virsh net-autostart default --disable
Network default unmarked as autostarted

# virsh net-list --all
Name                 State      Autostart
-----------------------------------------
default              inactive   no
2. Create bridge manually, aka let the os network configuration scripts make them 
# cat >/etc/sysconfig/network-scripts/ifcfg-virbr0 <<EOF
DEVICE=virbr0
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.233.254
NETMASK=255.255.255.0
DELAY=0
STP=off
EOF
# cat >/etc/sysconfig/network-scripts/ifcfg-virbr1 <<EOF
DEVICE=virbr1
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.244.254
NETMASK=255.255.255.0
DELAY=0
STP=off
EOF
# service network restart
3. If you would like to have a DHCP server on these subnets, configure and start dnsmasq as well: 
# cat >/etc/dnsmasq.conf <<EOF
domain=example.com
domain=dmz.example.com,192.68.233.90,192.168.233.99
dhcp-range=dmz,192.168.233.90,192.168.233.99, 12h
dhcp-option=dmz,15,dmz.example.com
domain=virt.example.com,192.68.244.90,192.168.244.99
dhcp-range=virt,192.168.244.90,192.168.244.99, 12h
dhcp-option=virt,15,virt.example.com

bind-interfaces
domain-needed
bogus-priv
resolv-file=/etc/dnsmasq.d/resolv.conf
strict-order
listen-address=192.168.233.254
log-facility=/var/log/dnsmasq.log
#log-queries
dns-forward-max=150
cache-size=10000
EOF
# cat >/etc/dnsmasq.d/resolv.conf <<EOF
nameserver 8.8.8.8
nameserver 8.8.4.4
EOF
# chkconfig dnsmasq on
# service dnsmasq start
4. Ensure that you have one-interface config installed in /etc/shorewall.
Update: one interface config is the default, so you can skip this step.
# cd /usr/share/doc/shorewall-4.*/Samples/one-interface
# cp interfaces policy rules shorewall.conf zones /etc/shorewall
5. Append virtual zones definitions to zones file: 
# TAB="$(printf '\t')"
# cat >>/etc/shorewall/zones <<EOF
virt${TAB}ipv4
dmz${TAB}ipv4
EOF
6. Append interfaces definitions to interfaces file: 
# TAB="$(printf '\t')"
# cat >>/etc/shorewall/interfaces <<EOF
net${TAB}${TAB}eth0${TAB}${TAB}${TAB}dhcp,tcpflags,logmartians,nosmurfs
dmz${TAB}${TAB}virbr0${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback,bridge
dmz${TAB}${TAB}vnet0${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback
virt${TAB}${TAB}virbr1${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback,bridge
virt${TAB}${TAB}vnet1${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback
EOF
7. Rewrite policy definitions according to your needs. For example: 
# TAB="$(printf '\t')"
# cat >>/etc/shorewall/policy <<EOF
# Connecto from firewall
\$FW${TAB}ALL${TAB}ACCEPT

# Connect from DMZ machines

dmz${TAB}dmz${TAB}ACCEPT
dmz${TAB}net${TAB}ACCEPT
dmz${TAB}\$FW${TAB}DROP${TAB}${TAB}info

# Connect from VIRT machines
virt${TAB}virt${TAB}ACCEPT
virt${TAB}all${TAB}DROP${TAB}${TAB}info

# Connect from net
net${TAB}all${TAB}DROP${TAB}${TAB}info

# The FOLLOWING POLICY MUST BE LAST
all${TAB}all${TAB}REJECT${TAB}${TAB}info
EOF
8. You should re-configure your sshd on the host to listen on different port and define an alternate SSH macro in shorewall: Warning! After ths step your host's sshd will listen on non default port! Be sure, that you can reach it on this different ports! 
# sed -i "s/#Port 22/Port 39022/" /etc/ssh/sshd_config
# service sshd restart
# TAB="$(printf '\t')"
# cat >/etc/shorewall/macro.SSHAlt <<EOF
###############################################################################
#ACTION SOURCE  DEST    PROTO   DEST    SOURCE  RATE    USER
#                               PORT(S) PORT(S) LIMIT   GROUP
PARAM${TAB}-${TAB}-${TAB}tcp${TAB}39022
EOF
9. Define an admin host in params file 
# cat >>/etc/shorewall/params <<EOF
ADMIN_HOSTS=11.22.33.44,11.22.66.44
DMZ_GUEST1=192.168.233.1
EOF
10. Enabe masquerading from dmz zone. 
echo -e "eth0\t\t\t192.168.233.0/24" >>/etc/shorewall/masq
11. Append or rewrite rules file according to your configuration. For example: 
# TAB="$(printf '\t')"
# cat >>/etc/shorewall/rules <<EOF
VNC(ACCEPT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}\$FW
SSHAlt(ACCEPT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}\$FW
SSH(DNAT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}dmz:\$DMZ_GUEST1
EOF
This configuration let the admin connect from ADMIN_HOSTS to the:
  • host on 39022 port (ssh) 
  • host on 5900-5999 (vnc) 
  • DMZ_GUEST1 virtual machine on port 22 (ssh)
12. Enable shorewall startup: 
# sed -i "s/startup=0/startup=1/" /etc/sysconfig/shorewall
13. Start shorewall 
# shorewall start
14. Create your virtual machine using
  • --network bridge:virbr0 if you want to connect it to dmz network
  • --network bridge:virbr1 to connect it to virt network.
15. Enjoy!
Posted by at No comments:
Labels: , , , , , , , ,

Create local CentOS repository


If you have to install a lot if machines with CentOS or you just play with kickstart script you should create a local repository to speed the process up. All you need as prerequisite is a configured web server and a script which copies the changes only from the public repository.
This script handles the CentOS and EPEL repository as well.

Here comes the script:
# cat > /usr/local/bin/repo-sync.sh
#!/bin/bash

rsync=/usr/bin/rsync
rsyncopts='--progress -avHSP --bwlimit=512 --delete'
rsyncexcl='--exclude=openoffice*.rpm --exclude=fonts-chinese*.rpm 
--exclude=fonts-japanese*.rpm --exclude=fonts-korean*.rpm 
--exclude=libreoffice*.rpm --exclude=firefox*.rpm 
--exclude=thunderbird*.rpm --exclude=ImageMagick*.rpm --exclude=/SRPMS/ 
--exclude=/headers/ --exclude=*.src.rpm --exclude=*.drpm 
--exclude=*-debuginfo-*.rpm --exclude=/debug/ --exclude=/repoview/'

if [ -f /var/lock/subsys/repo-sync ]; then
    echo "Updates via rsync already running."
    exit 0
fi

mirror=ftp.fsn.hu::linux/centos
verlist="6 5"
archlist="i386 x86_64"
baselist="updates os"
local=/var/www/html/mirror/centos/

for ver in $verlist; do
  for arch in $archlist; do
    for base in $baselist; do
      echo "==================================="
      echo "===== CentOS $ver/$base/$arch ====="
      echo "==================================="
      lrepo=$local/$ver/$base/$arch
      /bin/mkdir $lrepo 2> /dev/null
      /bin/touch /var/lock/subsys/rsync_sync
      remote=$mirror/$ver/$base/$arch/
      $rsync $rsyncopts $rsyncexcl $remote $lrepo
      newpkgs=`/usr/bin/find $lrepo -ctime -1 | wc -l`
      if [ $newpkgs -gt 0 ]; then
       /usr/bin/createrepo $lrepo #>> /dev/null
      fi
    done
  done
done

mirror=ftp.linux.cz::pub/linux/fedora/epel
verlist="6 5"
archlist="i386 x86_64"
baselist="."
local=/var/www/html/mirror/epel/

for ver in $verlist; do
  for arch in $archlist; do
    for base in $baselist; do
      echo "==================================="
      echo "===== EPEL $ver/$base/$arch ====="
      echo "==================================="
      lrepo=$local/$ver/$base/$arch
      /bin/mkdir $lrepo 2> /dev/null
      /bin/touch /var/lock/subsys/rsync_sync
      remote=$mirror/$ver/$base/$arch/
      $rsync $rsyncopts $rsyncexcl $remote $lrepo
      newpkgs=`/usr/bin/find $lrepo -ctime -1 | wc -l`
      if [ $newpkgs -gt 0 ]; then
       /usr/bin/createrepo $lrepo #>> /dev/null
      fi
    done
  done
done

Ctrl-D
Some explanation:
- there are some exclusions at the beginning of the script because I do not need any graphical programs for servers
- you should replace mirror hosts and path with one which is closer to you
- be sure that you choose a mirror which supports rsync protocol. http and/or ftp is not necessary for this script. - you can select which version (verlist) and which architecture (archlist) to sync

Posted by at No comments:
Labels: , , , ,

Tuesday, September 11, 2012

Extract filaname from path with regex

Extracting filname from the full path when the path is OPTIONAL is not so easy as it looks at first look.
I would like to extract cc.cc from
/aa/bb/cc.cc 
either from
cc.cc
as well.

Solution:

$ echo "/aa/bb/cc.cc" | perl -pe "s|(?:.*/)?([^/]*)|\1|"
cc.cc
$ echo "/aa/cc.cc" | perl -pe "s|(?:.*/)?([^/]*)|\1|"
cc.cc
$ echo "cc.cc" | perl -pe "s|(?:.*/)?([^/]*)|\1|"
cc.cc

Some explanation: 
  • ?: start non capturing group
  • [^/] any character except slash
Posted by at No comments:
Labels: , , , , ,
Subscribe to: Posts (Atom)