CentOS6 KVM with Shorewall

I wanted to install a CentOS6 host and some vitrual machines in it but I have only one public ip address so I cannot use the most common method to make a bridge as public interface.
There are a lot of guides describing how to install KVM with bridged public interface but just a few of them mentioned my situation.


By default there is just a simple firewall configuration tool on CentOS. I am not an iptables guru that is why I started using shorewall as firewall configuration tool long time ago and I am very satisfied with it..

According to libvirt's documentation when a libvirt manageded network bridge starts, some iptables rules applies and it can confuse administrators when there are two iptables configuration tool working on the same time.
It look like it is the time to set up the virtual network manually...

In shorewall there will be four defined zones:
- net: Internet on host's eth0 interface
- fw: the host computer itself
- dmz: virtual machines with configured portfowards from public ip (public services), interface: virbr0
- virt: virtual machines without portforwarding (testing purpose), interface: virbr1

1. First of all after installing libvirt stop and disable autostarting default network:

# virsh net-destroy default
Network default destroyed

# virsh net-autostart default --disable
Network default unmarked as autostarted

# virsh net-list --all
Name                 State      Autostart
-----------------------------------------
default              inactive   no

2. Create bridge manually, aka let the os network configuration scripts make them

# cat >/etc/sysconfig/network-scripts/ifcfg-virbr0 <<EOF
DEVICE=virbr0
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.233.254
NETMASK=255.255.255.0
DELAY=0
STP=off
EOF
# cat >/etc/sysconfig/network-scripts/ifcfg-virbr1 <<EOF
DEVICE=virbr1
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.244.254
NETMASK=255.255.255.0
DELAY=0
STP=off
EOF
# service network restart

3. If you would like to have a DHCP server on these subnets, configure and start dnsmasq as well:

# cat >/etc/dnsmasq.conf <<EOF
domain=example.com
domain=dmz.example.com,192.68.233.90,192.168.233.99
dhcp-range=dmz,192.168.233.90,192.168.233.99, 12h
dhcp-option=dmz,15,dmz.example.com
domain=virt.example.com,192.68.244.90,192.168.244.99
dhcp-range=virt,192.168.244.90,192.168.244.99, 12h
dhcp-option=virt,15,virt.example.com

bind-interfaces
domain-needed
bogus-priv
resolv-file=/etc/dnsmasq.d/resolv.conf
strict-order
listen-address=192.168.233.254
log-facility=/var/log/dnsmasq.log
#log-queries
dns-forward-max=150
cache-size=10000
EOF
# cat >/etc/dnsmasq.d/resolv.conf <<EOF
nameserver 8.8.8.8
nameserver 8.8.4.4
EOF
# chkconfig dnsmasq on
# service dnsmasq start

4. Ensure that you have one-interface config installed in /etc/shorewall.
Update: one interface config is the default, so you can skip this step.

# cd /usr/share/doc/shorewall-4.*/Samples/one-interface
# cp interfaces policy rules shorewall.conf zones /etc/shorewall

5. Append virtual zones definitions to zones file:

# TAB="$(printf '\t')"
# cat >>/etc/shorewall/zones <<EOF
virt${TAB}ipv4
dmz${TAB}ipv4
EOF

6. Append interfaces definitions to interfaces file:

# TAB="$(printf '\t')"
# cat >>/etc/shorewall/interfaces <<EOF
net${TAB}${TAB}eth0${TAB}${TAB}${TAB}dhcp,tcpflags,logmartians,nosmurfs
dmz${TAB}${TAB}virbr0${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback,bridge
dmz${TAB}${TAB}vnet0${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback
virt${TAB}${TAB}virbr1${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback,bridge
virt${TAB}${TAB}vnet1${TAB}${TAB}${TAB}dhcp,tcpflags,nosmurfs,routeback
EOF

7. Rewrite policy definitions according to your needs. For example:

# TAB="$(printf '\t')"
# cat >>/etc/shorewall/policy <<EOF
# Connecto from firewall
\$FW${TAB}ALL${TAB}ACCEPT

# Connect from DMZ machines

dmz${TAB}dmz${TAB}ACCEPT
dmz${TAB}net${TAB}ACCEPT
dmz${TAB}\$FW${TAB}DROP${TAB}${TAB}info

# Connect from VIRT machines
virt${TAB}virt${TAB}ACCEPT
virt${TAB}all${TAB}DROP${TAB}${TAB}info

# Connect from net
net${TAB}all${TAB}DROP${TAB}${TAB}info

# The FOLLOWING POLICY MUST BE LAST
all${TAB}all${TAB}REJECT${TAB}${TAB}info
EOF

8. You should re-configure your sshd on the host to listen on different port and define an alternate SSH macro in shorewall: Warning! After ths step your host's sshd will listen on non default port! Be sure, that you can reach it on this different ports!

# sed -i "s/#Port 22/Port 39022/" /etc/ssh/sshd_config
# service sshd restart
# TAB="$(printf '\t')"
# cat >/etc/shorewall/macro.SSHAlt <<EOF
###############################################################################
#ACTION SOURCE  DEST    PROTO   DEST    SOURCE  RATE    USER
#                               PORT(S) PORT(S) LIMIT   GROUP
PARAM${TAB}-${TAB}-${TAB}tcp${TAB}39022

EOF

9. Define an admin host in params file

# cat >>/etc/shorewall/params <<EOF
ADMIN_HOSTS=11.22.33.44,11.22.66.44
DMZ_GUEST1=192.168.233.1
EOF

10. Enabe masquerading from dmz zone.

echo -e "eth0\t\t\t192.168.233.0/24" >>/etc/shorewall/masq

11. Append or rewrite rules file according to your configuration. For example:

# TAB="$(printf '\t')"
# cat >>/etc/shorewall/rules <<EOF
VNC(ACCEPT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}\$FW
SSHAlt(ACCEPT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}\$FW
SSH(DNAT)${TAB}${TAB}net:\$ADMIN_HOSTS${TAB}${TAB}dmz:\$DMZ_GUEST1
EOF

This configuration let the admin connect from ADMIN_HOSTS to the:

• host on 39022 port (ssh) 
• host on 5900-5999 (vnc) 
• DMZ_GUEST1 virtual machine on port 22 (ssh)

12. Enable shorewall startup:

# sed -i "s/startup=0/startup=1/" /etc/sysconfig/shorewall

13. Start shorewall

# shorewall start

14. Create your virtual machine using

• --network bridge:virbr0 if you want to connect it to dmz network
• --network bridge:virbr1 to connect it to virt network.

15. Enjoy!