Edited: 2022-06-30, added capability to block multiple subnets.

First things first!

1
2

mkdir -p ~/Bench/libvirt-nwfilter-test
cd ~/Bench/libvirt-nwfilter-test

Preface

Until I learn how to tame VXLAN for a virtualization cluster, I need a dirty way of seperating infrastructure network and the VM network.

The solution presented in this post is to apply libvirt’s network filters (nwfilter) to drop packages from and to the “uplink”, namely home LAN (10.0.0.0/18), for any guest connected to the NAT (192.168.123.0/24). This approach is analogous to setting firewall rules on a router, it just is virtual. Remember to adjust your subnets!

Disclaimer: I make use of sudo while running virsh this may not be necessary if you have added yourself to proper groups, I did not bother.If so, kindly delete them before executing.

Pitfalls

Firstly, make sure you don’t make a typo unlike me (looking at you “<filterref type=‘drop-subnet’>”).

Secondly, make use of virsh dumpxml (or virt-manager’s XML tab under NIC) to validate your filter was indeed applied to VM definition.

Lastly, ensure you have a single filter applied per NIC (at least I could not apply two of them). Define an umbrella filter which includes all rules and other filters.

Shortcomings

I suppose, given enough eagerness, one could find a way of proxying the connection between the two subnets, namely “home LAN” and “guest NAT”.

One such case would be using the public IP address of a DMZ host, however, one would assume the DMZ host to be already hardened. Yet, be warned to think of such cases before implementing your internet-connected virus test lab.

Virtual NAT Firewall (nwfilter)

Let us define a parametrized filter, named drop-subnet, to be tested shortly after.

nwfilter-drop-subnet.xml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

<filter name='drop-subnet' chain='ipv4' priority='-700'>
  <!-- For spoofing prevention, a default filter -->
  <filterref filter='clean-traffic'/>

  <!-- Drop packages originating from the subnet -->
  <rule action='drop' direction='in' priority='500'>
    <all match='yes' srcipaddr='$SUBNET_IP[@1]' srcipmask='$SUBNET_MASK[@1]'/>
  </rule>

  <!-- Drop packages sent to the subnet -->
  <rule action='drop' direction='out' prority='500'>
    <all match='yes' dstipaddr='$SUBNET_IP[@1]' dstipmask='$SUBNET_MASK[@1]'/>
  </rule>
</filter>

Define the filter:

1

sudo virsh nwfilter-define nwfilter-drop-subnet.xml

For more details see chapter nwfilter concepts, better read it all tough.

Preparations

1. Define a seperate NAT network to be filtered

I chose to disable DHCP and DNS as I would be using static IPs and enforcing against spoofing using. See disabling DNS in Libvirt, disabling DHCP in Libvirt and especially virtual network docs for more.

net-nat-filtered.xml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

<network>
  <name>nat-filtered</name>
  <forward dev='enp1s0' mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
    <interface dev='enp1s0'/>
  </forward>
  <bridge name='virbr-filtered' stp='on' delay='0'/>
  <mac address='52:54:CB:42:00:00'/>
  <dns enable='no'/>
  <ip address='192.168.123.1' netmask='255.255.255.0'>
    <!--
    <dhcp>
      <range start='192.168.123.100' end='192.168.123.254'/>
      <host mac="52:54:CB:42:00:02" name="guest-02" ip="192.168.123.12"/>
    </dhcp>
    -->
  </ip>
</network>

You can omit <mac> and <nat> as well as commented dhcp example. Remember to change enp1s0 into what your physical interface or bridge is. Could be eth1, enp3s0, br0, etc.

Now let us import the XML:

1

sudo virsh net-define net-nat-filtered.xml

Lastly, activate and enable it:

1
2

sudo virsh net-start nat-filtered
sudo virsh net-autostart nat-filtered

P.S. Do not to use KVM’s default (192.168.122.0/24) or any other existing network’s IP block.

2. Create a VM

It is assumed that you prepared a VM connected to the network nat-filtered with name guest-01 and IP 192.168.123.11.

If in need of assistance, can follow Server World’s tutorials (select OS of your choice at Other OS Configs). I spun one up with GUI application virt-manager.

Now, one could make use of either of the below:

  1. virsh edit
  2. virsh dumpxml followed by virsh define

I will go for the latter out of personal preference (seems easier to script if need ever arises).

1

sudo virsh dumpxml guest-01 > vm-guest-01.xml

Below use of ... is to denote numerous lines irrelevant to the task at hand.

vm-guest-01.xml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

<domain type='kvm' id='2'>
  <name>guest-01</name>
  ...
  <devices>
    ...
    <interface type='network'>
      ... (this is where we are intrested)
    </interface>
    ...
  </devices>
  ...
</domain>

Your interface type might be a bridge or ethernet (e.g. <interface type='bridge'>) as described in nwfilter concepts, it is irrelevant.

Now make the addendum into existing <interface>:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

    <interface type='network'>
      ...
      <filterref filter='drop-subnet'>
        <-- Used by filter "clean-traffic" -->
        <parameter name='IP'          value='192.168.123.11'/>
        <-- For blacklisting target subnet -->
        <parameter name='SUBNET_IP'   value='10.0.0.0'/>
        <parameter name='SUBNET_MASK' value='255.255.192.0'/>
	<-- Yet another subnet to blacklist, Cloudflare's DNS server in particular-->
        <parameter name='SUBNET_IP'   value='1.1.1.1'/>
        <parameter name='SUBNET_MASK' value='255.255.255.255'/>
      </filterref>
      ...
    </interface>

See my answer to “libvirt nwfilter, multiple parameters” for more insight.

At this point, shut down the VM if already running:

1

sudo virsh shutdown guest-01

Then, (re)define VM using the xml:

1

sudo virsh define vm-guest-01.xml

Start it to start testing:

1

sudo virsh start guest-01

Testing

Connect to guest-01 one way or another. Below is accessing serial console from the hypervisor:

1

sudo virsh console guest-01

Test NAT-local connectivity, should work:

1

ping 192.168.123.1

Test internet connectivity, should work unless DNS is 1.1.1.1:

1

ping blog.cbugk.com

Test connectivity to blacklisted subnet (home LAN: 10.0.0.0/18), should not work:

1

ping 10.0.0.1  # or any host on the IP block

Test DNS at 1.1.1.1/32:

1
2

dig blog.cbugk.com @8.8.8.8 # should work
dig blog.cbugk.com @1.1.1.1 # should NOT work

That’s it.

If you wish to test spoofing as well, change the IP address of guest-01 and try pinging the NAT-local gateway.

“This does not float my boat!”

If this approach isn’t suitable, you might try setting up an OPNSense/ VyOS/ IPFire guest with multiple NICs to act as a properly virtualized router/ firewall OS. This would put a bit more overhead, but might be more apt for more complicated requirements. Of course an isolated network must be used in such a case. To refresh your color-coded zone knowlegde: IPFire’s color table (red, green, orange, blue).

Cheers !