Using Linux KVM, Expose Virtual Guests On The LAN

Audience: The blog post is for beginner to intermediary Linux system administrators.

• You should have a thorough understanding of the shell commands and
• be comfortable on the command line
• be able to install and configure packages, etc.
• be able to start and stop services using systemd
• be familiar with Linux configuration files
• be able to set Linux kernel parameters using sysctl
• be able to enable and disable Kernel modules
• be comfortable installing and using guest VMs using libvirt

You should have a rudimentary understanding of networking concepts and tooling such as

• The OSI model.
• IP addresses and subnet masks.
• Routing.
• Firewall, iptables, ufw, Firewalld(firewall-cmd).
• HTTP, ARP, DNS, ping, dig, nslookup, etc.
• Familiarity with the ip command is a must.
• You should be familiar with Netplan and nmcli.

We will run web servers on the virtual guests and expose them to the LAN. Any device or computer on the LAN will be able to access the web servers running on the virtual machines.

We have the two host computers on the LAN(Local Area Network):

• linux-host-01 labelled Linux Host 01 having the IP address 192.168.0.5. This host computer has the OS Ubuntu 22.04. The network interface is called enp2s0.
• linux-host-02 labelled Linux Host 02 having the IP address 192.168.0.6. The OS on this computer could be anything.

On the LAN, we have a router and internet gateway having IP address 192.168.0.1.

Virtual Guests:

On Linux Host 01, we have two virtual guests:

• guest-vm-01 labelled Guest VM 01 having IP address 192.168.0.11. The guest VM has the OS Ubuntu 22.04.
• guest-vm-02 labelled Guest VM 02 having IP address 192.168.0.12. The guest VM has the OS CentOS 9 Stream.

Prepare The Host Machine

Install The Linux Virtualization Packages

sudo apt install bridge-utils libvirt-clients libvirt-daemon-driver-qemu libvirt-daemon-system libvirt-daemon  qemu qemu-kvm  virt-manager
sudo systemctl start libvirtd
sudo systemctl enable libvirtd

bridge-utils: This package provides a set of command-line utilities for configuring and managing network bridges on Linux systems. Bridges are used to connect multiple network interfaces together into a single logical network segment.

libvirt-clients: This package provides command-line tools and libraries for interacting with the libvirt virtualization API. It includes utilities for managing virtual machines, storage pools, and networks.

libvirt-daemon-driver-qemu: This package provides the QEMU driver for the libvirt daemon. The libvirt daemon is a system service that manages virtualization resources and provides an API for managing virtual machines.

libvirt-daemon-system: This package provides the system-wide configuration files and service units for the libvirt daemon.

libvirt-daemon: This package provides the libvirt daemon itself, which is responsible for managing virtualization resources on the system.

qemu: QEMU is a generic and open source machine emulator and virtualizer. It provides hardware emulation for a wide range of architectures and devices, and can be used to run virtual machines on a host system.

qemu-kvm: This package provides a kernel module that allows QEMU to use the Kernel-based Virtual Machine (KVM) acceleration feature of modern Linux kernels. This provides faster and more efficient virtualization performance on hardware that supports it.

virt-manager: This is a graphical user interface for managing virtual machines using libvirt. It allows you to create, configure, and launch virtual machines from a user-friendly interface.

Create The Bridge

A bridge operates on the data link layer (layer 2) of the OSI (Open Systems Interconnection) model. The data link layer is responsible for the reliable transfer of data between two nodes on the same physical network. Bridges operate by examining the data link layer address (MAC address) of each incoming packet and forwarding it to the appropriate destination based on the address. This allows bridges to connect multiple physical networks into a single logical network, providing a way for devices on different physical networks to communicate with each other as if they were on the same network.

In our setup, we add the network interface enp2s0 to the bridge br0. Later, we will use the same bridge br0 to connect our virtual machines to the same bridge. That way, we share the physical network interface of the host machine with the guests on layer 2.

Create the Netplan configuration file /etc/netplan/01-netcfg.yaml with the following contents:

network:
  version: 2
  renderer: NetworkManager
  ethernets:
    enp2s0:
      dhcp4: no
  bridges:
    br0:
      interfaces: [enp2s0]
      dhcp4: no
      addresses: [192.168.0.5/24]
      routes:
        - to: 0.0.0.0/0
          via: 192.168.0.1
      nameservers:
        addresses: [192.168.0.1. 8.8.8.8]

sudo netplan apply

This configuration file sets up a bridge interface named br0 that includes the enp2s0 interface. The bridge interface is assigned the IP address 192.168.0.5, and it also specifies the gateway and DNS servers. The Linux host 01 uses this bridge and IPV4 settings. The bridge uses a static IP address and configuration as opposed to DHCP. A route to anywhere, ie 0.0.0.0/0 AKA default is defined on the bridge. All network destinations take this route via the gateway 192.168.0.1.

Note that we are using NetworkManager as the rendered. It could as well be Networkd. Netplan operates on a higher level and can manage both Networkd and NetworkManager.

Configure The Bridge To Bypass Netfilter Subsystem

Enable the br_netfilter kernel module:

cat <<EOF | sudo tee /etc/modules-load.d/br-netfilter-enable.conf
br_netfilter
EOF

sudo modprobe br_netfilter

Create the file /etc/sysctl.d/99-network-bridge.conf and the following contents:

net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

Apply sysctl parameters without reboot

sudo sysctl --system

These lines are sysctl settings or kernel parameters that affect how the Linux kernel handles network packets when they pass through a network bridge.

net.bridge.bridge-nf-call-ip6tables, net.bridge.bridge-nf-call-iptables, and net.bridge.bridge-nf-call-arptables are options that control whether the kernel should pass the packet to the corresponding Netfilter subsystem (in this case, iptables, ip6tables, and arptables) for further processing. If set to 1, the kernel will pass the packet to the Netfilter subsystem. If set to 0, the kernel will not pass the packet to the Netfilter subsystem.

Disabling these options can improve network performance by reducing the amount of processing overhead required by the kernel. However, it can also reduce the security of the system since these Netfilter subsystems provide important functions such as firewalling and packet filtering.

In the context of a virtualization setup like KVM, it is generally recommended to disable these options to improve network performance. We disable this setting to also ensure that a firewall such as iptables/ufw on the host does not matter. We simply bypass iptables as far as our bridge is concerned.

Create The Host Bridge

Create a host-bridge network in KVM using virsh. Create a new XML file named host-bridge.xml with the following contents:

<network>
  <name>host-bridge</name>
  <forward mode="bridge"/>
  <bridge name="br0"/>
</network>

This will define a new network called host-bridge that uses bridged networking with the br0 bridge.

Define The New Network

sudo virsh net-define host-bridge.xml

The virsh net-define command is used to define a new virtual network on a libvirt-based virtualization host. The host-bridge.xml part of the command refers to an XML file that contains the definition of the virtual network.

When you run the sudo virsh net-define host-bridge.xml command, you are instructing the virsh command-line tool to read the host-bridge.xml file and create a new virtual network based on the definition contained within the file.

Once the virtual network has been defined using the virsh net-define command, it can be started or stopped using the virsh net-start and virsh net-stop commands, respectively. Additionally, you can use the virsh net-list command to view a list of all virtual networks defined on the system, including the one you just defined using virsh net-define.

Start The New Network

sudo virsh net-start host-bridge

Ensure The New Network Starts On Boot

sudo virsh net-autostart host-bridge

Now you have successfully created a new host-bridge network in KVM using virsh. You can use this network to connect your guest VMs to the host network.

Create And Configure The Guest Virtual Machines

Prepare Ubuntu 22.04 Guest

Download ISO Image

sudo su
cd /var/lib/libvirt/images
wget https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso

The sudo su command is used to switch the current user to the root user in a Linux or Unix-like operating system. This allows the user to execute commands with administrative privileges, such as installing software or modifying system configurations.

The cd /var/lib/libvirt/images command changes the current working directory to /var/lib/libvirt/images. This is the default location where virtual machine images are stored on a system that uses the libvirt virtualization framework.

The wget command is a command-line utility used to download files from the internet. In this case, the command is being used to download the Ubuntu 22.04.2 Live Server ISO image from the official Ubuntu release website. The URL for the ISO image is https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso.

Install The Guest

qemu-img create -f qcow2 /var/lib/libvirt/images/ubuntu2204test.img 10G
virt-install \
--name guest-vm-01 \
--ram 1024 \
--vcpus 1 \
--location /var/lib/libvirt/images/ubuntu-22.04.2-live-server-amd64.iso,kernel=casper/vmlinuz,initrd=casper/initrd \
--disk path=/var/lib/libvirt/images/ubuntu2204test.qcow2,format=qcow2,bus=virtio \
--network network=host-bridge \
--graphics none \
--noautoconsole \
--os-variant ubuntu22.04 \
--extra-args='console=ttyS0,115200n8 --- console=ttyS0,115200n8'

The qemu-img command is used to create, convert, and modify disk images used by QEMU-based virtualization tools. The create subcommand is used to create a new disk image file, while the -f option is used to specify the format of the disk image.

In this case, the command qemu-img create -f qcow2 /var/lib/libvirt/images/centos9streamtest.img 10G is creating a new disk image file in the qcow2 format at the path /var/lib/libvirt/images/centos9streamtest.img with a size of 10 gigabytes.

The qcow2 format is a popular disk image format used by virtualization tools like QEMU and KVM. It supports features like snapshots and encryption, and provides a more efficient use of disk space than other formats like raw disk images.

After the disk image has been created, it can be used as the storage device for a new virtual machine using a tool like virt-manager. The virtual machine can then be configured to use the disk image as its primary storage device, allowing the guest operating system to be installed and run within the virtual machine.

Once the guest is created, open it in Virtual Machine Manager. Open a terminal in the guest. Configure its networking.

Create the file /etc/netplan/mynet.yaml with the following contents:

network:
  version: 2
  renderer: networkd
  ethernets:
    ens3:
      dhcp4: no
      addresses: [192.168.0.11/24]
      routes:
        - to: 0.0.0.0/0
          via: 192.168.0.1
      nameservers:
        addresses: [192.168.0.1, 8.8.8.8]

Prepare CentOS 9 Stream Guest

Download ISO Image

sudo su
cd /var/lib/libvirt/images
wget 'https://mirrors.centos.org/mirrorlist?path=/9-stream/BaseOS/x86_64/iso/CentOS-Stream-9-latest-x86_64-dvd1.iso&redirect=1&protocol=https' -O CentOS-Stream-x86_64-9-20211212-dvd1.iso

Install The Guest

qemu-img create -f qcow2 /var/lib/libvirt/images/centos9streamtest.img 10G
virt-install \
--name guest-vm-02 \
--ram 1024 \
--vcpus 1 \
--location /var/lib/libvirt/images/CentOS-Stream-x86_64-9-20211212-dvd1.iso \
--disk path=/var/lib/libvirt/images/centos9streamtest.qcow2,format=qcow2,bus=virtio \
--network network=host-bridge \
--graphics none \
--noautoconsole \
--os-variant centos \
--extra-args='console=ttyS0,115200n8 --- console=ttyS0,115200n8'

Once the guest is created, open it in Virtual Machine Manager. Open a terminal in the guest. Configure its networking.

Run the following command to show the available network interfaces:

nmcli device status

Identify the name of the interface you want to configure. In this example, we will use the name ens3.

ip link show

Run the following command to set the interface to use the host bridge:

sudo nmcli connection modify ens3 ipv4.addresses 192.168.0.12/24
sudo nmcli connection modify ens3 ipv4.gateway 192.168.0.1
sudo nmcli connection modify ens3 ipv4.method manual
sudo nmcli connection modify ens3 ipv4.dns "192.168.0.1, 8.8.8.8"
sudo nmcli connection modify ens3 connection.autoconnect yes
sudo nmcli connection up ens3

Restart the networking service:

sudo systemctl restart NetworkManager.service

Testing

• Install Apache web server on both the guests. Put up some unique web pages.
• Access the web servers from host machine 02 via curl:

curl http://192.168.0.11
curl http://192.168.0.12

The networking should also allow connections from host to guest. On Linux host 01, try the same curl command as above. You should see a proper HTTP response.

Guest Network Test

Launch the terminal on the guest and perform some networking checks:

Resolve DNS on the guest:

dig google.com

Reach the Internet

ping google.com

References

• Linux Virtualization
• sysctl
• Automating Virtual Machine Installation Using libvirt, virsh and cloud-init

Resources

• The OSI Model
• iproute2
• NetworkManager
• systemd-networkd
• Netplan

• Linux
• System Administration
• Devops
• Virtualization
• Libvirt
• Kvm

Sudheer Satyanarayana

Veteran DevOps leader who builds real systems and teaches from experience.

Hire me  ·  LinkedIn