How to configuring Linux interface so it can send and receive traffic with and without VLAN id in frames.

Often when deploying servers that are used as hypervisors you need to create VLANS on the software switch (on the hypervisor) as well as on the physical witches to isolate traffic between VMs running on different hypervisors.

There need to be as well a management network for the hypervisors itself. This should be isolated from the VM traffic of course.

Below is small example how a network diagram can look like for a setup with 2 hypervisors:

• 1 FW for security control as well as to isolate and route between networks
• 1 hypervisor cluster built out of 2 servers
• 1 server in the inside segment
• 1 server in the dmz segment

FW interface configuration:

FW# sh ip
System IP Addresses:
Interface                Name                   IP address      Subnet mask     Method
Ethernet0/0              outside                1.1.1.1         255.255.255.192 CONFIG
Ethernet0/1              inside                 192.168.100.1   255.255.255.0   manual
Ethernet0/2              dmz                    192.168.99.1    255.255.255.0   manual
Ethernet0/3              mgmt                   10.10.0.1       255.255.255.0   manual
Ethernet0/4.201          vm1                    172.168.1.1     255.255.255.0   manual
Ethernet0/4.202          vm2                    172.168.2.1     255.255.255.0   manual

The management network (10.10.0.0/24)
hyp1 host 10.10.0.2
hyp2 host  10.10.0.3
dmz host  10.10.0.4.
inside host  10.10.0.5

The inside network
inside host 192.168.100.5

The dmz network
dmz host  192.168.99.4

Relevant switch config (some details are not provided to keep it simple):

# sh run int Gi 0/1
interface FastEthernet0/1
 switchport trunk native vlan 300
 switchport trunk allowed vlan 300
 switchport trunk allowed vlan add 100,101,201,202
 switchport mode trunk
 speed 1000
 duplex full
 spanning-tree portfast
end

# sh run int Gi 0/2
 switchport trunk native vlan 300
 switchport trunk allowed vlan 300
 switchport trunk allowed vlan add 201,202,300
 speed 1000
 duplex full
 no cdp enable
 spanning-tree portfast

# sh run int Gi 0/3
 switchport access vlan 100
 switchport mode access
 speed 1000
 duplex full
 no cdp enable
 spanning-tree portfast

# sh run int Gi 0/4
 switchport access vlan 101
 switchport mode access
 speed 1000
 duplex full
 no cdp enable
 spanning-tree portfast

Problem

Each hypervisor have 2 physical interfaces only. Eth0 interface is used for the private VLAN communication. Eth1 should be used for management (VLAN 300) as well as for communication in the vm1 and vm2 networks. That way on a single physical interface we will manage 3 different networks: the untagged management and 2 tagged configured on the hypervisors to isolate traffic between VMs.

How to configure a Linux interface so it can sent and accept traffic with and without VLAN ID tags.

Analisis and results description

A default interface configuration in Linux doesn't use VLAN. From a switch perspective this interface operates in access mode. Once switch receives frames on such interface it uses its VLAN DB configuration to forward it to the next port.

# for the hyp1 server
# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE="eth1"
IPADDR=10.10.0.2
NETMASK=255.255.255.0
NM_CONTROLLED="yes"
ONBOOT="yes"

Base on our switch config above interface Gi 0/2 is configured as trunk with a native VLAN id 300. That means that all incoming frames without VLAN id will be classified and assign to VLAN 300. That way the servers and FW are able to communicate using the IP addresses 10.10.0.0/24 and don't worry about any VLAN tagging at all.

To instruct Linux to sent Ethernet frames that include VLAN id we need to configure following subinterfaces.

# for the hyp1 server

# cat ifcfg-eth1.201
DEVICE=eth0.201
BOOTPROTO=static
ONBOOT=yes
IPADDR=172.168.1.2
NETMASK=255.255.255.0
VLAN=yes

# cat ifcfg-eth1.202
DEVICE=eth0.202
BOOTPROTO=static
ONBOOT=yes
IPADDR=172.168.2.2
NETMASK=255.255.255.0
VLAN=yes

Now when Linux needs to communicate within 172.168.1.0/24 or 172.168.2.0/24 network it uses one of the appropriate interfaces above. All  ethernet frames will be encapsulated using 802.1q standard  The switch will see an ingress traffic encapsulated in 802.1q with VLAN and according to its trunk config it will decapsulation it and distribute it in the right VLAN or drop.

In other words, traffic will traverse down to the switch port Gi 0/2 and base on the trunk configuration the switch should accept it (it will be discarded if the vlan ID doesn't match 20X). Accepted frames will follow standard path inside switch and after CAM table lookup they will be forwarded to next port on the same VLAN base on the destination MAC address.

References

1. https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s2-networkscripts-interfaces_802.1q-vlan-tagging.html
2. https://learningnetwork.cisco.com/thread/8721