Often when deploying servers that are used as hypervisors you need to create VLANS on the software switch (on the hypervisor) as well as on the physical witches to isolate traffic between VMs running on different hypervisors.
There need to be as well a management network for the hypervisors itself. This should be isolated from the VM traffic of course.
Below is small example how a network diagram can look like for a setup with 2 hypervisors:
- 1 FW for security control as well as to isolate and route between networks
- 1 hypervisor cluster built out of 2 servers
- 1 server in the inside segment
- 1 server in the dmz segment
FW interface configuration:
FW# sh ip
System IP Addresses:
Interface Name IP address Subnet mask Method
Ethernet0/0 outside 1.1.1.1 255.255.255.192 CONFIG
Ethernet0/1 inside 192.168.100.1 255.255.255.0 manual
Ethernet0/2 dmz 192.168.99.1 255.255.255.0 manual
Ethernet0/3 mgmt 10.10.0.1 255.255.255.0 manual
Ethernet0/4.201 vm1 172.168.1.1 255.255.255.0 manual
Ethernet0/4.202 vm2 172.168.2.1 255.255.255.0 manual
The management network (10.10.0.0/24)
hyp1 host 10.10.0.2
hyp2 host 10.10.0.3
dmz host 10.10.0.4.
inside host 10.10.0.5
The inside network
inside host 192.168.100.5
The dmz network
dmz host 192.168.99.4
Relevant switch config (some details are not provided to keep it simple):
# sh run int Gi 0/1
interface FastEthernet0/1
switchport trunk native vlan 300
switchport trunk allowed vlan 300
switchport trunk allowed vlan add 100,101,201,202
switchport mode trunk
speed 1000
duplex full
spanning-tree portfast
end
# sh run int Gi 0/2
switchport trunk native vlan 300
switchport trunk allowed vlan 300
switchport trunk allowed vlan add 201,202,300
speed 1000
duplex full
no cdp enable
spanning-tree portfast
# sh run int Gi 0/3
switchport access vlan 100
switchport mode access
speed 1000
duplex full
no cdp enable
spanning-tree portfast
# sh run int Gi 0/4
switchport access vlan 101
switchport mode access
speed 1000
duplex full
no cdp enable
spanning-tree portfast
Problem
Each hypervisor have 2 physical interfaces only. Eth0 interface is used for the private VLAN communication. Eth1 should be used for management (VLAN 300) as well as for communication in the vm1 and vm2 networks. That way on a single physical interface we will manage 3 different networks: the untagged management and 2 tagged configured on the hypervisors to isolate traffic between VMs.
How to configure a Linux interface so it can sent and accept traffic with and without VLAN ID tags.
Analisis and results description
A default interface configuration in Linux doesn't use VLAN. From a switch perspective this interface operates in access mode. Once switch receives frames on such interface it uses its VLAN DB configuration to forward it to the next port.
# for the hyp1 server
# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE="eth1"
IPADDR=10.10.0.2
NETMASK=255.255.255.0
NM_CONTROLLED="yes"
ONBOOT="yes"
Base on our switch config above interface Gi 0/2 is configured as trunk with a
native VLAN id 300. That means that all incoming frames without VLAN id will be classified and assign to VLAN 300. That way the servers and FW are able to communicate using the IP addresses 10.10.0.0/24 and don't worry about any VLAN tagging at all.
To instruct Linux to sent Ethernet frames that include VLAN id we need to configure following subinterfaces.
# for the hyp1 server
# cat ifcfg-eth1.201
DEVICE=eth0.201
BOOTPROTO=static
ONBOOT=yes
IPADDR=172.168.1.2
NETMASK=255.255.255.0
VLAN=yes
# cat ifcfg-eth1.202
DEVICE=eth0.202
BOOTPROTO=static
ONBOOT=yes
IPADDR=172.168.2.2
NETMASK=255.255.255.0
VLAN=yes
Now when Linux needs to communicate within 172.168.1.0/24 or 172.168.2.0/24 network it uses one of the appropriate interfaces above. All ethernet frames will be encapsulated using 802.1q standard The switch will see an ingress traffic encapsulated in 802.1q with VLAN and according to its trunk config it will decapsulation it and distribute it in the right VLAN or drop.
In other words, traffic will traverse down to the switch port Gi 0/2 and base on the trunk configuration the switch should accept it (it will be discarded if the vlan ID doesn't match 20X). Accepted frames will follow standard path inside switch and after
CAM table lookup they will be forwarded to next port on the same VLAN base on the destination MAC address.
References
- https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s2-networkscripts-interfaces_802.1q-vlan-tagging.html
- https://learningnetwork.cisco.com/thread/8721
Posts
