Search This Blog

Showing posts with label hypervisor. Show all posts
Showing posts with label hypervisor. Show all posts

Sunday, January 5, 2014

Nested virtualization support on Rackspace public cloud

We have found the CPU hardware architecture that the public cloud is running on in this blog: Hypervisor hardware differences on Openstack Rackspace Cloud.

Problem

Does Rackspace public cloud support nested visualization?

Results discussion
  • Public Cloud
Of course for the cloud to exists the physical sever where the hypervisor runs (Xen or KVM for example) needs to have in-hardware virtualization support (Intel VT-x or AMD-V). This is the only way to provide a high performance cloud servers.

But once the cloud server boots up the cloud virtual CPU no longer exports the hardware CPU virtualization capabilities. You can verify this with this little script below.

egrep -i 'vmx|svm|ept|vpid|npt|tpr_shadow|flexpriority|vnmi'

That means you can't use your cloud server to run another, a guest hypervisor ( called as well nested hypervisor).
  • Private cloud 
The nested virtualization can be enabled. As an example this link describe some of the steps for Linux KVM:
Another solution

If your cloud server doesn't offer nested virtualization support you can always use the emulation mode. Qemu supports running VM that way.


References

http://www.ibm.com/developerworks/cloud/library/cl-nestedvirtualization/
https://www.diigo.com/user/rtomaszewski/nested_virtualization?type=all&snapshot=no&sort=updated
http://en.wikipedia.org/wiki/X86_virtualization

Hypervisor hardware differences on Openstack Rackspace Cloud

You can spin up test cloud servers and extract CPU flags with the help of this little script using csplit.
 
cat /proc/cpuinfo | csplit -z  - '/processor/' '{*}'
diff xx0*
grep flags xx01 | cut -d ':' -f 2 | xargs -n1 echo | sort > flags.txt

By comparing the results we can definitely say that:
  • Performance1 and performance2 cloud servers are running on the same hardware.
  • New performance cloud servers are hosted on Intel CPU.
  • The standard (next generation) series are being hosted on AMD CPU.
The CPU flags for comparison.
 
processor       : 1
vendor_id       : AuthenticAMD
cpu family      : 16
model           : 4
model name      : Quad-Core AMD Opteron(tm) Processor 2374 HE
stepping        : 2
microcode       : 0x1000086
cpu MHz         : 2200.096
cache size      : 512 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu de tsc msr pae cx8 cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow rep_good nopl pni cx16 popcnt hypervisor lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch hw_pstate
bogomips        : 4400.19
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping        : 7
microcode       : 0x70d
cpu MHz         : 2600.068
cache size      : 20480 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 43
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt tsc_deadline_timer aes hypervisor lahf_lm arat pln pts dtherm
bogomips        : 5200.13
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

Sunday, June 9, 2013

SR-IOV technology enables low level network virtualization

In the virtualization space the SR-IOV technology was introduced in about 2008/2010 [1]. The technical details can be found under the links in the reference section but in plain English the technology allows to create many virtual devices base on single a physical device. For this to work the hardware (CPU, North chip) and operating system need to have SR-IOV support.

Below is a video demonstrating packet processing for an Intel Ethernet card that supports SR-IOV.


Interesting slides showing the concept from the video and  reference links:
  • After the frame enters the physical port on the NIC the low level driver/firmware (supporting the SR-IOV) distributes the packet (based on header classifications/hash value/etc) to separate virtual queues 
  • Each virtual queue is assigned directly to a virtual device 
  • Once the packet is in the queue it can be deliver to the VM DIRECTLY without the usual software hypervisor overhead
  • Packets don't have to be copied from physical port buffer(s) to OS RAM and than from OS RAM to VM OS buffers. The data can be sent directly from the physical port to VM OS buffers. That way the hypervisor processing overhead can be minimised.
  • Critical part for the technology is CPU and chipset virtualization support
  • As access to physical RAM need to be protected between hypervisor and VMs as well as VMs themselves the virtual memory address is translated to the physical location by the north chipset
  • For the DMA request to copy the packets the address translation between the hypervisor address space and the VM address space is transparent (north chip take care of it)
  • Another view how the packet is delivered from physical port to the VM
References
  1. http://www.intel.com/content/dam/doc/application-note/pci-sig-sr-iov-primer-sr-iov-technology-paper.pdf
  2. http://www.intel.com/content/dam/doc/white-paper/pci-sig-single-root-io-virtualization-support-in-virtualization-technology-for-connectivity-paper.pdf
  3. http://communities.intel.com/community/wired/blog/2010/09/07/sr-iov-explained

Friday, March 22, 2013

Openstack deployment options

Openstack wants to be as flexible as possible. It means that each service that is developed under the umbrella of Openstack has to be written in a modular way to accept different backend system depending on user preferences. From a high level point of view it means that a service need to have a well defined internal and external API, and the more specific technical implementation details are left for backend systems.

The Openstack specific public (external) API is one milestone for every project. If the API is not reach, flexible and useful enough it will not empower uses when consuming the service. On the other side, if the internal service level API is badly design it may cause issues for example when integrating with other components, exchanging messages, causing bottlenecks or hinder vertical scalability to expand system capacity.

Below is a list of deployment options for Folsom and Grizzly Openstack release.
  • External API
    •  XML
    •  JSON
  • Possible hypervisors for OpenStack Compute
    • KVM
    • Xen
    • Citrix XenServer
    • Microsoft HyperV
    • VMware ESX
    • LXC
  • Possible OpenStack Block Storage drivers 
    • Coraid
    • EMC
    • GlusterFS
    • Huawei
    • LVM
    • NetApp
    • Nexenta
    • NFS
    • Ceph RBD
    • SAN/HP
    • SAN/Solaris
    • Scality
    • Sheepdog
    • SolidFire
    • Storwize
    • Windows
    • Xenapi
    • XIV
    • Zadara
  • Possible OpenStack Network backends 
    • Big Switch
    • Brocade
    • Cisco
    • Hyper-V
    • Linux Bridge
    • MidoNet
    • NEC
    • Nicira
    • Open vSwitch
    • PLUMgrid
    • Ryu
  • OpenStack Identity drivers
    • LDAP
    • SQL
    • PAM
    • KVS