Midonet packet processing

The few slides below from Midokura account on slideshare.net show the internal architecture design and packet processing that is happening whey you deploy Midonet as a cloud virtualization engine.

• Midonet is based on an IP overlay concept 

• By implementing in IP encapsulation it can provide tenant isolation and more advance features

•  By pushing the intelligence to the edge of the network (like hypervisors) it can handle the packet forwarding efficiently with out having to consult the with any external system
• All Midonet processes built a single logical distributed system within each single Midonet node is capable of finding and implementing the right action for a packet

•  As an example if VM tries to communicate with a peer that doesn't belong to the tenant the verification can be done at the edge without having to sent the packet out

References

1. http://www.slideshare.net/midokura/12th-japan-cloudstack-user-group-meetup
2. http://www.slideshare.net/midokura/cloudstack-collab-talk
3. http://www.slideshare.net/midokura/midonet-us-launch-oct-15-14706786

Midokura Midonet software

The network virtualization movement is getting bigger and stronger. Looking how serious the VMware is looking into SDN concept ( VMware NSX Network Virtualization ) is is only a matter of time before we start seeing this on a regular basis in data centers.

With this message in mind let's take a look at one of the SDN vendors like Midokura and its software Midonet.

1. General availability 

During Openstack Summit in April 2013 Midokura sent a strong message that its Midonet software is available to download. Midonet is Midokura the SDN implementation of the cloud network virtualization concept.


2. Software

At the moment there is only a little bit of information what it is.

MidoNet pushes intelligence to the edge of the network, as it takes an overlay-based approach to network virtualization and sits on top of any IP-connected network. [1]

Its technology is functionally similar to VMware-backed Nicira's, but the approach is different: Midokura has a Level 3 network gateway, whereas Nicira is Level 2. Both companies offer distributed switching at Level 2 [2]

Midokura uses a continuous-licensing basis for its network virtualization software. The technology is a 5-to-10MB download that runs on top of a JVM on standard server hardware. [2]


3. Midonet network architecture
Below, on the left we see the logical and on the right the physical topology.



The main logical concept is based on having a virtual router that your VMs are talking to. As Midonet software is using/is based on Open vSwitch that pacifically means that there is going to be a virtual port attached to Virtual router and VM in Open vSwitch. That way the VM is directly (virtualy) connected to its virtual router / default gateway. The building layout of the physical topology seems to confirms that assumptions.

The physical topology help us better understanding the Midonet distributed model as well: Midonet software architecture.

The backend network is a standard IP base physical network infrastructure that aims to provide IP connectivity between the Midonet nodes. The power of Midonet comes from the way it manages its distributed NW State DB (network state data base). This is the critical part and place where is decided what to do with an Ethernet frame/IP packet from or to VM when there is no flow entry on the routers or hypervisor virtual switches.

This is the link for a full Midokura Midonet presentation (in Japan).

References

1. http://www.midokura.com/press-releases/midonet_launch_at_openstacksummit2013/
2. http://www.theregister.co.uk/2013/04/02/midokura_sdn_funding/

Network virtualization jargon

I've read recently this blog post Virtual Network Domain and Network Abstractions. In the post PlumGrid  CTO describes challenges and opportunities that currently exists in cloud network. In my opinion what makes it unique and interesting in comparison to other blogs concentrating on network virtualization is the language he uses.

Below are some of the examples:

• Distributed Virtual Switch (DVS)
• Physical Network Infrastructure (PNI)
• Virtual Broadcast Domains (aka Distributed VLANs) 
• Virtual Network Infrastructure (VNI)
• Virtual Network Domains (VND)

Do you believe that these concepts are more understandable when we speak about networking in the cloud. Typically described today as network virtualization, virtual network, cloud network, hybrid network or private network.

References

1. http://umairhoodbhoy.net/2013/02/12/what-is-plumgrid-up-to/
2. http://en.wikipedia.org/wiki/Network_virtualization

Architecture frameworks

Did you see big systems implementations?
Did you participated in a system implementation or deployment?
What is your current role?
What is your carrier goal?

Every company is organised in a different way. Every company is build in a different way. But it all starts from a vision that is going to be mapped into an architecture. Once the concept or the first sketches of the architecture are established the implementation process is going to follow.

It is an example how it can look like. If you are interested in other options it is worth to take a look at some of the established (enterprise) architecture frameworks. Generally speaking, it is rather a quite difficult document to read as it tries to address the issue of an enterprise company. But even though its complexities it still gives a good overview how complex some processes, implementations or deployments can be. Below is a table from FEAF framework that list key roles and teams for an organisation.

This one below shows the concept of an architecture from a high level point of view.

The more detailed description of the architecture domains can be found here. Personally, I'm specializing in the area of technical infrastructure architecture. With the specialization of  network virtualization and cloud network in IaaS.

Technical architecture or infrastructure architecture: The structure and behaviour of the technology infrastructure. Covers the client and server nodes of the hardware configuration, the infrastructure applications that run on them, the infrastructure services they offer to applications, the protocols and networks that connect applications and nodes.

References

1. http://en.wikipedia.org/wiki/Solution_architecture
2. http://en.wikipedia.org/wiki/Architecture_domain
3. http://en.wikipedia.org/wiki/TOGAF
4. http://pubs.opengroup.org/architecture/togaf8-doc/arch/
5. http://en.wikipedia.org/wiki/Multitier_architecture
6. http://en.wikipedia.org/wiki/Peer-to-peer

Openssl cheat sheet

How to extract certificates

Usual certificate files have not only the certificate itself but include the chain as well.

# example chain cert file

-----BEGIN CERTIFICATE-----
687f687asfafaufyaufasfyfyasifyayfafvG74WlTANBgkqhkiG9w0BAQUFADBm
678fa6auyasfyasf8a7fa9f7a9sfiauy987safasffQgSW5jMRkwFwYDVQQLExB3
....
UW9iatnbVzOcOdJJaBK7obGALVFBAQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
76af5at7fat8f7astfuyasftuftauftaufasfasfOzANBgkqhkiG9w0BAQUFADBs
....
tfiuytaiutauisftiaustfiuasftuiastfasf2oWGU4K8K2Eyl2Us1p292E=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
tfuastfuiatfutfuiatfiuatsifuastifuayrkYldzANBgkqhkiG9w0BAQUFADBs
....
tuaftiautfuaftiuatfiuastfuastfiuastfiastfuatsifutafutafutasuftap
+OkuE6N36B9K
-----END CERTIFICATE-----

To extract each certificate and save it in a separate file you can use this little tick.

 
root@server:~# csplit -k cert.txt '%-----BEGIN CERTIFICATE-----%' '/-----END CERTIFICATE-----/+1' {9}
2362
2260
1367
csplit: `/-----END CERTIFICATE-----/+1': match not found on repetition 3
1

root@server:~# ll 
-rw-r--r-- 1 root root 2362 May 21 16:50 cert.txt
-rw-r--r-- 1 root root 2362 May 21 16:50 xx00
-rw-r--r-- 1 root root 2260 May 21 16:50 xx01
-rw-r--r-- 1 root root 1367 May 21 16:50 xx02
-rw-r--r-- 1 root root    1 May 21 16:50 xx03

# because it is irrelevant
root@server:~# rm xx03 

How to verify that the certificate and key belong together

 
$ openssl x509 -noout -modulus -in server.crt | openssl md5
$ openssl rsa -noout -modulus -in server.key | openssl md5

References

1. https://kb.wisc.edu/middleware/page.php?id=4064

How do I control vm placement in Openstack cluster

If you have built your owns Openstack cluster than it is up to you as well how do you tune and configure it to meet your requirements.

If you use your cluster primarily to spin up and spin down VM you need often some level of control where the VM are going to be built. In the latest Openstack release

There are to options should look at: availability zones and host aggregate filter.

Host aggregates filter

It was introduced in Essex (Release Note, all Essex blueprints)

• Host aggregates (direct link to blueprint) 

It was next modified and extended in Folsom as we can see this in the Folsom Release Notes and Folsom blueprints here:

• General Host Aggregates were implemented, allowing metadata to be set dynamically for hosts and pased to the scheduler general-host-aggregates
• Lots of improvements were made to the scheduler, Including filters for image architecture and scheduling based on capabilities from aggregates. See doc.

Going through the Grizzly Release note and the Grizzly blueprint I see only these blueprints that are relevant:

• host aggregate based availability zones
• show all availibility_zone in a region and show availability_zone field in instance detail information

Availability zones

I didn't have a chance to investigate where this feature was introduced and how was it implemented  For these who are interested this link should give a good starting point: Nova release notes and blueprints

Example

Below are some links to examples with nova commands to show how the both feature work.

Controlling volume and instance placement in OpenStack (take 1)
Controlling volume and instance placement in OpenStack (take 2)
Controlling where instances run (Openstack compute administration guide - grizzly)
How do I group compute nodes to control workload placement? (ask openstack forum)
http://devstack.org/exercises/aggregates.sh.html

Nova release notes and blueprints

To navigate between various systems and versions in Openstack you need understand the concept of blueprints and how they link to the source code.

For every project that is developed under the umbrella of Openstack there is a list of features people are working on. We call them blueprints and they are tracked in a written form on launchpad.

Release notes and blueprints for Openstack Nova project

All Nova blueprints.

Blueprints sorted by the Nova release.

Essex: Release Note, Blueprints
Folsom: Release Note, Blueprints
Grizzly: Release Note, Blueprints

Searching and matching

This isn't a simple task when you want to track down in reverse order what was implemented.
One way of doing this is to search for the blueprint on the blueprint list or in the release note and then browse through the Gerrit code reviews.

In the example below you can see the blueprint name and a link (see the yellow text). Unfortunately the name used in Gerrit and the links are not always pointing directly to each other.

The Project and branch (red color) are not links to the code with the changes but rather indication of the name for project and branch at the time this review was going on.

If you want to download or checkout the code to play with it or test you need to scroll down the the 'Download' section. Alternatively you can see the diff and changes in browser.

https://review.openstack.org/#/c/3035/

Change Iceb27609: blueprint host-aggregates

Change-Id: Iceb27609dc53bf4305c02d7cbc436fba4c4a7256 Owner Armando Migliaccio Project openstack/nova Branch master Topic bp/host-aggregates Uploaded Jan 13, 2012 5:05 PM Updated Jan 17, 2012 9:37 PM Status Merged Permalink

blueprint host-aggregates This is the first of a series of commits that add the host-aggregates capability, as described on the blueprint page. This commit, more precisely, introduces changes to the Nova model: model classes related to aggregates have been added, as well as DB API methods to interact with the model; a sqlalchemy migration script plus a bunch of tests are also part of this changeset. Commits that will follow are going to add: - Extensions to OSAPI Admin, and related python_novaclient mappings - Implementation of the XenAPI virt layer - Integration of OSAPI and virt layer, via the compute_api - smoketests - openstack-manuals documentation These commits will be pushed for review not necessarily in this exact order. Change-Id: Iceb27609dc53bf4305c02d7cbc436fba4c4a7256

Example 'Download' section.

Download

checkoutpullcherry-pickpatch Anonymous HTTP git fetch https://review.openstack.org/openstack/nova refs/changes/35/3035/8 && git checkout FETCH_HEAD