Concurrency and parallelism in python

Difference between concurrency and parallelism

The GIL problem is a well know limitation in CPython. Below is on of the video from Heroku conference that shows why this is important (as a bonus you get as well a demo of how to write code in Go language if you want)

The further consequences of this design limitations can be seen in this excellent Mirantis blog post that analyses the performance of an python program: Edge of the Stack: Improve Performance of Python Programs by Restricting Them to a Single CPU.

So what can you do about it? Well until there is GIL in Cpython (and you want or need to stick with this version of python) you may want to chose another library/module for better concurrency support. A long list of available options can be found here: https://wiki.python.org/moin/Concurrency/

At the end to finish up our discussion I can refer you to an practical benchmark that shows a code and do performance analyzes with dealing with concurrency in python: Gevent, Threads, and Benchmarks

How to estimate hardware requirements for your private Openstack deployment

I've found this little tool on the Mirantis web page: http://www.mirantis.com/openstack-services/bom-calculator/. You  can play with it to estimate how much hardware (and money as well :)) you need to build your own private Openstack cloud infrastructure. It gives as well as an overview about potential vendors and theirs hardware.

Simple example

Top 10 data center network vendor list

In previous post was saw how handful the Gartner magic quadrants can be when learning and researching particular technology or industry trend:

• New 2012 Gartner Magic Quadrant for Cloud Infrastructure as a Service (Iaas)
• Rackspace and Brocade pushing ADX platform

Below is an another comparison chart, this time targeting vendor network equipment in data centers.

By the way I was investigating this because of the networks vendor selection on this cloud hardware deployment calculator (How to estimate hardware requirements for your private Openstack deployment ) : Dell, Cisco, HP, Arista, Juniper, Brocade.

New features in OpenStack Grizzly

There is a new Openstack released called Grizzly available. In this post I would like to share some info with you to answer the following questions:

• What changed
• What new features were implemented
• What improvements were made 

Let's summarize first couple of facts we can confirm about Grizzly.

1. For every new release there is a release note (for Grizzly here). As an example links to Nova related releases and blueprints check this Nova release notes and blueprints link.
2. There are 2 webcasts (with video and slides) from Mirantis that discuss changes in Folsom and Grizzly.

The new video and slides from Mirantis for Grizzly are really informative: What’s new in OpenStack Grizzly. Some of the slides I found interesting for Nova and Quantum are copied below.

• Blueprints comparison between Grizzly and Folsom. For the Nova project more detailed info is available here as well: Nova release notes and blueprints.

• In Nova project there were many enhancements for VM placements and improvements for  large scale deployments support

• New awaited features for network virtualization in Quantum project 

• In Folsom there were certain limitation on how to deploy Quantum in a highly available and scalable manner. It was improved as detailed by the slide below and this article: A new agent management approach for Quantum in OpenStack Grizzly.

• There were number of new vendor plugin announcements for Quantum to give people choice how to implement network virtualization. More info about Midokura (here) and Nicira (here) is available as well.

References

1. http://rtomaszewski.blogspot.co.uk/search/label/grizzly
2. http://rtomaszewski.blogspot.co.uk/search/label/folsom

Understanding network architecture in Openstack

With the new Folsom Openstack release we have now access to new networking architectures called Quantum. Even more, network is no longer only an internal part of nova but rather become a new core component [1].

It is going to take some time before Quantum become the main single option for network configuration. Many small to medium deployments will probably never require such a complicated mechanism to manage network creation, ports assignments or IPs allocation.

Problem

• How many network configuration options do we have in Folsom.
• How does the old nova-network work.

Solution

In Folsom we have a choice and we can use either nova-network [1] or quantum [3] service for our  cloud infrastructure.

Depending on the complexity and flexibility we can use the the following configurations in nova-network [2]:

Configuring Flat Networking

Configuring Flat DHCP Networking

Configuring VLAN Networking

This picture shows an example interaction between 2 VMs. More in depth description of how the network flows look like and what role plays the hypervisor in routing or restricting the traffic can be found in [4].

References

1. http://docs.openstack.org/trunk/openstack-compute/admin/content/networking-options.html
2. http://docs.openstack.org/trunk/openstack-compute/admin/content/ch_networking.html
3. http://wiki.openstack.org/Quantum
4. Mirrantis 
http://www.mirantis.com/blog/openstack-networking-flatmanager-and-flatdhcpmanager/
http://www.mirantis.com/blog/openstack-networking-single-host-flatdhcpmanager/"
http://www.mirantis.com/blog/vlanmanager-network-flow-analysis/ 
http://www.mirantis.com/blog/openstack-networking-vlanmanager/
5. IBM OpenStack

What is new in Folsom Openstack release

On the latest Openstack design summit there was a lot of talking about the future roadmap of what the next Openstack release (code name Grizzly) is going to be. It is nice to know where the project is heading to but more importantly it is good to know what new features and improvements we made and coded in Folsom.

These 3 links should give you enough information to start.

1. Mirantis Webcast with a recorded sound and video What’s new in OpenStack Folsom
2. Slides to the video http://www.slideshare.net/mirantis/slideshare-folsom-qawebinar
3. FAQ after presentation OpenStack Cloud Folsom Webcast: Followup on your questions