Concurrency and parallelism in python

Difference between concurrency and parallelism

The GIL problem is a well know limitation in CPython. Below is on of the video from Heroku conference that shows why this is important (as a bonus you get as well a demo of how to write code in Go language if you want)

The further consequences of this design limitations can be seen in this excellent Mirantis blog post that analyses the performance of an python program: Edge of the Stack: Improve Performance of Python Programs by Restricting Them to a Single CPU.

So what can you do about it? Well until there is GIL in Cpython (and you want or need to stick with this version of python) you may want to chose another library/module for better concurrency support. A long list of available options can be found here: https://wiki.python.org/moin/Concurrency/

At the end to finish up our discussion I can refer you to an practical benchmark that shows a code and do performance analyzes with dealing with concurrency in python: Gevent, Threads, and Benchmarks

Network operating system architecture

In my previous posts we took a look at how hardware networking vendors design and build modern systems. We concentrated mainly on the OS and software integration on the appliance:

• Arista network operating system switch architecture
• Network appliance architecture

Today I found another good article on Arista blog that takes this one step further and again advocates an open operating system design: Linux as a Switch Operating System: Five Lessons Learned. This is what Arista says:

• It’s okay to leave the door unlocked - get net admin access to the underlying Linux operating system tools
• Preserve the integrity of the Linux core - keep your product specific changes as small as possible to allow integration with already existing software out there
• Focus on state, not messages - being only maybe an average programmer I will not comment on this as this is clearly not my area of expertise
• Keep your hands out of the kernel - why to complicate your code if it doesn't bring any revolutionary benefit; besides it is much easier to find developers who know how to program in linux/libc than in Linux kernel.
• Provide familiar interfaces to ease adoption - everyone knows the IOS CLI so why to invent something new.

Lesson learned from Arista: reuse, integrate and trust your customers.

The company success would not be possible without a clear and consistent OS and tools set design. From my experience they are the second company (I saw this first on BigIP - F5 load balancers).

Network appliance architecture

Networking is a fascinating topic. There are thousands of books and RFC describing protocols from layer 1 to layer 7. In every big network we find variety of network devices that handle traffic and provide additional enhanced services. Example of such devices can be switches, routers, load balancers, traffic accelerators, firewalls, IDS, DDOS mitigation devices and others. Services could be QoS, security and traffic deduplication etc.

Looking at the network devices someone could ask one interesting questions: how to build a network device; how would you describe an internal device architecture?

Of course every company has its own patents, secrets, methods etc. how they built, manage and operate network appliances. Below are some information I found when working with them or reading about them.

• Arista network operating system switch architecture
• Dell DNOS or FTOS architecture
• F5 Network has its Linux based TMOS architecture (link1/link2)

How to understand Apache MPM processes during the starting phase

One of the great flexibility that comes with Apache HTTP server is its incredible modularity. But sometimes it take a little time and experimenting before you understand how your favourite MPM model works.

So let's take a look how many processes there are going to be created when we start Apache after not customized installation?

All descriptions and examples are based on the default configuration in the Ubuntu:

# cat /etc/lsb-release

DISTRIB_ID=Ubuntu

DISTRIB_RELEASE=10.04

DISTRIB_CODENAME=lucid

DISTRIB_DESCRIPTION="Ubuntu 10.04.1 LTS"

The installation itself is as simple as:

# aptitude install apache2

By default when you start Apache with its default config it will look like this:

# ps -AHl | egrep 'apach|httpd'
1 S     0 25528     1  0  80   0 -  1348 poll_s ?        00:00:00   apache2
5 S    33 25530 25528  0  80   0 -  1291 skb_re ?        00:00:00     apache2
5 S    33 25532 25528  0  80   0 - 56702 pipe_w ?        00:00:00     apache2
5 S    33 25533 25528  0  80   0 - 56702 pipe_w ?        00:00:00     apache2

A closer look at the configuration in /etc/apache2/apache2.conf reveals more then one section that could be in control of your http server. So which one is our?

# grep -B7 'IfModule mpm_prefork_module'  apache2.conf 

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves


# grep -A8 'IfModule mpm_'  apache2.conf 

    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients          150
    MaxRequestsPerChild   0


# worker MPM
--

    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75 
    ThreadLimit          64
    ThreadsPerChild      25
    MaxClients          150
    MaxRequestsPerChild   0

--

    StartServers          2
    MaxClients          150
    MinSpareThreads      25
    MaxSpareThreads      75 
    ThreadLimit          64
    ThreadsPerChild      25
    MaxRequestsPerChild   0

Well, if you paid attention when installing the server then you know what MPM you installed. If you don't remember than we can find this by running:

# apache2ctl -l
Compiled in modules:
  core.c
  mod_log_config.c
  mod_logio.c
  worker.c  < - - - this is the one we can customize
  http_core.c
  mod_so.c

From the doc Apache MPM worker we know what the above options should mean as well.

    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75 
    ThreadLimit          64
    ThreadsPerChild      25
    MaxClients          150
    MaxRequestsPerChild   0

Base on line # 2 (StartServers) we should see 2 processes. But when you scroll up we can see 4! So what is going on there? To understand the situation we need to look at the processes and threads statistics together.

# ps -lFmC apache2
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN    RSS PSR STIME TTY          TIME CMD
1 - root     25528     1  0   -   - -  1348 -       2580   - Apr04 ?        00:00:00 /usr/sbin/apache2 -k start
1 S root         -     -  0  80   0 -     - poll_s     -   0 Apr04 -        00:00:00 -
5 - www-data 25530 25528  0   -   - -  1291 -       1784   - Apr04 ?        00:00:00 /usr/sbin/apache2 -k start
5 S www-data     -     -  0  80   0 -     - skb_re     -   0 Apr04 -        00:00:00 -
5 - www-data 25532 25528  0   -   - - 56736 -       2776   - Apr04 ?        00:00:00 /usr/sbin/apache2 -k start
5 S www-data     -     -  0  80   0 -     - pipe_w     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - inet_c     -   1 Apr04 -        00:00:00 -
5 - www-data 25533 25528  0   -   - - 56736 -       2768   - Apr04 ?        00:00:00 /usr/sbin/apache2 -k start
5 S www-data     -     -  0  80   0 -     - pipe_w     -   0 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - futex_     -   1 Apr04 -        00:00:00 -
1 S www-data     -     -  0  80   0 -     - inet_c     -   1 Apr04 -        00:00:00 -

At the end everything is as it suppose to be. From all the 4 Apache processes the first 2 are for management and communication and then the last 2 are actually the one we defined in our config file to process the client's requests. All lines with the 'futex' show as well that there is 25 worker threads in the last to processes so this is controlled by the parameter ThreadsPerChild and is correct as well.