Search This Blog

Tuesday, July 31, 2012

What does the GIL lock means when I use threads in Python

Looking for a way to introduce parallelism to your code you are quickly going to find that there are two different solutions: either you can use threads or processes.

Although python supports both [1] and [2] models the support for threads has a following warning:

[1] CPython implementation detail: Due to the Global Interpreter Lock, in CPython only one thread can execute Python code at once.

It took me a while to understand what this actualy means. This is a serious limitation for a multithreaded programming and it still exists even in the latest python version 3.x [3]. To better understanding what this means this is an example from one of the links below I found:

The GIL enforces Python's requirement that only a single bytecode operation is executed at a time.

References

  1. http://docs.python.org/library/threading.html
  2. http://docs.python.org/library/multiprocessing.html#module-multiprocessing
  3. http://docs.python.org/py3k/library/threading.html

  4. Google search about GIL has many link, I personaly found these very useful
  5. http://www.grouplens.org/node/244
    http://linuxgazette.net/107/pai.html

    The section "Write multithreaded or multiprocess code"
    http://www.scipy.org/ParallelProgramming

    http://stackoverflow.com/questions/10344529/is-the-max-thread-limit-actually-a-non-relevant-issue-for-python-linux
    http://stackoverflow.com/questions/7053284/what-is-more-suitable-in-performance-aspect-multithreading-or-multiprocessing

  6. These below are a little longer to read but they talk about the problem and around it on a very low level
  7. http://www.jeffknupp.com/blog/2012/03/31/pythons-hardest-problem/
    http://jessenoller.com/2009/02/01/python-threads-and-the-global-interpreter-lock/

No comments:

Post a Comment