Because synchrony is harmony It was a magical ‘aha’ moment for me when I learned about multithreading for the first time. The fact that I can ask my computer to do actions in a parallel manner delighted me (although it should be noted here that things don’t happen precisely in a parallel manner on a single core computer, and more importantly, they don’t precisely execute in a parallel sense in Python due to the language's Global Interpreter Lock). Multithreading opens new dimensions for computing, but with power comes responsibility. There are obvious troubles one can imagine with multithreading — many threads trying to access the same piece of data can lead to problems like making data inconsistent or getting garbled output (like having in place of on your console). Such problems can arise when we don’t tell the computer how to mange threads in an organized manner. HWeolrldo Hello World But how can we ‘tell’ the computer to keep the threads of our program in synchrony? We do so by using These are simple software mechanisms to ensure that your threads run in a harmonious manner with each other. synchronization primitives. This post presents some of the most popular synchronization primitives in Python, defined in it’s standard module. Most of the blocking methods (i.e., the methods which block execution of a particular thread until some condition is met) of these primitives provide the optional functionality of timeout, but I haven’t included it here for simplicity. Also I’ve just included the principal functionalities of these objects, again for the sake of simplicity. This post assumes you have a basic knowledge of implementing multithreading using Python. threading.py We’ll be learning about , , , , and . Of course, you can construct your own custom synchronization primitives by subclassing these classes. We’ll start with as they are the simplest primitives and gradually we’ll move on to primitives with more and more sophistication. Locks RLocks Semaphores Events Conditions Barriers Locks Locks s are perhaps the simplest synchronization primitives in Python. A has only two states — locked and unlocked (surprise). It is created in the unlocked state and has two principal methods — and . The method locks the and blocks execution until the method in some other coroutine sets it to unlocked. Then it locks the again and returns . The method should only be called in the locked state, it sets the state to unlocked and returns immediately. If is called in the unlocked state, a is raised. Lock Lock acquire() release() acquire() Lock release() Lock True release() release() RunTimeError Here’s the code which uses a primitive for securely accessing a shared variable: Lock #lock_tut.py from threading import Lock, Threadlock = Lock()g = 0 def add_one():"""Just used for demonstration. It’s bad to use the ‘global’statement in general.""" global glock.acquire()g += 1lock.release() def add_two():global glock.acquire()g += 2lock.release() threads = []for func in [add_one, add_two]:threads.append(Thread(target=func))threads[-1].start() for thread in threads:"""Waits for threads to complete before moving on with the mainscript."""thread.join() print(g) This simply gives an output of 3, but now we are sure that the two functions are not changing the value of the global variable simultaneously although they run on two different threads. Thus, s can be used to avoid inconsistent output by allowing only one thread to modify data at a time. g Lock RLocks The standard doesn’t know which thread is currently holding thelock. If the lock is held, any thread that attempts to acquire it willblock, even if the same thread itself is already holding the lock.In such cases, (re-entrant lock) is used. You can extend the code in the following snippet by adding output statements for demonstrating how s can prevent unwanted blocking. Lock RLock RLock #rlock_tut.py import threading num = 0lock = Threading.Lock() lock.acquire()num += 1lock.acquire() # This will block.num += 2lock.release() # With RLock, that problem doesn’t happen.lock = Threading.RLock() lock.acquire()num += 3lock.acquire() # This won’t block.num += 4lock.release()lock.release() # You need to call release once for each call to acquire. One good use case for s is recursion, when a parent call of a function would otherwise block its nested call. Thus, the main use for s is nested access to shared resources. RLock RLock Semaphores Semaphores are simply advanced counters. An call to a semaphore will block only after a number of threads have ed it. The associated counter decreases per call, and increases per call. A will occur if calls try to increment the counter beyond it’s assigned maximum value (which is the number of threads that can the semaphore before blocking occurs). Following code demonstrates the use of semaphores in a simple producer-consumer problem. acquire() acquire() acquire() release() ValueError release() acquire() #semaphores_tut.py import random, timefrom threading import BoundedSemaphore, Thread max_items = 5 """Consider 'container' as a container, of course, with a capacity of 5items. Defaults to 1 item if 'max_items' is passed."""container = BoundedSemaphore(max_items) def producer(nloops):for i in range(nloops):time.sleep(random.randrange(2, 5))print(time.ctime(), end=": ")try:container.release()print("Produced an item.")except ValueError:print("Full, skipping.") def consumer(nloops):for i in range(nloops):time.sleep(random.randrange(2, 5))print(time.ctime(), end=": ") """  
    In the following if statement we disable the default  
    blocking behaviour by passing False for the blocking flag.  
    """

    if container.acquire(False):  
        print("Consumed an item.")  
    else:  
        print("Empty, skipping.") threads = []nloops = random.randrange(3, 6)print("Starting with %s items." % max_items)threads.append(Thread(target=producer, args=(nloops,)))threads.append(Thread(target=consumer, args=(random.randrange(nloops, nloops+max_items+2),))) for thread in threads:  # Starts all the threads.thread.start()for thread in threads:  # Waits for threads to complete before moving on with the main script.thread.join()print("All done.") semaphore_tut.py in action The module also provides the simple class. A provides a non-bounded counter which allows you to call any number of times for incrementing. However, to avoid programming errors, it’s usually a correct choice to use , which raises an error if a call tries to increase the counter beyond it’s maximum size. threading Semaphore Semaphore release() BoundedSemaphore release() Semaphores are typically used for limiting a resource, like limiting a server to handle only 10 clients at a time. In such a case, multiple thread connections compete for a limited resource (in our example, it is the server). Events The synchronization primitive acts as a simple communicator between threads. They are based on an internal flag which threads can or . Other threads can for the internal flag to be . The method blocks until the flag becomes true. Following snippet demonstrates how s can be used to trigger actions. Event set() clear() wait() set() wait() Event #event_tut.py import random, timefrom threading import Event, Thread event = Event() def waiter(event, nloops):for i in range(nloops):print(“%s. Waiting for the flag to be set.” % (i+1))event.wait() # Blocks until the flag becomes true.print(“Wait complete at:”, time.ctime())event.clear() # Resets the flag.print() def setter(event, nloops):for i in range(nloops):time.sleep(random.randrange(2, 5)) # Sleeps for some time.event.set() threads = []nloops = random.randrange(3, 6) threads.append(Thread(target=waiter, args=(event, nloops)))threads[-1].start()threads.append(Thread(target=setter, args=(event, nloops)))threads[-1].start() for thread in threads:thread.join() print(“All done.”) Execution of event_tut.py Conditions A object is simply a more advanced version of the object. It too acts as a communicator between threads and can be used to other threads about a change in the state of the program. For example, it can be used to signal the availability of a resource for consumption. Other threads must also the condition (and thus its related lock) before ing for the condition to be satisfied. Also, a thread should a once it has completed the related actions, so that other threads can acquire the condition for their purposes. Following code demonstrates the implementation of another simple producer-consumer problem with the help of the object. Condition Event notify() acquire() wait() release() Condition Condition #condition_tut.py import random, timefrom threading import Condition, Thread """'condition' variable will be used to represent the availability of a produceditem.""" condition = Condition() box = [] def producer(box, nitems):for i in range(nitems):time.sleep(random.randrange(2, 5))  # Sleeps for some time.condition.acquire()num = random.randint(1, 10)box.append(num)  # Puts an item into box for consumption.condition.notify()  # Notifies the consumer about the availability.print("Produced:", num)condition.release() def consumer(box, nitems):for i in range(nitems):condition.acquire()condition.wait()  # Blocks until an item is available for consumption.print("%s: Acquired: %s" % (time.ctime(), box.pop()))condition.release() threads = [] """'nloops' is the number of times an item will be produced andconsumed.""" nloops = random.randrange(3, 6)for func in [producer, consumer]:threads.append(Thread(target=func, args=(box, nloops)))threads[-1].start()  # Starts the thread. for thread in threads:"""Waits for the threads to complete before moving onwith the main script."""thread.join()print("All done.") Output of condition_tut.py There can be other uses of s. I think they will be useful when you’re developing a streaming API which notifies a waiting client once a piece of data is available. Condition Barriers A barrier is a simple synchronization primitive which can be used by different threads to wait for each other. Each thread tries to pass a barrier by calling the method, which will block until all of threads have made that call. As soon as that happens, the threads are released simultaneously. Following snippet demonstrates the use of s. wait() Barrier #barrier_tut.py from random import randrangefrom threading import Barrier, Threadfrom time import ctime, sleep num = 4# 4 threads will need to pass this barrier to get released.b = Barrier(num)names = [“Harsh”, “Lokesh”, “George”, “Iqbal”] def player():name = names.pop()sleep(randrange(2, 5))print(“%s reached the barrier at: %s” % (name, ctime()))b.wait() threads = []print(“Race starts now…”) for i in range(num):threads.append(Thread(target=player))threads[-1].start() """Following loop enables waiting for the threads to complete before moving on with the main script.""" for thread in threads:thread.join()print()print(“Race over!”) Here’s the output of barrier_tut.py Barriers can find many uses; one of them being synchronizing a server and aclient — as the server has to wait for the client after initializing itself. With that, we have reached the end of our discussion on synchronization primitives in Python. I wrote this post as a in the book “Core Python Applications Programming” by Wesley Chun. If you liked this post, consider having a look at on GitHub and starring the repository 🙂. The gists for code mentioned in this article are also available at my profile. solution to an exercise my other works from this book Sources: , , effbot.org bogotobogo.com Python Docs I’m new to blogging, so constructive criticism is not only welcomed, but very much wanted! Did you like the read? Medium doesn’t offer partner program in my country―so I ask people to buy me coffee instead.

Target

Another Way to Find Max Partitions

In Defense of Version Control

Too Long; Didn't Read

Make resilience your competitive advantage

Let’s Synchronize Threads in Python

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Celebrating a Joyful JavaScript Certification

A Guide to NSOperation and NSOperationQueue in iOS Development

An Introduction to Multi-Threading

Backend Development 101: Prime Numbers and Multi-threading

Concurrency Black-Art: Excerpts from Real-world concurrency

Celebrating a Joyful JavaScript Certification

A Guide to NSOperation and NSOperationQueue in iOS Development

An Introduction to Multi-Threading

Backend Development 101: Prime Numbers and Multi-threading

Concurrency Black-Art: Excerpts from Real-world concurrency

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps