Hi Folks !! Hope you all programming geeks are doing well. In this post, we  will discuss about concurrency and Parallelism in python. Here, we will look at Multithreading , Multiprocessing , asynchronous programming , concurrency and parallelism and how we can use these concepts to speed up computation tasks in python.  So, without wasting time, lets get started . [ I am already assuming that you have a fair knowledge of python. If not, I would recommend you to read this post before moving forward : ] https://hackernoon.com/pythonic-way-of-doing-things-code-kq2b430vv Parallelism It means performing multiple tasks at same time and in same order . It means distributing your tasks over CPU cores [ type <lscpu> in terminal to check the number of cores in your computer. ]. For any CPU bound tasks ( like — doing numerical computations ), we can use python’s multiprocessing module . We simply create a Pool object in multiprocessing which offers a convenient means to parallelize the execution of a function across multiple input values. Let's look at it with the help of an example : 1. Multiprocessing: multiprocessing os time numpy np dot_product = np.dot(A[ ],A[ ]) List = [[np.arange( ).reshape( , ),np.arange( ).reshape( , )],
         [np.arange( ).reshape( , ),np.arange( ).reshape( , )],
         [np.arange( ).reshape( , ),np.arange( ).reshape( , )]] __name__ == : start = time.time()
    B = list(map(DotProduct,List))
    end = time.time() - start
    print( , end , ) start = time.time()
    pool = multiprocessing.cpu_count() multiprocessing.Pool(pool) p:
        print(p.map(DotProduct,List))
    end = time.time() - start
    print( , end , ) import import import import as : def DotProduct (A) 0 1 return 1000000 5000 200 1000000 200 5000 1000000 500 2000 1000000 2000 500 1000000 5000 200 1000000 200 5000 if "__main__" # executing a code without multiprocessing .. ie. on single core . "Full time taken : " "seconds" # lets look at executing same code with multiprocesing module on multiple cores .. with as "Full time taken : " "seconds" ## output // Full time taken :  23.593358993530273 seconds Full time taken :  14.405884027481079 seconds Concurrency It means performing multiple tasks at same time but in overlapping or different or same order . (Python is not great at handling concurrency ) but it does a pretty decent job . running different/multiple threads to perform tasks on a single processor . Multithreading is really good for performing IO bound tasks (like — Sending multiple request to servers concurrently etc ..). Every new thread created will have a PID (process ID) and it will have a start function . join() function of the thread can be used, if want to run loc after thread finishes its job. Python has a very complicated relationship with its GIL and the output of the code vary a lot . 1. Multithreading : In Python, Async IO is a single threaded - single process design paradigm that somehow manages to achieve concurrency . 2. Async IO : Lets look at it with the help of an example . threading os time numpy np number*number number** number**power number*(number+ )/ print( , square( ))
    print( , cube( ))
    print( , nth_power( , ))
    print( , sum_of_n_numbers( )) A = np.arange( ).reshape( , )
    B = np.arange( ).reshape( , )
    Dot = np.dot(A,B) __name__ == : start = time.time()
        BasicOperation()
        Mid = time.time() - start
        print( , Mid , )
        DotProduct()
        end = time.time() - start
        print( , end , ) start = time.time()
        Thread_1 = threading.Thread(target = BasicOperation, name = ) 
        Thread_2 = threading.Thread(target = DotProduct , name= )
        Thread_1.start() 
        Thread_2.start() 
        Thread_1.join() 
        Mid = time.time() - start
        print( , Mid , ) 
        Thread_2.join()
        end = time.time() - start
        print( , end , ) import import import import as : def BasicOperation () # square of number : def square (number) return # cube of a number : def cube (number) return 3 # nth power of a number : def nth_power (number,power) return # sum of n numbers : def sum_of_n_numbers (number) return 1 2 # using functions to drive a program ... "square of 5 is " 5 "cube of 5 is " 5 "5 raise to power 2 is " 5 2 "sum of first 5 numbers is" 5 : def DotProduct () 1000000 5000 200 1000000 200 5000 if "__main__" # without threading ... "Mid time taken : " "seconds" "Full time taken : " "seconds" # with threading ... ' Basic Operation Thread ' ' Dot Product Thread ' "Mid time taken : " "seconds" "Full time taken : " "seconds" ## output // square of 5 is  25 cube of 5 is  125 5 raise to power 2 is  25 sum of first 5 numbers is 15.0 Mid time taken :  0.0006113052368164062 seconds Full time taken :  square of 5 is 10.373110294342041  25seconds cube of 5 is Mid time taken :   1250.0015938282012939453 5 raise to power 2 is seconds 25 sum of first 5 numbers is 15.0 Full time taken :  12.598262786865234 seconds Summary We use python’s multiprocessing module to achieve parallelism whereas concurrency in Python is achieved with the help of threading and Async IO modules . A program running in parallel will be called as concurrent but the reverse is not true . That's it. Thank you for taking your time and reading my post. I hope you liked it.

Geeks

Target

Parallelism and Concurrency in Python (Concepts + Code)

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

7 Programming Concepts Everyone Should Know (With Code)

What No One Told Me About Being a Product Manager at an Early Stage Startup

What Are Convolution Neural Networks? [ELI5]

Rules of Thumb for Software Engineering

10,331,579,614 Records Leaked in 2019 And Counting...

10 Ways to Future-Proof Your Business With Cloud

7 Programming Concepts Everyone Should Know (With Code)

What No One Told Me About Being a Product Manager at an Early Stage Startup

What Are Convolution Neural Networks? [ELI5]

Rules of Thumb for Software Engineering

10,331,579,614 Records Leaked in 2019 And Counting...

10 Ways to Future-Proof Your Business With Cloud

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps