Answering the 12 Most Common Questions About Python

Python is an open-source high-level programming language that is easy to learn and user-friendly. It is one of the first choices of many programmers be it a beginner or experienced. So, today we have prepared a list of most asked questions on Python programming language. 12 Most Asked Questions On Python 1. What does the “yield” keyword do? Answer: is a keyword that is used like , except the function will return a generator. yield return >>> def createGenerator(): ... mylist = range( ) ... for i mylist: ... yield i*i ... >>> mygenerator = createGenerator() # create a generator >>> print(mygenerator) # mygenerator is an object! 3 in >>> for i in mygenerator: ... print(i) 0 1 4 generator object createGenerator at 0xb7555c34 It’s handy when you know your function will return a huge set of values that you will only need to read once. To master , you must understand that The function only returns the generator object. yield when you call the function, the code you have written in the function body does not run. Then, your code will continue from where it left off each time uses the generator. for Now the hard part: The first time the calls the generator object created from your function, it will run the code in your function from the beginning until it hits , then it’ll return the first value of the loop. Then, each subsequent call will run another iteration of the loop you have written in the function and return the next value. This will continue until the generator is considered empty, which happens when the function runs without hitting . That can be because the loop has come to an end, or because you no longer satisfy an " ". for yield yield if/else Alternative Answer: You can also think of it this way. An iterator is just a fancy-sounding term for an object that has a method. So a yield-ed function ends up being something like this: next() Original version: def some_function(): i xrange( ): i i some_function(): print i for in 4 yield for in This is basically what the Python interpreter does with the above code: = # The __iter__ method will be called once by the loop. # The rest the magic happens on the object returned by method. # In it is the object itself. def __iter__(self): self # The next method will be called repeatedly by the loop # until it raises . def next(self): self.count += self.count < : self.count : # A exception is raised # to signal that the iterator is done. # This is caught implicitly by the loop. raise def some_func(): it() i some_func(): print i : ( ): # -1 0 1 . . class it def __init__ self Start at so that we get when we add below self count -1 'for' of this this case return 'for' StopIteration 1 if 4 return else StopIteration 'for' StopIteration return for in For more insight as to what’s happening behind the scenes, the loop can be rewritten to this: for iterator = some_func() : : print iterator.next() except : pass try while 1 StopIteration 2. Does Python have a ternary conditional operator? Answer: Yes, it was in version 2.5. The expression syntax is: added a condition b if else First is evaluated, then exactly one either a or is evaluated and returned based on the value of . If evaluates to , then is evaluated and returned but is ignored, or else when is evaluated and returned but is ignored. condition of b Boolean condition condition True a b b a This allows short-circuiting because when is true only is evaluated and is not evaluated at all, but when is false only is evaluated and is not evaluated at all. condition a b condition b a For example: >>> True >>> False 'true' if else 'false' 'true' 'true' if else 'false' 'false' Note that conditionals are an , not a . This means you can’t use assignment statements or or other within a conditional : expression statement pass statements expression >>> pass False x = File , line pass False x = ^ : invalid syntax if else 3 " " 1 if else 3 SyntaxError You can, however, use conditional expressions to assign a variable like so: x = a True b if else Think of the conditional expression as switching between two values. It is very useful when you’re in a ‘one value or another’ situation, it but doesn’t do much else. If you need to use statements, you have to use a normal instead of a if statement conditional expression. Keep in mind that it’s frowned upon by some Pythonistas for several reasons: The order of the arguments is different from those of the classic ternary operator from many other languages (such as C, C++, Go, Perl, Ruby, Java, Javascript, etc.), which may lead to bugs when people unfamiliar with Python’s “surprising” behavior use it (they may reverse the argument order). condition ? a : b Some find it “unwieldy”, since it goes contrary to the normal flow of thought (thinking of the condition first and then the effects). Stylistic reasons. (Although the ‘inline ‘ can be useful, and make your script more concise, it really does complicate your code). if really If you’re having trouble remembering the order, then remember that when read aloud, you (almost) say what you mean. For example, is read aloud as . x = 4 if b > 8 else 9 x will be 4 if b is greater than 8 otherwise 9 Alternative Answer: You can index into a tuple: (falseValue, trueValue)[test] needs to return or . test True False It might be safer to always implement it as: (falseValue, trueValue)[test == True] or you can use the built-in to assure a Boolean value: bool() falseValue, trueValue)[bool( )] expression 3. What are metaclasses in Python? Answer: A metaclass is the class of a class. A class defines how an instance of the class (i.e. an object) behaves while a metaclass defines how a class behaves. A class is an instance of a metaclass. While in Python you can use arbitrary callables for metaclasses, the better approach is to make it an actual class itself. is the usual metaclass in Python. is itself a class, and it is its own type. You won’t be able to recreate something like purely in Python, but Python cheats a little. To create your own metaclass in Python you really just want to subclass . type type type type A metaclass is most commonly used as a class-factory. When you create an object by calling the class, Python creates a new class (when it executes the ‘class’ statement) by calling the metaclass. Combined with the normal and methods, metaclasses, therefore, allow you to do ‘extra things’ when creating a class, like registering the new class with some registry or replace the class with something else entirely. __init__ __new__ When the statement is executed, Python first executes the body of the statement as a normal block of code. The resulting namespace (a dict) holds the attributes of the class-to-be. The metaclass is determined by looking at the baseclasses of the class-to-be (metaclasses are inherited), at the attribute of the class-to-be (if any) or the global variable. The metaclass is then called with the name, bases and attributes of the class to instantiate it. class class __metaclass__ __metaclass__ However, metaclasses actually define the of a class, not just a factory for it, so you can do much more with them. You can, for instance, define normal methods on the metaclass. These metaclass-methods are like classmethods in that they can be called on the class without an instance, but they are also not like classmethods in that they cannot be called on an instance of the class. . is an example of a method on the metaclass. You can also define the normal ‘magic’ methods, like , and , to implement or change how the class behaves. type type __subclasses__() type __add__ __iter__ __getattr__ Here’s an aggregated example of the bits and pieces: def make_hook(f): f.is_hook = f = {} attrname, attrvalue attrs.iteritems(): getattr(attrvalue, , ): newattrs[ % attrname] = attrvalue : newattrs[attrname] = attrvalue (MyType, mcls).__new__(mcls, name, bases, newattrs) def __init__(self, name, bases, attrs): (MyType, self).__init__(name, bases, attrs) # classregistry.register(self, self.interfaces) print % self def __add__(self, other): {}) def unregister(self): "" "Decorator to turn 'foo' method into '__foo__'" "" 1 return ( ): ( , , , ): . (' '): # . class MyType type def __new__ mcls name bases attrs if name startswith None return None Go over attributes and see if they should be renamed newattrs for in if 'is_hook' 0 '__%s__' else return super super "Would register class %s now." ( , ): # , : # ( . + . , ( , ), class AutoClass self other pass return AutoClass Alternatively to autogenerate the classname as well as the class return type self __name__ other __name__ self other 4. How to check whether a file exists without exceptions? Answer: If the reason you’re checking is so you can do something like , it’s safer to use a around the attempt to open it. Checking and then opening risks the file being deleted or moved or something between when you check and when you try to open it. if file_exists: open_it() try If you’re not planning to open the file immediately, you can use os.path.isfile Return if path is an existing regular file. This follows symbolic links, so both and can be true for the same path. True islink() isfile() os.path os.path.isfile(fname) import if you need to be sure it’s a file. Starting with Python 3.4, the offers an object-oriented approach (backported to in Python 2.7): module pathlib pathlib2 pathlib Path my_file = Path( ) my_file.is_file(): # file exists from import "/path/to/file" if To check a directory, do: my_file.exists(): # path exists if To check whether a object exists independently of whether is it a file or directory, use : Path exists() my_file.exists(): # path exists if You can also use in a block: resolve(strict=True) try : my_abs_path = my_file.resolve(strict=True) except FileNotFoundError: # doesn try 't exist else: # exists Alternative Answer: You have the function: os.path.exists os.path os.path.exists(file_path) import This returns for both files and directories but you can instead use True os.path.isfile(file_path) to test if it’s a file specifically. It follows symlinks. 5. How to call an external command from within a Python script? Answer: Look at the module in the standard library: subprocess subprocess subprocess.run([ , ]) import "ls" "-l" The advantage of vs. is that it is more flexible (you can get the , , the “real” status code, better error handling, etc…). subprocess system stdout stderr The recommends the module over the alternative : official documentation subprocess os.system() The module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function [ ]. subprocess os.system() The section in the documentation may have some helpful recipes. Replacing Older Functions with the subprocess Module subprocess For versions of Python before 3.5, use : call subprocess subprocess.call([ , ]) import "ls" "-l" Alternative Answer: Here’s a summary of the ways to call external programs and the advantages and disadvantages of each: passes the command and arguments to your system’s shell. This is nice because you can actually run multiple commands at once in this manner and set up pipes and input/output redirection. For example: os.system("some_command with args") os.system("some_command output_file") However, while this is convenient, you have to manually handle the escaping of shell characters such as spaces, etc. On the other hand, this also lets you run commands which are simply shell commands and not actually external programs. See . the documentation will do the same thing as except that it gives you a file-like object that you can use to access standard input/output for that process. There are 3 other variants of popen that all handle the i/o slightly differently. If you pass everything as a string, then your command is passed to the shell; if you pass them as a list then you don’t need to worry about escaping anything. See . stream = os.popen("some_command with args") os.system the documentation The class of the module. This is intended as a replacement for but has the downside of being slightly more complicated by virtue of being so comprehensive. For example, you’d say: instead of: but it is nice to have all of the options there in one unified class instead of 4 different popen functions. See . Popen subprocess os.popen print subprocess.Popen("echo Hello World", shell=True, stdout=subprocess.PIPE).stdout.read() print os.popen("echo Hello World").read() the documentation The function from the module. This is basically just like the class and takes all of the same arguments, but it simply waits until the command completes and gives you the return code. For example: See . call subprocess Popen return_code = subprocess.call("echo Hello World", shell=True) the documentation If you’re on Python 3.5 or later, you can use the new function, which is a lot like the above but even more flexible and returns a object when the command finishes executing. subprocess.run CompletedProcess The os module also has all of the fork/exec/spawn functions that you’d have in a C program, but we don’t recommend using them directly. The module should probably be what you use. subprocess Finally please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it. if any part of the string that you pass can not be fully trusted. For example, if a user is entering some/any part of the string. If you are unsure, only use these methods with constants. To give you a hint of the implications considers this code: There are serious security implications print subprocess.Popen( % user_input, stdout=PIPE).stdout.read() "echo %s " and imagine that the user enters something “my mama didnt love me && rm -rf /” which could erase the whole filesystem. 6. How to create a nested directory safely? Answer: On Python ≥ 3.5, use : pathlib.Path.mkdir pathlib Path Path( ).mkdir(parents=True, exist_ok=True) from import "/my/directory" For older versions of Python, we see two answers with good qualities, each with a small flaw, so we will give our take on it: Try , and consider for the creation. os.path.exists os.makedirs os not os.path.exists(directory): os.makedirs(directory) import if As noted, there’s a race condition – if the directory is created between the and the calls, the will fail with an . Unfortunately, blanket-catching and continuing is not foolproof, as it will ignore a failure to create the directory due to other factors, such as insufficient permissions, full disk, etc. os.path.exists os.makedirs os.makedirs OSError OSError One option would be to trap the and examine the embedded error code (see ): OSError Is there a cross-platform way of getting information from Python’s OSError os, errno : os.makedirs(directory) except OSError e: e.errno != errno.EEXIST: raise import try as if Alternatively, there could be a second , but suppose another created the directory after the first check, then removed it before the second one – we could still be fooled. os.path.exists Depending on the application, the danger of concurrent operations may be more or less than the danger posed by other factors such as file permissions. The developer would have to know more about the particular application being developed and its expected environment before choosing an implementation. Modern versions of Python improve this code quite a bit, both by exposing (in 3.3+)… FileExistsError : os.makedirs( ) except FileExistsError: # directory already exists pass try "path/to/directory" and by allowing a keyword argument to os.makedirs called exist_ok os.makedirs( , exist_ok=True) # succeeds even directory exists. ( +). "path/to/directory" if in 3.2 Alternative Answer: Python 3.5+: pathlib pathlib.Path( ).mkdir(parents=True, exist_ok=True) import '/my/directory' as used above recursively creates the directory and does not raise an exception if the directory already exists. If you don’t need or want the parents to be created, skip the argument. pathlib.Path.mkdir parents Python 3.2+: Using pathlib : If you can, install the current backport named . Do not install the older unmaintained backport named . Next, refer to the Python 3.5+ section above and use it the same. pathlib pathlib2 pathlib If using Python 3.4, even though it comes with , it is missing the useful option. The backport is intended to offer a newer and superior implementation of which includes this missing option. pathlib exist_ok mkdir Using os : os os.makedirs(path, exist_ok=True) import as used above recursively creates the directory and does not raise an exception if the directory already exists. It has the optional argument only if using Python 3.2+, with a default value of . This argument does not exist in Python 2.x up to 2.7. As such, there is no need for manual exception handling as with Python 2.7. os.makedirs exist_ok False Python 2.7+: Using pathlib : If you can, install the current backport named . Do not install the older unmaintained backport named . Next, refer to the Python 3.5+ section above and use it the same. pathlib pathlib2 pathlib Using os : os : os.makedirs(path) except OSError: not os.path.isdir(path): raise import try if While a naive solution may first use followed by , the solution above reverses the order of the two operations. In doing so, it prevents a common race condition having to do with a duplicated attempt at creating the directory, and also disambiguates files from directories. os.path.isdir os.makedirs Note that capturing the exception and using is of limited usefulness because , i.e. , is raised for both files and directories. It is more reliable simply to check if the directory exists. errno OSError: [Errno 17] File exists errno.EEXIST Alternative: creates the nested directory, and does nothing if the directory already exists. This works in both Python 2 and 3. mkpath distutils.dir_util distutils.dir_util.mkpath(path) import Per , a severe limitation of this alternative is that it works only once per python process for a given path. In other words, if you use it to create a directory, then delete the directory from inside or outside Python, then use again to recreate the same directory, will simply silently use its invalid cached info of having previously created the directory, and will not actually make the directory again. In contrast, doesn’t rely on any such cache. This limitation may be okay for some applications. Bug 10948 mkpath mkpath os.makedirs 7. Does Python have a string ‘contains’ substring method? Answer: You can use the : operator in not somestring: if "blah" in continue Alternative Answer: If it’s just a substring search you can use . string.find("substring") You do have to be a little careful with , , and though, as they are substring searches. In other words, this: find index in s = s.find( ) == : print( ) : print( ) "This be a string" if "is" -1 "No 'is' here!" else "Found 'is' in the string." It would print . Similarly, would evaluate to . This may or may not be what you want. Found 'is' in the string if "is" in s: True 8. How to access the index in ‘for’ loops? Answer: Using an additional state variable, such as an index variable (which you would normally use in languages such as C or PHP), is considered non-pythonic. The better option is to use the built-in function , available in both Python 2 and 3: enumerate() idx, val enumerate(ints): print(idx, val) for in Check out for more. PEP 279 Alternative Answer: Use to get the index with the element as you iterate: enumerate index, item enumerate(items): print(index, item) for in And note that Python’s indexes start at zero, so you would get 0 to 4 with the above. If you want the count, 1 to 5, do this: count, item enumerate(items, start= ): print(count, item) for in 1 9. What is the difference between staticmethod and classmethod? Answer: Maybe a bit of example code will help. Notice the difference in the call signatures of , and : foo class_foo static_foo = A() ( ): ( , ): " (% , % )" % ( , ) @ ( , ): " (% , % )" % ( , ) @ ( ): " (% )" % class A object def foo self x print executing foo s s self x classmethod def class_foo cls x print executing class_foo s s cls x staticmethod def static_foo x print executing static_foo s x a Below is the usual way an object instance calls a method. The object instance, , is implicitly passed as the first argument. a a.foo( ) # executing foo( 1 ,1) __main__.A object at 0xb7dbef0c , the class of the object instance is implicitly passed as the first argument instead of . With classmethods self a.class_foo( ) # executing class_foo( 1 ,1) class __main__.A You can also call using the class. In fact, if you define something to be a classmethod, it is probably because you intend to call it from the class rather than from a class instance. would have raised a TypeError, but works just fine: class_foo A.foo(1) A.class_foo(1) A.class_foo( ) # executing class_foo( 1 ,1) class __main__.A One use people have found for class methods is to create . inheritable alternative constructors , neither (the object instance) nor (the class) is implicitly passed as the first argument. They behave like plain functions except that you can call them from an instance or the class: With staticmethods self cls a.static_foo( ) # executing static_foo( ) A.static_foo( ) # executing static_foo(hi) 1 1 'hi' Staticmethods are used to group functions which have some logical connection with a class to the class. is just a function, but when you call you don’t just get the function, you get a “partially applied” version of the function with the object instance bound as the first argument to the function. expects 2 arguments, while only expects 1 argument. foo a.foo a foo a.foo is bound to . That is what is meant by the term “bound” below: a foo print(a.foo) # > of 0xb7d52f0c With , a is not bound to , rather the class is bound to . a.class_foo class_foo A class_foo print(a.class_foo) # > class __main__ A Here, with a staticmethod, even though it is a method, just returns a good old function with no arguments bound. expects 1 argument, and expects 1 argument too. a.static_foo static_foo a.static_foo print(a.static_foo) # function static_foo at xb7d479cc And of course the same thing happens when you call with the class instead. static_foo A print(A.static_foo) # function static_foo at xb7d479cc Additional Answer: A is a method that knows nothing about the class or instance it was called on. It just gets the arguments that were passed, no implicit first argument. It is basically useless in Python — you can just use a module function instead of a staticmethod. staticmethod A , on the other hand, is a method that gets passed the class it was called on, or the class of the instance it was called on, as the first argument. This is useful when you want the method to be a factory for the class: since it gets the actual class it was called on as the first argument, you can always instantiate the right class, even when subclasses are involved. Observe for instance how , a classmethod, returns an instance of the subclass when called on a subclass: classmethod dict.fromkeys() >>> { : None, : None, : None} >>> DictSubclass.fromkeys( ) DictSubclass >>> ( ): ... ( ): ... " " ... >>> . (" ") class DictSubclass dict def __repr__ self return DictSubclass dict fromkeys abc 'a' 'c' 'b' "abc" 10. How to list all files of a directory? Answer: will get you everything that’s in a directory – and . os.listdir() files directories If you want files, you could either filter this down using : just os.path os listdir os.path isfile, join onlyfiles = [f f listdir(mypath) isfile(join(mypath, f))] from import from import for in if or you could use which will for each directory it visits – splitting into and for you. If you only want the top directory you can just break the first time it yields os.walk() yield two lists files dirs os walk f = [] (dirpath, dirnames, filenames) walk(mypath): f.extend(filenames) from import for in break Alternative Answer: You can also prefer using the module, as it does pattern matching and expansion. glob glob print(glob.glob( )) import "/home/adam/*.txt" It will return a list with the queried files: [ , , .... ] '/home/adam/file1.txt' '/home/adam/file2.txt' 11. How to make a flat list out of list of lists? Answer: Given a list of lists , l flat_list = [item sublist l item sublist] for in for in which means: flat_list = [] sublist l: item sublist: flat_list.append(item) for in for in is faster than the shortcuts posted so far. ( is the list to flatten.) l Here is the corresponding function: flatten = lambda l: [item sublist l item sublist] for in for in As evidence, you can use the module in the standard library: timeit $ python -mtimeit -s loops, best : usec per loop $ python -mtimeit -s loops, best : usec per loop $ python -mtimeit -s loops, best : msec per loop 'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' '[item for sublist in l for item in sublist]' 10000 of 3 143 'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' 'sum(l, [])' 1000 of 3 969 'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' 'reduce(lambda x,y: x+y,l)' 1000 of 3 1.1 The shortcuts based on (including the implied use in ) are, of necessity, when there are L sublists — as the intermediate result list keeps getting longer, at each step a new intermediate result list object gets allocated, and all the items in the previous intermediate result must be copied over (as well as a few new ones added at the end). So, for simplicity and without actual loss of generality, say you have L sublists of I items each: the first I items are copied back and forth L-1 times, the second I items L-2 times, and so on; total number of copies is I times the sum of x for x from 1 to L excluded, i.e., . Explanation: + sum O(L**2) I * (L**2)/2 The list comprehension just generates one list, once, and copies each item over (from its original place of residence to the result list) also exactly once. Alternative Answer: You can use : itertools.chain() itertools list2d = [[ , , ], [ , , ], [ ], [ , ]] merged = list(itertools.chain(*list2d)) import 1 2 3 4 5 6 7 8 9 Or you can use which doesn’t require unpacking the list with the : itertools.chain.from_iterable() operator * itertools list2d = [[ , , ], [ , , ], [ ], [ , ]] merged = list(itertools.chain.from_iterable(list2d)) import 1 2 3 4 5 6 7 8 9 12. How to check if a list is empty? Answer: not a: print( ) if "List is empty" Using the of the empty is quite pythonic. implicit booleanness list Alternative Answer: The pythonic way to do it is from the (where means “recommended” and means “not recommended”): PEP 8 style guide Yes No For sequences, (strings, lists, tuples), use the fact that empty sequences are false. Yes: not seq: seq: No: len(seq): not len(seq): if if if if In Conclusion These are the 12 most commonly asked questions about Python. If you have any suggestions regarding the article, please feel free to comment below. If you need any help, then we would be glad to help you. Hope this article helped you.