825 reads

Answering the 12 Most Common Questions About Python

by KiranNovember 10th, 2020

Too Long; Didn't Read

Python is an open-source high-level programming language that is easy to learn and user-friendly. It is one of the first choices of many programmers be it a beginner or experienced. Today we have prepared a list of most asked questions on Python programming language. The list includes: What does the “yield’s keyword do? What are metaclasses in Python? What do you know about Python's "yield" keyword? How do you use it? What does it do?

Company Mentioned

Coins Mentioned

featured image - Answering the 12 Most Common Questions About Python

12 Most Asked Questions On Python

1. What does the “yield” keyword do?

Answer:

yield

is a keyword that is used like

return

, except the function will return a generator.

>>> def createGenerator():
...    mylist = range(3)
...    for i in mylist:
...        yield i*i
...
>>> mygenerator = createGenerator() # create a generator
>>> print(mygenerator) # mygenerator is an object!
<generator object createGenerator at 0xb7555c34>
>>> for i in mygenerator:
...     print(i)
0
1
4

It’s handy when you know your function will return a huge set of values that you will only need to read once.

To master

yield

, you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object.

Then, your code will continue from where it left off each time

for

uses the generator.

Now the hard part:

The first time the

for

calls the generator object created from your function, it will run the code in your function from the beginning until it hits

yield

, then it’ll return the first value of the loop. Then, each subsequent call will run another iteration of the loop you have written in the function and return the next value. This will continue until the
generator is considered empty, which happens when the function runs
without hitting

yield

. That can be because the loop has come to an end, or because you no longer satisfy an "

if/else

Alternative Answer:

You can also think of it this way.

An iterator is just a fancy-sounding term for an object that has a

next()

method. So a yield-ed function ends up being something like this:

Original version:

def some_function():
    for i in xrange(4):
        yield i

for i in some_function():
    print i

This is basically what the Python interpreter does with the above code:

class it:
    def __init__(self):
        # Start at -1 so that we get 0 when we add 1 below.
        self.count = -1

    # The __iter__ method will be called once by the 'for' loop.
    # The rest of the magic happens on the object returned by this method.
    # In this case it is the object itself.
    def __iter__(self):
        return self

    # The next method will be called repeatedly by the 'for' loop
    # until it raises StopIteration.
    def next(self):
        self.count += 1
        if self.count < 4:
            return self.count
        else:
            # A StopIteration exception is raised
            # to signal that the iterator is done.
            # This is caught implicitly by the 'for' loop.
            raise StopIteration

def some_func():
    return it()

for i in some_func():
    print i

For more insight as to what’s happening behind the scenes, the

for

loop can be rewritten to this:

iterator = some_func()
try:
    while 1:
        print iterator.next()
except StopIteration:
    pass

2. Does Python have a ternary conditional operator?

Answer:

Yes, it was added in version 2.5. The expression syntax is:

a if condition else b

First

condition

is evaluated, then exactly one

of

either a or

is evaluated and returned based on the Boolean value of

condition

. If

condition

evaluates to

True

, then

is evaluated and returned but

is ignored, or else when

is evaluated and returned but

is ignored.

This allows short-circuiting because when

condition

is true only

is evaluated and

is not evaluated at all, but when

condition

is false only

is evaluated and

is not evaluated at all.

For example:

>>> 'true' if True else 'false'
'true'
>>> 'true' if False else 'false'
'false'

Note that conditionals are an expression, not a statement. This means you can’t use assignment statements or

pass

or other statements within a conditional expression:

>>> pass if False else x = 3
  File "<stdin>", line 1
    pass if False else x = 3
          ^
SyntaxError: invalid syntax

You can, however, use conditional expressions to assign a variable like so:

x = a if True else b

Think of the conditional expression as switching between two values. It is very useful when you’re in a ‘one value or another’ situation, it but doesn’t do much else.

If you need to use statements, you have to use a normal

if

statement instead of a conditional expression.

Keep in mind that it’s frowned upon by some Pythonistas for several reasons:

The order of the arguments is different from those of the classic
```
condition ? a : b
```
ternary operator from many other languages (such as C, C++, Go, Perl, Ruby, Java, Javascript, etc.), which may lead to bugs when people unfamiliar with Python’s “surprising” behavior use it (they may reverse the argument order).
Some find it “unwieldy”, since it goes contrary to the normal flow of thought (thinking of the condition first and then the effects).
Stylistic reasons. (Although the ‘inline
```
if
```
‘ can be really useful, and make your script more concise, it really does complicate your code).

If you’re having trouble remembering the order, then remember that
when read aloud, you (almost) say what you mean. For example,

x = 4 if b > 8 else 9

is read aloud as

x will be 4 if b is greater than 8 otherwise 9

Alternative Answer:

You can index into a tuple:

(falseValue, trueValue)[test]

test

needs to return True or False.

It might be safer to always implement it as:

(falseValue, trueValue)[test == True]

or you can use the built-in

bool()

to assure a Boolean value:

falseValue, trueValue)[bool(<expression>)]

3. What are metaclasses in Python?

Answer:

A metaclass is the class of a class. A class defines how an instance of the class (i.e. an object) behaves while a metaclass defines how a class behaves. A class is an instance of a metaclass.

While in Python you can use arbitrary callables for metaclasses, the better approach is to make it an actual class itself.

type

is the usual metaclass in Python.

type

is itself a class, and it is its own type. You won’t be able to recreate something like

type

purely in Python, but Python cheats a little. To create your own metaclass in Python you really just want to subclass

type

A metaclass is most commonly used as a class-factory. When you create an object by calling the class, Python creates a new class (when it executes the ‘class’ statement) by calling the metaclass. Combined with the normal

__init__

and

__new__

methods, metaclasses, therefore, allow you to do ‘extra things’ when creating a class, like registering the new class with some registry or replace the class with something else entirely.

When the

class

statement is executed, Python first executes the body of the

class

statement as a normal block of code. The resulting namespace (a dict) holds the attributes of the class-to-be. The metaclass is determined by looking at the baseclasses of the class-to-be (metaclasses are inherited), at the

__metaclass__

attribute of the class-to-be (if any) or the

__metaclass__

global variable. The metaclass is then called with the name, bases and attributes of the class to instantiate it.

However, metaclasses actually define the type of a class, not just a factory for it, so you can do much more with them. You can, for instance, define normal methods on the metaclass. These metaclass-methods are like classmethods in that they can be called on the class without an instance, but they are also not like classmethods in that they cannot be called on an instance of the class.

type

__subclasses__()

is an example of a method on the

type

metaclass. You can also define the normal ‘magic’ methods, like

__add__

__iter__

and

__getattr__

, to implement or change how the class behaves.

Here’s an aggregated example of the bits and pieces:

def make_hook(f):
    """Decorator to turn 'foo' method into '__foo__'"""
    f.is_hook = 1
    return f

class MyType(type):
    def __new__(mcls, name, bases, attrs):

        if name.startswith('None'):
            return None

        # Go over attributes and see if they should be renamed.
        newattrs = {}
        for attrname, attrvalue in attrs.iteritems():
            if getattr(attrvalue, 'is_hook', 0):
                newattrs['__%s__' % attrname] = attrvalue
            else:
                newattrs[attrname] = attrvalue

        return super(MyType, mcls).__new__(mcls, name, bases, newattrs)

    def __init__(self, name, bases, attrs):
        super(MyType, self).__init__(name, bases, attrs)

        # classregistry.register(self, self.interfaces)
        print "Would register class %s now." % self

    def __add__(self, other):
        class AutoClass(self, other):
            pass
        return AutoClass
        # Alternatively, to autogenerate the classname as well as the class:
        # return type(self.__name__ + other.__name__, (self, other), {})

    def unregister(self):

4. How to check whether a file exists without exceptions?

Answer:

If the reason you’re checking is so you can do something like

if file_exists: open_it()

, it’s safer to use a

try

around the attempt to open it. Checking and then opening risks the file being deleted or moved or something between when you check and when you try to open it.

If you’re not planning to open the file immediately, you can use

os.path.isfile

Return
True
if path is an existing regular file. This follows symbolic links, so both islink() and isfile() can be true for the same path.

import os.path
os.path.isfile(fname)

if you need to be sure it’s a file.

Starting with Python 3.4, the

pathlib

module offers an object-oriented approach (backported to

pathlib2

in Python 2.7):

from pathlib import Path

my_file = Path("/path/to/file")
if my_file.is_file():
    # file exists

To check a directory, do:

if my_file.exists():
    # path exists

To check whether a

Path

object exists independently of whether is it a file or directory, use

exists()

if my_file.exists():
    # path exists

You can also use

resolve(strict=True)

in a

try

block:

try:
    my_abs_path = my_file.resolve(strict=True)
except FileNotFoundError:
    # doesn't exist
else:
    # exists

Alternative Answer:

You have the

os.path.exists

function:

import os.path
os.path.exists(file_path)

This returns

True

for both files and directories but you can instead use

os.path.isfile(file_path)

to test if it’s a file specifically. It follows symlinks.

5. How to call an external command from within a Python script?

Answer:

Look at the subprocess module in the standard library:

import subprocess
subprocess.run(["ls", "-l"])

The advantage of

subprocess

vs.

system

is that it is more flexible (you can get the

stdout

stderr

, the “real” status code, better error handling, etc…).

The official documentation recommends the

subprocess

module over the alternative

os.system()

The
subprocess
module provides more powerful facilities
for spawning new processes and retrieving their results; using that
module is preferable to using this function [
os.system()
].

The Replacing Older Functions with the subprocess Module section in the

subprocess

documentation may have some helpful recipes.

For versions of Python before 3.5, use

call

import subprocess
subprocess.call(["ls", "-l"])

Alternative Answer:

Here’s a summary of the ways to call external programs and the advantages and disadvantages of each:

```
os.system("some_command with args") 
```
passes the command and arguments to your system’s shell. This is nice because you can actually run multiple commands at once in this manner and set up pipes and input/output redirection. For example:
```
 os.system("some_command < input_file | another_command > output_file")
```

However, while this is convenient, you have to manually handle the escaping of shell characters such as spaces, etc. On the other hand,
this also lets you run commands which are simply shell commands and not
actually external programs. See the documentation.

```
stream = os.popen("some_command with args")
```
will do the same thing as
```
os.system
```
except that it gives you a file-like object that you can use to access standard input/output for that process. There are 3 other variants of popen that all handle the i/o slightly differently. If you pass everything as a string, then your command is passed to the shell; if you pass them as a list then you don’t need to worry about escaping
anything. See the documentation.
The
```
Popen 
```
class of the
```
subprocess
```
module. This is intended as a replacement for
```
os.popen
```
but has the downside of being slightly more complicated by virtue of being so comprehensive. For example, you’d say:
```
print subprocess.Popen("echo Hello World", shell=True, stdout=subprocess.PIPE).stdout.read()
```
instead of:
```
print os.popen("echo Hello World").read()
```
but it is nice to have all of the options there in one unified class instead of 4 different popen functions. See the documentation.
The
```
call
```
function from the
```
subprocess
```
module. This is basically just like the
```
Popen
```
class and takes all of the same arguments, but it simply waits until the command completes and gives you the return code. For example:
```
return_code = subprocess.call("echo Hello World", shell=True)
```
See the documentation.
If you’re on Python 3.5 or later, you can use the new
```
subprocess.run
```
function, which is a lot like the above but even more flexible and returns a
```
CompletedProcess
```
object when the command finishes executing.
The os module also has all of the fork/exec/spawn functions that you’d have in a C program, but we don’t recommend using them directly.

The

subprocess

module should probably be what you use.

Finally please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it. There are serious security implications if any part of the string that you pass can not be fully trusted. For example, if a user is entering some/any part of the string. If you are unsure, only use these methods with constants. To give you a hint of the implications considers this code:

print subprocess.Popen("echo %s " % user_input, stdout=PIPE).stdout.read()

and imagine that the user enters something “my mama didnt love me && rm -rf /” which could erase the whole filesystem.

6. How to create a nested directory safely?

Answer:

On Python ≥ 3.5, use

pathlib.Path.mkdir

from pathlib import Path
Path("/my/directory").mkdir(parents=True, exist_ok=True)

For older versions of Python, we see two answers with good qualities, each with a small flaw, so we will give our take on it:

Try

os.path.exists

, and consider

os.makedirs

for the creation.

import os
if not os.path.exists(directory):
    os.makedirs(directory)

As noted, there’s a race condition – if the directory is created between the

os.path.exists

and the

os.makedirs

calls, the

os.makedirs

will fail with an

OSError

. Unfortunately, blanket-catching

OSError

and continuing is not foolproof, as it will ignore a failure to create the directory due to other factors, such as insufficient permissions, full disk, etc.

One option would be to trap the

OSError

and examine the embedded error code (see Is there a cross-platform way of getting information from Python’s OSError):

import os, errno

try:
    os.makedirs(directory)
except OSError as e:
    if e.errno != errno.EEXIST:
        raise

Alternatively, there could be a second

os.path.exists

, but suppose another created the directory after the first check, then removed it before the second one – we could still be fooled.

Depending on the application, the danger of concurrent operations may be more or less than the danger posed by other factors such as file permissions. The developer would have to know more about the particular
application being developed and its expected environment before choosing an implementation.

Modern versions of Python improve this code quite a bit, both by exposing

FileExistsError

(in 3.3+)…

try:
    os.makedirs("path/to/directory")
except FileExistsError:
    # directory already exists
    pass

and by allowing a keyword argument to os.makedirs called

exist_ok

os.makedirs("path/to/directory", exist_ok=True)  # succeeds even if directory exists. (in 3.2+).

Alternative Answer:

Python 3.5+:

import pathlib
pathlib.Path('/my/directory').mkdir(parents=True, exist_ok=True)

pathlib.Path.mkdir

as used above recursively creates the directory and does not raise an exception if the directory already exists. If you don’t need or want the parents to be created, skip the

parents

argument.

Python 3.2+:

Using

pathlib

If you can, install the current

pathlib

backport named

pathlib2

. Do not install the older unmaintained backport named

pathlib

. Next, refer to the Python 3.5+ section above and use it the same.

If using Python 3.4, even though it comes with

pathlib

, it is missing the useful

exist_ok

option. The backport is intended to offer a newer and superior implementation of

mkdir

which includes this missing option.

Using

os

import os
os.makedirs(path, exist_ok=True)

os.makedirs

as used above recursively creates the directory and does not raise an exception if the directory already exists. It has the optional

exist_ok

argument only if using Python 3.2+, with a default value of

False

. This argument does not exist in Python 2.x up to 2.7. As such, there is no need for manual exception handling as with Python 2.7.

Python 2.7+:

Using

pathlib

If you can, install the current

pathlib

backport named

pathlib2

. Do not install the older unmaintained backport named

pathlib

. Next, refer to the Python 3.5+ section above and use it the same.

Using

os

import os
try: 
    os.makedirs(path)
except OSError:
    if not os.path.isdir(path):
        raise

While a naive solution may first use

os.path.isdir

followed by

os.makedirs

, the solution above reverses the order of the two operations. In doing so, it prevents a common race condition having to do with a duplicated attempt at creating the directory, and also disambiguates files from directories.

Note that capturing the exception and using

errno

is of limited usefulness because

OSError: [Errno 17] File exists

, i.e.

errno.EEXIST

, is raised for both files and directories. It is more reliable simply to check if the directory exists.

Alternative:

mkpath

creates the nested directory, and does nothing if the directory already exists. This works in both Python 2 and 3.

import distutils.dir_util
distutils.dir_util.mkpath(path)

Per Bug 10948, a severe limitation of this alternative is that it works only once per python process for a given path. In other words, if you use it to create a directory, then delete the directory from inside or outside Python,
then use

mkpath

again to recreate the same directory,

mkpath

will simply silently use its invalid cached info of having previously created the directory, and will not actually make the directory again. In contrast,

os.makedirs

doesn’t rely on any such cache. This limitation may be okay for some applications.

7. Does Python have a string ‘contains’ substring method?

Answer:

You can use the

in

operator:

if "blah" not in somestring: 
    continue

Alternative Answer:

If it’s just a substring search you can use

string.find("substring")

You do have to be a little careful with

find

index

, and

in

though, as they are substring searches. In other words, this:

s = "This be a string"
if s.find("is") == -1:
    print("No 'is' here!")
else:
    print("Found 'is' in the string.")

It would print

Found 'is' in the string

. Similarly,

 if "is" in s:

would evaluate to

True

. This may or may not be what you want.

8. How to access the index in ‘for’ loops?

Answer:

Using an additional state variable, such as an index variable (which you would normally use in languages such as C or PHP), is considered non-pythonic.

The better option is to use the built-in function

enumerate()

, available in both Python 2 and 3:

for idx, val in enumerate(ints):
    print(idx, val)

Check out PEP 279 for more.

Alternative Answer:

Use

enumerate

to get the index with the element as you iterate:

for index, item in enumerate(items):
    print(index, item)

And note that Python’s indexes start at zero, so you would get 0 to 4 with the above. If you want the count, 1 to 5, do this:

for count, item in enumerate(items, start=1):
    print(count, item)

9. What is the difference between staticmethod and classmethod?

Answer:

Maybe a bit of example code will help. Notice the difference in the call signatures of

foo

class_foo

and

static_foo

class A(object):
    def foo(self, x):
        print "executing foo(%s, %s)" % (self, x)

    @classmethod
    def class_foo(cls, x):
        print "executing class_foo(%s, %s)" % (cls, x)

    @staticmethod
    def static_foo(x):
        print "executing static_foo(%s)" % x    

a = A()

Below is the usual way an object instance calls a method. The object instance,

, is implicitly passed as the first argument.

a.foo(1)
# executing foo(<__main__.A object at 0xb7dbef0c>,1)

With classmethods, the class of the object instance is implicitly passed as the first argument instead of

self

a.class_foo(1)
# executing class_foo(<class '__main__.A'>,1)

You can also call

class_foo

using the class. In fact, if you define something to be a classmethod, it is probably because you intend to call it from the class rather than from a class instance.

A.foo(1)

would have raised a TypeError, but

A.class_foo(1)

works just fine:

A.class_foo(1)
# executing class_foo(<class '__main__.A'>,1)

One use people have found for class methods is to create inheritable alternative constructors.

With staticmethods, neither

self

(the object instance) nor

cls

(the class) is implicitly passed as the first argument. They behave like plain functions except that you can call them from an instance or the class:

a.static_foo(1)
# executing static_foo(1)

A.static_foo('hi')
# executing static_foo(hi)

Staticmethods are used to group functions which have some logical connection with a class to the class.

foo

is just a function, but when you call

a.foo

you don’t just get the function, you get a “partially applied” version of the function with the object instance

bound as the first argument to the function.

foo

expects 2 arguments, while

a.foo

only expects 1 argument.

is bound to

foo

. That is what is meant by the term “bound” below:

print(a.foo)
# <bound method A.foo of <__main__.A object at 0xb7d52f0c>>

With

a.class_foo

, a is not bound to

class_foo

, rather the class

is bound to

class_foo

print(a.class_foo)
# <bound method type.class_foo of <class '__main__.A'>>

Here, with a staticmethod, even though it is a method,

a.static_foo

just returns a good old function with no arguments bound.

static_foo

expects 1 argument, and

a.static_foo

expects 1 argument too.

print(a.static_foo)
# <function static_foo at 0xb7d479cc>

And of course the same thing happens when you call

static_foo

with the class

instead.

print(A.static_foo)
# <function static_foo at 0xb7d479cc>

Additional Answer:

A staticmethod is a method that knows nothing about the class or instance it was called on. It just gets the arguments that were passed, no implicit first argument. It is basically useless in Python — you can just use a module function instead of a staticmethod.

A classmethod, on the other hand, is a method that gets passed the class it was called on, or the class of the instance it was called on, as the first argument. This is useful when you want the method to be a factory for the class: since it gets the actual class it was called on as the first argument, you can always instantiate the right class, even when subclasses are involved. Observe for instance how

dict.fromkeys()

, a classmethod, returns an instance of the subclass when called on a subclass:

>>> class DictSubclass(dict):
...     def __repr__(self):
...         return "DictSubclass"
... 
>>> dict.fromkeys("abc")
{'a': None, 'c': None, 'b': None}
>>> DictSubclass.fromkeys("abc")
DictSubclass
>>>

10. How to list all files of a directory?

Answer:

os.listdir()

will get you everything that’s in a directory – files and directories.

If you want just files, you could either filter this down using

os.path

from os import listdir
from os.path import isfile, join
onlyfiles = [f for f in listdir(mypath) if isfile(join(mypath, f))]

or you could use

os.walk()

which will yield two lists for each directory it visits – splitting into files and dirs for you. If you only want the top directory you can just break the first time it yields

from os import walk

f = []
for (dirpath, dirnames, filenames) in walk(mypath):
    f.extend(filenames)
    break

Alternative Answer:

You can also prefer using the

glob

module, as it does pattern matching and expansion.

import glob
print(glob.glob("/home/adam/*.txt"))

It will return a list with the queried files:

['/home/adam/file1.txt', '/home/adam/file2.txt', .... ]

11. How to make a flat list out of list of lists?

Answer:

Given a list of lists

flat_list = [item for sublist in l for item in sublist]

which means:

flat_list = []
for sublist in l:
    for item in sublist:
        flat_list.append(item)

is faster than the shortcuts posted so far. (

is the list to flatten.)

Here is the corresponding function:

flatten = lambda l: [item for sublist in l for item in sublist]

As evidence, you can use the

timeit

module in the standard library:

$ python -mtimeit -s'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' '[item for sublist in l for item in sublist]'
10000 loops, best of 3: 143 usec per loop
$ python -mtimeit -s'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' 'sum(l, [])'
1000 loops, best of 3: 969 usec per loop
$ python -mtimeit -s'l=[[1,2,3],[4,5,6], [7], [8,9]]*99' 'reduce(lambda x,y: x+y,l)'
1000 loops, best of 3: 1.1 msec per loop

Explanation: The shortcuts based on

(including the implied use in

sum

) are, of necessity,

O(L**2)

when there are L sublists — as the intermediate result list keeps getting longer, at each step a new intermediate result list object gets allocated, and all the items in the previous intermediate result must be copied over (as well as a few new ones added at the end). So, for simplicity and without actual loss of generality, say you have L sublists of I items each: the first I items are copied back and forth L-1 times, the second I items L-2 times, and so on; total number of copies is I times the sum of x for x from 1 to L excluded, i.e.,

I * (L**2)/2

The list comprehension just generates one list, once, and copies each item over (from its original place of residence to the result list) also exactly once.

Alternative Answer:

You can use

itertools.chain()

import itertools
list2d = [[1,2,3], [4,5,6], [7], [8,9]]
merged = list(itertools.chain(*list2d))

Or you can use

itertools.chain.from_iterable()

which doesn’t require unpacking the list with the

operator:

import itertools
list2d = [[1,2,3], [4,5,6], [7], [8,9]]
merged = list(itertools.chain.from_iterable(list2d))

12. How to check if a list is empty?

Answer:

if not a:
  print("List is empty")

Using the implicit booleanness of the empty

list

is quite pythonic.

Alternative Answer:

The pythonic way to do it is from the PEP 8 style guide (where Yes means “recommended” and No means “not recommended”):

For sequences, (strings, lists, tuples), use the fact that empty sequences are false.

Yes: if not seq:
     if seq:

No:  if len(seq):
     if not len(seq):

In Conclusion

These are the 12 most commonly asked questions about Python. If you have any suggestions regarding the article, please feel free to comment below. If you need any help, then we would be glad to help you.

Hope this article helped you.