If you are a Python programmer, it is quite likely that you have experience in shell scripting. It is not uncommon to face a task that seems trivial to solve with a shell command. Therefore, it is useful to be familiar with how to call these commands efficiently from your Python code and know their limitations. In this short article, I discuss how to use the older (although still relatively common) command and the newer command. I will show some of their potential risks, limitations and provide complete examples of their use. os.system subprocess External commands might be an attractive option for small tasks if you are familiar with them and the pythonic alternative would imply more learning/implementation time. However, Python functions will save you time and trouble in the long run. Option 1: OS.system (deprecated) I show below a simple example of to print the first line of a file provided by the user, using . os.system head -n 1 os command = filename = input( ) return_value = os.system(command+filename)
print( )
print( , return_value) #!/usr/bin/env python3 import # Define command and options wanted "head -n 1 " # Ask user for file name(s) - SECURITY RISK: susceptible to shell injection "Please introduce name of file of interest:\n" # Run os.system and save return_value '###############' 'Return Value:' The function is easy to use and interpret: simply use as input of the function the same command you would use in a shell. However, it is deprecated and now. os.system it is recommended to use subprocess Note that this function will simply execute the shell command and the result will be printed to the standard output, but the output that the function returns is the (0 if it ran OK, and different than 0 otherwise). return value 1.1: Susceptibility to shell injection Among other drawbacks, directly executes the command in a shell, which means that it is susceptible to (aka command injection). You can read more about it in by Anthony Shaw). os.system shell injection 10 common security gotchas in Python and how to avoid them Shell injection is an issue any time that is receiving unformatted input, like for example when a user can introduce a filename, as in the example above. You can try to execute the script above and give the following input: os.system dummy; touch harmful_file This will result in the shell doing: head -n 1 dummy; touch harmful_file As a result, the program will first execute , as expected, but then it will execute the command to create a file named 'harmful_file'. head -n 1 dummy touch harmful_file Granted that this empty file is not much of a threat, but you can imagine a user adding extra commands to create a file with actual nefarious purposes. This shell injection can also be used to simply transfer or delete information. For example, one could use , or any other potentially dangerous command (please try these!). ;rm -rf ~ ;rm -rf / do not Option 2: subprocess.call (recommended for Python<3.5) 2.1: Not-secure way to run external shell commands A preferable alternative is . As os.sys, the function subprocess.call returns the as its output. A naive first approach to subprocess is using the option. This way, the desired command will also be run in a subshell. Note that this is recommended, as it has the same potential security risks as . For example, the code below would be equivalent to the previous example. subprocess.call return value shell=True not os.system os.system subprocess command = filename = input( ) return_value = subprocess.call(command+filename, shell= )
print( )
print( , return_value) #!/usr/bin/env python3 import # Define command and options wanted "head -n 1 " # Ask user for file name(s) - SECURITY RISK: susceptible to shell injection "Please introduce name of file of interest:\n" # Run subprocess.call and save return_value True '###############' 'Return value:' 2.2: Preferred way to run external commands A preferable way to run is by using (it is the default option, so there is no need to specify it). Then, we can simply call , where contains the command, and contains all the extra options to the command. subprocess.call shell=False subprocess.call(args) agrs[0] args[1:] subprocess command = options = filename = input( ) args=[]
args.append(command)
args.append(options) i filename.split():
    args.append(i) return_value = subprocess.call(args)
print( )
print( , return_value) #!/usr/bin/env python3 import # Define command and options wanted "head" "-n 1" # Ask user for file name(s) - now it's safe from shell injection "Please introduce name(s) of file(s) of interest:\n" # Create list with arguments for subprocess.call for in # Run subprocess.call and save return_value '###############' 'Return value:' 2.3: Getting standard output: subprocess.check_output As mentioned before, returns the . However, sometimes we might be interested in the standard output returned by the shell command. In this case, we can make use of . subprocess.call return value subprocess.check_output To make our script more robust, we can add a try/exclude statement to deal with situations when raises an error (in this case, for example, when no file is found). check_output As shown below, we can use in the except clause to deal with the error in a controlled manner and get information about it. subprocess.CalledProcessError subprocess command = options = filename = input( ) args=[]
args.append(command)
args.append(options) i filename.split():
    args.append(i) :
    output = subprocess.check_output(args) print( )
    print( , output.decode( )) subprocess.CalledProcessError error:
    print( , error.returncode, , error.output.decode( )) #!/usr/bin/env python3 import # Define command and options wanted "head" "-n 1" # Ask user for file name(s) - now it's safe from shell injection "Please introduce name(s) of file(s) of interest:\n" # Create list with arguments for subprocess.check_output for in # Run subprocess.check_output and save command output try # use decode function to convert to string '###############' 'Output:' "utf-8" # If check_output returns an error: except as 'Error code:' '. Output:' "utf-8" Option 3: subprocess.run (recommended since Python 3.5) The , since Python 3.5, is with the function. It is more secure and user-friendly than the previous options discussed. recommended way to execute external shell commands subprocess.run By default, this function returns an object with the input command and the return code. One can very easily also get the standard output by using the option , and finally retrieve the return code and command output using: and , respectively. capture_output=True output.returncode output.stdout subprocess command = options = filename = input( ) args=[]
args.append(command)
args.append(options) i filename.split():
    args.append(i) output = subprocess.run(args,capture_output= )
print( )
print( , output.returncode) print( ,output.stdout.decode( )) #!/usr/bin/env python3 import # Define command and options wanted "head" "-n 1" # Ask user for file name(s) - now it's safe from shell injection "Please introduce name(s) of file(s) of interest:\n" # Create list with arguments for subprocess.run for in # Run subprocess.run and save output object True '###############' 'Return code:' # use decode function to convert to string 'Output:' "utf-8" Final Note on Shell Commands in Python If you are thinking about using any of the methods discussed here to call an external command, it might be worth considering if there is a standard Python function that allows you to do the same task and will avoid creating a new process. Note that using external commands also makes your program less cross-platform friendly. External commands might be an attractive option for small tasks if you are familiar with them and the pythonic alternative would imply more learning/implementation time. However, sticking to Python functions will save you computation time and trouble in the long run, so external commands should be saved for tasks that cannot be achieved with standard Python libraries. I hope this article helped you to call external commands from your python code! Featured image by on , used with permission. JPuckett Design Instagram

Instagram

Visit my science and programming blog

Read My Stories

Too Long; Didn't Read

Learn how to continuously improve your codebase

Calling Shell Commands from Python: OS.system vs Subprocess

Too Long; Didn't Read

People Mentioned

Company Mentioned

@marcosdelcueto

RELATED STORIES