An Introduction to Python Packages for Absolute Beginners

Knowing how to work with packages is an important skill for any Python developer. However, unlike the language itself, there is nothing Zen about packaging in Python. Packaging is one of the hardest things to learn in Python. That’s why we asked two Mighty Avengers to help us. We’ll work with Clint and Natasha on a small application, starting with a couple of modules and finish with our very own python package. The code examples in this article can be found . here Modules We break down our code into modules for modularity. A python module is just one file containing code. Modular code is easier to work with. Why? Less stuff to keep in (human) memory while coding, less scrolling, knowing exactly where to look when you have to find a particular thing, breaking big problems into smaller ones, et cetera. Let’s meet Clint. Clint is working on a new app, which prints a message based on the command line argument entered. The app uses a custom print function. Clint asks Natasha to write that code for him. Natasha creates the module for Clint, who imports it in his module. bprint app """01/app.pyMain app module.""" import sys import bprint commands = {"reboot": "Rebooting launch control system.","halt": "Halting all systems.",} try:command = sys.argv[1]except IndexError:bprint.upper_print("No command provided. Terminating.")sys.exit(1)else:print(commands.get(command, "Doing absolutely nothing.")) Here’s the module. bprint """01/bprint.pyCool print functions.""" def upper_print(message):print(message.upper()) if __name__ == '__main__':print("Hello little child.")upper_print("You have run bprint as a script.")print("That seems great. Goodnight.") Let’s see Clint’s directory structure. .├── app.py└── bprint.py 0 directories, 2 files Clint can run his app using: python3 app.py Awesome! Modules are pretty helpful. Clint and Natasha did a great job together. It doesn’t seem like they needed any packages yet. Packages In the next version of his app, Clint decides that he needs more custom print functions to display his messages. Again, he asks Natasha to help him out. Natasha writes a module containing the code that Clint requires. sprint Clint can just copy the files in the same directory along with his and use them. app.py .├── app.py├── bprint.py└── sprint.py 0 directories, 3 files But now Clint feels uneasy, his code is mixed up with Natasha’s code. This will cause problems as the size of the code base increases. He decides to re-organize it. So he moves Natasha’s code into a sub-folder and modifies the imports in the file. app.py .├── app.py└── print_helpers├── bprint.py└── sprint.py 1 directory, 3 files Clint’s modified file. app.py """02/app.pyClint's new main app module.""" import sys from print_helpers import bprint, sprint commands = {"reboot": "Rebooting launch control system.","halt": "Halting all systems.",} try:command = sys.argv[1]except IndexError:bprint.upper_print("No command provided. Terminating.")sys.exit(1)else:sprint.tacky_print(commands.get(command, "Doing absolutely nothing.")) And that’s it. Anyone who wants to use Natasha’s code can (with her permission) copy the directory to their code and . print_helpers import Clint just created a package of Natasha’s code. A python package is collection of modules. Developers generally put modules related to each other in the same package. Distribution Natasha is not satisfied with this. Copy pasting complete packages is error prone. It feels like a temporary hack. Natasha tries to learn more about packages and finds that with just a little more work, she can distribute her code in a better way. By providing a install script. setup.py A lot can go inside the file. Here is the minimal example that Natasha uses. setup.py """03/setup.pyNatasha's initial setup.py""" from setuptools import setup, find_packages setup(name='print-helpers',version='0.1',packages=find_packages(),) Natasha also creates an empty file named inside the directory. This is required by python. A directory containing a is identified as a package. __init__.py print_helpers __init__.py .├── print_helpers│ ├── bprint.py│ ├── __init__.py│ └── sprint.py└── setup.py 1 directory, 4 files Natasha can now pack all of her code into a single archive file and distribute it. Wheels are the recommended format for distributing packages. Natasha can create a wheel for her package by running python3 setup.py bdist_wheel This will generate a lot of new files along with the in the dist/ directory. wheel .├── build│ ├── bdist.linux-x86_64│ └── lib│ └── print_helpers│ ├── bprint.py│ ├── __init__.py│ └── sprint.py├── dist│ └── print_helpers-0.1-py3-none-any.whl├── print_helpers│ ├── bprint.py│ ├── __init__.py│ └── sprint.py├── print_helpers.egg-info│ ├── dependency_links.txt│ ├── PKG-INFO│ ├── SOURCES.txt│ └── top_level.txt└── setup.py 7 directories, 12 files Natasha can then send this file to Clint. .whl Using packages Clint can now use to install Natasha’s package. Pip is a command line utility to manage python packages. Here’s the command Clint runs. pip pip install ./print_helpers-0.1-py3-none-any.whl Installing a package will automatically copy Natasha’s code files from the package to specific locations on Clint’s computer. These are the locations where Python looks for modules when it encounters an import statement in the code. pip Check the output of to examine the location of the newly installed package. pip show $ pip show print-helpers Name: print-helpersVersion: 0.1Summary: UNKNOWNHome-page: UNKNOWNAuthor: UNKNOWNAuthor-email: UNKNOWNLicense: UNKNOWNLocation: /home/ramit/my_py_env/lib/python3.6/site-packagesRequires:Required-by: This location will vary from one computer to the next, but you will also find this location in which is a list of directories where python searches for modules. sys.path Importing a package Any Python code running on Clint’s computer can now import Natasha’s package. Clint does not need to maintain a copy of Natasha’s code along with his own in his projects. """04/app.py""" import sys from print_helpers import bprint, sprint commands = {"reboot": "Rebooting launch control system.","halt": "Halting all systems.",} try:command = sys.argv[1]except IndexError:bprint.upper_print("No command provided. Terminating.")sys.exit(1)else:sprint.tacky_print(commands.get(command, "Doing absolutely nothing.")) Directory structure of Clint’s app. .└── app.py 0 directories, 1 file Packaging can do more PyPI Pip can install packages from PyPI, local or remote archives, Github repos or project directories. Python packages are generally uploaded to which serves as a repository for python packages. Natasha can upload her package to PyPI and anyone wanting to use her package can install the package directly using , eliminating the need to acquire a package archive manually. This is what most of us are familiar with. PyPI pip Extra files Try finding a few python packages on Github. Here’s if you can’t remember any. Here’s . Examine the directory structure of the repository linked above. one another .├── LICENSE.txt├── MANIFEST.in├── README.md├── requirements.txt├── setup.py├── src│ └── fsanitize│ ├── app.py│ ├── __init__.py│ ├── logmgr.py│ └── sanitize.py└── tests├── __init__.py└── test_all.py 3 directories, 11 files These contain a lot of files along with . These can include tests, test configs, manifest files, license information, documentation, build scripts and much more. These are distributed along with the python modules inside the package. setup.py Application Packages Ideally, Clint would also create a package for his app and specify Natasha’s package as a dependency. Dependencies get installed along with the main package automatically. Learn More Verify that that install location of is present in python’s print-helpers sys.path Create a wheel from Natasha’s package and install it. Run from the command line. bprint Find out what and do. console_scripts entry_points Try packaging Natasha’s app without the file and running from the command line. __init__.py bprint Create a package for Clint’s app specifying Natasha’s package as a dependency. Try finding out what the keyword argument does. install_requires Learn more about . python packaging The python packaging ecosystem is constantly evolving. Be sure to check out , a new tool to package python apps. poetry