Phasing out Python runtimes gracefully

Dropping support of a Python runtime can be a tricky task for any package maintainer.

Python 2.6 is out of maintenance, 3.3 is now out of maintenance and both have syntax gaps with the common subset found in 2.7 and 3.4. When you might want to later only offer Python 3 support on a package, you don’t want to completely leave your Python 2 users in the cold.

The Python Packaging Authority have been working on solving that issue, with a combination of updates to PyPi (the packaging service), the Packaging API, Pip (the command line installer), setuptools (the library used to create the metadata) and the metadata format. All of these changes have come together to allow package maintainers to publish “maintenance releases” to old Python runtimes and new releases to supported Python runtimes.

How does it work today

To publish a package to PyPi, you need to follow 3 steps:

1. Define a setup.py file that includes distutils or setuptools and calls setup( with some information about the package
2. Call python setup.py sdist or python setup.py sdist bdist_wheel if you want to include a wheel file
3. Upload the package to PyPi using twine upload dist/* .

Defining the PKG-INFO

Within a Python source package (the zip or the tar-gz file you download) is a text file called PKG-INFO.

This file is generated by distutils or setuptools when it generates the source package. The file contains a set of keys and values, the list of keys is part of the PyPa standard metadata format. You can see the contents of the generated file like this:

tar xvfz dist/my-package-1.0.0.tar.gz -O | cat */PKG-INFO

At the top of the file should be the header Metadata-Version, which depends upon the version of setuptools or distutils you have installed. So the first thing you should do is upgrade setuptools and stop using distutils.

pip install — upgrade setuptools

Setuptools is the package being maintained by PyPa now, so make sure at the top of setup.py you are importing the setup method from the setuptools module.

If you have upgraded correctly, the Metadata-Version value should be 1.2 or higher. Metadata 1.2 introduced a new field for the version of Python required for this package.

You can specify version ranges and exclusion rules, such as at least Python 3. Or, Python 2.7 and 3.4 beyond.

Requires-Python: >=3Requires-Python: >2.7,!=3.0.*,!=3.1.*, !=3.2.*,!=3.3.*

The way to set those values is within the call to setup within your setup.py script.

Now, calling python setup.py sdist and then twine upload dist/* will publish version 1 of this package specifying that only Python 2.7+ is supported.

Verifying before you publish

Since it’s almost impossible to edit a published package on PyPi, you really want to check first, so run this command to validate the metadata has been generated correctly.

tar xvfz dist/my-package-1.0.0.tar.gz -O | grep “Requires-Python”

Using Twine to publish

Anyone used to using setup.py to publish the package to PyPi, you can also stop doing that now. Twine has a number of advantages, apart from being faster is now the supported method for publishing packages.

Make sure you are using the newest version of twine (1.9 at the time of writing).

Dropping a Python release

Once you have published a package with the metadata, you can then make a further update removing that Python runtime from support.

It must be done in this order for the automated fail-back to work.

This is a pretty typical timeline

1. You publish 1.0 of your package before reading this article
2. You publish 1.1.0 after the reading the PEP on semantic versioning
3. You read this article and added python_requires in setup.py for all versions 2.7+
4. Many seasons pass, versions go by
5. You publish 1.6.0 supporting Python 2.7+
6. You announce to your users that you are dropping Python 2 support in a month
7. You modify the python_requires field to >3.3 and publish version 2.0.0 of your package

Now, if the user has pip >9.0 and they install your package from Python 2 without specifying a version number, they get the last version of the package to support their Python distribution!

If a user has an in-support Python distribution as specified in your setup.py then they get the latest version.

Try it out on a package I published to PyPi called friendly-deprecation-test.

But wait…

Someone raises a critical bug against v1.6.0 and you don’t want to leave your Python 2 users at risk. What do you do?

1. Assuming you have 1.6 in a branch, apply the fix. Bump the version on the branch to 1.6.1 and publish it with the requires_python >= 2.7 attribute. Release it to PyPi.
2. Apply the fix to the master branch and release 2.0.1 with the requires_python >= 3.4 attribute.

Now the same process applies, all Python 2 users will get 1.6.1 and Python 3.4+ will get 2.0.1.