Publishing Python Packages on Pip and PyPI

Library diversity might be the trigger of being popular of python programming language nowadays. This encourages open source culture as well. Herein, PyPI (acronym of The Python Package Index) is the de facto package index standard even though some popular alternatives exist such as conda. You just need to run pip install command to install a package. Recently, I’ve published some of my open source projects via PyPI. In contrast to easy of installation, publishing a package is a troublesome for the first time. In this post, you will learn how to publish a package on PyPI.

pip_big
pip

Installation guide

Setup.py in the main directory will handle the installation on clients.


Visit Deep Learning Enabled Art Exhibition: Digital Van Gogh




Name must be unique in PyPI. It reserves the name for the first requestor.

This will put the content of README.md to long description. Herein, version is important variable because you have to increase this number even a code line changed. You cannot deploy different code with this version anymore. However, you can still delete this version if you realize a major bug later.

import setuptools

with open("README.md", "r", encoding="utf-8") as fh:
    long_description = fh.read()

setuptools.setup(
    name="deepface",  
    version="0.0.11",
    author="Sefik Ilkin Serengil",
    author_email="sxxx@gmail.com",
    description="Deep Face Anaylsis Framework for Face Recognition and Demography",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://github.com/serengil/deepface",
    packages=setuptools.find_packages(),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.5.5',
    install_requires=["numpy>=1.14.0", "pandas>=0.23.4", "tqdm>=4.30.0", "gdown>=3.10.1", "matplotlib>=2.2.2", "opencv-python>=3.4.4", "Pillow>=5.2.0", "tensorflow>=1.9.0", "keras>=2.2.0"]
)

Besides, if your package has some external packages as requirements, you can specify them install_requires variable. In this way, pip handles these requirements. For example, this deepface package needs numpy (min 1.14.0). Numpy will be installed as well when you call pip install deepface. You should put a folder having same name of your package in the root directory. Your unit tests should be put in the root directory as well.

You should specify the license information of your code here. LICENSE file should also exist in the root folder as well.

Initialization

You have to put dummy __init__.py files in all folders including root folder. Content of these files should be empty.

deepface/__init__.py

deepface/basemodels/__init__.py

deepface/extendedmodels/__init__.py





deepface/commons/__init__.py

Folder structure

Firstly, you should rearrange your folder structure. Suppose that you have DeepFace.py and basemodels folder in the same directory.

pip-root-folder
root folder
pip-subsdiary-folder
basemodels folder

You could normally import basemodels/Facenet.py from root/DeepFace.py in your regular python code as shown below.

from basemodels import Facenet #you should not import like this anymore

Instead of this direct usage, you have to specify that basemodels folder is under the deepface. Herein, deepface is the package name. In other words, you will install this package with the command pip install deepface.

from deepface.basemodels import Facenet

Local testing

If you rearrange your code based on these instructions, your code is ready to publish on pypi. However, you should test your code before publishing. You should change directory where setup.py exists in command prompt. Then call the following command. This will install your package locally.

pip install -e .

Then, you can test your code. For example, I can import the deepface/DeepFace.py file as demonstrated below.

from deepface import DeepFace

Before publishing

You will need some python packages to publish your package on pip. Call the following commands in your command prompt. This will install if they were not installed before. Otherwise, it will update installed versions.

python -m pip install --user --upgrade setuptools wheel
python -m pip install --user --upgrade twine

You need to register PyPI as well. You will use PyPI username and password.

Publishing

Open command prompt and change your directory where setup.py exists.

python3 setup.py sdist bdist_wheel

This will create a dist folder and put the compressed version of your code in here.





pip-dist
distribution folder

Then, you will need twine’s executable. It is installed C:\Users\Sefik\AppData\Roaming\Python\Python36\Scripts\twine.exe in my computer. You can add this path as environment variable as well.

We will upload everything under the dist folder to PyPI.

twine upload dist/*

Calling this command asks PyPI username and password.

So, publishing process is done. Anyone can install your package with just calling pip install command. In this way, the more people can access and use your code. Do not forget that open source is the future!


Like this blog? Support me on Patreon

Buy me a coffee