Build a private pypi server

To run onyxia without internet, we need to build a private pypi server which can respond all the service which need to run pip install.

To build a private pypi server, we need three modules: 1. download(mirror) the required python packages from official pypi server, and build the local package index 2. Serve the local index and the packages behind it. 3. A cron job which sync the private server with the official server.

Existing solutions:

For now, the below list show the available tools * pypi-server: https://github.com/pypiserver/pypiserver * devpi: https://github.com/devpi/devpi (focus on releasing private python package) * bandersnatch: https://github.com/pypa/bandersnatch (focus on mirroring the official repo ) * localshop: https://github.com/jazzband/localshop (too old, no release since 2015) * JFog Artifactory: https://jfrog.com/artifactory (commercial license, the free version is too broken to meet our requirements) * Warehouse: https://warehouse.pypa.io/application.html (The official codebase of pypi server)

Https certificates

The pip client does not use the system trust store. So to make pip to accept a not trusted certificate is not simple.

The official doc can be found https://pip.pypa.io/en/stable/topics/https-certificates/

Dependency resolution of pip

https://pip.pypa.io/en/latest/topics/dependency-resolution/

Our solution

For now we choose Bandersnatch to build our private pypi repository. Because it offers a mirror client and can build the package index for the private pypi server. Bandersnatch also implments the PEP 381 + PEP 503 + PEP 691 and PEP381.

Version features: * bandersnatch >=6.0 implements PEP691 * bandersnatch >=4.0 supports Linux, MacOSX + Windows

The official documentation can be foun here