From 215ee011d33cc9df89103f327d5e2962e3905824 Mon Sep 17 00:00:00 2001 From: Ethan Paul <24588726+enpaul@users.noreply.github.com> Date: Wed, 23 Sep 2020 23:24:59 -0400 Subject: [PATCH] Update problem and usecase description to reflect wider context This version also not written at 3am --- README.md | 144 +++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 109 insertions(+), 35 deletions(-) diff --git a/README.md b/README.md index 5230921..cd144b1 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ dependencies to be installed using [Poetry](https://python-poetry.org/) using it * [Installation and Usage](#installation-and-usage) * [Limitations](#limitations) -* [What problem does this solve?](#what-problems-does-this-solve) (Why would I use this?) +* [Why would I use this?](#what-problems-does-this-solve) (What problems does this solve?) * [Developing](#developing) * [Contributing](#contributing) * [Roadmap](#roadmap) @@ -84,52 +84,126 @@ poetry run tox --recreate in the Tox configuration. -## What problems does this solve? +## Why would I use this? -[The point of using a lockfile is to create reproducable builds](https://docs.gradle.org/current/userguide/dependency_locking.html). One of the main points of Tox is to [allow a Python -package to be built and tested in multiple environments](https://tox.readthedocs.io/en/latest/#what-is-tox). However, in the Tox configuration file the dependencies are specified with -standard dynamic ranges and passed directly to Pip. This means that the reproducability -a lockfile brings to a project is circumvented when running the tests. +**Introduction** -The obvious solution to this problem is to add the dependencies required for testing to the -lockfile as development dependencies so that they are locked along with the primary dependencies -of the project. The only remaining question however, is how to install the dev-dependencies from -the lockfile into the Tox environment when Tox sets it up. [For very good reason](https://dev.to/elabftw/stop-using-sudo-pip-install-52mn) Tox uses independent -[virtual environments](https://docs.python.org/3/tutorial/venv.html) for each environment a -project defines, so there needs to be a way to install a locked dependency into a Tox -environment. +The lockfile is a file generated by a package manager for a project that lists what +dependencies are installed, the versions of those dependencies, and additional metadata that +the package manager can use to recreate the local project environment. This allows developers +to have confidence that a bug they are encountering that may be caused by one of their +dependencies will be reproducible on another device. In addition, installing a project +environment from a lockfile gives confidence that automated systems running tests or performing +builds are using the same environment that a developer is. -This is where this plugin comes in. +[Poetry](https://python-poetry.org/) is a project dependency manager for Python projects, and +as such it creates and manages a lockfile so that its users can benefit from all the features +described above. [Tox](https://tox.readthedocs.io/en/latest/#what-is-tox) is an automation tool +that allows Python developers to run tests suites, perform builds, and automate tasks within +self contained [Python virtual environments](https://docs.python.org/3/tutorial/venv.html). +To make these environments useful, Tox supports installing per-environment dependencies. +However, since these environments are created on the fly and Tox does not maintain a lockfile, +there can be subtle differences between the dependencies a developer is using and the +dependencies Tox uses. -Traditionally Tox environments specify dependencies and their corresponding versions inline in -[PEP-440](https://www.python.org/dev/peps/pep-0440/) format like below: +This is where this plugin comes into play. + +By default Tox uses [Pip](https://docs.python.org/3/tutorial/venv.html) to install the +PEP-508 compliant dependencies to a test environment. A more robust way to do this is to +install dependencies directly from the lockfile so that the version installed to the Tox +environment always matches the version Poetry specifies. This plugin overwrites the default +Tox dependency installation behavior and replaces it with a Poetry-based installation using +the dependency metadata from the lockfile. + +**The Problem** + +Environment dependencies for a Tox environment are usually done in PEP-508 format like the +below example ```ini +# tox.ini +... + [testenv] -description = Run the tests +description = Some very cool tests deps = - foo == 1.2.3 - bar >=1.3,<2.0 - baz + foo == 1.2.3 + bar >=1.3,<2.0 + baz + +... ``` -This runs into the problem outlined above: many different versions of the `bar` dependency -could be installed depending on what the latest version is that matches the defined range. The -`baz` dependency is entirely unpinned making it a true wildcard, and even the seemingly static -`foo` dependency could result in subtly different files being downloaded depending on what's -available in the upstream mirrors. +Perhaps these dependencies are also useful during development, so they can be added to the +Poetry environment using this command: -However these same versions, specified in the [pyproject.toml](https://snarky.ca/what-the-heck-is-pyproject-toml/) file, result in reproducible -installations when using `poetry install` because they each have a specific version and file -hash specified in the lockfile. The versions specified in the lockfile are updated only when -`poetry update` is run. + ``` + poetry add foo==1.2.3 bar>=1.3,<2.0 baz --dev + ``` -This plugin allows environment dependencies to be specified in the [tox.ini](https://tox.readthedocs.io/en/latest/config.html) configuration file -just by name. The package is automatically retrieved from the lockfile and the Poetry backend -is used to install the singular locked package version to the Tox environment. When the -lockfile is updated, the Tox environment will automatically install the newly locked package -as well. All dependency requirements are specified in one place (pyproject.toml), all -dependencies have a locked version, and everything is installed from that source of truth. + However there are three potential problems that could arise from each of these environment + dependencies that would _only_ appear in the Tox environment and not in the Poetry + environment: + + * **The `foo` dependency is pinned to a specific version:** let's imagine a security + vulnerability is discovered in `foo` and the maintainers release version `1.2.4` to fix + it. A developer can run `poetry remove foo && poetry add foo^1.2` to get the new version, + but the Tox environment is left unchanged. The developer environment specified by the + lockfile is now patched against the vulnerability, but the Tox environment is not. + +* **The `bar` dependency specifies a dynamic range:** a dynamic range allows a range of + versions to be installed, but the lockfile will have an exact version specified so that + the Poetry environment is reproducible; this allows versions to be updated with + `poetry update` rather than with the `remove` and `add` used above. If the maintainers of + `bar` release version `1.6.0` then the Tox environment will install it because it is valid + for the specified version range, meanwhile the Poetry environment will continue to install + the version from the lockfile until `poetry update bar` explicitly updates it. The + development environment is now has a different version of `bar` than the Tox environment. + +* **The `baz` dependency is unpinned:** unpinned dependencies are + [generally a bad idea](https://python-poetry.org/docs/faq/#why-are-unbound-version-constraints-a-bad-idea), + but here it can cause real problems. Poetry will interpret an unbound dependency using + [the carrot requirement](https://python-poetry.org/docs/dependency-specification/#caret-requirements) + but Pip (via Tox) will interpret it as a wildcard. If the latest version of `baz` is `1.0.0` + then `poetry add baz` will result in a constraint of `baz>=1.0.0,<2.0.0` while the Tox + environment will have a constraint of `baz==*`. The Tox environment can now install an + incompatible version of `baz` that cannot be easily caught using `poetry update`. + +All of these problems can apply not only to the dependencies specified for a Tox environment, +but also to the dependencies of those dependencies, and so on. + +**The Solution** + +This plugin requires that all dependencies specified for all Tox environments be unbound +with no version constraint specified at all. This seems counter-intuitive given the problems +outlined above, but what it allows the plugin to do is offload all version management to +Poetry. + +On initial inspection, the environment below appears less stable than the one presented above +because it does not specify any versions for its dependencies: + +```ini +# tox.ini +... + +[testenv] +description = Some very cool tests +deps = + foo + bar + baz + +... +``` + +However with the `tox-poetry-installer` plugin installed this instructs Tox to install these +dependencies using the Poetry lockfile so that the version installed to the Tox environment +exactly matches the version Poetry is managing. When `poetry update` updates the lockfile +with new dependency versions, Tox will automatically install these new versions without needing +any changes to the configuration. + +All dependencies are specified in one place (the lockfile) and dependency version management is +handled by a tool dedicated to that task (Poetry). ## Developing