mirror of
https://github.com/enpaul/tox-poetry-installer.git
synced 2024-10-29 19:47:00 +00:00
Proofreading and editing fixes
Check spelling more thoroughly Improve clarity in a few places Fix grammar mistakes Fix docs that didn't get updated for the 0.2 overhaul
This commit is contained in:
parent
2e1d5fc922
commit
5411025612
151
README.md
151
README.md
@ -1,9 +1,9 @@
|
||||
# tox-poetry-installer
|
||||
|
||||
A plugin for [Tox](https://tox.readthedocs.io/en/latest/) that allows test environment
|
||||
dependencies to be installed using [Poetry](https://python-poetry.org/) using its lockfile.
|
||||
dependencies to be installed using [Poetry](https://python-poetry.org/) from its lockfile.
|
||||
|
||||
⚠️ **This project is alpha software and should not be used in a production capacity** ⚠️
|
||||
⚠️ **This project is alpha software and should not be used in production environments** ⚠️
|
||||
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
[](https://pypi.org/project/tox-poetry-installer/)
|
||||
@ -31,7 +31,7 @@ Related resources:
|
||||
|
||||
## Installation
|
||||
|
||||
Add the plugin as a development dependency a project using Poetry:
|
||||
Add the plugin as a development dependency of a Poetry project:
|
||||
|
||||
```
|
||||
~ $: poetry add tox-poetry-installer --dev
|
||||
@ -43,11 +43,10 @@ Confirm that the plugin is installed, and Tox recognizes it, by checking the Tox
|
||||
~ $: poetry run tox --version
|
||||
3.20.0 imported from .venv/lib64/python3.8/site-packages/tox/__init__.py
|
||||
registered plugins:
|
||||
tox-poetry-installer-0.2.0 at .venv/lib64/python3.8/site-packages/tox_poetry_installer.py
|
||||
tox-poetry-installer-0.2.2 at .venv/lib64/python3.8/site-packages/tox_poetry_installer.py
|
||||
```
|
||||
|
||||
If using in a CI/automation environment using Pip, ensure that the plugin is installed to the
|
||||
same environment as Tox:
|
||||
If using Pip, ensure that the plugin is installed to the same environment as Tox:
|
||||
|
||||
```
|
||||
# Calling the virtualenv's 'pip' binary directly will cause pip to install to that virtualenv
|
||||
@ -77,8 +76,8 @@ commands = ...
|
||||
```
|
||||
|
||||
To require specific dependencies be installed from the Poetry lockfile, and let the rest be
|
||||
installed using the default Tox installation method, add the suffix `@poetry` to the dependencies.
|
||||
In the example below the `pytest`, `pytest-cov`, and `black` dependencies will be installed using
|
||||
installed using the default Tox installation backend, add the suffix `@poetry` to the dependencies.
|
||||
In the example below the `pytest`, `pytest-cov`, and `black` dependencies will be installed from
|
||||
the lockfile while `pylint` and `mypy` will be installed using the versions specified here:
|
||||
|
||||
```ini
|
||||
@ -100,7 +99,7 @@ one Tox is testing) will always be installed from the lockfile.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
After installing the plugin to a project, your Tox automation is already benefiting from the
|
||||
After installing the plugin to a project your Tox automation is already benefiting from the
|
||||
lockfile: when Tox installs your project package to one of your environments, all the dependencies
|
||||
of your project package will be installed using the versions specified in the lockfile. This
|
||||
happens automatically and requires no configuration changes.
|
||||
@ -108,7 +107,7 @@ happens automatically and requires no configuration changes.
|
||||
But what about the rest of your Tox environment dependencies?
|
||||
|
||||
Let's use an example `tox.ini` file, below, that defines two environments: the main `testenv` for
|
||||
running the project tests and `testenv:check` for running some other helpful checks:
|
||||
running the project tests and `testenv:check` for running some other helpful tools:
|
||||
|
||||
```ini
|
||||
[tox]
|
||||
@ -152,9 +151,9 @@ Running Tox using this config gives us this error:
|
||||
tox_poetry_installer.LockedDepVersionConflictError: Locked dependency 'pylint >=2.4.4,<2.6.0' cannot include version specifier
|
||||
```
|
||||
|
||||
This is because we told the Tox environment to require all dependencies to be locked, but then also
|
||||
This is because we told the Tox environment to require all dependencies be locked, but then also
|
||||
specified a specific version constraint for Pylint. With the `require_locked_deps = true` setting
|
||||
Tox expects all dependencies to take their version from the lockfile, so when it got conflicting
|
||||
Tox expects all dependencies to take their version from the lockfile, so when it gets conflicting
|
||||
information it errors. We can fix this by simply removing all version specifiers from the
|
||||
environment dependency list:
|
||||
|
||||
@ -175,9 +174,9 @@ recreated.
|
||||
|
||||
Now let's look at the `testenv` environment. Let's make the same changes to the `testenv`
|
||||
environment that we made to `testenv:check` above; remove the PyTest version and add
|
||||
`require_locked_deps = true`. Then imagine that we want to add a new (made up) tool the test
|
||||
environment called `crash_override` to the environment: we can add `crash-override` as a dependency
|
||||
of the test environment, but this will cause an error:
|
||||
`require_locked_deps = true`. Then imagine that we want to add the
|
||||
[Requests](https://requests.readthedocs.io/en/master/) library to the test environment: we
|
||||
can add `requests` as a dependency of the test environment, but this will cause an error:
|
||||
|
||||
```ini
|
||||
[testenv]
|
||||
@ -185,30 +184,29 @@ description = Run the tests
|
||||
require_locked_deps = true
|
||||
deps =
|
||||
pytest
|
||||
crash-override
|
||||
requests
|
||||
commands = ...
|
||||
```
|
||||
|
||||
Running Tox with this config gives us this error:
|
||||
|
||||
```
|
||||
tox_poetry_installer.LockedDepNotFoundError: No version of locked dependency 'crash-override' found in the project lockfile
|
||||
tox_poetry_installer.LockedDepNotFoundError: No version of locked dependency 'requests' found in the project lockfile
|
||||
```
|
||||
|
||||
This is because `crash-override` is not in our lockfile. Tox will refuse to install a dependency
|
||||
This is because `requests` is not in our lockfile yet. Tox will refuse to install a dependency
|
||||
that isn't in the lockfile to an an environment that specifies `require_locked_deps = true`. We
|
||||
could fix this (if `crash-override` was a real package) by running
|
||||
`poetry add crash-override --dev` to add it to the lockfile.
|
||||
can fix this by running `poetry add requests --dev` to add it to the lockfile.
|
||||
|
||||
Now let's combine dependencies from the lockfile ("locked dependencies") with dependencies that are
|
||||
specified inline in the environment configuration ("unlocked dependencies").
|
||||
[This isn't generally recommended of course](#why-would-i-use-this), but it's a valid use case and
|
||||
fully supported by this plugin. Let's modify the `testenv` configuration to install PyTest from the
|
||||
lockfile but then install an older version of the
|
||||
[Requests](https://requests.readthedocs.io/en/master/) library.
|
||||
Now let's combine dependencies from the lockfile with dependencies that are
|
||||
specified in-line in the Tox environment configuration.
|
||||
[This isn't generally recommended](#why-would-i-use-this), but it is a valid use case and
|
||||
fully supported by this plugin. Let's modify the `testenv` configuration to install PyTest
|
||||
from the lockfile but then install an older version of the Requests library.
|
||||
|
||||
The first thing to do is remove the `require_locked_deps = true` setting so that we can install
|
||||
Requests as an unlocked dependency. Then we can add our version of requests to the dependency list:
|
||||
Requests as an unlocked dependency. Then we can add our version specifier to the `requests`
|
||||
entry in the dependency list:
|
||||
|
||||
```ini
|
||||
[testenv]
|
||||
@ -220,7 +218,8 @@ commands = ...
|
||||
```
|
||||
|
||||
However we still want `pytest` to be installed from the lockfile, so the final step is to tell Tox
|
||||
to install it from the lockfile by adding the suffix `@poetry` to it:
|
||||
to install it from the lockfile by adding the suffix `@poetry` to the `pytest` entry in the
|
||||
dependency list:
|
||||
|
||||
```ini
|
||||
[testenv]
|
||||
@ -233,7 +232,7 @@ commands = ...
|
||||
|
||||
Now when the `testenv` environment is created it will install PyTest (and all of its dependencies)
|
||||
from the lockfile while it will install Requests (and all of its dependencies) using the default
|
||||
Tox installation backend using Pip.
|
||||
Tox installation backend.
|
||||
|
||||
|
||||
## Known Drawbacks and Problems
|
||||
@ -256,7 +255,7 @@ Tox installation backend using Pip.
|
||||
|
||||
* Tox environments automatically inherit their settings from the main `testenv` environment. This
|
||||
means that if the `require_locked_deps = true` is specified for the `testenv` environment then
|
||||
all environments will also require locked dependencies. This can be overridden by explicitly
|
||||
all environments will also require locked dependencies. This can be overwritten by explicitly
|
||||
specifying `require_locked_deps = false` on child environments where unlocked dependencies are
|
||||
needed.
|
||||
|
||||
@ -272,20 +271,20 @@ Tox installation backend using Pip.
|
||||
|
||||
**Introduction**
|
||||
|
||||
The lockfile is a file generated by a package manager for a project that lists what
|
||||
dependencies are installed, the versions of those dependencies, and additional metadata that
|
||||
the package manager can use to recreate the local project environment. This allows developers
|
||||
The lockfile is a file generated by a package manager for a project that records what
|
||||
dependencies are installed, the versions of those dependencies, and any additional metadata that
|
||||
the package manager needs to recreate the local project environment. This allows developers
|
||||
to have confidence that a bug they are encountering that may be caused by one of their
|
||||
dependencies will be reproducible on another device. In addition, installing a project
|
||||
environment from a lockfile gives confidence that automated systems running tests or performing
|
||||
builds are using the same environment that a developer is.
|
||||
builds are using the same environment as a developer.
|
||||
|
||||
[Poetry](https://python-poetry.org/) is a project dependency manager for Python projects, and
|
||||
as such it creates and manages a lockfile so that its users can benefit from all the features
|
||||
so it creates and manages a lockfile so that its users can benefit from all the features
|
||||
described above. [Tox](https://tox.readthedocs.io/en/latest/#what-is-tox) is an automation tool
|
||||
that allows Python developers to run tests suites, perform builds, and automate tasks within
|
||||
self contained [Python virtual environments](https://docs.python.org/3/tutorial/venv.html).
|
||||
To make these environments useful, Tox supports installing per-environment dependencies.
|
||||
self-contained [Python virtual environments](https://docs.python.org/3/tutorial/venv.html).
|
||||
To make these environments useful Tox supports installing dependencies in each environment.
|
||||
However, since these environments are created on the fly and Tox does not maintain a lockfile,
|
||||
there can be subtle differences between the dependencies a developer is using and the
|
||||
dependencies Tox uses.
|
||||
@ -293,19 +292,17 @@ dependencies Tox uses.
|
||||
This is where this plugin comes into play.
|
||||
|
||||
By default Tox uses [Pip](https://docs.python.org/3/tutorial/venv.html) to install the
|
||||
PEP-508 compliant dependencies to a test environment. A more robust way to do this is to
|
||||
install dependencies directly from the lockfile so that the version installed to the Tox
|
||||
environment always matches the version Poetry specifies. This plugin overwrites the default
|
||||
Tox dependency installation behavior and replaces it with a Poetry-based installation using
|
||||
the dependency metadata from the lockfile.
|
||||
PEP-508 compliant dependencies to a test environment. This plugin extends the default
|
||||
Tox dependency installation behavior to support installing dependencies using a Poetry-based
|
||||
installation method that makes use of the dependency metadata from Poetry's lockfile.
|
||||
|
||||
**The Problem**
|
||||
|
||||
Environment dependencies for a Tox environment are usually done in PEP-508 format like the
|
||||
below example
|
||||
Environment dependencies for a Tox environment are usually specified in PEP-508 format, like
|
||||
the below example:
|
||||
|
||||
```ini
|
||||
# tox.ini
|
||||
# from tox.ini
|
||||
...
|
||||
|
||||
[testenv]
|
||||
@ -318,31 +315,35 @@ deps =
|
||||
...
|
||||
```
|
||||
|
||||
Perhaps these dependencies are also useful during development, so they can be added to the
|
||||
Let's assume these dependencies are also useful during development, so they can be added to the
|
||||
Poetry environment using this command:
|
||||
|
||||
```
|
||||
poetry add foo==1.2.3 bar>=1.3,<2.0 baz --dev
|
||||
poetry add --dev \
|
||||
foo==1.2.3 \
|
||||
bar>=1.3,<2.0 \
|
||||
baz
|
||||
```
|
||||
|
||||
However there are three potential problems that could arise from each of these environment
|
||||
However there is a potential problem that could arise from each of these environment
|
||||
dependencies that would _only_ appear in the Tox environment and not in the Poetry
|
||||
environment:
|
||||
environment in use by a developer:
|
||||
|
||||
* **The `foo` dependency is pinned to a specific version:** let's imagine a security
|
||||
vulnerability is discovered in `foo` and the maintainers release version `1.2.4` to fix
|
||||
it. A developer can run `poetry remove foo && poetry add foo^1.2` to get the new version,
|
||||
but the Tox environment is left unchanged. The developer environment specified by the
|
||||
lockfile is now patched against the vulnerability, but the Tox environment is not.
|
||||
it. A developer can run `poetry remove foo` and then `poetry add foo^1.2` to get the new
|
||||
version, but the Tox environment is left unchanged. The development environment, as defined by
|
||||
the lockfile, is now patched against the vulnerability but the Tox environment is not.
|
||||
|
||||
* **The `bar` dependency specifies a dynamic range:** a dynamic range allows a range of
|
||||
versions to be installed, but the lockfile will have an exact version specified so that
|
||||
the Poetry environment is reproducible; this allows versions to be updated with
|
||||
`poetry update` rather than with the `remove` and `add` used above. If the maintainers of
|
||||
`bar` release version `1.6.0` then the Tox environment will install it because it is valid
|
||||
for the specified version range, meanwhile the Poetry environment will continue to install
|
||||
the version from the lockfile until `poetry update bar` explicitly updates it. The
|
||||
development environment is now has a different version of `bar` than the Tox environment.
|
||||
`poetry update` rather than with the `remove` and `add` commands used above. If the
|
||||
maintainers of `bar` release version `1.6.0` then the Tox environment will install it
|
||||
because it is valid for the specified version range. Meanwhile the Poetry environment will
|
||||
continue to install the version from the lockfile until `poetry update bar` explicitly
|
||||
updates it. The development environment is now has a different version of `bar` than the Tox
|
||||
environment.
|
||||
|
||||
* **The `baz` dependency is unpinned:** unpinned dependencies are
|
||||
[generally a bad idea](https://python-poetry.org/docs/faq/#why-are-unbound-version-constraints-a-bad-idea),
|
||||
@ -351,27 +352,26 @@ Poetry environment using this command:
|
||||
but Pip (via Tox) will interpret it as a wildcard. If the latest version of `baz` is `1.0.0`
|
||||
then `poetry add baz` will result in a constraint of `baz>=1.0.0,<2.0.0` while the Tox
|
||||
environment will have a constraint of `baz==*`. The Tox environment can now install an
|
||||
incompatible version of `baz` that cannot be easily caught using `poetry update`.
|
||||
incompatible version of `baz` and any errors that causes cannot be replicated using `poetry update`.
|
||||
|
||||
All of these problems can apply not only to the dependencies specified for a Tox environment,
|
||||
but also to the dependencies of those dependencies, and so on.
|
||||
but also to the dependencies of those dependencies, those dependencies' dependencies, and so on.
|
||||
|
||||
**The Solution**
|
||||
|
||||
This plugin requires that all dependencies specified for all Tox environments be unbound
|
||||
with no version constraint specified at all. This seems counter-intuitive given the problems
|
||||
outlined above, but what it allows the plugin to do is offload all version management to
|
||||
Poetry.
|
||||
|
||||
On initial inspection, the environment below appears less stable than the one presented above
|
||||
because it does not specify any versions for its dependencies:
|
||||
This plugin allows dependencies specified in Tox environment take their version directly from
|
||||
the Poetry lockfile without needing an independent version to be specified in the Tox
|
||||
environment configuration. The modified version of the example environment given below appears
|
||||
less stable than the one presented above because it does not specify any versions for its
|
||||
dependencies:
|
||||
|
||||
```ini
|
||||
# tox.ini
|
||||
# from tox.ini
|
||||
...
|
||||
|
||||
[testenv]
|
||||
description = Some very cool tests
|
||||
require_locked_deps = true
|
||||
deps =
|
||||
foo
|
||||
bar
|
||||
@ -380,19 +380,16 @@ deps =
|
||||
...
|
||||
```
|
||||
|
||||
However with the `tox-poetry-installer` plugin installed this instructs Tox to install these
|
||||
dependencies using the Poetry lockfile so that the version installed to the Tox environment
|
||||
exactly matches the version Poetry is managing. When `poetry update` updates the lockfile
|
||||
with new dependency versions, Tox will automatically install these new versions without needing
|
||||
any changes to the configuration.
|
||||
|
||||
All dependencies are specified in one place (the lockfile) and dependency version management is
|
||||
handled by a tool dedicated to that task (Poetry).
|
||||
However with the `tox-poetry-installer` plugin installed the `require_locked_deps = true`
|
||||
setting means that Tox will install these dependencies from the Poetry lockfile so that the
|
||||
version installed to the Tox environment exactly matches the version Poetry is managing. When
|
||||
`poetry update` updates the lockfile with new versions of these dependencies, Tox will
|
||||
automatically install these new versions without needing any changes to the configuration.
|
||||
|
||||
|
||||
## Developing
|
||||
|
||||
This project requires Poetry-1.0+, see the [installation instructions here](https://python-poetry.org/docs/#installation).
|
||||
This project requires Poetry version 1.0+, see the [installation instructions here](https://python-poetry.org/docs/#installation).
|
||||
|
||||
```bash
|
||||
# Clone the repository...
|
||||
@ -435,7 +432,7 @@ releases on PyPI.
|
||||
## Roadmap
|
||||
|
||||
This project is under active development and is classified as alpha software, not yet ready
|
||||
usage in production systems.
|
||||
for usage in production environments.
|
||||
|
||||
* Beta classification will be assigned when the initial feature set is finalized
|
||||
* Stable classification will be assigned when the test suite covers an acceptable number of
|
||||
|
||||
Loading…
Reference in New Issue
Block a user