Python @programming.dev logging_strict @programming.dev 2 days ago

PEP 735 does dependency group solve anything?

peps.python.org PEP 735 – Dependency Groups in pyproject.toml | peps.python.org

This PEP specifies a mechanism for storing package requirements in pyproject.toml files such that they are not included in any built distribution of the project.

PEP 735 what is it's goal? Does it solve our dependency hell issue?

A deep dive and out comes this limitation

The mutual compatibility of Dependency Groups is not guaranteed.

-- https://peps.python.org/pep-0735/#lockfile-generation

Huh?! Why not?

mutual compatibility or go pound sand!

pip install -r requirements/dev.lock
pip install -r requirements/kit.lock -r requirements/manage.lock

The above code, purposefully, does not afford pip a fighting chance. If there are incompatibilities, it'll come out when trying randomized combinations.

Without a means to test for and guarantee mutual compatibility, end users will always find themselves in dependency hell.

Any combination of requirement files (or dependency groups), intended for the same venv, MUST always work!

What if this is scaled further, instead of one package, a chain of packages?!

26 comments

I think you’re expecting this PEP to be something that it is not. It is not supposed to be a full solution to dependency hell (which I’m not really sure that there is). It is supposed to just allow a static method to declare dependencies, notably supporting both Python package and non-Python-package dependencies. There are plenty of use cases for when you might want incompatible sets of dependencies, for a simple example consider a graphics library with both a Vulkan and OpenGL backend. You could reasonably argue that you should allow both to coexist and just select the best one at runtime, but when you’re dealing with native libraries that’s not always possible, and there is no way to make a guarantee about compatibility without excluding all non-pure Python dependencies.
- Wasn't aware that this PEP is including non-Python package dependencies or how that affects dependency resolution.
  
  Managing python dependencies is challenging enough
Limitations of requirements.txt files

-- https://peps.python.org/pep-0735/#limitations-of-requirements-txt-files

The only benefit i can see, is to attempt to bring requirements files into pyproject.toml and an additional layer to abstract away from pip.

^^ this does not keep me awake at night nor is it a substitute for porn

The PEP author's intentions are good, puts focus on a little discussed topic.

The outcome ... questionable

If this is all it's doing, that's not enough. Ignores the elephant in the room.

Which are

fixing dependency conflicts

mutual compatibility

Fixing dependency conflicts would be easier if can work with existing proven tooling. Which acts upon r/w files rather than pyproject.toml, a config file; shouldn't constantly be updated.
- additional layer to abstract away from pip
  
  reqs.txt files are not standardized and pip can read from a pyproject.toml - which is - using pip install .
  
  there are still many unresolved matters with dependency resolution, but we need to leave requirements.txt files behind.
  
  Throwing out an alternative. Not making the assumption that more TOML is better. Cuz the contents of those requirements.txt files are rw, not ro. I see pyproject.toml as a ro configuration file.
  
  Do you agree or should pyproject.toml be rw?
  
  Another option, strictly validated YAML.
  
  For the configuration section, before parsing occurs, strict validation occurs against a schema.
  
  TOML vs strictyaml -- https://hitchdev.com/strictyaml/why-not/toml/

You've viewed 26 comments.