[gentoo-python] RFC: Redesign for 'best implementation' in python-r1

[Michał Górny]

Hello,

The current design for getting the 'best implementation' is far from
pretty. It pretty much relies on constant preference order, setting
Python 2.7 over other implementations for practical reasons. As a side
effect, various ebuilds rely on that particular order, e.g.:

DEPEND="doc? ( dev-python/epydoc[$(python_gen_usedep 'python2*')] )"
REQUIRED_USE="doc? ( $(python_gen_useflags 'python2*') )"

which pretty much assumes that *if python2 is enabled*, then any
version of it will be the best impl. I find this disgusting, and I'd
like to replace it with something more explicit.


Specifically:

1. python_export_best becomes deprecated for good,

2. python_setup changes API to:

python_setup [<impl-wildcard>...]

alike python_gen_* functions, defaulting to '*'. Now it chooses
the best impl from implementations matching the specified patterns,
e.g.:

python_setup 'python2*'

would get the best version of CPython 2.

3. we introduce extra variable for distutils-r1 (oh cruel world, how
should we name it?!) that applies wildcards to python_*_all() phases.


As for the choice within the list the algo needs to stay as-is for now.
However, in the future we could either:

a. respect EPYTHON and fallback to the other algo,

b. respect pre-defined order i.e. 3.4 > 2.7,

c. respect the order in PYTHON_COMPAT -- however that could be a little
surprising to devs.


What are your thoughts?

[Michał Górny]

Dnia 2014-12-27, o godz. 14:50:43

Post by Michał Górny
python_setup [<impl-wildcard>...]
alike python_gen_* functions, defaulting to '*'. Now it chooses
the best impl from implementations matching the specified patterns,
python_setup 'python2*'
would get the best version of CPython 2.

Patch for this one attached.

[Mike Gilbert]

Post by Michał Górny
Hello,
The current design for getting the 'best implementation' is far from
pretty. It pretty much relies on constant preference order, setting
Python 2.7 over other implementations for practical reasons. As a side
DEPEND="doc? ( dev-python/epydoc[$(python_gen_usedep 'python2*')] )"
REQUIRED_USE="doc? ( $(python_gen_useflags 'python2*') )"
which pretty much assumes that *if python2 is enabled*, then any
version of it will be the best impl. I find this disgusting, and I'd
like to replace it with something more explicit.

...

Yeah, relying python_compile_all running under python2.7 is no good.

Post by Michał Górny
1. python_export_best becomes deprecated for good,
python_setup [<impl-wildcard>...]
alike python_gen_* functions, defaulting to '*'. Now it chooses
the best impl from implementations matching the specified patterns,
python_setup 'python2*'
would get the best version of CPython 2.
3. we introduce extra variable for distutils-r1 (oh cruel world, how
should we name it?!) that applies wildcards to python_*_all() phases.

This sounds like a reasonable idea, and a natural extension of the
python_setup function.

Post by Michał Górny
As for the choice within the list the algo needs to stay as-is for now.
a. respect EPYTHON and fallback to the other algo,
b. respect pre-defined order i.e. 3.4 > 2.7,
c. respect the order in PYTHON_COMPAT -- however that could be a little
surprising to devs.
What are your thoughts?

Starting/ending with EPYTHON from the environment seems wrong, and
doesn't seem to add value anyway.

I would prefer to stick with a predefined order unless someone can
present a case where the ebuild author really needs to control it.

[Michał Górny]

Dnia 2014-12-31, o godz. 11:41:45

Post by Mike Gilbert

Post by Michał Górny
As for the choice within the list the algo needs to stay as-is for now.
a. respect EPYTHON and fallback to the other algo,
b. respect pre-defined order i.e. 3.4 > 2.7,
c. respect the order in PYTHON_COMPAT -- however that could be a little
surprising to devs.
What are your thoughts?

Starting/ending with EPYTHON from the environment seems wrong, and
doesn't seem to add value anyway.
I would prefer to stick with a predefined order unless someone can
present a case where the ebuild author really needs to control it.

...

Well, it isn't really about the ebuild author but about the user.
EPYTHON idea is pretty much a reference to the python-any-r1 behavior.
IOW, having both eclasses use the same preference handling.

[Michał Górny]

Dnia 2014-12-27, o godz. 14:50:43

Post by Michał Górny
3. we introduce extra variable for distutils-r1 (oh cruel world, how
should we name it?!) that applies wildcards to python_*_all() phases.

Ok, I see a problem with this one. Consider the common use:

DEPEND="doc? ( dev-python/epydoc[$(python_gen_usedep 'python2*')] )"
REQUIRED_USE="doc? ( $(python_gen_useflags 'python2*') )"

python_compile_all() {
use doc && esetup.py doc # runs epydoc
}

So we'd like the _all() phase to be run with python2* because of epydoc.
But OTOH we don't need any python2* if USE=-doc.

If we used a simple variable to control impl for _all(), it would
unnecessarily force python2* with USE=-doc, and python2* may not be
enabled at all then. So we either set it conditionally (+ extra phase,
added complexity), or make it non-fatal. Non-fatal is ugly since it
means unpredictable behavior.

So we can use a complex variable instead. But this way, we're in
python.eclass territory soon. Alternatively, add some function that's
called random-early by other python stuff, and that can be used to set
this variable -- but this doesn't look like good API design.

Maybe calling python_setup inside _all() phases -- but this implies
setting at least two environments, and we'd have to handle extra stuff
that distutils-r1 sets like PYTHONPATH. Still doesn't sound good.

So any ideas? :P