I tried to install matplotlib by using pip and it failed with the following output:
2021-08-18T11:52:26,171 Collecting matplotlib
2021-08-18T11:52:26,173 Created temporary directory: /tmp/pip-unpack-wuth2u0e
2021-08-18T11:52:26,565 Using cached matplotlib-3.4.3.tar.gz (37.9 MB)
2021-08-18T11:52:38,659 Added matplotlib from https://files.pythonhosted.org/packages/21/37/ 197e68df384ff694f78d687a49ad39f96c67b8d75718bc61503e1676b617/matplotlib-3.4.3.tar. gz#sha256=fc4f526dfdb31c9bd6b8ca06bf9fab663ca12f3ec9cdf4496fb44bc680140318 to build tracker '/tmp/pip- req-tracker-u30x8pht'
2021-08-18T11:52:38,660 Running setup.py (path:/tmp/pip-install-afiyiers/ matplotlib_8e717e38862f4976a3d6cb1832ba3261/setup.py) egg_info for package matplotlib
2021-08-18T11:52:38,661 Created temporary directory: /tmp/pip-pip-egg-info-kbtiezxq
2021-08-18T11:52:38,662 Running command python setup.py egg_info
2021-08-18T11:52:38,831 Traceback (most recent call last):
2021-08-18T11:52:38,832 File "<string>", line 1, in <module>
2021-08-18T11:52:38,832 File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 8, in <module>
2021-08-18T11:52:38,833 ModuleNotFoundError: No module named '_distutils_hack'
More details about it:
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968410
Once I applied the patch from the bug tracker, I got further to install
it.
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Python works with GNU DBM and with Berkley DBM.
Berkley DBM has been under Oracle for some time.
And it's not clear how many Python users actually use DBM.
In the packages feed, we have both libdb47 (which is now under Oracle) and
GNU DBM. The GNU DBM has a compatibility layer for Berkley DBM.
There are newer versions than libdb47, but it's probably not worth having
them yet. The libbd47 tarball is ~40+ MB. Odds are newer versions will be
bigger and more bloated.
This change merges the old `python3-gdbm` package into the `python3-dbm`
package, since they are effectively using the same underlying library now,
i.e. gdbm.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Python3 comes with a built-in readline module. It wasn't included up until
now; mostly because it wasn't considered.
This change introduces it as a sub-package of the main Python3 package.
readline support is included in Python.
libreadline pulls libncursesw as a package, so python3-ncurses was
updated to pull libncursesw as well.
It should be the same package; mostly done for consistency.
Resolves the issue reported here:
https://forum.openwrt.org/t/python3-repl-missing-readline/90039
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Refreshed Python patches.
Updated pip & setuptools version.
For pip, patch '001-pep517-pyc-fix.patch' was reworked.
Also, the current version of the bundled pip (21.1.1) no longer supports
Python2, so the 'py2.py3' suffix gets replaced with just py3.
For setuptools, there is no longer a script/module:
https://github.com/pypa/setuptools/pull/2544
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Refreshed patches.
Dropped 'patches-setuptools/004-site-patch.patch'
Does not apply anymore. Setuptools has removed site.py support:
https://github.com/pypa/setuptools/issues/2165
If this is still needed, we may need to re-think it's implementation.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This version includes fixes for:
* CVE-2020-15801 - Fixes python3x._pth being ignored on Windows
* CVE-2019-20907 - Avoid infinite loop when reading specially crafted
TAR files using the tarfile module
This also:
* Remove patches that are included in the update
* Add a dependency in python3-distutils for python3-email[1]
[1]: https://github.com/python/cpython/blob/v3.8.5/Lib/distutils/dist.py#L10
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Previously, binaries installed by Python packages will have a
non-suffixed Python 2 version and a suffixed Python 3 version, e.g. pip
and pip3. With the removal of Python 2, the non-suffixed names are no
longer taken.
This adds symlinks for the non-suffixed names linking to the suffixed
scripts (or in the case of pip, easy_install, and python-config, to the
fully-versioned scripts).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
When a Python package is installed from source (i.e. using setup.py)
into a custom location (with --home), setuptools may want to create a
site.py file in the custom location. This file is created based on the
source code of site-patch.py, a file bundled with setuptools.
Because the normal OpenWrt setuptools package does not contain Python
source code, this file is missing and the installation will end with an
error.
This copies site-patch.py to site-patch.py.txt so that it will be
included in python3-setuptools, and patches setuptools to look for this
file.
See https://github.com/openwrt/packages/issues/12223
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
The ssl module assumes OpenSSL can load the default trust anchors (root
CA certificates).
From https://github.com/openwrt/packages/issues/12209
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Python will record the values of CC, CXX, AR, and READELF (and other
configure options) used during compilation. pip will use these programs
when asked to compile extension modules on the target device.
* If ccache is used during build, CC and CXX will be ccache_cc and
ccache_cxx, respectively, which are not available on-device (#11912).
* If an external toolchain is used during build, the values of these
variables will contain the external toolchain prefix, which may not be
available on target.
* If the normal toolchain is used during build, AR and READELF will
contain the toolchain prefix, but the names of ar and readelf
on-device do not contain the prefix; they are named "ar" and
"readelf".
This changes the values of these variables in Python's files to match
the names available on-device, and without any toolchain prefix.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
PYTHON_VERSION is a holdover from Python 2; all Python 3 variables are
prefixed with PYTHON3 (or some variation with "3").
This updates all uses of PYTHON_VERSION to PYTHON3_VERSION.
This also sets PYTHON3_PKG_BUILD:=0 before python3-package.mk is
included in the python3 Makefile.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Currently, python3-pip installs the same script as pip3 and pip3.8 to
usr/bin. This changes pip3 to be a symlink to pip3.8.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This package is required by other packages to run some binaries via
`load_entry_point`.
So, this splits this package away from setuptools.
setuptools is pretty big, akd pkg-resources is also big, but not as big.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This uses two find commands to delete __pycache__ contents then the
__pycache__ directories, rather than a for loop.
The second command omits a -empty test, so that if the first command
doesn't remove all directory contents for some reason, the second
command will return an error (find will not delete a non-empty
directory).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This changes the --prefix option, passed to host pip when "installing"
target setuptools and pip, to /usr, in case the prefix is recorded in
the packages.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Fixes: https://github.com/openwrt/packages/issues/8301
This seems to have slipped for some time. No idea if it ever worked.
It could be that this worked at some point.
In any case, the shebang is properly updated now.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This is the result of this discussion:
https://github.com/openwrt/packages/issues/8285
`urllib.request` requires the `email` module/lib, which was part of
python3-light.
This change moves the Lib/urllib folder from the python3-light into it's
own package, making it lighter. At least this way, users that want `urllib`
(on top of `python3-light`) will be forced to install it via opkg and this
will make sure `python3-email` gets installed as well.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This extends the Python[3] shebang fixup to all packages.
Only Python scripts in `/usr/bin` will be handled at the moment. Later it
may make sense to also cover executables in `/bin`, though typically Python
executables shouldn't be placed there.
Previously the shebang handling was only done for python[3]-pip &
python[3]-setuptools.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Piping to xargs does not handle spaces in paths too well, because it splits
up the paths.
For deleting empty dirs, we also need to do several retries, otherwise
`find` will try to go through the directories after they're deleted.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Since `lang/python` is it's own folder of Python packages
(for both Python 2 & 3), and these build rules are needed
in a lot of packages [especially Python packages],
putting them here makes sense architecturally,
to be shared.
This also helps get rid of the `include_mk` construct
which relies on OpenWrt core to provide, and seems
like a broken design idea that has persisted for a while.
Reason is: it requires that Python 2/3 be built to provide
these mk files for other Python packages,
which seems like a bad idea.
Long-term, there could be an issue where some other feeds
would require these mk files [e.g. telephony] for
some Python packages.
We'll see how we handle this a bit later.
For now we limit this to this feed.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This reverts commits 4333d1dcbf and
074d2863be, making Python packages
discoverable again by pkg_resources.
Fixes#5361.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Re-worked patch `003-do-not-run-distutils-tests.patch`
to reduce patch-size.
Removed `011-fix-ncursesw-definition-colisions.patch`
it is fixed upstream.
Refreshed with `make package/python3/refresh`
Resetting PKG_RELEASE to 1.
This variable was never used for pip3 & setuptools, since
VERSION is specified in the package definitions.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
That way some python packages can choose
to keep their egg-info dirs, if they want to, or they're needed.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
The host pip install should have the host's CFLAGS, LDFLAGS, etc
available.
And not the target's flags.
Otherwise, weird things can happen when installing
packages (host-side) that need to build C code.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
To install Python packages host side, that
may be needed for a build.
The intent, is to try to reduce host-side Python
packages being installed via LEDE/OpenWrt build system.
Because those seem like a pain to maintain.
The idea is adapted from Yousong's `python-packages`
package.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Depending on execution order the `python-package-install.sh`
script would return a non-zero err code.
So, this enforces that all commands in the script
don't fail (via the `set -e` directive).
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Similar to LEDE/OpenWrt's Build/Compile/Default rule,
and other similarities like this.
This should allow Python packages to define
PyBuild/Compile rules to do specific stuff per
package.
The advantage of using these (over just overriding
Build/Compile) is the VARIANT mechanism that is
in place to support packaging both for Python & Python3.
So, PyBuild/Compile will get picked up for the Python
variant build, and Py3Build/Compile will get picked
up for the Python3 variant build.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This is in essence fixes pip3.
That means pip3 will ship without Python byte-codes
for a while, until I'll find a better way to fix it.
I couldn't think of a not-very hack-ish way of doing it.
The only draw-back of this, will be that pip3 will run
a bit slower ; but that should be ok for a while.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
python3-pip & python3-setuptools have slightly
different installation mechanisms.
We need to remove the __pycache__ folders.
Seems they're generated.
This also reduces the size of the python3-pip &
python3-setuptools packages.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
I admit this may be be a bit aggressive, but the lang
folder is getting cluttered/filled up with Python, PHP, Perl,
Ruby, etc. packages.
Makes sense to try to group them into per-lang folders.
I took the Pythons.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
