This removes 014-remove-platform-so-suffix.patch and
016-adjust-config-paths.patch, restoring the platform triplet to paths
for:
* C extensions (*.cpython-311-*.so)
* Build config data directory (/usr/lib/python3.11/config-3.11-*/)
* sysconfig data file (/usr/lib/python3.11/_sysconfigdata_*.py)
Setting `_PYTHON_SYSCONFIGDATA_NAME` during package builds ensures that
sysconfig data for target Python is loaded, in particular so that C
extensions built will have the correct extension / platform triplet.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* Fix default Python package description not included in individual
package descriptions
* Update default Python package description text (from General Python
FAQ, "What is Python?")
* Add package descriptions for Python module packages
* Reduce duplication in package titles
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Packaging setuptools from a separate source package allows it to stay
updated with upstream.
Host setuptools will remain installed as part of python3. Host
setuptools is used in a much more controlled way and so is less critical
for it to track upstream.
setuptools was in a separate source package that was removed in
a53d0c5a403d1669e2cf6c59c2be6a9d3ed633a0; this work is not based on that
earlier package.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Packaging pip from a separate source package allows it to stay updated
with upstream.
Host pip will remain installed as part of python3. Host pip is used in a
much more controlled way and so is less critical for it to track
upstream.
This also removes the python-pip-conf package and installs the pip.conf
file as part of python3-pip.
The patch 003-disable-pip-version-check.patch is originally from Debian:
bb079efb8c/debian/patches/disable-pip-version-check.patch
pip was in a separate source package that was removed in
a53d0c5a403d1669e2cf6c59c2be6a9d3ed633a0; this work is not based on that
earlier package.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds a new subpackage for the venv module. This also moves the
ensurepip module from python3-pip into python3-venv, as ensurepip is not
necessary for pip but often used for venv.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* Removed patches:
* 001-enable-zlib.patch: zlib module should be enabled automatically
* 007-distutils-do-not-adjust-path.patch: Not necessary since we
process shebang lines for all scripts (in python3-package.mk)
* 030-bpo-43112-detect-musl-as-a-separate-SOABI-GH-24502.patch:
Already merged
* Move configure vars from config.site back into Makefile
Centralizing all build information into one file makes it easier to
maintain
* No longer set ac_cv_header_uuid_h=yes as configure should detect
libuuid
* Order configure args by enable-/disable-/with-/without-, then
alphabetically
* Set ac_cv_working_openssl_hashlib=yes for host configure to bypass the
OpenSSL API tests with LibreSSL
* Use the default Host/Compile recipe instead of picking out specific
targets to make
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Fixes:
https://github.com/openwrt/packages/issues/12707
Seems to work.
Looking into the 'venv' lib, it seems it's installing pip & setuptools
inside a virtual environment.
`python3-pip` is already ~6 MB.
This adds another ~3 MB.
But, this gives users the ability to run Python virtual environments, which
is a pretty common feature of Python in production cases (usually web
stuff).
Signed-off-by: Alexandru Ardelean <alex@shruggie.ro>
Manually re-applied:
008-distutils-use-python-sysroot.patch
016-adjust-config-paths.patch
Drop patch: 003-do-not-run-distutils-tests.patch
There is now a configure option '--disable-test-modules'
And seems we left the '_ctypes_test' around for quite some time.
Dropped now.
Refs:
https://bugs.python.org/issue27640https://bugs.python.org/issue43282
Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch
Python build honors SOURCE_DATE_EPOCH pretty well now.
Drop setuptools patches. Setuptools should be reproducible with Python 3.6+
according to a mention here:
https://github.com/pypa/setuptools/pull/1690#issuecomment-536517456
It's time to let upstream fix Setuptools reproduce-ability.
Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch
I can't seem to fully remember why it's there.
And it seem to build fine without it.
Drop patch: 015-abort-on-failed-modules.patch
Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var
option.
Add patch: 026-openssl-feature-flags.patch
We need to keep this in our tree for a while.
See:
https://bugs.python.org/issue45627
Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch
Link: https://github.com/python/cpython/pull/29353
Fixes the build for uuid C module.
Add patch: 028-host-python-support-ssl-with-libressl.patch
We need the _ssl module working on the host-side with LibreSSL for pip to
work to download from https://pypi.org
Refs: https://github.com/openwrt/openwrt/pull/4749
Add patch: 029-disable-deprecation-warning.patch
Fixes apparmor build. The warning causes a configure error.
Refreshed the rest of patches.
Some old build-flags were removed. They don't seem to be necessary anymore.
Split python3-uuid from python3-light. To better manage the libuuid library
(if needed). Also, fixing the uuid C module build. Seems this was failing,
and was falling back to using hashlib.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
I tried to install matplotlib by using pip and it failed with the following output:
2021-08-18T11:52:26,171 Collecting matplotlib
2021-08-18T11:52:26,173 Created temporary directory: /tmp/pip-unpack-wuth2u0e
2021-08-18T11:52:26,565 Using cached matplotlib-3.4.3.tar.gz (37.9 MB)
2021-08-18T11:52:38,659 Added matplotlib from https://files.pythonhosted.org/packages/21/37/ 197e68df384ff694f78d687a49ad39f96c67b8d75718bc61503e1676b617/matplotlib-3.4.3.tar. gz#sha256=fc4f526dfdb31c9bd6b8ca06bf9fab663ca12f3ec9cdf4496fb44bc680140318 to build tracker '/tmp/pip- req-tracker-u30x8pht'
2021-08-18T11:52:38,660 Running setup.py (path:/tmp/pip-install-afiyiers/ matplotlib_8e717e38862f4976a3d6cb1832ba3261/setup.py) egg_info for package matplotlib
2021-08-18T11:52:38,661 Created temporary directory: /tmp/pip-pip-egg-info-kbtiezxq
2021-08-18T11:52:38,662 Running command python setup.py egg_info
2021-08-18T11:52:38,831 Traceback (most recent call last):
2021-08-18T11:52:38,832 File "<string>", line 1, in <module>
2021-08-18T11:52:38,832 File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 8, in <module>
2021-08-18T11:52:38,833 ModuleNotFoundError: No module named '_distutils_hack'
More details about it:
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968410
Once I applied the patch from the bug tracker, I got further to install
it.
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Python works with GNU DBM and with Berkley DBM.
Berkley DBM has been under Oracle for some time.
And it's not clear how many Python users actually use DBM.
In the packages feed, we have both libdb47 (which is now under Oracle) and
GNU DBM. The GNU DBM has a compatibility layer for Berkley DBM.
There are newer versions than libdb47, but it's probably not worth having
them yet. The libbd47 tarball is ~40+ MB. Odds are newer versions will be
bigger and more bloated.
This change merges the old `python3-gdbm` package into the `python3-dbm`
package, since they are effectively using the same underlying library now,
i.e. gdbm.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Python3 comes with a built-in readline module. It wasn't included up until
now; mostly because it wasn't considered.
This change introduces it as a sub-package of the main Python3 package.
readline support is included in Python.
libreadline pulls libncursesw as a package, so python3-ncurses was
updated to pull libncursesw as well.
It should be the same package; mostly done for consistency.
Resolves the issue reported here:
https://forum.openwrt.org/t/python3-repl-missing-readline/90039
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Refreshed Python patches.
Updated pip & setuptools version.
For pip, patch '001-pep517-pyc-fix.patch' was reworked.
Also, the current version of the bundled pip (21.1.1) no longer supports
Python2, so the 'py2.py3' suffix gets replaced with just py3.
For setuptools, there is no longer a script/module:
https://github.com/pypa/setuptools/pull/2544
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Refreshed patches.
Dropped 'patches-setuptools/004-site-patch.patch'
Does not apply anymore. Setuptools has removed site.py support:
https://github.com/pypa/setuptools/issues/2165
If this is still needed, we may need to re-think it's implementation.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This version includes fixes for:
* CVE-2020-15801 - Fixes python3x._pth being ignored on Windows
* CVE-2019-20907 - Avoid infinite loop when reading specially crafted
TAR files using the tarfile module
This also:
* Remove patches that are included in the update
* Add a dependency in python3-distutils for python3-email[1]
[1]: https://github.com/python/cpython/blob/v3.8.5/Lib/distutils/dist.py#L10
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Previously, binaries installed by Python packages will have a
non-suffixed Python 2 version and a suffixed Python 3 version, e.g. pip
and pip3. With the removal of Python 2, the non-suffixed names are no
longer taken.
This adds symlinks for the non-suffixed names linking to the suffixed
scripts (or in the case of pip, easy_install, and python-config, to the
fully-versioned scripts).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
When a Python package is installed from source (i.e. using setup.py)
into a custom location (with --home), setuptools may want to create a
site.py file in the custom location. This file is created based on the
source code of site-patch.py, a file bundled with setuptools.
Because the normal OpenWrt setuptools package does not contain Python
source code, this file is missing and the installation will end with an
error.
This copies site-patch.py to site-patch.py.txt so that it will be
included in python3-setuptools, and patches setuptools to look for this
file.
See https://github.com/openwrt/packages/issues/12223
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
The ssl module assumes OpenSSL can load the default trust anchors (root
CA certificates).
From https://github.com/openwrt/packages/issues/12209
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Python will record the values of CC, CXX, AR, and READELF (and other
configure options) used during compilation. pip will use these programs
when asked to compile extension modules on the target device.
* If ccache is used during build, CC and CXX will be ccache_cc and
ccache_cxx, respectively, which are not available on-device (#11912).
* If an external toolchain is used during build, the values of these
variables will contain the external toolchain prefix, which may not be
available on target.
* If the normal toolchain is used during build, AR and READELF will
contain the toolchain prefix, but the names of ar and readelf
on-device do not contain the prefix; they are named "ar" and
"readelf".
This changes the values of these variables in Python's files to match
the names available on-device, and without any toolchain prefix.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
PYTHON_VERSION is a holdover from Python 2; all Python 3 variables are
prefixed with PYTHON3 (or some variation with "3").
This updates all uses of PYTHON_VERSION to PYTHON3_VERSION.
This also sets PYTHON3_PKG_BUILD:=0 before python3-package.mk is
included in the python3 Makefile.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Currently, python3-pip installs the same script as pip3 and pip3.8 to
usr/bin. This changes pip3 to be a symlink to pip3.8.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This package is required by other packages to run some binaries via
`load_entry_point`.
So, this splits this package away from setuptools.
setuptools is pretty big, akd pkg-resources is also big, but not as big.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This uses two find commands to delete __pycache__ contents then the
__pycache__ directories, rather than a for loop.
The second command omits a -empty test, so that if the first command
doesn't remove all directory contents for some reason, the second
command will return an error (find will not delete a non-empty
directory).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This changes the --prefix option, passed to host pip when "installing"
target setuptools and pip, to /usr, in case the prefix is recorded in
the packages.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Fixes: https://github.com/openwrt/packages/issues/8301
This seems to have slipped for some time. No idea if it ever worked.
It could be that this worked at some point.
In any case, the shebang is properly updated now.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This is the result of this discussion:
https://github.com/openwrt/packages/issues/8285
`urllib.request` requires the `email` module/lib, which was part of
python3-light.
This change moves the Lib/urllib folder from the python3-light into it's
own package, making it lighter. At least this way, users that want `urllib`
(on top of `python3-light`) will be forced to install it via opkg and this
will make sure `python3-email` gets installed as well.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This extends the Python[3] shebang fixup to all packages.
Only Python scripts in `/usr/bin` will be handled at the moment. Later it
may make sense to also cover executables in `/bin`, though typically Python
executables shouldn't be placed there.
Previously the shebang handling was only done for python[3]-pip &
python[3]-setuptools.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Piping to xargs does not handle spaces in paths too well, because it splits
up the paths.
For deleting empty dirs, we also need to do several retries, otherwise
`find` will try to go through the directories after they're deleted.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Since `lang/python` is it's own folder of Python packages
(for both Python 2 & 3), and these build rules are needed
in a lot of packages [especially Python packages],
putting them here makes sense architecturally,
to be shared.
This also helps get rid of the `include_mk` construct
which relies on OpenWrt core to provide, and seems
like a broken design idea that has persisted for a while.
Reason is: it requires that Python 2/3 be built to provide
these mk files for other Python packages,
which seems like a bad idea.
Long-term, there could be an issue where some other feeds
would require these mk files [e.g. telephony] for
some Python packages.
We'll see how we handle this a bit later.
For now we limit this to this feed.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This reverts commits 4333d1dcbf and
074d2863be, making Python packages
discoverable again by pkg_resources.
Fixes#5361.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Re-worked patch `003-do-not-run-distutils-tests.patch`
to reduce patch-size.
Removed `011-fix-ncursesw-definition-colisions.patch`
it is fixed upstream.
Refreshed with `make package/python3/refresh`
Resetting PKG_RELEASE to 1.
This variable was never used for pip3 & setuptools, since
VERSION is specified in the package definitions.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
That way some python packages can choose
to keep their egg-info dirs, if they want to, or they're needed.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
The host pip install should have the host's CFLAGS, LDFLAGS, etc
available.
And not the target's flags.
Otherwise, weird things can happen when installing
packages (host-side) that need to build C code.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
To install Python packages host side, that
may be needed for a build.
The intent, is to try to reduce host-side Python
packages being installed via LEDE/OpenWrt build system.
Because those seem like a pain to maintain.
The idea is adapted from Yousong's `python-packages`
package.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Depending on execution order the `python-package-install.sh`
script would return a non-zero err code.
So, this enforces that all commands in the script
don't fail (via the `set -e` directive).
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Similar to LEDE/OpenWrt's Build/Compile/Default rule,
and other similarities like this.
This should allow Python packages to define
PyBuild/Compile rules to do specific stuff per
package.
The advantage of using these (over just overriding
Build/Compile) is the VARIANT mechanism that is
in place to support packaging both for Python & Python3.
So, PyBuild/Compile will get picked up for the Python
variant build, and Py3Build/Compile will get picked
up for the Python3 variant build.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This is in essence fixes pip3.
That means pip3 will ship without Python byte-codes
for a while, until I'll find a better way to fix it.
I couldn't think of a not-very hack-ish way of doing it.
The only draw-back of this, will be that pip3 will run
a bit slower ; but that should be ok for a while.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Jeffery To