setuptools provides a local copy of distutils and when building a C
extension, this distutils will add the target LIBDIR (/usr/lib) to the
list of library paths.
If the build system has a libpython3.11.so in /usr/lib, then the linker
will try to link to this shared library and fail.
This adapts 008-distutils-use-python-sysroot.patch for host setuptools
to add the correct library directory.
Fixes: https://github.com/openwrt/packages/issues/22330
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Move the order in which BuildPackage is called, so that the libpython
package is built ahead of the module packages, to avoid forcing a
clean-build of the package when 'make package/python3/compile' is called
a second time without changes.
The library must be built first, so that when the buildsystem checks for
ABI version changes using libpython3.version, its timestamp should be
older than the dependent package's STAMP_PREPARED file.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Includes fix for CVE-2023-40217 (Bypass TLS handshake on closed
sockets).
This also:
* Remove 027-install-python3-symlink-after-stdlib.patch
This was merged upstream in
https://github.com/python/cpython/pull/104693.
* Remove fix for unnecessary linking with libbsd
(60bf01cb60)
This was fixed upstream in
https://github.com/python/cpython/pull/105236.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This removes 014-remove-platform-so-suffix.patch and
016-adjust-config-paths.patch, restoring the platform triplet to paths
for:
* C extensions (*.cpython-311-*.so)
* Build config data directory (/usr/lib/python3.11/config-3.11-*/)
* sysconfig data file (/usr/lib/python3.11/_sysconfigdata_*.py)
Setting `_PYTHON_SYSCONFIGDATA_NAME` during package builds ensures that
sysconfig data for target Python is loaded, in particular so that C
extensions built will have the correct extension / platform triplet.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Currently, configure does not find the correct platform triplet for musl
as the default build/host values passed by OpenWrt buildroot does not
contain the text "linux-musl".
This backports
c163d7f0b6
to add detection for mips soft float and musl.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* Support wildcards in install (`+`) paths
* Add fourth parameter to set directory permissions
If file permissions are given (third parameter), these will now apply
to files only.
* Add non-recursive set permissions command (`==`)
* Be more strict about filespec format
Blank lines and lines starting with `#` will be ignored. Other errors
(unknown command, missing path parameter, etc.) will cause the script
to exit.
* Be more strict about ensuring paths exist for all commands
* Avoid spawning subshells
This also removes outdated filespec paths in the python3 package; these
paths delete files that are no longer present.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* Fix default Python package description not included in individual
package descriptions
* Update default Python package description text (from General Python
FAQ, "What is Python?")
* Add package descriptions for Python module packages
* Reduce duplication in package titles
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
There is no need to use Py3Package for python3 as it does not package
any Python files; it is an empty package with dependencies to install
the full Python installation.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Packaging setuptools from a separate source package allows it to stay
updated with upstream.
Host setuptools will remain installed as part of python3. Host
setuptools is used in a much more controlled way and so is less critical
for it to track upstream.
setuptools was in a separate source package that was removed in
a53d0c5a403d1669e2cf6c59c2be6a9d3ed633a0; this work is not based on that
earlier package.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Packaging pip from a separate source package allows it to stay updated
with upstream.
Host pip will remain installed as part of python3. Host pip is used in a
much more controlled way and so is less critical for it to track
upstream.
This also removes the python-pip-conf package and installs the pip.conf
file as part of python3-pip.
The patch 003-disable-pip-version-check.patch is originally from Debian:
bb079efb8c/debian/patches/disable-pip-version-check.patch
pip was in a separate source package that was removed in
a53d0c5a403d1669e2cf6c59c2be6a9d3ed633a0; this work is not based on that
earlier package.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds a new subpackage for the venv module. This also moves the
ensurepip module from python3-pip into python3-venv, as ensurepip is not
necessary for pip but often used for venv.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
configure will try to link with libbsd for the flock function, even when
flock is available without it.
Fixes: https://github.com/openwrt/packages/issues/21161
Fixes: 2445fe9fb2 ("python3: Update to 3.11.3, refresh/redo patches")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* Removed patches:
* 001-enable-zlib.patch: zlib module should be enabled automatically
* 007-distutils-do-not-adjust-path.patch: Not necessary since we
process shebang lines for all scripts (in python3-package.mk)
* 030-bpo-43112-detect-musl-as-a-separate-SOABI-GH-24502.patch:
Already merged
* Move configure vars from config.site back into Makefile
Centralizing all build information into one file makes it easier to
maintain
* No longer set ac_cv_header_uuid_h=yes as configure should detect
libuuid
* Order configure args by enable-/disable-/with-/without-, then
alphabetically
* Set ac_cv_working_openssl_hashlib=yes for host configure to bypass the
OpenSSL API tests with LibreSSL
* Use the default Host/Compile recipe instead of picking out specific
targets to make
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
026-openssl-feature-flags.patch and
028-host-python-support-ssl-with-libressl.patch were removed in
4ecd9d67e9 to fix the ssl module after
libressl was upgraded to 3.7.0[1].
However, the cause of the ssl module build failure was only
028-host-python-support-ssl-with-libressl.patch.
Removing 026-openssl-feature-flags.patch caused a build failure for the
hashlib module.
This restores 026-openssl-feature-flags.patch with an updated version of
the patch from OpenBSD[2].
[1]: https://github.com/openwrt/packages/issues/20107
[2]: 26a04435bf/lang/python/3.10/patches/patch-Modules__hashopenssl_c
Fixes: 4ecd9d67e9 ("python3: fix ssl support by removing libressl patches")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds $(STAGING_DIR_HOST)/include/e2fsprogs to HOST_CFLAGS and
HOST_CPPFLAGS so that configure can find uuid/uuid.h.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
By default, the Python build process will add /usr/local/{lib,include},
and multiarch paths (e.g. /usr/{lib,include}/x86_64-linux-gnu) if
building on Debian/Ubuntu, to its library and includes paths.
006-remove-multi-arch-and-local-paths.patch was added in
84202f17e1 to stop the Python build
process from adding these paths.
006-remove-multi-arch-and-local-paths.patch was removed in
48277ec915.
006-do-not-add-multiarch-paths-when-cross-compiling.patch was added in
0c8b0b0bf7 to stop the Python build
process from adding these paths for target Python.
These paths are still added by the Python build process when building
host Python.
This replaces the cross-compiling-only patch with the original patch,
renamed slightly and adapted for Python 3.10.
Fixes: 48277ec915 ("python3: bump to version 3.8")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
When doing parallel builds, host Python can install the python3 symlink
before the Python standard library is installed completely.
When this occurs, it is possible for other packages to detect the
python3 symlink and try to use host Python before it is fully installed.
This adds a patch to make commoninstall (where the standard library is
installed) a prerequisite of bininstall (where the python3 symlink is
installed), so that commoninstall is fully completed before bininstall
begins.
Patch has been submitted upstream:
https://github.com/python/cpython/pull/104693
Fixes: https://github.com/openwrt/packages/issues/19241
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
The Makefile lines to add READELF to TARGET_CONFIGURE_OPTS was removed
in 4e05541782.
Without setting READELF, configure finds the symlink to
$(TARGET_CROSS)readelf (e.g. arm-openwrt-linux-readelf) instead of
$(TARGET_CROSS)readelf (e.g. arm-openwrt-linux-muslgnueabi-readelf).
This leads to the symlink name being saved to _sysconfigdata.py, and so
the readelf name is not replaced correctly (in
Py3Package/python3-base/install).
This restores the removed Makefile lines.
Fixes: 4e05541782 ("python3: bump to version 3.10.0")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
003-do-not-run-distutils-tests.patch was removed in
4e05541782. This patch stopped "make
install" from, among other things, running compileall.
When this patch was removed, "make install" ran compileall as normal and
created bytecode files in __pycache__ directories. These files were then
packaged in python3-light.
This adds a patch to stop compileall from being run during "make
install".
Fixes: 4e05541782 ("python3: bump to version 3.10.0")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
--without-pymalloc was added in 7bf1ae65a8
because leaving it enabled added an "m" flag/suffix to file names.
This flag/suffix was removed in Python 3.8[1], so disabling pymalloc is
no longer necessary.
[1]: https://docs.python.org/3.8/whatsnew/3.8.html#build-and-c-api-changes
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This removes the changes made in
61f202c017 and adds actual support for
pyproject.toml-based (PEP 517) builds of Python packages.
Packages can force the use of the old build process by setting
PYTHON3_PKG_FORCE_DISTUTILS_SETUP:=1; this should only be a temporary
workaround until the package can be updated/fixed to use the new build
process.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Fixes:
https://github.com/openwrt/packages/issues/12707
Seems to work.
Looking into the 'venv' lib, it seems it's installing pip & setuptools
inside a virtual environment.
`python3-pip` is already ~6 MB.
This adds another ~3 MB.
But, this gives users the ability to run Python virtual environments, which
is a pretty common feature of Python in production cases (usually web
stuff).
Signed-off-by: Alexandru Ardelean <alex@shruggie.ro>
Remove libressl specific patches. With commit
("tools/libressl: update to 3.7.0") they are no longer needed,
rather they cause python3 to be compiled without working ssl-support.
Fixes: #20107
Suggested-by: Andre Heider <a.heider@gmail.com>
Signed-off-by: Nick Hainke <vincent@systemli.org>
Patch 030:
Backported from Python main branch[^1] for Python to distinguish between glibc and musl libc SOABI.
Patch 131:
Changes PLATFORM_TRIPLET -gnu/-musl suffix detection (performed by the backported patch)
to be based on the target OS instead of the building OS.
See included patches for more detailed descriptions.
Specifically this fixes cross-compilation for mpc8548 CPUs with SPE instructions[^2] enabled.
[^1]: merged to python:main as https://github.com/python/cpython/pull/24502 'bpo-43112: detect musl as a separate SOABI'
[^2]: https://www.nxp.com/docs/en/reference-manual/SPEPEM.pdf
Co-authored-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Šimon Bořek <simon.borek@nic.cz>
Includes fixes for:
* Windows builds updated to bzip2 1.0.8 to mitigate CVE-2016-3189 and
CVE-2019-12900
* CVE-2022-26488: Escalation of privilege via Windows Installer
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Removed patches:
* 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch
Already merged.
* 029-disable-deprecation-warning.patch
Packages should be patched/fixed to remove the use of distutils
instead of disabling this warning.
Also:
* Updates PKG_LICENSE to use the correct SPDX license identifier
* Fixes build for mipsel_24kc_24kf
Fixes https://github.com/openwrt/packages/issues/17217.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
There's been a bit of overlapping opinions on some of these packages.
The best thing to do here is to reduce ownership and relinquish my
control.
This patch does that.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Manually re-applied:
008-distutils-use-python-sysroot.patch
016-adjust-config-paths.patch
Drop patch: 003-do-not-run-distutils-tests.patch
There is now a configure option '--disable-test-modules'
And seems we left the '_ctypes_test' around for quite some time.
Dropped now.
Refs:
https://bugs.python.org/issue27640https://bugs.python.org/issue43282
Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch
Python build honors SOURCE_DATE_EPOCH pretty well now.
Drop setuptools patches. Setuptools should be reproducible with Python 3.6+
according to a mention here:
https://github.com/pypa/setuptools/pull/1690#issuecomment-536517456
It's time to let upstream fix Setuptools reproduce-ability.
Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch
I can't seem to fully remember why it's there.
And it seem to build fine without it.
Drop patch: 015-abort-on-failed-modules.patch
Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var
option.
Add patch: 026-openssl-feature-flags.patch
We need to keep this in our tree for a while.
See:
https://bugs.python.org/issue45627
Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch
Link: https://github.com/python/cpython/pull/29353
Fixes the build for uuid C module.
Add patch: 028-host-python-support-ssl-with-libressl.patch
We need the _ssl module working on the host-side with LibreSSL for pip to
work to download from https://pypi.org
Refs: https://github.com/openwrt/openwrt/pull/4749
Add patch: 029-disable-deprecation-warning.patch
Fixes apparmor build. The warning causes a configure error.
Refreshed the rest of patches.
Some old build-flags were removed. They don't seem to be necessary anymore.
Split python3-uuid from python3-light. To better manage the libuuid library
(if needed). Also, fixing the uuid C module build. Seems this was failing,
and was falling back to using hashlib.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Bluetooth support requires bluez-libs present, but they are only required
for the build, and don't seem to be needed to be present on the target.
There isn't any linking required to libbluetooth. It's only the bluetooth.h
header that is required for building BT support into Python.
For testing, this snippet was used from `Lib/test/test_socket.py` (inside
cpython):
```
def _have_socket_bluetooth():
"""Check whether AF_BLUETOOTH sockets are supported on this host."""
try:
# RFCOMM is supported by all platforms with bluetooth support. Windows
# does not support omitting the protocol.
s = socket.socket(socket.AF_BLUETOOTH, socket.SOCK_STREAM, socket.BTPROTO_RFCOMM)
except (AttributeError, OSError):
return False
else:
s.close()
return True
```
Fixes: https://github.com/openwrt/packages/issues/16544
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Refreshed patches.
And apply hack for line-endings in pep517 (from pip).
Hack comment:
# FIXME: [1] get rid of this asap; 'patch' doesn't like Windows endings, and this file is full of them...
# I actually tried this in a number of ways and the only way to fix this is to implement
# a poor-man's dos2unix using sed.
# The issue is with the pip package; it seems that it throws in some Windows line-endings
# and 'patch' won't handle them. So, we do a "dos2unix" and then patch.
# We can get rid of this once this is solved upstream and in pip:
# https://github.com/pypa/pep517/pull/130
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
I tried to install matplotlib by using pip and it failed with the following output:
2021-08-18T11:52:26,171 Collecting matplotlib
2021-08-18T11:52:26,173 Created temporary directory: /tmp/pip-unpack-wuth2u0e
2021-08-18T11:52:26,565 Using cached matplotlib-3.4.3.tar.gz (37.9 MB)
2021-08-18T11:52:38,659 Added matplotlib from https://files.pythonhosted.org/packages/21/37/ 197e68df384ff694f78d687a49ad39f96c67b8d75718bc61503e1676b617/matplotlib-3.4.3.tar. gz#sha256=fc4f526dfdb31c9bd6b8ca06bf9fab663ca12f3ec9cdf4496fb44bc680140318 to build tracker '/tmp/pip- req-tracker-u30x8pht'
2021-08-18T11:52:38,660 Running setup.py (path:/tmp/pip-install-afiyiers/ matplotlib_8e717e38862f4976a3d6cb1832ba3261/setup.py) egg_info for package matplotlib
2021-08-18T11:52:38,661 Created temporary directory: /tmp/pip-pip-egg-info-kbtiezxq
2021-08-18T11:52:38,662 Running command python setup.py egg_info
2021-08-18T11:52:38,831 Traceback (most recent call last):
2021-08-18T11:52:38,832 File "<string>", line 1, in <module>
2021-08-18T11:52:38,832 File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 8, in <module>
2021-08-18T11:52:38,833 ModuleNotFoundError: No module named '_distutils_hack'
More details about it:
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968410
Once I applied the patch from the bug tracker, I got further to install
it.
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Python works with GNU DBM and with Berkley DBM.
Berkley DBM has been under Oracle for some time.
And it's not clear how many Python users actually use DBM.
In the packages feed, we have both libdb47 (which is now under Oracle) and
GNU DBM. The GNU DBM has a compatibility layer for Berkley DBM.
There are newer versions than libdb47, but it's probably not worth having
them yet. The libbd47 tarball is ~40+ MB. Odds are newer versions will be
bigger and more bloated.
This change merges the old `python3-gdbm` package into the `python3-dbm`
package, since they are effectively using the same underlying library now,
i.e. gdbm.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
I seem to forget to check/select setuptools and pip (that come bundled with
Python).
This change will do a simple 'ls' on the 2 wheel files, so that the build
fails even if just building Python.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Python3 comes with a built-in readline module. It wasn't included up until
now; mostly because it wasn't considered.
This change introduces it as a sub-package of the main Python3 package.
readline support is included in Python.
libreadline pulls libncursesw as a package, so python3-ncurses was
updated to pull libncursesw as well.
It should be the same package; mostly done for consistency.
Resolves the issue reported here:
https://forum.openwrt.org/t/python3-repl-missing-readline/90039
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Refreshed Python patches.
Updated pip & setuptools version.
For pip, patch '001-pep517-pyc-fix.patch' was reworked.
Also, the current version of the bundled pip (21.1.1) no longer supports
Python2, so the 'py2.py3' suffix gets replaced with just py3.
For setuptools, there is no longer a script/module:
https://github.com/pypa/setuptools/pull/2544
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Related to discussion:
https://github.com/openwrt/packages/pull/14060
Every once in a while a version bump will occur that requires an ABI
change. Example: Python 3.8 to 3.9. When this happens some Python packages
would need to be rebuilt. In setups where everything gets rebuilt, this
isn't a problem.
It's usually a bigger problem when needing to upgrade something via
opkg.
To accommodate for this, we add a libpython with it's own ABI_VERSION
flag. If this ABI_VERSION changes, then this should propagate forward.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Jeffery To