* Support wildcards in install (`+`) paths
* Add fourth parameter to set directory permissions
If file permissions are given (third parameter), these will now apply
to files only.
* Add non-recursive set permissions command (`==`)
* Be more strict about filespec format
Blank lines and lines starting with `#` will be ignored. Other errors
(unknown command, missing path parameter, etc.) will cause the script
to exit.
* Be more strict about ensuring paths exist for all commands
* Avoid spawning subshells
This also removes outdated filespec paths in the python3 package; these
paths delete files that are no longer present.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* Fix default Python package description not included in individual
package descriptions
* Update default Python package description text (from General Python
FAQ, "What is Python?")
* Add package descriptions for Python module packages
* Reduce duplication in package titles
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
There is no need to use Py3Package for python3 as it does not package
any Python files; it is an empty package with dependencies to install
the full Python installation.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Packaging setuptools from a separate source package allows it to stay
updated with upstream.
Host setuptools will remain installed as part of python3. Host
setuptools is used in a much more controlled way and so is less critical
for it to track upstream.
setuptools was in a separate source package that was removed in
a53d0c5a403d1669e2cf6c59c2be6a9d3ed633a0; this work is not based on that
earlier package.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Packaging pip from a separate source package allows it to stay updated
with upstream.
Host pip will remain installed as part of python3. Host pip is used in a
much more controlled way and so is less critical for it to track
upstream.
This also removes the python-pip-conf package and installs the pip.conf
file as part of python3-pip.
The patch 003-disable-pip-version-check.patch is originally from Debian:
bb079efb8c/debian/patches/disable-pip-version-check.patch
pip was in a separate source package that was removed in
a53d0c5a403d1669e2cf6c59c2be6a9d3ed633a0; this work is not based on that
earlier package.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds a new subpackage for the venv module. This also moves the
ensurepip module from python3-pip into python3-venv, as ensurepip is not
necessary for pip but often used for venv.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
configure will try to link with libbsd for the flock function, even when
flock is available without it.
Fixes: https://github.com/openwrt/packages/issues/21161
Fixes: 2445fe9fb2 ("python3: Update to 3.11.3, refresh/redo patches")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* Removed patches:
* 001-enable-zlib.patch: zlib module should be enabled automatically
* 007-distutils-do-not-adjust-path.patch: Not necessary since we
process shebang lines for all scripts (in python3-package.mk)
* 030-bpo-43112-detect-musl-as-a-separate-SOABI-GH-24502.patch:
Already merged
* Move configure vars from config.site back into Makefile
Centralizing all build information into one file makes it easier to
maintain
* No longer set ac_cv_header_uuid_h=yes as configure should detect
libuuid
* Order configure args by enable-/disable-/with-/without-, then
alphabetically
* Set ac_cv_working_openssl_hashlib=yes for host configure to bypass the
OpenSSL API tests with LibreSSL
* Use the default Host/Compile recipe instead of picking out specific
targets to make
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
026-openssl-feature-flags.patch and
028-host-python-support-ssl-with-libressl.patch were removed in
4ecd9d67e9 to fix the ssl module after
libressl was upgraded to 3.7.0[1].
However, the cause of the ssl module build failure was only
028-host-python-support-ssl-with-libressl.patch.
Removing 026-openssl-feature-flags.patch caused a build failure for the
hashlib module.
This restores 026-openssl-feature-flags.patch with an updated version of
the patch from OpenBSD[2].
[1]: https://github.com/openwrt/packages/issues/20107
[2]: 26a04435bf/lang/python/3.10/patches/patch-Modules__hashopenssl_c
Fixes: 4ecd9d67e9 ("python3: fix ssl support by removing libressl patches")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds $(STAGING_DIR_HOST)/include/e2fsprogs to HOST_CFLAGS and
HOST_CPPFLAGS so that configure can find uuid/uuid.h.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
By default, the Python build process will add /usr/local/{lib,include},
and multiarch paths (e.g. /usr/{lib,include}/x86_64-linux-gnu) if
building on Debian/Ubuntu, to its library and includes paths.
006-remove-multi-arch-and-local-paths.patch was added in
84202f17e1 to stop the Python build
process from adding these paths.
006-remove-multi-arch-and-local-paths.patch was removed in
48277ec915.
006-do-not-add-multiarch-paths-when-cross-compiling.patch was added in
0c8b0b0bf7 to stop the Python build
process from adding these paths for target Python.
These paths are still added by the Python build process when building
host Python.
This replaces the cross-compiling-only patch with the original patch,
renamed slightly and adapted for Python 3.10.
Fixes: 48277ec915 ("python3: bump to version 3.8")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
The Makefile lines to add READELF to TARGET_CONFIGURE_OPTS was removed
in 4e05541782.
Without setting READELF, configure finds the symlink to
$(TARGET_CROSS)readelf (e.g. arm-openwrt-linux-readelf) instead of
$(TARGET_CROSS)readelf (e.g. arm-openwrt-linux-muslgnueabi-readelf).
This leads to the symlink name being saved to _sysconfigdata.py, and so
the readelf name is not replaced correctly (in
Py3Package/python3-base/install).
This restores the removed Makefile lines.
Fixes: 4e05541782 ("python3: bump to version 3.10.0")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
003-do-not-run-distutils-tests.patch was removed in
4e05541782. This patch stopped "make
install" from, among other things, running compileall.
When this patch was removed, "make install" ran compileall as normal and
created bytecode files in __pycache__ directories. These files were then
packaged in python3-light.
This adds a patch to stop compileall from being run during "make
install".
Fixes: 4e05541782 ("python3: bump to version 3.10.0")
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
--without-pymalloc was added in 7bf1ae65a8
because leaving it enabled added an "m" flag/suffix to file names.
This flag/suffix was removed in Python 3.8[1], so disabling pymalloc is
no longer necessary.
[1]: https://docs.python.org/3.8/whatsnew/3.8.html#build-and-c-api-changes
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This removes the changes made in
61f202c017 and adds actual support for
pyproject.toml-based (PEP 517) builds of Python packages.
Packages can force the use of the old build process by setting
PYTHON3_PKG_FORCE_DISTUTILS_SETUP:=1; this should only be a temporary
workaround until the package can be updated/fixed to use the new build
process.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Fixes:
https://github.com/openwrt/packages/issues/12707
Seems to work.
Looking into the 'venv' lib, it seems it's installing pip & setuptools
inside a virtual environment.
`python3-pip` is already ~6 MB.
This adds another ~3 MB.
But, this gives users the ability to run Python virtual environments, which
is a pretty common feature of Python in production cases (usually web
stuff).
Signed-off-by: Alexandru Ardelean <alex@shruggie.ro>
Remove libressl specific patches. With commit
("tools/libressl: update to 3.7.0") they are no longer needed,
rather they cause python3 to be compiled without working ssl-support.
Fixes: #20107
Suggested-by: Andre Heider <a.heider@gmail.com>
Signed-off-by: Nick Hainke <vincent@systemli.org>
Includes fixes for:
* Windows builds updated to bzip2 1.0.8 to mitigate CVE-2016-3189 and
CVE-2019-12900
* CVE-2022-26488: Escalation of privilege via Windows Installer
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Removed patches:
* 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch
Already merged.
* 029-disable-deprecation-warning.patch
Packages should be patched/fixed to remove the use of distutils
instead of disabling this warning.
Also:
* Updates PKG_LICENSE to use the correct SPDX license identifier
* Fixes build for mipsel_24kc_24kf
Fixes https://github.com/openwrt/packages/issues/17217.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
There's been a bit of overlapping opinions on some of these packages.
The best thing to do here is to reduce ownership and relinquish my
control.
This patch does that.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Manually re-applied:
008-distutils-use-python-sysroot.patch
016-adjust-config-paths.patch
Drop patch: 003-do-not-run-distutils-tests.patch
There is now a configure option '--disable-test-modules'
And seems we left the '_ctypes_test' around for quite some time.
Dropped now.
Refs:
https://bugs.python.org/issue27640https://bugs.python.org/issue43282
Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch
Python build honors SOURCE_DATE_EPOCH pretty well now.
Drop setuptools patches. Setuptools should be reproducible with Python 3.6+
according to a mention here:
https://github.com/pypa/setuptools/pull/1690#issuecomment-536517456
It's time to let upstream fix Setuptools reproduce-ability.
Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch
I can't seem to fully remember why it's there.
And it seem to build fine without it.
Drop patch: 015-abort-on-failed-modules.patch
Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var
option.
Add patch: 026-openssl-feature-flags.patch
We need to keep this in our tree for a while.
See:
https://bugs.python.org/issue45627
Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch
Link: https://github.com/python/cpython/pull/29353
Fixes the build for uuid C module.
Add patch: 028-host-python-support-ssl-with-libressl.patch
We need the _ssl module working on the host-side with LibreSSL for pip to
work to download from https://pypi.org
Refs: https://github.com/openwrt/openwrt/pull/4749
Add patch: 029-disable-deprecation-warning.patch
Fixes apparmor build. The warning causes a configure error.
Refreshed the rest of patches.
Some old build-flags were removed. They don't seem to be necessary anymore.
Split python3-uuid from python3-light. To better manage the libuuid library
(if needed). Also, fixing the uuid C module build. Seems this was failing,
and was falling back to using hashlib.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Bluetooth support requires bluez-libs present, but they are only required
for the build, and don't seem to be needed to be present on the target.
There isn't any linking required to libbluetooth. It's only the bluetooth.h
header that is required for building BT support into Python.
For testing, this snippet was used from `Lib/test/test_socket.py` (inside
cpython):
```
def _have_socket_bluetooth():
"""Check whether AF_BLUETOOTH sockets are supported on this host."""
try:
# RFCOMM is supported by all platforms with bluetooth support. Windows
# does not support omitting the protocol.
s = socket.socket(socket.AF_BLUETOOTH, socket.SOCK_STREAM, socket.BTPROTO_RFCOMM)
except (AttributeError, OSError):
return False
else:
s.close()
return True
```
Fixes: https://github.com/openwrt/packages/issues/16544
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Refreshed patches.
And apply hack for line-endings in pep517 (from pip).
Hack comment:
# FIXME: [1] get rid of this asap; 'patch' doesn't like Windows endings, and this file is full of them...
# I actually tried this in a number of ways and the only way to fix this is to implement
# a poor-man's dos2unix using sed.
# The issue is with the pip package; it seems that it throws in some Windows line-endings
# and 'patch' won't handle them. So, we do a "dos2unix" and then patch.
# We can get rid of this once this is solved upstream and in pip:
# https://github.com/pypa/pep517/pull/130
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Python works with GNU DBM and with Berkley DBM.
Berkley DBM has been under Oracle for some time.
And it's not clear how many Python users actually use DBM.
In the packages feed, we have both libdb47 (which is now under Oracle) and
GNU DBM. The GNU DBM has a compatibility layer for Berkley DBM.
There are newer versions than libdb47, but it's probably not worth having
them yet. The libbd47 tarball is ~40+ MB. Odds are newer versions will be
bigger and more bloated.
This change merges the old `python3-gdbm` package into the `python3-dbm`
package, since they are effectively using the same underlying library now,
i.e. gdbm.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
I seem to forget to check/select setuptools and pip (that come bundled with
Python).
This change will do a simple 'ls' on the 2 wheel files, so that the build
fails even if just building Python.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Python3 comes with a built-in readline module. It wasn't included up until
now; mostly because it wasn't considered.
This change introduces it as a sub-package of the main Python3 package.
readline support is included in Python.
libreadline pulls libncursesw as a package, so python3-ncurses was
updated to pull libncursesw as well.
It should be the same package; mostly done for consistency.
Resolves the issue reported here:
https://forum.openwrt.org/t/python3-repl-missing-readline/90039
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Refreshed Python patches.
Updated pip & setuptools version.
For pip, patch '001-pep517-pyc-fix.patch' was reworked.
Also, the current version of the bundled pip (21.1.1) no longer supports
Python2, so the 'py2.py3' suffix gets replaced with just py3.
For setuptools, there is no longer a script/module:
https://github.com/pypa/setuptools/pull/2544
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Related to discussion:
https://github.com/openwrt/packages/pull/14060
Every once in a while a version bump will occur that requires an ABI
change. Example: Python 3.8 to 3.9. When this happens some Python packages
would need to be rebuilt. In setups where everything gets rebuilt, this
isn't a problem.
It's usually a bigger problem when needing to upgrade something via
opkg.
To accommodate for this, we add a libpython with it's own ABI_VERSION
flag. If this ABI_VERSION changes, then this should propagate forward.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Includes fixes for:
* CVE-2021-3177 - ctypes: Buffer overflow in PyCArg_repr
* CVE-2021-23336 - urllib parse_qsl(): Web cache poisoning - semicolon
as a query args separator
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
The Python version of python-config is installed when building on macOS,
rather than the shell script version when building on Linux. When run on
macOS, the Python version of python-config will return values with
Mac-specific customizations.
This patches the python-config install recipe so that which version is
installed can be controlled by the package makefile. When building on
macOS, this installs the Python version for host Python and the shell
script version for target Python.
This also updates Host/Compile and Host/Install to use the default host
build recipes, so that the various HOST_* variables are taken into
account automatically.
Fixes https://github.com/openwrt/packages/issues/14652
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Because of e52d0487e88c3c8c57e1310d1a02b18eae0d142e , these flags no
longer get passed. Add them back to fix compilation.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Refreshed patches.
Dropped 'patches-setuptools/004-site-patch.patch'
Does not apply anymore. Setuptools has removed site.py support:
https://github.com/pypa/setuptools/issues/2165
If this is still needed, we may need to re-think it's implementation.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This version includes fixes for:
* CVE-2020-15801 - Fixes python3x._pth being ignored on Windows
* CVE-2019-20907 - Avoid infinite loop when reading specially crafted
TAR files using the tarfile module
This also:
* Remove patches that are included in the update
* Add a dependency in python3-distutils for python3-email[1]
[1]: https://github.com/python/cpython/blob/v3.8.5/Lib/distutils/dist.py#L10
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This version includes fixes for:
* CVE-2020-14422: Hash collisions in IPv4Interface and IPv6Interface
* CVE-2020-15523: Python uses invalid DLL path after calling Py_SetPath
on Windows
This version also includes support for OpenSSL 1.1.x builds that use
'no-deprecated' and '--api=1.1.0'[1], and so this removes the previous
OpenSSL-related patches.
This also backports fixes for security issues, including:
* CVE-2019-20907: Infinite loop in the tarfile module
This also updates the setuptools and pip packages to 47.1.0 and 20.1.1,
respectively.
[1]: https://github.com/python/cpython/pull/20566
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This lets the Python build process set _PYTHON_HOST_PLATFORM instead of
forcing an explicit value.
Also:
* Save the target _PYTHON_HOST_PLATFORM value during Build/InstallDev
for use when building target Python packages (in python3-package.mk).
* Use the (mostly) default PYTHON_FOR_BUILD value, instead patch
configure to remove the platform triplet from the sysconfigdata file
name.
* Remove the "CROSS_COMPILE=yes" make variable (there is no indication
that this variable is necessary).
* Force host pip to build packages from source instead of downloading
binary wheels.
Previously, host pip can download universal (platform-independent)
wheels but not platform-specific wheels, because of the custom
_PYTHON_HOST_PLATFORM value. (Packages that do not have universal
wheels would be compiled from source.)
With a correct _PYTHON_HOST_PLATFORM, host pip can install
platform-specific wheels as well. However, the pre-built shared object
(.so) files in these wheels will have the host's platform triplet in
their file names. When target Python packages are built (using the
target's _PYTHON_HOST_PLATFORM), Python will not use these shared
object files.
By forcing host pip to build packages from source, the built shared
object files will not have the platform triplet in their file names.
(Host Python has been patched to remove the platform triplet from file
names.) This allows these packages to be used when building target
Python packages.
(The net effect of this complete change is that platform-dependent
packages will continue to be compiled from source, while
platform-independent packages will now also be compiled from source.)
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Previously, binaries installed by Python packages will have a
non-suffixed Python 2 version and a suffixed Python 3 version, e.g. pip
and pip3. With the removal of Python 2, the non-suffixed names are no
longer taken.
This adds symlinks for the non-suffixed names linking to the suffixed
scripts (or in the case of pip, easy_install, and python-config, to the
fully-versioned scripts).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Jeffery To