Now that most cases do the same thing in SetupHostCommand, merge them
together into one. To allow moving the generic symlink check, invert the
check and let it check for relative links by matching on link targets
that do not start with a slash.
This then allows us to also drop the absolute link case, shortening the
case statement further.
This reorders the check to
* if it is not a symlink, do not change it
* if it is a symlink and it points to the found command, do not change it
* if it is a symlink with a relative path, do not change it
* else, update/replace it
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
To avoid replacing host built binaries with symlinks again, a check for
an appropriate stamp was added in 729909c07f ("prereq-build: do not
replace binaries with symlinks"). Unfortunately the stamp directory does
not exist in the SDK, so the fix was ineffective there.
This caused the packages builders to e.g. use the host tar again, which
in turn made the tarballs created different since it may lack
reproducibility fixes, or implement these differently, causing spurious
hash failures on source repository based packages.
Fix this by dropping the stamp dir check, and just check that the file
is usable.
Fixes: 729909c07f ("prereq-build: do not replace binaries with symlinks")
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
Some programs installed to staging_dir/host/bin
also install some symlinks to itself
for an alternative name.
Some of those new symlinks are overwriting
symlinks that were installed by prereq stage.
If prereq stage were to somehow be run again,
it should not be overwriting symlinks
that point to programs that are already built.
To filter that out, catch all symlinks
after first catching all symlinks
that have an absolute target
after all other cases in the case statement,
make sure it is not broken, and if so exit successfully.
Suggested-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Some programs, like bash and patch, are checked by prereq stage
and have a symlink installed, but then is later built from source.
Now that the prereq-build checks are not successful
just by finding the file alone, it is possible for
a new symlink to overwrite the installed binary.
If a normal file is found in staging_dir/host/bin,
let the check look for the associated stampfile, and if found,
skip creation of a symlink and exit successfully.
Suggested-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Michael Pratt <mcpratt@pm.me>
This reverts commit 7855378fcd.
The return "exit 1" was intentional and actually just
makes the symlink checks much more strict.
This new level of strictness added to the checks revealed
what was a confusing regression with prereq stage that
already existed but was not presenting itself
because of the simple way that checks used to be done before.
Either way, reverting to "exit 0" was a nice workaround
until the true root cause was discovered, so as to not interfere
with others' pull requests and builds in the meantime.
It turns out that this problem was the inconsistent value of $PATH
between different commands within the SetupHostCommand recipe,
now fixed in the parent commit, using the variable created
in the parent of the parent commit.
Ref: f75204036c ("prereq-build: allow host command symlinks to update")
Signed-off-by: Michael Pratt <mcpratt@pm.me>
In the recipe SetupHostCommand for checking
and creating symlinks, $PATH was only overridden
for one of several commands.
This causes the symlinks to be included
in the paths to pick a program from
when the check was repeated, because
staging_dir/host/bin was included in $PATH,
but only *sometimes*.
When the check ran again, the command succeded
with a $PATH including the symlink,
(eval "$$$$$$$$cmd")
while the path to the program was evaluated
with a $PATH NOT including the symlink,
(bin=...)
causing the symlink to be relinked incorrectly,
instead of passing as exactly the same.
Coincidentally, this was only a problem
if the symlink targeted the alternative
program with a different name.
By overriding the value of $PATH at the invocation of Make,
it will apply to the entire environment of the checks.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Tony has reported, that CI tools job is failing for him in macOS
container due to prereq check failure for GNU `install` utility.
Michael diagnosed it and from his traces it was clear, that the issue is
caused by a wrong return value in the success check case, so lets fix it
accordingly.
Fixes: f75204036c ("prereq-build: allow host command symlinks to update")
Reported-by: Tony Ambardar <itugrok@yahoo.com>
Diagnosed-by: Michael Pratt <mcpratt@pm.me>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
This reverts commit 3b68fb57c9.
After refactoring build checks to update old symlinks,
and after a long time of no python 2 support,
this is no longer needed.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
This makes the prereq stage update the symlinks
installed into staging_dir/host/bin
by rearrainging the way they are verified.
Before, seeing or installing a symlink would result in
a successful exit code, and not installing a symlink
would result is a failed exit code. However,
that is not able to account for the difference
between existing good and bad links, or whether
the link would be the same if it was reinstalled,
because the check can match the program to a different path.
Instead, let a success exit code represent
identifying an existing symlink as exactly the same
as what would be installed if it did not exist,
and let a fail exit code represent
needing to install the symlink
or not having a match to the check criteria.
The failing exit code is caught by a new second attempt
for all of the check-* targets which will then indicate
to the user that there was an update by having a success
exit code when the check is run again and the link is the same.
When there is nothing to update, the checks will run only once.
This relies on the ls command to be POSIX-conformant with long format:
"path/to/link -> target/of/link"
Also, make sure the symlink is executable, not just a file,
and the directory only needs to be created once.
Fixes: #12610
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Add RequireCHeader helper that will try to compile a fake c program with
the requested header included.
This is useful to check if a specific header is present in the system
without checking for the specific path.
This is a generilized version of the current ncurses test.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Ubuntu started to flag which as deprecated and it
seems which is not really standard and may vary
across Distro.
Drop the use of which and use the standard 'command -v'
for this simple task.
Which is still present in the prereq if some package/script
still use which.
A utility script called command_all.sh is implemented that
will just mimic the output of which -a.
Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
This reverts commit c7aec47e5e.
The original commit replaces 'which' with 'command'. Sadly most of
them are not equivalent and for 'which -a', there is no easy
replacements that would not reimplement PATH parsing logic. Hence
revert. Keeping a dependency on which is absolutely fine.
Signed-off-by: Clemens Fruhwirth <clemens@endorphin.org>
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
`which` utility is not shipped by default for example on recent Arch
Linux and then any steps relying on its presence fails, like for example
following Python3 prereq build check:
$ python3 --version
Python 3.9.1
$ make
/bin/sh: line 1: which: command not found
/bin/sh: line 1: which: command not found
/bin/sh: line 1: which: command not found
...
Checking 'python3'... failed.
...
Fix this by switching to Bash builtin `command` which should provide
same functionality.
Fixes: FS#3525
Signed-off-by: Petr Štetiar <ynezz@true.cz>
When bumping buildroot to Python 3, we need to assure, that Python
symlink in staging bin directory points to Python >= 3.5 as well.
We can't rely completly just on SetupHostCommand as its executed only in
cases when the $(STAGING_DIR_HOST)/bin/python doesn't already exist, so
we need to remove it before running SetupHostCommand.
Acked-by: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Commit d6d3db0543 added more gcc version probes, exceeding the argument
limit of the SetupHostCommand macro, leading to failing GCC/LLVM tests
on OS X.
Extend the handled number of arguments to restore proper functionality.
Fixes FS#1470
Fixes d6d3db0543 ("build: Improve GCC version detection")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Perl might not be present while checking prereqs, e.g. on a fresh
FreeBSD install.
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
SVN-Revision: 44276
Those macros allow testing various commands on the host system, also process the
prereq tests in the order they're declared.
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
SVN-Revision: 44269
Jonas Gorski