Skip to content

L
linux
Switch branch/tag
  1. 28 Sep, 2022 17 commits
    • Masahiro Yamada's avatar
      kbuild: use objtool-args-y to clean up objtool arguments · 9ec6ab6e
      Masahiro Yamada authored 
      Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well.

Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/Suggested-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
Reviewed-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      9ec6ab6e
    • Masahiro Yamada's avatar
      kbuild: fix and refactor single target build · cc306abd
      Masahiro Yamada authored 
      The single target build has a subtle bug for the combination for
an individual file and a subdirectory.

[1] 'make kernel/fork.i' builds only kernel/fork.i

  $ make kernel/fork.i
    CALL    scripts/checksyscalls.sh
    DESCEND objtool
    CPP     kernel/fork.i

[2] 'make kernel/' builds only under the kernel/ directory.

  $ make kernel/
    CALL    scripts/checksyscalls.sh
    DESCEND objtool
    CC      kernel/fork.o
    CC      kernel/exec_domain.o
       [snip]
    CC      kernel/rseq.o
    AR      kernel/built-in.a

But, if you try to do [1] and [2] in a single command, you will get
only [1] with a weird log:

  $ make kernel/fork.i kernel/
    CALL    scripts/checksyscalls.sh
    DESCEND objtool
    CPP     kernel/fork.i
  make[2]: Nothing to be done for 'kernel/'.

With 'make kernel/fork.i kernel/', you should get both [1] and [2].

Rewrite the single target build.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      cc306abd
    • Owen Rafferty's avatar
      kbuild: rewrite check-local-export in sh/awk · 033a52d0
      Owen Rafferty authored 
      Remove the bash build dependency for those who otherwise do not
have it installed. This also provides a significant speedup:

$ make defconfig
$ make yes2modconfig

...

$ find  .  -name "*.o" | grep -v vmlinux | wc
     3169      3169     89615
$ export NM=nm
$ time sh -c 'find . -name "*.o" | grep -v vmlinux | xargs -n1
./scripts/check-local-export'

Without patch:
    0m15.90s real     0m12.17s user     0m05.28s system

With patch:
dash + nawk
    0m02.16s real     0m02.92s user     0m00.34s system

dash + busybox awk
    0m02.36s real     0m03.36s user     0m00.34s system

dash + gawk
    0m02.07s real     0m03.26s user     0m00.32s system

bash + gawk
    0m03.55s real     0m05.00s user     0m00.54s system
Signed-off-by: default avatarOwen Rafferty <owen@owenrafferty.com>
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      033a52d0
    • Masahiro Yamada's avatar
      Revert "kbuild: Make scripts/compile.h when sh != bash" · a6c26e38
      Masahiro Yamada authored 
      This reverts commit [1] in the pre-git era.

I do not know what problem happened in the script when sh != bash
because there is no commit message.

Now that this script is much simpler than it used to be, let's revert
it, and let' see. (If this turns out to be problematic, fix the code
with proper commit description.)

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=11acbbbb8a50f4de7dbe4bc1b5acc440dfe81810Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      a6c26e38
    • Masahiro Yamada's avatar
      scripts/mkcompile_h: move LC_ALL=C to '$LD -v' · c7b594f5
      Masahiro Yamada authored 
      Minimize the scope of LC_ALL=C like before commit 87c94bfb ("kbuild:
override build timestamp & version").

Give LC_ALL=C to '$LD -v' to get the consistent version output, as commit
bcbcf50f ("kbuild: fix ld-version.sh to not be affected by locale")
mentioned the LD version is affected by locale.

While I was here, I merged two sed invocations.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      c7b594f5
    • Masahiro Yamada's avatar
      kbuild: generate include/generated/compile.h in top Makefile · a55f283e
      Masahiro Yamada authored 
      Now that UTS_VERSION was separated out, this header can be generated
much earlier, and probably the top Makefile is a better place to do it
than init/Makefile.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      a55f283e
    • Masahiro Yamada's avatar
      kbuild: build init/built-in.a just once · 2df8220c
      Masahiro Yamada authored 
      Kbuild builds init/built-in.a twice; first during the ordinary
directory descending, second from scripts/link-vmlinux.sh.

We do this because UTS_VERSION contains the build version and the
timestamp. We cannot update it during the normal directory traversal
since we do not yet know if we need to update vmlinux. UTS_VERSION is
temporarily calculated, but omitted from the update check. Otherwise,
vmlinux would be rebuilt every time.

When Kbuild results in running link-vmlinux.sh, it increments the
version number in the .version file and takes the timestamp at that
time to really fix UTS_VERSION.

However, updating the same file twice is a footgun. To avoid nasty
timestamp issues, all build artifacts that depend on init/built-in.a
are atomically generated in link-vmlinux.sh, where some of them do not
need rebuilding.

To fix this issue, this commit changes as follows:

[1] Split UTS_VERSION out to include/generated/utsversion.h from
    include/generated/compile.h

    include/generated/utsversion.h is generated just before the
    vmlinux link. It is generated under include/generated/ because
    some decompressors (s390, x86) use UTS_VERSION.

[2] Split init_uts_ns and linux_banner out to init/version-timestamp.c
    from init/version.c

    init_uts_ns and linux_banner contain UTS_VERSION. During the ordinary
    directory descending, they are compiled with __weak and used to
    determine if vmlinux needs relinking. Just before the vmlinux link,
    they are compiled without __weak to embed the real version and
    timestamp.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      2df8220c
    • Masahiro Yamada's avatar
      init/version.c: remove #include <linux/version.h> · 561daaac
      Masahiro Yamada authored 
      This is unneeded since commit 073a9ecb ("init/version.c: remove
Version_<LINUX_VERSION_CODE> symbol").
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      561daaac
    • Masahiro Yamada's avatar
      kbuild: move 'PHONY += modules_prepare' to the common part · 7f371813
      Masahiro Yamada authored 
      Unify the code between in-tree builds and external module builds.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      7f371813
    • Masahiro Yamada's avatar
      kbuild: refactor single builds of *.ko · f110e5a2
      Masahiro Yamada authored 
      Remove the potentially invalid modules.order instead of using
the temporary file.

Also, KBUILD_MODULES is don't care for single builds. No need to
cancel it.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      f110e5a2
    • Masahiro Yamada's avatar
      kbuild: remove duplicated dependency between modules and modules_check · f75a0334
      Masahiro Yamada authored 
      The dependency, "modules: modules_check" is specified twice.
Commit 1a998be6 ("kbuild: check module name conflict for external
modules as well") missed to clean it up.

'PHONY += modules' also appears twice.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      f75a0334
    • Masahiro Yamada's avatar
      nios2: move core-y in arch/nios2/Makefile to arch/nios2/Kbuild · e30d4487
      Masahiro Yamada authored 
      Use obj-y to clean up Makefile.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      e30d4487
    • Masahiro Yamada's avatar
      kbuild: do not deduplicate modules.order · d724b578
      Masahiro Yamada authored 
      The AWK code was added to deduplicate modules.order in case $(obj-m)
contains the same module multiple times, but it is actually unneeded
since commit b2c88554 ("kbuild: update modules.order only when
contained modules are updated").

The list is already deduplicated before being processed by AWK because
$^ is the deduplicated list of prerequisites.
(Please note the real-prereqs macro uses $^)

Yet, modules.order will contain duplication if two different Makefiles
build the same module:

  foo/Makefile:

      obj-m += bar/baz.o

  foo/bar/Makefile:

      obj-m += baz.o

However, the parallel builds cannot properly handle this case in the
first place. So, it is better to let it fail (as already done by
scripts/modules-check.sh).
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      d724b578
    • Masahiro Yamada's avatar
      kbuild: check sha1sum just once for each atomic header · b10fdeea
      Masahiro Yamada authored 
      It is unneeded to check the sha1sum every time.

Create the timestamp files to manage it.

Add '.' to clean-dirs because 'make clean' must visit ./Kbuild to
clean up the timestamp files.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      b10fdeea
    • Masahiro Yamada's avatar
      kbuild: hard-code KBUILD_ALLDIRS in scripts/Makefile.package · a3c4d4ab
      Masahiro Yamada authored 
      My future plan is to list subdirectories in ./Kbuild. When it occurs,
$(vmlinux-alldirs) will not contain all subdirectories.

Let's hard-code the directory list until I get around to implementing
a more sophisticated way for generating a source tarball.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
Tested-by: default avatarNick Desaulniers <ndesaulniers@google.com>
Reviewed-by: default avatarNicolas Schier <nicolas@fjasle.eu>
      a3c4d4ab
    • Masahiro Yamada's avatar
      kbuild: add phony targets to ./Kbuild · ed7ceac1
      Masahiro Yamada authored 
      missing-syscalls and old-atomics are meant to be phony targets.
Adding them to always-y is odd. (always-y should generate something).

Add a new phony target 'prepare', which depends on all the other.
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      ed7ceac1
    • Masahiro Yamada's avatar
      kbuild: remove the target in signal traps when interrupted · a7f3257d
      Masahiro Yamada authored 
      When receiving some signal, GNU Make automatically deletes the target if
it has already been changed by the interrupted recipe.

If the target is possibly incomplete due to interruption, it must be
deleted so that it will be remade from scratch on the next run of make.
Otherwise, the target would remain corrupted permanently because its
timestamp had already been updated.

Thanks to this behavior of Make, you can stop the build any time by
pressing Ctrl-C, and just run 'make' to resume it.

Kbuild also relies on this feature, but it is equivalently important
for any build systems that make decisions based on timestamps (if you
want to support Ctrl-C reliably).

However, this does not always work as claimed; Make immediately dies
with Ctrl-C if its stderr goes into a pipe.

  [Test Makefile]

    foo:
            echo hello > $@
            sleep 3
            echo world >> $@

  [Test Result]

    $ make                         # hit Ctrl-C
    echo hello > foo
    sleep 3
    ^Cmake: *** Deleting file 'foo'
    make: *** [Makefile:3: foo] Interrupt

    $ make 2>&1 | cat              # hit Ctrl-C
    echo hello > foo
    sleep 3
    ^C$                            # 'foo' is often left-over

The reason is because SIGINT is sent to the entire process group.
In this example, SIGINT kills 'cat', and 'make' writes the message to
the closed pipe, then dies with SIGPIPE before cleaning the target.

A typical bad scenario (as reported by [1], [2]) is to save build log
by using the 'tee' command:

    $ make 2>&1 | tee log

This can be problematic for any build systems based on Make, so I hope
it will be fixed in GNU Make. The maintainer of GNU Make stated this is
a long-standing issue and difficult to fix [3]. It has not been fixed
yet as of writing.

So, we cannot rely on Make cleaning the target. We can do it by
ourselves, in signal traps.

As far as I understand, Make takes care of SIGHUP, SIGINT, SIGQUIT, and
SITERM for the target removal. I added the traps for them, and also for
SIGPIPE just in case cmd_* rule prints something to stdout or stderr
(but I did not observe an actual case where SIGPIPE was triggered).

[Note 1]

The trap handler might be worth explaining.

    rm -f $@; trap - $(sig); kill -s $(sig) $$

This lets the shell kill itself by the signal it caught, so the parent
process can tell the child has exited on the signal. Generally, this is
a proper manner for handling signals, in case the calling program (like
Bash) may monitor WIFSIGNALED() and WTERMSIG() for WCE although this may
not be a big deal here because GNU Make handles SIGHUP, SIGINT, SIGQUIT
in WUE and SIGTERM in IUE.

  IUE - Immediate Unconditional Exit
  WUE - Wait and Unconditional Exit
  WCE - Wait and Cooperative Exit

For details, see "Proper handling of SIGINT/SIGQUIT" [4].

[Note 2]

Reverting 392885ee ("kbuild: let fixdep directly write to .*.cmd
files") would directly address [1], but it only saves if_changed_dep.
As reported in [2], all commands that use redirection can potentially
leave an empty (i.e. broken) target.

[Note 3]

Another (even safer) approach might be to always write to a temporary
file, and rename it to $@ at the end of the recipe.

   <command>  > $(tmp-target)
   mv $(tmp-target) $@

It would require a lot of Makefile changes, and result in ugly code,
so I did not take it.

[Note 4]

A little more thoughts about a pattern rule with multiple targets (or
a grouped target).

    %.x %.y: %.z
            <recipe>

When interrupted, GNU Make deletes both %.x and %.y, while this solution
only deletes $@. Probably, this is not a big deal. The next run of make
will execute the rule again to create $@ along with the other files.

[1]: https://lore.kernel.org/all/YLeot94yAaM4xbMY@gmail.com/
[2]: https://lore.kernel.org/all/20220510221333.2770571-1-robh@kernel.org/
[3]: https://lists.gnu.org/archive/html/help-make/2021-06/msg00001.html
[4]: https://www.cons.org/cracauer/sigint.html

Fixes: 392885ee ("kbuild: let fixdep directly write to .*.cmd files")
Reported-by: default avatarIngo Molnar <mingo@kernel.org>
Reported-by: default avatarRob Herring <robh@kernel.org>
Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
Tested-by: default avatarIngo Molnar <mingo@kernel.org>
Reviewed-by: default avatarNicolas Schier <nicolas@fjasle.eu>
      a7f3257d
  3. 25 Sep, 2022 8 commits
  5. 24 Sep, 2022 10 commits
  7. 23 Sep, 2022 5 commits
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7