Avoid unneeded recreation of these in the incremental build.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

These explicit rules are unneeded because scripts/Makefile.build
provides a pattern rule to create %.s from %.c
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Clean up the Makefile. Equivalent *.gen.S files are still produced.

 - Use filechk to remove ugly wordsize_deps

 - Get FWNAME, FWSTR, ASM_WORD, ASM_ALIGN, and PROGBITS out of the
   recipe for readability

 - Remove 'mkdir' because filechk takes care of it
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Commit df85b2d7 ("firmware: Restore support for built-in firmware")
should not have restored this .gitignore blindly.

*.gen.S is only pattern that should be ignored by git.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Commit c512d254 ("gitignore: ignore scripts/ihex2fw") was unneeded.
ihex2fw was generated in firmware/ instead of scripts/ at that time
although ihex2fw.c was pushed back and forth between those directories
in the past.

check-lc_ctype was removed by commit cb43fb57 ("docs: remove
DocBook from the building system").
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This is a leftover of commit ecba97d4 ("[PATCH] uml makefiles
sanitized").
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The LLVM/Clang project provides many tools for analyzing C source code.
Many of these tools are based on LibTooling
(https://clang.llvm.org/docs/LibTooling.html), which depends on a
database of compiler flags. The standard container for this database is
compile_commands.json, which consists of a list of JSON objects, each
with "directory", "file", and "command" fields.

Some build systems, like cmake or bazel, produce this compilation
information directly. Naturally, Makefiles don't. However, the kernel
makefiles already create .<target>.o.cmd files that contain all the
information needed to build a compile_commands.json file.

So, this commit adds scripts/gen_compile_commands.py, which recursively
searches through a directory for .<target>.o.cmd files and extracts
appropriate compile commands from them. It writes a
compile_commands.json file that LibTooling-based tools can use.

By default, gen_compile_commands.py starts its search in its working
directory and (over)writes compile_commands.json in the working
directory. However, it also supports --output and --directory flags for
out-of-tree use.

Note that while gen_compile_commands.py enables the use of clang-based
tools, it does not require the kernel to be compiled with clang. E.g.,
the following sequence of commands produces a compile_commands.json file
that works correctly with LibTooling.

make defconfig
make
scripts/gen_compile_commands.py

Also note that this script is written to work correctly in both Python 2
and Python 3, so it does not specify the Python version in its first
line.

For an example of the utility of this script: after running
gen_compile_commands.json on the latest kernel version, I was able to
use Vim + the YouCompleteMe pluging + clangd to automatically jump to
definitions and declarations. Obviously, cscope and ctags provide some
of this functionality; the advantage of supporting LibTooling is that it
opens the door to many other clang-based tools that understand the code
directly and do not rely on regular expressions and heuristics.

Tested: Built several recent kernel versions and ran the script against
them, testing tools like clangd (for editor/LSP support) and clang-check
(for static analysis). Also extracted some test .cmd files from a kernel
build and wrote a test script to check that the script behaved correctly
with all permutations of the --output and --directory flags.
Signed-off-by: Tom Roeder <tmroeder@google.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This flag is documented in the GCC 4.6 manual, and recognized by
Clang as well. Let's rip off the cc-option switch.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

-Wstrict-prototypes is added to KBUILD_CFLAGS first, then overridden
by -Werror=strict-prototypes later.

Let's add -Werror=strict-prototypes unconditionally because it is
supported by GCC 4.6, and also by Clang.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

This flag is documented in the GCC 4.6 manual, and recognized by
Clang as well. Let's rip off the cc-option switch.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

"Assignment" requires the assigned value before the place that
value is stored into.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Some code may overall use 0 and 1, so don't introduce occasional
uses of true and false in these cases.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The single targets *.ll and *.symtypes have never been treated as
a module. Fix it.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Use $(foreach ...) to make it shorter.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

part-of-module and quiet_modtag are set for the same targets.
Define quiet_modtag based on part-of-module.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

All objects in $(obj-m) are contained in $(real-obj-m) as well.

It is true composite objects are only contained in $(obj-m),
but [M] is hard-coded in quiet_cmd_link_multi-m.

This line is redundant.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

 - Use conventional $(MAKE) $(asm-generic)=<dir> style
   for directory descending

 - Remove unneeded FORCE since "all" is a phony target

 - Remove unneeded "_dummy :=" assignment

 - Skip $(shell mkdir ...) when headers exist in the directory

 - Misc cleanups
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>

Signed-off-by: Eugen Cazacu <green182day@hotmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

When CONFIG_HEADERS_CHECK is enabled, the headers_check is executed
as a serialized task in the vmlinux recipe.

Make it independent of vmlinux so that parallel building can process
the headers_check and other build targets simultaneously.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

scripts/Makefile.headersinst takes care of *.agh just for

  arch/cris/include/uapi/arch-v10/arch/sv_addr.agh

because renaming exported headers is difficult (or impossible).

This code is no longer necessary thanks to commit c690eddc ("CRIS:
Drop support for the CRIS port").
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

In commit 54a702f7 ("kbuild: mark $(targets) as .SECONDARY and
remove .PRECIOUS markers"), I missed one important feature of the
.SECONDARY target:

    .SECONDARY with no prerequisites causes all targets to be
    treated as secondary.

... which agrees with the policy of Kbuild.

Let's move it to scripts/Kbuild.include, with no prerequisites.

Note:
If an intermediate file is generated by $(call if_changed,...), you
still need to add it to "targets" so its .*.cmd file is included.

The arm/arm64 crypto files are generated by $(call cmd,shipped),
so they do not need to be added to "targets", but need to be added
to "clean-files" so "make clean" can properly clean them away.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

These three cmd_* are invoked in the $(call cmd,*) form.

Now that 'set -e' moved to the 'cmd' macro, they do not need to
explicitly give 'set -e'.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

'@set -e; $(echo-cmd) $(cmd_$(1)' can be replaced with '$(cmd)'.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

With the change of rule_cc_o_c / rule_as_o_S in the last commit, each
command is executed in a separate subshell. Rip off unneeded semicolons.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The 'define' ... 'endef' directive is useful to confine a series of
shell commands into a single macro:

  define foo
          [action1]
          [action2]
          [action3]
  endif

Each action is executed in a separate subshell.

However, rule_cc_o_c and rule_as_o_S in scripts/Makefile.build are
written as follows (with a trailing semicolon in each cmd_*):

  define rule_cc_o_c
          [action1] ; \
          [action2] ; \
          [action3] ;
  endef

All shell commands are concatenated with '; \' so that it looks like
a single command from the Makefile point of view. This does not
exploit the benefits of 'define' ... 'endef' form because a single
shell command can be more simply written, like this:

  rule_cc_o_c = \
          [action1] ; \
          [action2] ; \
          [action3] ;

I guess the intention for the command concatenation was to let the
'@set -e' in if_changed_rule cover all the commands.

We can improve the readability by moving '@set -e' to the 'cmd' macro.
The combo of $(call echo-cmd,*) $(cmd_*) in rule_cc_o_c and rule_as_o_S
have been replaced with $(call cmd,*). The trailing back-slashes have
been removed.

Here is a note about the performance: the commands in rule_cc_o_c and
rule_as_o_S were previously executed all together in a single subshell,
but now each line in a separate subshell. This means Make will spawn
extra subshells [1]. I measured the build performance for
  x86_64_defconfig + CONFIG_MODVERSIONS + CONFIG_TRIM_UNUSED_KSYMS
and I saw slight performance regression, but I believe code readability
and maintainability wins.

[1] Precisely, GNU Make may optimize this by executing the command
    directly instead of forking a subshell, if no shell special
    characters are found in the command line and omitting the subshell
    will not change the behavior.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

My main motivation of this commit is to clean up scripts/Kbuild.include
and scripts/Makefile.build.

Currently, CONFIG_TRIM_UNUSED_KSYMS works with a tricky gimmick;
possibly exported symbols are detected by letting $(CPP) replace
EXPORT_SYMBOL* with a special string '=== __KSYM_*===', which is
post-processed by sed, and passed to fixdep. The extra preprocessing
is costly, and hacking cmd_and_fixdep is ugly.

I came up with a new way to find exported symbols; insert a dummy
symbol __ksym_marker_* to each potentially exported symbol. Those
dummy symbols are picked up by $(NM), post-processed by sed, then
appended to .*.cmd files. I collected the post-process part to a
new shell script scripts/gen_ksymdeps.sh for readability. The dummy
symbols are put into the .discard.* section so that the linker
script rips them off the final vmlinux or modules.

A nice side-effect is building with CONFIG_TRIM_UNUSED_KSYMS will
be much faster.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>

Let $(CC) compile objects into normal files *.o instead of .tmp_*.o
whether CONFIG_MODVERSIONS is enabled or not. With this, the input
file for objtool is always *.o so objtool_o can go away.

I guess the reason of using .tmp_*.o for intermediate objects was
to avoid leaving incomplete *.o file (, whose timestamp says it is
up-to-date) when the genksyms tool failed for some reasons.

It no longer matters because any targets are deleted on errors since
commit 9c2af1c7 ("kbuild: add .DELETE_ON_ERROR special target").
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This is executed inside the if_changed_rule, which already sets
'set -e'.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The filechk macro in scripts/Kbuild.include already sets 'set -e'.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, fixdep writes dependencies to .*.tmp, which is renamed to
.*.cmd after everything succeeds. This is a very safe way to avoid
corrupted .*.cmd files. The if_changed_dep has carried this safety
mechanism since it was added in 2002.

If fixdep fails for some reasons or a user terminates the build while
fixdep is running, the incomplete output from the fixdep could be
troublesome.

This is my insight about some bad scenarios:

[1] If the compiler succeeds to generate *.o file, but fixdep fails
    to write necessary dependencies to .*.cmd file, Make will miss
    to rebuild the object when headers or CONFIG options are changed.
    In this case, fixdep should not generate .*.cmd file at all so
    that 'arg-check' will surely trigger the rebuild of the object.

[2] A partially constructed .*.cmd file may not be a syntactically
    correct makefile. The next time Make runs, it would include it,
    then fail to parse it. Once this happens, 'make clean' is be the
    only way to fix it.

In fact, [1] is no longer a problem since commit 9c2af1c7 ("kbuild:
add .DELETE_ON_ERROR special target"). Make deletes a target file on
any failure in its recipe. Because fixdep is a part of the recipe of
*.o target, if it fails, the *.o is deleted anyway. However, I am a
bit worried about the slight possibility of [2].

So, here is a solution. Let fixdep directly write to a .*.cmd file,
but allow makefiles to include it only when its corresponding target
exists.

This effectively reverts commit 2982c953 ("kbuild: remove redundant
$(wildcard ...) for cmd_files calculation"), and commit 00d78ab2
("kbuild: remove dead code in cmd_files calculation in top Makefile")
because now we must check the presence of targets.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Now that 'archprepare' depends on 'scripts', Kbuild can descend into
scripts/gcc-plugins in a more standard way.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>

Before start descending, Kbuild needs to run 'prepare' and 'scripts',
which has been orthogonal to each other.

Going forward, let's consider 'scripts' is a part of the preparation.
This will allow more cleanups.

Move 'scripts' to the prerequisite of 'archprepare', where UML starts
compiling target *.c files.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

I am eagar to build under the scripts/ directory only with $(HOSTCC),
but scripts/mod/ highly depends on the $(CC) and target arch headers.
That it why the 'scripts' target must depend on 'asm-generic',
'gcc-plugins', and $(autoksyms_h).

Move it to the 'prepare0' stage. I know this is a cheesy workaround,
but better than the current situation.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

UML fails to build with CONFIG_GCC_PLUGINS=y.

$ make -s ARCH=um mrproper
$ make -s ARCH=um allmodconfig
$ make ARCH=um
  UPD     include/generated/uapi/linux/version.h
  WRAP    arch/x86/include/generated/uapi/asm/bpf_perf_event.h
  WRAP    arch/x86/include/generated/uapi/asm/poll.h
  WRAP    arch/x86/include/generated/asm/dma-contiguous.h
  WRAP    arch/x86/include/generated/asm/early_ioremap.h
  WRAP    arch/x86/include/generated/asm/export.h
  WRAP    arch/x86/include/generated/asm/mcs_spinlock.h
  WRAP    arch/x86/include/generated/asm/mm-arch-hooks.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
  HOSTCC  scripts/unifdef
  CC      arch/x86/um/user-offsets.s
cc1: error: cannot load plugin ./scripts/gcc-plugins/cyc_complexity_plugin.so
./scripts/gcc-plugins/cyc_complexity_plugin.so: cannot open shared object file: No such file or directory
cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so
./scripts/gcc-plugins/structleak_plugin.so: cannot open shared object file: No such file or directory
cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared object file: No such file or directory
cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open shared object file: No such file or directory
make[1]: *** [scripts/Makefile.build;119: arch/x86/um/user-offsets.s] Error 1
make: *** [arch/um/Makefile;152: arch/x86/um/user-offsets.s] Error 2

Reorder the preparation stage (with cleanups) to make sure gcc-plugins
is built before descending to arch/x86/um/.

Fixes: 6b90bd4b ("GCC plugin infrastructure")
Reported-by: kbuild test robot <lkp@intel.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This will fit better in check_exports() than add_versions().
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Probably, this is just a matter of the order of error/warning
messages. Merge the two for-loops.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

You do not need to iterate over all modules for resetting ->seen flag
because add_depends() is only interested in modules that export symbols
referenced from the given 'mod'.

This also avoids shadowing the 'modules' parameter of add_depends().
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Use specific prototype instead of an opaque pointer so that the
compiler can catch function prototype mismatch.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Mathieu Malaterre <malat@debian.org>

Commit e49ce141 ("modpost: use linker section to generate table.")
was not so cool as we had expected first; it ended up with ugly section
hacks when commit dd2a3aca ("mod/file2alias: make modpost compile
on darwin again") came in.

Given a certain degree of unknowledge about the link stage of host
programs, I really want to see simple, stupid table lookup so that
this works in the same way regardless of the underlying executable
format.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Mathieu Malaterre <malat@debian.org>

Building a single target in an external module fails due to missing
.tmp_versions directory.

For example,

  $ make -C /lib/modules/$(uname -r)/build M=$PWD foo.o

will fail in the following way:

  CC [M]  /home/masahiro/foo/foo.o
/bin/sh: 1: cannot create /home/masahiro/foo/.tmp_versions/foo.mod: Directory nonexistent

This is because $(cmd_crmodverdir) is executed only before building
/, %/, %.ko single targets of external modules. Create .tmp_versions
in the 'prepare' target.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>