GitLab

Since commit 040fcc81 ("kbuild: improved modversioning support for
external modules"), the external module build reads Module.symvers in
the directory of the module itself, then dumps symbols back into it.
It accumulates stale symbols in the file when you build an external
module incrementally.

The idea behind it was, as the commit log explained, you can copy
Modules.symvers from one module to another when you need to pass symbol
information between two modules. However, the manual copy of the file
sounds questionable to me, and containing stale symbols is a downside.

Some time later, commit 0d96fb20

 ("kbuild: Add new Kbuild variable
KBUILD_EXTRA_SYMBOLS") introduced a saner approach.

So, this commit removes the former one. Going forward, the external
module build dumps symbols into Module.symvers to be carried via
KBUILD_EXTRA_SYMBOLS, but never reads it automatically.

With the -I option removed, there is no one to set the external_module
flag unless KBUILD_EXTRA_SYMBOLS is passed. Now the -i option does it
instead.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

When building external modules, $(objtree)/Module.symvers is scanned
for symbol information of vmlinux and in-tree modules.

Additionally, vmlinux is parsed if it exists in $(objtree)/.
This is totally redundant since all the necessary information is
contained in $(objtree)/Module.symvers.

Do not parse vmlinux at all for external module builds. This makes
sense because vmlinux is deleted by 'make clean'.

'make clean' leaves all the build artifacts for building external
modules. vmlinux is unneeded for that.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The comment line "When building external modules ..." explains
the same thing as "Include the module's Makefile ..." a few lines
below.

The comment "they may be used when building the .mod.c file" is no
longer true; .mod.c file is compiled in scripts/Makefile.modfinal
since commit 9b9a3f20

 ("kbuild: split final module linking out
into Makefile.modfinal"). I still keep the code in case $(obj) or
$(src) is used in the external module Makefile.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

A script that uses the '<module>.ns_deps' files generated by modpost to
automatically add the required symbol namespace dependencies to each
module.

Usage:
1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
   DEFAULT_SYMBOL_NAMESPACE
2) Run 'make' (or 'make modules') and get warnings about modules not
   importing that namespace.
3) Run 'make nsdeps' to automatically add required import statements
   to said modules.

This makes it easer for subsystem maintainers to introduce and maintain
symbol namespaces into their codebase.
Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Jessica Yu <jeyu@kernel.org>

I think splitting the modpost and linking modules into separate
Makefiles will be useful especially when more complex build steps
come in. The main motivation of this commit is to integrate the
proposed klp-convert feature cleanly.

I moved the logging 'Building modules, stage 2.' to Makefile.modpost
to avoid the code duplication although I do not know whether or not
this message is needed in the first place.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, the timestamp of module linker scripts are not checked.
Add them to the dependency of modules so they are correctly rebuilt.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This builds module objects, so [M] makes sense.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Update the build scripts and the version magic to reflect when
CONFIG_PREEMPT_RT is enabled in the same way as CONFIG_PREEMPT is treated.

The resulting version strings:

  Linux m 5.3.0-rc1+ #100 SMP Fri Jul 26 ...
  Linux m 5.3.0-rc1+ #101 SMP PREEMPT Fri Jul 26 ...
  Linux m 5.3.0-rc1+ #102 SMP PREEMPT_RT Fri Jul 26 ...

The module vermagic:

  5.3.0-rc1+ SMP mod_unload modversions
  5.3.0-rc1+ SMP preempt mod_unload modversions
  5.3.0-rc1+ SMP preempt_rt mod_unload modversions
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

I removed the single target %.ko in commit ff9b45c5 ("kbuild:
modpost: read modules.order instead of $(MODVERDIR)/*.mod") because
the modpost stage does not work reliably. For instance, the module
dependency, modversion, etc. do not work if we lack symbol information
from the other modules.

Yet, some people still want to build only one module in their interest,
and it may be still useful if it is used within those limitations.

Fixes: ff9b45c5

 ("kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod")
Reported-by: Don Brace <don.brace@microsemi.com>
Reported-by: Arend Van Spriel <arend.vanspriel@broadcom.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Since commit ff9b45c5 ("kbuild: modpost: read modules.order instead
of $(MODVERDIR)/*.mod"), 'make vmlinux' emits a warning, like this:

$ make defconfig vmlinux
  [ snip ]
  LD      vmlinux.o
cat: modules.order: No such file or directory
  MODPOST vmlinux.o
  MODINFO modules.builtin.modinfo
  KSYM    .tmp_kallsyms1.o
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map

When building only vmlinux, KBUILD_MODULES is not set. Hence, the
modules.order is not generated. For the vmlinux modpost, it is not
necessary at all.

Separate scripts/Makefile.modpost for the vmlinux/modules stages.
This works more efficiently because the vmlinux modpost does not
need to include .*.cmd files.

Fixes: ff9b45c5

 ("kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod")
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

__modpost is a phony target. The dependency on FORCE is pointless.
All the objects have been built in the previous stage, so the
dependency on the objects are not necessary either.

Count the number of modules in a more straightforward way.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

KBUILD_EXTRA_SYMBOLS makes sense only when building external modules.
Moreover, the modpost sets 'external_module' if the -e option is given.

I replaced $(patsubst %, -e %,...) with simpler $(addprefix -e,...)
while I was here.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

If a build rule fails, the .DELETE_ON_ERROR special target removes the
target, but does nothing for the .*.cmd file, which might be corrupted.
So, .*.cmd files should be included only when the corresponding targets
exist.

Commit 392885ee ("kbuild: let fixdep directly write to .*.cmd
files") missed to fix up this file.

Fixes: 392885ee

 ("kbuild: let fixdep directly write to .*.cmd")
Cc: <stable@vger.kernel.org> # v5.0+
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

While descending directories, Kbuild produces objects for modules,
but do not link final *.ko files; it is done in the modpost.

To keep track of modules, Kbuild creates a *.mod file in $(MODVERDIR)
for every module it is building. Some post-processing steps read the
necessary information from *.mod files. This avoids descending into
directories again. This mechanism was introduced in 2003 or so.

Later, commit 551559e1 ("kbuild: implement modules.order") added
modules.order. So, we can simply read it out to know all the modules
with directory paths. This is easier than parsing the first line of
*.mod files.

$(MODVERDIR) has a flat directory structure, that is, *.mod files
are named only with base names. This is based on the assumption that
the module name is unique across the tree. This assumption is really
fragile.

Stephen Rothwell reported a race condition caused by a module name
conflict:

  https://lkml.org/lkml/2019/5/13/991

I...

Towards the goal of removing MODVERDIR, read out modules.order to get
the list of modules to be processed. This is simpler than parsing *.mod
files in $(MODVERDIR).

For external modules, $(KBUILD_EXTMOD)/modules.order should be read.

I removed the single target %.ko from the top Makefile. To make sure
modpost works correctly, vmlinux and the other modules must be built.
You cannot build a particular .ko file alone.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Commit ea837f1c

 ("kbuild: make modpost processing configurable")
was intended to give KBUILD_MODPOST_WARN flexibility to be configurable.
Right now KBUILD_MODPOST_WARN gets just ignored when KBUILD_EXTMOD is
set which happens per default when building modules out of the tree.

This change gives the opportunity to define module build behaving also
in case of out of tree builds and default will become exit on error.
Errors which can be detected by the build should be trapped out of the box
there, unless somebody wants to notice broken stuff later at runtime.

As this patch changes the default behaving from warning to error,
users can consider to fix it for external module builds by:
- providing module symbol table via KBUILD_EXTRA_SYMBOLS for
  modules which are dependent
- OR getting old behaving back by passing KBUILD_MODPOST_WARN to the build
Signed-off-by: Wladislav Wiebe <wladislav.wiebe@nokia.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Unless CONFIG_DEBUG_SECTION_MISMATCH is enabled, modpost only shows
the number of section mismatches.

If you want to know the symbols causing the issue, you need to rebuild
with CONFIG_DEBUG_SECTION_MISMATCH. It is tedious.

I think it is fine to show annoying warning when a new section mismatch
comes in.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

In Kbuild, if_changed and friends must have FORCE as a prerequisite.

Hence, $(filter-out FORCE,$^) or $(filter-out $(PHONY),$^) is a common
idiom to get the names of all the prerequisites except phony targets.

Add real-prereqs as a shorthand.

Note:
We cannot replace $(filter %.o,$^) in cmd_link_multi-m because $^ may
include auto-generated dependencies from the .*.cmd file when a single
object module is changed into a multi object module. Refer to commit
69ea912f

 ("kbuild: remove unneeded link_multi_deps"). I added some
comment to avoid accidental breakage.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Rob Herring <robh@kernel.org>

Commit a0f97e06 ("kbuild: enable 'make CFLAGS=...' to add
additional options to CC") renamed CFLAGS to KBUILD_CFLAGS.

Commit 222d394d ("kbuild: enable 'make AFLAGS=...' to add
additional options to AS") renamed AFLAGS to KBUILD_AFLAGS.

Commit 06c5040c

 ("kbuild: enable 'make CPPFLAGS=...' to add
additional options to CPP") renamed CPPFLAGS to KBUILD_CPPFLAGS.

For some reason, LDFLAGS was not renamed.

Using a well-known variable like LDFLAGS may result in accidental
override of the variable.

Kbuild generally uses KBUILD_ prefixed variables for the internally
appended options, so here is one more conversion to sanitize the
naming convention.

I did not touch Makefiles under tools/ since the tools build system
is a different world.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>

The comment is the same as in the top-level Makefile.

Also, the comments contain typos:
  - the .PHONY variable  ->  the PHONY variable
  - se we can ...        ->  so we can ...

Instead of fixing the typos, just remove the duplicated comments.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

I do not see any reason why $(wildcard ...) needs to be called twice
for computing cmd_files.  Remove the first one.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier.  The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
 - file had no licensing information it it.
 - file was a */uapi/* one with no licensing information in it,
 - file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard...

Since commit 040fcc81

 ("kbuild: improved modversioning
support for external modules"), symverfile has been replaced
with kernelsymfile and modulesymfile.
Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Allow architectures to create arch/xxx/Makefile.postlink with targets
for vmlinux, modules.ko, and clean, which will be invoked after final
linking of vmlinux and modules.

powerpc will use this to check vmlinux linker relocations for sanity,
and may use it to fix up alternate instruction patch branch addresses.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.com>

The section mismatch warning can be easy to miss during the kernel build
process. Allow it to be marked as fatal to be easily caught and prevent
bugs from slipping in.

Setting CONFIG_SECTION_MISMATCH_WARN_ONLY=y causes these warnings to be
non-fatal, since there are a number of section mismatches when using
allmodconfig on some architectures, and we do not want to break these
builds by default.
Signed-off-by: Nicolas Boichat <drinkcat@chromium.org>
Change-Id: Ic346706e3297c9f0d790e3552aa94e5cff9897a6
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Commit ea4054a2 (modpost: handle huge numbers of modules) added
support for building a large number of modules.

Unfortunately, the commit changed the semantics of the makefile: Instead of
passing only existing object files to modpost, make now passes all expected
object files. If make was started with option -i, this results in a modpost
error if a single file failed to build.

Example with the current btrfs build falure on m68k:

fs/btrfs/btrfs.o: No such file or directory
make[1]: [__modpost] Error 1 (ignored)

This error is followed by lots of errors such as:

m68k-linux-gcc: error: arch/m68k/emu/nfcon.mod.c: No such file or directory
m68k-linux-gcc: fatal error: no input files
compilation terminated.
make[1]: [arch/m68k/emu/nfcon.mod.o] Error 1 (ignored)

This doesn't matter much for normal builds, but it is annoying for builds
started with "make -i" due to the large number of secondary errors.
Those errors unnececessari...

strace shows:
72102 execve("/bin/sh", ["/bin/sh", "-c", "echo '  scripts/mod/modpost -m -a
-o /cc/wfg/sound-compiletest/Module.symvers      -s'; scripts/
mod/modpost -m -a -o /cc/wfg/sound-compiletest/Module.symvers      -s vmlinux
arch/x86/crypto/ablk_helper.o arch/x86/crypto/aes-i586.o arch
/x86/crypto/aesni-intel.o arch/x86/crypto/crc32-pclmul.o
...
drivers/ata/sata_promise.o "...], [/* 119 vars */] <unfinished ...>
71827 wait4(-1,  <unfinished ...>
72102 <... execve resumed> )            = -1 E2BIG (Argument list too long)

So we re-run the shell command which produces the list and feed it into modpost -T -.
Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Use the target compiler to compute the offsets for the fields of the
device_id structures, so that it won't be broken by different alignments
between the host and target ABIs.

This also fixes missing endian corrections for some modaliases.
Signed-off-by: Andreas Schwab <schwab@linux-m68k.org>
Signed-off-by: Michal Marek <mmarek@suse.cz>

Linus deleted the old code and put signing on the install command,
I fixed it to extract the keyid and signer-name within sign-file
and cleaned up that script now it always signs in-place.

Some enthusiast should convert sign-key to perl and pull
x509keyid into it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

If CONFIG_MODULE_SIG is set, then this patch will cause all modules files to
to have signatures added.  The following steps will occur:

 (1) The module will be linked to foo.ko.unsigned instead of foo.ko

 (2) The module will be stripped using both "strip -x -g" and "eu-strip" to
     ensure minimal size for inclusion in an initramfs.

 (3) The signature will be generated on the stripped module.

 (4) The signature will be appended to the module, along with some information
     about the signature and a magic string that indicates the presence of the
     signature.

Step (3) requires private and public keys to be available.  By default these
are expected to be found in files:

	signing_key.priv
	signing_key.x509

in the base directory of the build.  The first is the private key in PEM form
and the second is the X.509 certificate in DER form as can be generated from
openssl:

	openssl req \
		-new -x509 -outform PEM -out signing_key.x509 \
		-keyout signing...

This error may happen when the user's id or path includes .ko string.
For example, user's id is xxx.ko and building test.ko module,
the test.mod file lists ko name and all object files.
   /home/xxx.ko/kernel_dev/device/drivers/test.ko
   /home/xxx.ko/kernel_dev/device/drivers/test_main.o
/home/xxx.ko/kernel_dev/device/drivers/test_io.o ...
Current Makefile.modpost and Makefile.modinst find and list up not
only test.ko but also other object files.
because all of object file's path includes .ko string.
This is a patch to fix it.
Signed-off-by: Gunho Lee <gunho.lee@lge.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

It's "include/linux/vermagic.h", not "include/vermagic.h"
Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Michal Marek <mmarek@suse.cz>

Fixes generated by 'codespell' and manually reviewed.
Signed-off-by: Lucas De Marchi <lucas.demarchi@profusion.mobi>

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Michal Marek <mmarek@suse.cz>

It is now possible to assign options to AS, CC and LD
on the command line - which is only used when building modules.

{A,C,LD}FLAGS_MODULE was all used both in the top-level Makefile
in the arch makefiles, thus users had no way to specify
additional options to AS, CC, LD when building modules
without overriding the original value.

Introduce a new set of variables KBUILD_{A,C,LD}FLAGS_MODULE
that is used by arch specific files and free up
{A,C,LD}FLAGS_MODULE so they can be assigned on
the command line.

All arch Makefiles that used the old variables has been updated.

Note: Previously we had a MODFLAGS variable for both
AS and CC. But in favour of consistency this was dropped.
So in some cases arch Makefile has one assignmnet replaced by
two assignmnets.

Note2: MODFLAGS was not documented and is dropped
without any notice. I do not expect much/any breakage
from this.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Denys Vlasenko <...

Now that the last users of markers have migrated to the event
tracer we can kill off the (now orphan) support code.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <20090917173527.GA1699@lst.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>

Taken from http://bugzilla.kernel.org/show_bug.cgi?id=11567



If you even define KBUILD_EXTRA_SYMBOLS in Makefile it will not be expanded
into command line argument for modpost.
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

When a kernel was rebuilt, the previous Module.markers was not cleared.
It caused markers with different format strings to appear as duplicates
when a markers was changed.  This problem is present since
scripts/mod/modpost.c started to generate Module.markers, commit
b2e3e658



It therefore applies to 2.6.25, 2.6.26 and linux-next.

I merely merged the patches from Roland, Wenji and Takashi here.

Credits to
Roland McGrath <roland@redhat.com>
Wenji Huang <wenji.huang@oracle.com>
and
Takashi Nishiie <t-nishiie@np.css.fujitsu.com>

for providing the individual fixes.

- Changelog :
  - Integrated Takashi's Makefile modification to clear Module.markers upon
    make clean.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Cc: Roland McGrath <roland@redhat.com>
Cc: Wenji Huang <wenji.huang@oracle.com>
Cc: Takashi Nishiie <t-nishiie@np.css.fujitsu.com>
Cc: <stable@kernel.org>		[2.6.25.x, 2.6.26.x]
Signed-off-by: Andrew Morton <a...

When we introduced support for KBUILD_EXTRA_SYMBOLS
we started to include the externam module's kbuild
file when doing the final modpost step.

As external modules often do:
ccflags-y := -I$(src)

We had problems because $(src) was unassinged and
gcc then used the next parameter for -I resulting in
strange build failures.

Fix is to assign $(src) and $(obj) when building
external modules.

This fixes: http://bugzilla.kernel.org/show_bug.cgi?id=10798

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Tvrtko <tvrtko.ursulin@sophos.com>
Cc: Andrea Arcangeli <andrea@qumranet.com>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>

-EVIUSER  ;-)
Signed-off-by: Adrian Bunk <bunk@kernel.org>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>