1. 24 Aug, 2019 1 commit
  2. 21 Aug, 2019 15 commits
    • Mark Brown's avatar
      merge_config.sh: Check error codes from make · cdfca821
      Mark Brown authored
      When we execute make after merging the configurations we ignore any
      errors it produces causing whatever is running merge_config.sh to be
      unaware of any failures.  This issue was noticed by Guillaume Tucker
      while looking at problems with testing of clang only builds in KernelCI
      which caused Kbuild to be unable to find a working host compiler.
      
      This implementation was suggested by Yamada-san.
      Suggested-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Reported-by: default avatarGuillaume Tucker <guillaume.tucker@collabora.com>
      Signed-off-by: default avatarMark Brown <broonie@kernel.org>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      cdfca821
    • Masahiro Yamada's avatar
      kbuild: move modkern_{c,a}flags to Makefile.lib from Makefile.build · eb27ea5c
      Masahiro Yamada authored
      Makefile.lib is included by Makefile.modfinal as well as Makefile.build.
      
      Move modkern_cflags to Makefile.lib in order to simplify cmd_cc_o_c
      in Makefile.modfinal. Move modkern_cflags as well for consistency.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      eb27ea5c
    • Masahiro Yamada's avatar
      kbuild: add CONFIG_ASM_MODVERSIONS · 2ff2b7ec
      Masahiro Yamada authored
      Add CONFIG_ASM_MODVERSIONS. This allows to remove one if-conditional
      nesting in scripts/Makefile.build.
      
      scripts/Makefile.build is run every time Kbuild descends into a
      sub-directory. So, I want to avoid $(wildcard ...) evaluation
      where possible although computing $(wildcard ...) is so cheap that
      it may not make measurable performance difference.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Acked-by: default avatarGeert Uytterhoeven <geert@linux-m68k.org>
      2ff2b7ec
    • Masahiro Yamada's avatar
      .gitignore: ignore modules.order explicitly · a564bdeb
      Masahiro Yamada authored
      The pattern '*.order' was added by commit c6025f4c ("kbuild: ignore
      *.order files") to ignore modules.order files.
      
      I do not see any other user of the '.order' extension.
      
      Ignore 'modules.order' explicitly instead of '*.order'.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      a564bdeb
    • Masahiro Yamada's avatar
      kbuild: split final module linking out into Makefile.modfinal · 9b9a3f20
      Masahiro Yamada authored
      I think splitting the modpost and linking modules into separate
      Makefiles will be useful especially when more complex build steps
      come in. The main motivation of this commit is to integrate the
      proposed klp-convert feature cleanly.
      
      I moved the logging 'Building modules, stage 2.' to Makefile.modpost
      to avoid the code duplication although I do not know whether or not
      this message is needed in the first place.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      9b9a3f20
    • Masahiro Yamada's avatar
      kbuild: rebuild modules when module linker scripts are updated · 10df0638
      Masahiro Yamada authored
      Currently, the timestamp of module linker scripts are not checked.
      Add them to the dependency of modules so they are correctly rebuilt.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      10df0638
    • Masahiro Yamada's avatar
      kbuild: move KBUILD_LDS, KBUILD_VMLINUX_{OBJS,LIBS} to makefiles.rst · 888f0c34
      Masahiro Yamada authored
      These three variables are not intended to be tweaked by users.
      Move them from kbuild.rst to makefiles.rst.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      888f0c34
    • Masahiro Yamada's avatar
      treewide: remove dummy Makefiles for single targets · cbdf59ad
      Masahiro Yamada authored
      Now that the single target build descends into sub-directories in the
      same way as the normal build, these dummy Makefiles are not needed
      any more.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      cbdf59ad
    • Masahiro Yamada's avatar
      kbuild: make single targets work more correctly · 394053f4
      Masahiro Yamada authored
      Currently, the single target build directly descends into the directory
      of the target. For example,
      
        $ make foo/bar/baz.o
      
      ... directly descends into foo/bar/.
      
      On the other hand, the normal build usually descends one directory at
      a time, i.e. descends into foo/, and then foo/bar/.
      
      This difference causes some problems.
      
      [1] miss subdir-asflags-y, subdir-ccflags-y in upper Makefiles
      
          The options in subdir-{as,cc}flags-y take effect in the current
          and its sub-directories. In other words, they are inherited
          downward. In the example above, the single target will miss
          subdir-{as,cc}flags-y if they are defined in foo/Makefile.
      
      [2] could be built in a different directory
      
          As Documentation/kbuild/modules.rst section 4.3 says, Kbuild can
          handle files that are spread over several sub-directories.
      
          The build rule of foo/bar/baz.o may not necessarily be specified in
          foo/bar/Makefile. It might be specifies in foo/Makefile as follows:
      
          [foo/Makefile]
          obj-y := bar/baz.o
      
          This often happens when a module is so big that its source files
          are divided into sub-directories.
      
          In this case, there is no Makefile in the foo/bar/ directory, yet
          the single target descends into foo/bar/, then fails due to the
          missing Makefile. You can still do 'make foo/bar/' for partial
          building, but cannot do 'make foo/bar/baz.s'. I believe the single
          target '%.s' is a useful feature for inspecting the compiler output.
      
          Some modules work around this issue by putting an empty Makefile
          in every sub-directory.
      
      This commit fixes those problems by making the single target build
      descend in the same way as the normal build does.
      
      Another change is the single target build will observe the CONFIG
      options. Previously, it allowed users to build the foo.o even when
      the corresponding CONFIG_FOO is disabled:
      
         obj-$(CONFIG_FOO) += foo.o
      
      In the new behavior, the single target build will just fail and show
      "No rule to make target ..." (or "Nothing to be done for ..." if the
      stale object already exists, but cannot be updated).
      
      The disadvantage of this commit is the build speed. Now that the
      single target build visits every directory and parses lots of
      Makefiles, it is slower than before. (But, I hope it will not be
      too slow.)
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      394053f4
    • Kees Cook's avatar
      kbuild: Parameterize kallsyms generation and correct reporting · 8959e392
      Kees Cook authored
      When kallsyms generation happens, temporary vmlinux outputs are linked
      but the quiet make output didn't report it, giving the impression that
      the prior command is taking longer than expected.
      
      Instead, report the linking step explicitly. While at it, this
      consolidates the repeated "kallsyms generation step" into a single
      function and removes the existing copy/pasting.
      Signed-off-by: default avatarKees Cook <keescook@chromium.org>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      8959e392
    • Masahiro Yamada's avatar
      kbuild: re-implement detection of CONFIG options leaked to user-space · c7c0eecf
      Masahiro Yamada authored
      scripts/headers_check.pl can detect references to CONFIG options in
      exported headers, but it has been disabled for more than a decade.
      
      Reverting commit 7e3fa561 ("kbuild: drop check for CONFIG_ in
      headers_check") would emit the following warnings for headers_check
      on x86:
      
      usr/include/mtd/ubi-user.h:283: leaks CONFIG_MTD_UBI_BEB_LIMIT to userspace where it is not valid
      usr/include/linux/cm4000_cs.h:26: leaks CONFIG_COMPAT to userspace where it is not valid
      usr/include/linux/pkt_cls.h:301: leaks CONFIG_NET_CLS_ACT to userspace where it is not valid
      usr/include/linux/videodev2.h:2465: leaks CONFIG_VIDEO_ADV_DEBUG to userspace where it is not valid
      usr/include/linux/bpf.h:249: leaks CONFIG_EFFICIENT_UNALIGNED_ACCESS to userspace where it is not valid
      usr/include/linux/bpf.h:819: leaks CONFIG_CGROUP_NET_CLASSID to userspace where it is not valid
      usr/include/linux/bpf.h:1011: leaks CONFIG_IP_ROUTE_CLASSID to userspace where it is not valid
      usr/include/linux/bpf.h:1742: leaks CONFIG_BPF_KPROBE_OVERRIDE to userspace where it is not valid
      usr/include/linux/bpf.h:1747: leaks CONFIG_FUNCTION_ERROR_INJECTION to userspace where it is not valid
      usr/include/linux/bpf.h:1936: leaks CONFIG_XFRM to userspace where it is not valid
      usr/include/linux/bpf.h:2184: leaks CONFIG_BPF_LIRC_MODE2 to userspace where it is not valid
      usr/include/linux/bpf.h:2210: leaks CONFIG_BPF_LIRC_MODE2 to userspace where it is not valid
      usr/include/linux/bpf.h:2227: leaks CONFIG_SOCK_CGROUP_DATA to userspace where it is not valid
      usr/include/linux/bpf.h:2311: leaks CONFIG_NET to userspace where it is not valid
      usr/include/linux/bpf.h:2348: leaks CONFIG_NET to userspace where it is not valid
      usr/include/linux/bpf.h:2422: leaks CONFIG_BPF_LIRC_MODE2 to userspace where it is not valid
      usr/include/linux/bpf.h:2528: leaks CONFIG_NET to userspace where it is not valid
      usr/include/linux/pktcdvd.h:37: leaks CONFIG_CDROM_PKTCDVD_WCACHE to userspace where it is not valid
      usr/include/linux/hw_breakpoint.h:27: leaks CONFIG_HAVE_MIXED_BREAKPOINTS_REGS to userspace where it is not valid
      usr/include/linux/raw.h:17: leaks CONFIG_MAX_RAW_DEVS to userspace where it is not valid
      usr/include/linux/elfcore.h:62: leaks CONFIG_BINFMT_ELF_FDPIC to userspace where it is not valid
      usr/include/linux/eventpoll.h:82: leaks CONFIG_PM_SLEEP to userspace where it is not valid
      usr/include/linux/atmdev.h:104: leaks CONFIG_COMPAT to userspace where it is not valid
      usr/include/asm-generic/unistd.h:651: leaks CONFIG_MMU to userspace where it is not valid
      usr/include/asm-generic/bitsperlong.h:9: leaks CONFIG_64BIT to userspace where it is not valid
      usr/include/asm-generic/fcntl.h:119: leaks CONFIG_64BIT to userspace where it is not valid
      usr/include/asm/auxvec.h:14: leaks CONFIG_IA32_EMULATION to userspace where it is not valid
      usr/include/asm/e820.h:14: leaks CONFIG_NODES_SHIFT to userspace where it is not valid
      usr/include/asm/e820.h:39: leaks CONFIG_X86_PMEM_LEGACY to userspace where it is not valid
      usr/include/asm/e820.h:49: leaks CONFIG_INTEL_TXT to userspace where it is not valid
      usr/include/asm/mman.h:7: leaks CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS to userspace where it is not valid
      
      Most of these are false positives because scripts/headers_check.pl
      parses comment lines.
      
      It is also false negative. arch/x86/include/uapi/asm/auxvec.h contains
      CONFIG_IA32_EMULATION and CONFIG_X86_64, but the only former is reported.
      
      It would be possible to fix scripts/headers_check.pl, of course.
      However, we already have some duplicated checks between headers_check
      and CONFIG_UAPI_HEADER_TEST. At this moment of time, there are still
      dozens of headers excluded from the header test (usr/include/Makefile),
      but we might be able to remove headers_check eventually.
      
      I re-implemented it in scripts/headers_install.sh by using sed because
      the most of code in scripts/headers_install.sh is written in sed.
      
      This patch works like this:
      
      [1] Run scripts/unifdef first because we need to drop the code
          surrounded by #ifdef __KERNEL__ ... #endif
      
      [2] Remove all C style comments. The sed code is somewhat complicated
          since we need to deal with both single and multi line comments.
      
          Precisely speaking, a comment block is replaced with a space just
          in case.
      
            CONFIG_FOO/* this is a comment */CONFIG_BAR
      
          should be converted into:
      
            CONFIG_FOO CONFIG_BAR
      
          instead of:
      
            CONFIG_FOOCONFIG_BAR
      
      [3] Match CONFIG_... pattern. It correctly matches to all CONFIG
          options that appear in a single line.
      
      After this commit, this would detect the following warnings, all of
      which are real ones.
      
      warning: include/uapi/linux/pktcdvd.h: leak CONFIG_CDROM_PKTCDVD_WCACHE to user-space
      warning: include/uapi/linux/hw_breakpoint.h: leak CONFIG_HAVE_MIXED_BREAKPOINTS_REGS to user-space
      warning: include/uapi/linux/raw.h: leak CONFIG_MAX_RAW_DEVS to user-space
      warning: include/uapi/linux/elfcore.h: leak CONFIG_BINFMT_ELF_FDPIC to user-space
      warning: include/uapi/linux/eventpoll.h: leak CONFIG_PM_SLEEP to user-space
      warning: include/uapi/linux/atmdev.h: leak CONFIG_COMPAT to user-space
      warning: include/uapi/asm-generic/fcntl.h: leak CONFIG_64BIT to user-space
      warning: arch/x86/include/uapi/asm/auxvec.h: leak CONFIG_IA32_EMULATION to user-space
      warning: arch/x86/include/uapi/asm/auxvec.h: leak CONFIG_X86_64 to user-space
      warning: arch/x86/include/uapi/asm/mman.h: leak CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS to user-space
      
      However, it is not nice to show them right now. I created a list of
      existing leakages. They are not warned, but a new leakage will be
      blocked by the 0-day bot.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Acked-by: default avatarSam Ravnborg <sam@ravnborg.org>
      c7c0eecf
    • Masahiro Yamada's avatar
      kbuild: unify clean-dirs rule for in-kernel and external module · 76cd306d
      Masahiro Yamada authored
      Factor out the duplicated code for in-kernel and external module
      cleaning.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      76cd306d
    • Masahiro Yamada's avatar
      kbuild: unify vmlinux-dirs and module-dirs rules · c99f3918
      Masahiro Yamada authored
      The in-kernel build and external module build have similar code
      for descending into sub-directories.
      
      Factor out the code into the common place.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      c99f3918
    • Masahiro Yamada's avatar
      kbuild: unset variables in top Makefile instead of setting 0 · 2042b548
      Masahiro Yamada authored
      There is no need to set 0 to variables such as config-targets,
      mixed-targets, etc.
      
      Unset instead of setting 0 in order to use 'ifdef' to test them.
      
      I also renamed:
      
        config-targets  ->  config-build
        mixed-targets   ->  mixed-build
        dot-config      ->  need-config
      
      to clarify what we are doing.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      2042b548
    • Masahiro Yamada's avatar
      kbuild: do not descend to ./Kbuild when cleaning · 125d059b
      Masahiro Yamada authored
      'make clean' descends into ./Kbuild, but does not clean anything
      since everything is added to no-clean-files.
      
      There is no need to descend to ./Kbuild in the first place.
      We can drop the no-clean-files assignment.
      
      With this, there is no more user of no-clean-files. I will keep it
      for a while to see whether a new user will appear.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      125d059b
  3. 14 Aug, 2019 6 commits
  4. 13 Aug, 2019 7 commits
    • Masahiro Yamada's avatar
      kbuild: add [M] marker for build log of *.mod.o · f6545bec
      Masahiro Yamada authored
      This builds module objects, so [M] makes sense.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      f6545bec
    • Thomas Gleixner's avatar
      Kbuild: Handle PREEMPT_RT for version string and magic · 4b950bb9
      Thomas Gleixner authored
      Update the build scripts and the version magic to reflect when
      CONFIG_PREEMPT_RT is enabled in the same way as CONFIG_PREEMPT is treated.
      
      The resulting version strings:
      
        Linux m 5.3.0-rc1+ #100 SMP Fri Jul 26 ...
        Linux m 5.3.0-rc1+ #101 SMP PREEMPT Fri Jul 26 ...
        Linux m 5.3.0-rc1+ #102 SMP PREEMPT_RT Fri Jul 26 ...
      
      The module vermagic:
      
        5.3.0-rc1+ SMP mod_unload modversions
        5.3.0-rc1+ SMP preempt mod_unload modversions
        5.3.0-rc1+ SMP preempt_rt mod_unload modversions
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      4b950bb9
    • Masahiro Yamada's avatar
      kbuild: move flex and bison rules to Makefile.host · cf8dfd15
      Masahiro Yamada authored
      Flex and bison are used for kconfig, dtc, genksyms, all of which are
      host programs. I never imagine the kernel embeds a parser or a lexer.
      
      Move the flex and bison rules to scripts/Makefile.host. This file is
      included only when hostprogs-y etc. is present in the Makefile in the
      directory. So, parsing these rules are skipped in most of directories.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      cf8dfd15
    • Masahiro Yamada's avatar
      kbuild: make bison create C file and header in a single pattern rule · 6ba7dc66
      Masahiro Yamada authored
      We generally expect bison to create not only a C file, but also a
      header, which will be included from the lexer.
      
      Currently, Kbuild generates them in separate rules. So, for instance,
      when building Kconfig, you will notice bison is invoked twice:
      
        HOSTCC  scripts/kconfig/conf.o
        HOSTCC  scripts/kconfig/confdata.o
        HOSTCC  scripts/kconfig/expr.o
        LEX     scripts/kconfig/lexer.lex.c
        YACC    scripts/kconfig/parser.tab.h
        HOSTCC  scripts/kconfig/lexer.lex.o
        YACC    scripts/kconfig/parser.tab.c
        HOSTCC  scripts/kconfig/parser.tab.o
        HOSTCC  scripts/kconfig/preprocess.o
        HOSTCC  scripts/kconfig/symbol.o
        HOSTLD  scripts/kconfig/conf
      
      Make handles such cases nicely in pattern rules [1]. Merge the two
      rules so that one invokcation of bison can generate both of them.
      
        HOSTCC  scripts/kconfig/conf.o
        HOSTCC  scripts/kconfig/confdata.o
        HOSTCC  scripts/kconfig/expr.o
        LEX     scripts/kconfig/lexer.lex.c
        YACC    scripts/kconfig/parser.tab.[ch]
        HOSTCC  scripts/kconfig/lexer.lex.o
        HOSTCC  scripts/kconfig/parser.tab.o
        HOSTCC  scripts/kconfig/preprocess.o
        HOSTCC  scripts/kconfig/symbol.o
        HOSTLD  scripts/kconfig/conf
      
      [1] Pattern rule
      
      GNU Make manual says:
      "Pattern rules may have more than one target. Unlike normal rules,
      this does not act as many different rules with the same prerequisites
      and recipe. If a pattern rule has multiple targets, make knows that
      the rule's recipe is responsible for making all of the targets. The
      recipe is executed only once to make all the targets. When searching
      for a pattern rule to match a target, the target patterns of a rule
      other than the one that matches the target in need of a rule are
      incidental: make worries only about giving a recipe and prerequisites
      to the file presently in question. However, when this file's recipe is
      run, the other targets are marked as having been updated themselves."
      
      https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.htmlSigned-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      6ba7dc66
    • Masahiro Yamada's avatar
      kbuild: use $(basename ...) for cmd_asn1_compiler · 49d5089d
      Masahiro Yamada authored
      $(basename ...) trims the last suffix. Using it is more intuitive in
      my opinion.
      
      This pattern rule makes %.asn1.c and %.asn1.h at the same time.
      Previously, the short log showed only either of them, depending on
      the target file in question.
      
      To clarify that two files are being generated by the single recipe,
      I changed the log as follows:
      
      Before:
      
        ASN.1   crypto/asymmetric_keys/x509.asn1.c
      
      After:
      
        ASN.1   crypto/asymmetric_keys/x509.asn1.[ch]
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      49d5089d
    • Thomas Gleixner's avatar
      kbuild: Fail if gold linker is detected · 75959d44
      Thomas Gleixner authored
      The gold linker has known issues of failing the build both in random and in
      predictible ways:
      
       - The x86/X32 VDSO build fails with:
      
         arch/x86/entry/vdso/vclock_gettime-x32.o:vclock_gettime.c:function do_hres:
         error: relocation overflow: reference to 'hvclock_page'
      
         That's a known issue for years and the usual workaround is to disable
         CONFIG_X86_32
      
       - A recent build failure is caused by turning a relocation into an
         absolute one for unknown reasons. See link below.
      
       - There are a couple of gold workarounds applied already, but reports
         about broken builds with ld.gold keep coming in on a regular base and in
         most cases the root cause is unclear.
      
      In context of the most recent fail H.J. stated:
      
        "Since building a workable kernel for different kernel configurations
         isn't a requirement for gold, I don't recommend gold for kernel."
      
      So instead of dealing with attempts to duct tape gold support without
      understanding the root cause and without support from the gold folks, fail
      the build when gold is detected.
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Acked-by: default avatarIngo Molnar <mingo@kernel.org>
      Link: https://lore.kernel.org/r/CAMe9rOqMqkQ0LNpm25yE_Yt0FKp05WmHOrwc0aRDb53miFKM+w@mail.gmail.comReviewed-by: default avatarNathan Chancellor <natechancellor@gmail.com>
      Tested-by: default avatarNathan Chancellor <natechancellor@gmail.com>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      75959d44
    • Denis Efremov's avatar
      modpost: check for static EXPORT_SYMBOL* functions · 15bfc234
      Denis Efremov authored
      This patch adds a check to warn about static EXPORT_SYMBOL* functions
      during the modpost. In most of the cases, a static symbol marked for
      exporting is an odd combination that should be fixed either by deleting
      the exporting mark or by removing the static attribute and adding the
      appropriate declaration to headers.
      
      This check could help to detect the following problems:
      1. 550113d4 ("i2c: add newly exported functions to the header, too")
      2. 54638c6e ("net: phy: make exported variables non-static")
      3. 98ef2046 ("mm: remove the exporting of totalram_pages")
      4. 73df167c ("s390/zcrypt: remove the exporting of ap_query_configuration")
      5. a57caf8c ("sunrpc/cache: remove the exporting of cache_seq_next")
      6. e4e47306 ("crypto: skcipher - remove the exporting of skcipher_walk_next")
      7. 14b4c48b ("gve: Remove the exporting of gve_probe")
      8. 9b79ee97 ("scsi: libsas: remove the exporting of sas_wait_eh")
      9. ...
      
      The build time impact is very limited and is almost at the unnoticeable
      level (< 1 sec).
      Acked-by: default avatarEmil Velikov <emil.l.velikov@gmail.com>
      Signed-off-by: default avatarDenis Efremov <efremov@linux.com>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      15bfc234
  5. 11 Aug, 2019 3 commits
  6. 10 Aug, 2019 8 commits