All files in lxdialog/ are licensed under GPL-2.0+, and the rest are
under GPL-2.0. I added GPL-2.0 tags to test scripts in tests/.

Documentation/process/license-rules.rst does not suggest anything
about the flex/bison files. Because flex does not accept the C++
comment style at the very top of a file, I used the C style for
zconf.l, and so for zconf.y for consistency.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Commit 7a88488b ("[PATCH] kconfig: use gperf for kconfig keywords")
introduced gperf for the keyword lookup.

Then, commit bb3290d9 ("Remove gperf usage from toolchain") killed
the gperf use. As a result, the linear keyword search was left behind.

If we do not use gperf, there is no reason to have the separate table
of the keywords. Move all keywords back to the lexer.

I also refactored the lexer to remove the COMMAND and PARAM states.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

To simplify the generated lexer, let the hand-made lexer update the
file name and line number for the parser.

I tested this with DEBUG_PARSE, and confirmed the same file names
and line numbers were dumped.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

To simplify the generated lexer, switch to the ASSIGN_VAL state in
the hand-made lexer.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The lexer has conventionally associated kconf_id data with yylval
to carry additional information to the parser.

No token is relying on this any more.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

T_ENDMENU, T_ENDCHOICE, T_ENDIF are the last users of kconf_id
associated with yylval. Refactor them to not use it.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

In my understanding, special characters such as '.' and '/' are
supported in unquoted words to use bare file paths in the "source"
statement.

With the previous commit surrounding all file paths with double
quotes, we can drop this.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The Kconfig lexer supports special characters such as '.' and '/' in
the parameter context. In my understanding, the reason is just to
support bare file paths in the source statement.

I do not see a good reason to complicate Kconfig for the room of
ambiguity.

The majority of code already surrounds file paths with double quotes,
and it makes sense since file paths are constant string literals.

Make it treewide consistent now.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Wolfram Sang <wsa@the-dreams.de>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Acked-by: Ingo Molnar <mingo@kernel.org>

I guess this is a constant value instead of a symbol.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Michal Simek <michal.simek@xilinx.com>

There is no grammatical ambiguity by using T_WORD for variables.
The parser can distinguish variables from symbols from the context.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, the lexer returns T_ASSIGN for all of =, :=, and +=
associating yylval with the flavor.

I want to make the generated lexer as simple as possible. So, the
lexer should convert keywords to tokens without thinking about the
meaning.

   =  ->  T_EQUAL
  :=  ->  T_COLON_EQUAL
  +=  ->  T_PLUS_EQUAL

Unfortunately, Kconfig uses = instead of == for the equal operator.
So, the same token T_EQUAL is used for assignment and comparison.
The parser can still distinguish them from the context.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

For the keywords "modules", "defconfig_list", and "allnoconfig_y",
the lexer should pass specific tokens instead of generic T_WORD.

This simplifies both the lexer and the parser.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This commit removes kconf_id::stype to prepare for the entire
removal of kconf_id.c

To simplify the lexer, I want keywords straight-mapped to tokens.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

These are already defined as %left.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Now the comment_stmt is the only user of depends_list. Rename it to
comment_option_list
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, "visible" and "depends on", if defined in a menu entry,
must appear in that order.

The real example is in drivers/media/tuners/Kconfig:

  menu "Customize TV tuners"
          visible if <expr1>
          depends on <expr2>

... is fine, but you cannot change the property order like this:

  menu "Customize TV tuners"
          depends on <expr2>
          visible if <expr1>

Kconfig does not require a specific order of properties. In this case,
menu_add_visibility(() and menu_add_dep() are orthogonal.

Loosen this unreasonable restriction.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The code block surrounded by "menu" ... "endmenu" is stmt_list.

Remove the redundant menu_block symbol entirely.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The code block surrounded by "if" ... "endif" is stmt_list.

Remove the redundant if_block symbol entirely.

Remove "stmt_list: stmt_list end" rule as well since it would
obviously cause conflicts.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This commit decreases 6 shift/reduce conflicts, and finally achieves
conflict-free parser.

Since Kconfig has no terminator for a config block, detecting the end
of config_stmt is not easy.

For example, there are two ways for handling the error in the following
code:

  1 config FOO
  2         =

 [A] Print "unknown option" error, assuming the line 2 is a part of
     config_option_list

 [B] Print "invalid statement", assuming the line 1 is reduced into
     a config_stmt by itself

Bison actually chooses [A] because it performs the shift rather than
the reduction where both are possible.

However, there is no reason to choose one over the other.

Let's remove the option_error, and let it fall back to [B].
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This commit decreases 15 shift/reduce conflicts.

The location of this error recovery is ambiguous.

For example, there are two ways to interpret the following code:

  1 config FOO
  2         bool "foo"

 [A] Both lines are reduced together into a config_stmt.

 [B] The only line 1 is reduced into a config_stmt, and the line 2
     matches to "option_name error T_EOL"

Of course, we expect [A], but [B] could be grammatically possible.

Kconfig has no terminator for a config block. So, we cannot detect its
end until we see a non-property keyword. People often insert a blank
line between two config blocks, but it is just a coding convention.
Blank lines are actually allowed anywhere in Kconfig files.

The real error is when a property keyword appears right after "endif",
"endchoice", "endmenu",  "source", "comment", or variable assignment.

Instead of fixing the grammatical ambiguity, I chose to simply remove
this error recovery.

The difference is

  unexpected option "bool"

... is turned into a more generic message:

  invalid statement
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

It would be nice to warn if a new line is missing at end of file.

We could do this by checkpatch.pl for arbitrary files, but new line
is rather essential as a statement terminator in Kconfig.

The warning message looks like this:

  kernel/Kconfig.preempt:60:warning: no new line at end of file

Currently, kernel/Kconfig.preempt is the only file with no new line
at end of file. Fix it.

I know there are some false negative cases. For example, no warning
is displayed when the last line contains some whitespaces/comments,
but no new line. Yet, this commit works well for most cases.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

A new file should always start in the INITIAL state.

When the lexer bumps into EOF, the lexer must get back to the INITIAL
state anyway. Remove the redundant <<EOF>> pattern in the PARAM state.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This commit decreases 8 shift/reduce conflicts.

A certain amount of grammatical ambiguity comes from how to reduce
excessive T_EOL tokens.

Let's take a look at the example code below:

  1  config A
  2          bool "a"
  3
  4          depends on B
  5
  6  config B
  7          def_bool y

The line 3 is melt into "config_option_list", but the line 5 can be
either a part of "config_option_list" or "common_stmt" by itself.

Currently, the lexer converts '\n' to T_EOL verbatim. In Kconfig,
a new line works as a statement terminator, but new lines in empty
lines are not critical since empty lines (or lines that contain only
whitespaces/comments) are just no-op.

If the lexer simply discards no-op lines, the parser will not be
bothered by excessive T_EOL tokens.

Of course, this means we are shifting the complexity from the parser
to the lexer, but it is much easier than tackling on shift/reduce
conflicts.

I introduced the second stage lexer to tweak the behavior.

Discard T_EOL if the previous token is T_EOL or T_HELPTEXT.
Two T_EOL tokens in a row is meaningless. T_HELPTEXT is a special
token that is reduced without T_EOL.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Here, similar matching patters are duplicated in order to look ahead
the '\n' character. If the next character is '\n', the lexer returns
T_WORD_QUOTE because it must be prepared to return T_EOL at the next
match.

Use unput('\n') trick to reduce the code duplication.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Whitespaces are consumed in the COMMAND state anyway.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

All line-oriented statements should be reduced when seeing a T_EOL
token. I guess missing T_EOL for the "visible" statement is just a
mistake. This commit decreases one shift/reduce conflict.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

An unterminated string literal followed by new line is passed to the
parser (with "multi-line strings not supported" warning shown), then
handled properly there.

On the other hand, an unterminated string literal at end of file is
never passed to the parser, then results in memory leak.

[Test Code]

  ----------(Kconfig begin)----------
  source "Kconfig.inc"

  config A
          bool "a"
  -----------(Kconfig end)-----------

  --------(Kconfig.inc begin)--------
  config B
          bool "b\No new line at end of file
  ---------(Kconfig.inc end)---------

[Summary from Valgrind]

  Before the fix:

    LEAK SUMMARY:
       definitely lost: 16 bytes in 1 blocks
       ...

  After the fix:

    LEAK SUMMARY:
       definitely lost: 0 bytes in 0 blocks
       ...

Eliminate the memory leak path by handling this case. Of course, such
a Kconfig file is wrong already, so I will add an error message later.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, warn_ignore_character() displays invalid file name and
line number.

The lexer should use current_file->name and yylineno, while the parser
should use zconf_curname() and zconf_lineno().

This difference comes from that the lexer is always going ahead
of the parser. The parser needs to look ahead one token to make a
shift/reduce decision, so the lexer is requested to scan more text
from the input file.

This commit fixes the warning message from warn_ignored_character().

[Test Code]

  ----(Kconfig begin)----
  /
  -----(Kconfig end)-----

[Output]

  Before the fix:

  <none>:0:warning: ignoring unsupported character '/'

  After the fix:

  Kconfig:1:warning: ignoring unsupported character '/'
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The only possibility of k_invalid being returned was when
expr_parse_sting() parsed S_OTHER type symbol. This actually never
happened, and this is even clearer since S_OTHER has gone.

Clean up unreachable code.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The S_OTHER type could be set only when conf_read_simple() is reading
include/config/auto.conf file.

For example, CONFIG_FOO=y exists in include/config/auto.conf but it is
missing from the currently parsed Kconfig files, sym_lookup() allocates
a new symbol, and sets its type to S_OTHER.

Strangely, it will be set to S_STRING by conf_set_sym_val() a few lines
below while it is obviously bool or tristate type. On the other hand,
when CONFIG_BAR="bar" is being dropped from include/config/auto.conf,
its type remains S_OTHER. Because for_all_symbols() omits S_OTHER
symbols, conf_touch_deps() misses to touch include/config/bar.h

This behavior has been a pretty mystery for me, and digging the git
histroy did not help. At least, touching depfiles is broken for string
type symbols.

I removed S_OTHER entirely, and reimplemented it more simply.

If CONFIG_FOO was visible in the previous syncconfig, but is missing
now, what we want to do is quite simple; just call conf_touch_dep()
to touch include/config/foo.h instead of allocating a new symbol data.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

conf_touch_deps() iterates over symbols, touching corresponding
include/config/*.h files as needed.

Split the part that touches a single file into a new helper so it can
be reused.

The new helper, conf_touch_dep(), takes a symbol name as a parameter,
and touches the corresponding include/config/*.h file.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

According to commit 2e3646e5 ("kconfig: integrate split config
into silentoldconfig"), this function was named after split-include
tool, which used to exist in old versions of Linux.

Setting aside the historical reason, rename it into a more intuitive
name. This function touches timestamp files under include/config/
in order to interact with the fixdep tool.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The two 'goto setsym' statements are reachable only when sym == NULL.

The code below the 'setsym:' label does nothing when sym == NULL
since there is just one if-block guarded by 'if (sym && ...)'.

Hence, 'goto setsym' can be replaced with 'continue'.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

In today's merge_config.sh the order of the config fragment files dictates
the output of a config option. With this approach we will get different
.config files depending on the order of the config fragment files.

So doing something like:
$ ./merge/kconfig/merge_config.sh selftest.config drm.config

Where selftest.config defines DRM=y and drm.config defines DRM=m, the
result will be "DRM=m".

Rework to add a switch to get builtin '=y' precedence over modules '=m',
this will result in "DRM=y". If we do something like this:

$ ./merge/kconfig/merge_config.sh -y selftest.config drm.config
Suggested-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Pull networking fixes from David Miller:
 "One last pull request before heading to Vancouver for LPC, here we have:

   1) Don't forget to free VSI contexts during ice driver unload, from
      Victor Raj.

   2) Don't forget napi delete calls during device remove in ice driver,
      from Dave Ertman.

   3) Don't request VLAN tag insertion of ibmvnic device when SKB
      doesn't have VLAN tags at all.

   4) IPV4 frag handling code has to accomodate the situation where two
      threads try to insert the same fragment into the hash table at the
      same time. From Eric Dumazet.

   5) Relatedly, don't flow separate on protocol ports for fragmented
      frames, also from Eric Dumazet.

   6) Memory leaks in qed driver, from Denis Bolotin.

   7) Correct valid MTU range in smsc95xx driver, from Stefan Wahren.

   8) Validate cls_flower nested policies properly, from Jakub Kicinski.

   9) Clearing of stats counters in mc88e6xxx driver doesn't retain
      important bits in the G1_STATS_OP register causing the chip to
      hang. Fix from Andrew Lunn"

* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net: (41 commits)
  act_mirred: clear skb->tstamp on redirect
  net: dsa: mv88e6xxx: Fix clearing of stats counters
  tipc: fix link re-establish failure
  net: sched: cls_flower: validate nested enc_opts_policy to avoid warning
  net: mvneta: correct typo
  flow_dissector: do not dissect l4 ports for fragments
  net: qualcomm: rmnet: Fix incorrect assignment of real_dev
  net: aquantia: allow rx checksum offload configuration
  net: aquantia: invalid checksumm offload implementation
  net: aquantia: fixed enable unicast on 32 macvlan
  net: aquantia: fix potential IOMMU fault after driver unbind
  net: aquantia: synchronized flow control between mac/phy
  net: smsc95xx: Fix MTU range
  net: stmmac: Fix RX packet size > 8191
  qed: Fix potential memory corruption
  qed: Fix SPQ entries not returned to pool in error flows
  qed: Fix blocking/unlimited SPQ entries leak
  qed: Fix memory/entry leak in qed_init_sp_request()
  inet: frags: better deal with smp races
  net: hns3: bugfix for not checking return value
  ...

Merge tag 'kbuild-fixes-v4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild fixes from Masahiro Yamada:

 - fix build errors in binrpm-pkg and bindeb-pkg targets

 - fix false positive matches in merge_config.sh

 - fix build version mismatch in deb-pkg target

 - fix dtbs_install handling in (bin)deb-pkg target

 - revert a commit that allows setlocalversion to write to source tree

* tag 'kbuild-fixes-v4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
  builddeb: Fix inclusion of dtbs in debian package
  Revert "scripts/setlocalversion: git: Make -dirty check more robust"
  kbuild: deb-pkg: fix too low build version number
  kconfig: merge_config: avoid false positive matches from comment lines
  kbuild: deb-pkg: fix bindeb-pkg breakage when O= is used
  kbuild: rpm-pkg: fix binrpm-pkg breakage when O= is used

Pull btrfs fixes from David Sterba:
 "Several fixes to recent release (4.19, fixes tagged for stable) and
  other fixes"

* tag 'for-4.20-rc1-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux:
  Btrfs: fix missing delayed iputs on unmount
  Btrfs: fix data corruption due to cloning of eof block
  Btrfs: fix infinite loop on inode eviction after deduplication of eof block
  Btrfs: fix deadlock on tree root leaf when finding free extent
  btrfs: avoid link error with CONFIG_NO_AUTO_INLINE
  btrfs: tree-checker: Fix misleading group system information
  Btrfs: fix missing data checksums after a ranged fsync (msync)
  btrfs: fix pinned underflow after transaction aborted
  Btrfs: fix cur_offset in the error case for nocow

Pull ext4 fixes from Ted Ts'o:
 "A large number of ext4 bug fixes, mostly buffer and memory leaks on
  error return cleanup paths"

* tag 'ext4_for_linus_stable' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4:
  ext4: missing !bh check in ext4_xattr_inode_write()
  ext4: fix buffer leak in __ext4_read_dirblock() on error path
  ext4: fix buffer leak in ext4_expand_extra_isize_ea() on error path
  ext4: fix buffer leak in ext4_xattr_move_to_block() on error path
  ext4: release bs.bh before re-using in ext4_xattr_block_find()
  ext4: fix buffer leak in ext4_xattr_get_block() on error path
  ext4: fix possible leak of s_journal_flag_rwsem in error path
  ext4: fix possible leak of sbi->s_group_desc_leak in error path
  ext4: remove unneeded brelse call in ext4_xattr_inode_update_ref()
  ext4: avoid possible double brelse() in add_new_gdb() on error path
  ext4: avoid buffer leak in ext4_orphan_add() after prior errors
  ext4: avoid buffer leak on shutdown in ext4_mark_iloc_dirty()
  ext4: fix possible inode leak in the retry loop of ext4_resize_fs()
  ext4: fix missing cleanup if ext4_alloc_flex_bg_array() fails while resizing
  ext4: add missing brelse() update_backups()'s error path
  ext4: add missing brelse() add_new_gdb_meta_bg()'s error path
  ext4: add missing brelse() in set_flexbg_block_bitmap()'s error path
  ext4: avoid potential extra brelse in setup_new_flex_group_blocks()

Pull x86 fixes from Thomas Gleixner:
 "A set of x86 fixes:

   - Cure the LDT remapping to user space on 5 level paging which ended
     up in the KASLR space

   - Remove LDT mapping before freeing the LDT pages

   - Make NFIT MCE handling more robust

   - Unbreak the VSMP build by removing the dependency on paravirt ops

   - Support broken PIT emulation on Microsoft hyperV

   - Don't trace vmware_sched_clock() to avoid tracer recursion

   - Remove -pipe from KBUILD CFLAGS which breaks clang and is also
     slower on GCC

   - Trivial coding style and typo fixes"

* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/cpu/vmware: Do not trace vmware_sched_clock()
  x86/vsmp: Remove dependency on pv_irq_ops
  x86/ldt: Remove unused variable in map_ldt_struct()
  x86/ldt: Unmap PTEs for the slot before freeing LDT pages
  x86/mm: Move LDT remap out of KASLR region on 5-level paging
  acpi/nfit, x86/mce: Validate a MCE's address before using it
  acpi/nfit, x86/mce: Handle only uncorrectable machine checks
  x86/build: Remove -pipe from KBUILD_CFLAGS
  x86/hyper-v: Fix indentation in hv_do_fast_hypercall16()
  Documentation/x86: Fix typo in zero-page.txt
  x86/hyper-v: Enable PIT shutdown quirk
  clockevents/drivers/i8253: Add support for PIT shutdown quirk