Since __HAVE_ARCH_* style guards have been depricated in favour of
defining the function name onto itself, convert pxxp_get().
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/Y2EUEBlQXNgaJgoI@hirez.programming.kicks-ass.net

pmdp_get_lockless() should itself imply any ordering required.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114425.298833095%40infradead.org

Recognise that set_64bit() is a special case of our previously
introduced pxx_xchg64(), so use that and get rid of set_64bit().
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114425.233481884%40infradead.org

The use of set_64bit() in X86_64 only code is pretty pointless, seeing
how it's a direct assignment. Remove all this nonsense.

[nathanchance: unbreak irte]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114425.168036718%40infradead.org

Given that ptep_get_and_clear() uses cmpxchg8b, and that should be by
far the most common case, there's no point in having an optimized
variant for pmd/pud.

Introduce the pxx_xchg64() helper to implement the common logic once.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114425.103392961%40infradead.org

Disallow write-tearing, that would be really unfortunate.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114425.038102604%40infradead.org

PAE implies CX8, write readable code.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114424.971450128%40infradead.org

On architectures where the PTE/PMD is larger than the native word size
(i386-PAE for example), READ_ONCE() can do the wrong thing. Use
pmdp_get_lockless() just like we use ptep_get_lockless().
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114424.906110403%40infradead.org

There's no point in having the identical routines for PTE/PMD have
different names.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114424.841277397%40infradead.org

Since it no longer applies to only PTEs, rename it to PXX.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114424.776404066%40infradead.org

AFAICT there's no reason to do anything different than what we do for
PTEs. Make it so (also affects SH).
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114424.711181252%40infradead.org

Just like 64bit pte_t, have a low/high split in pmd_t.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114424.645657294%40infradead.org

Instead of mucking about with at least 2 different ways of fudging
it, do the same thing we do for pte_t.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114424.580310787%40infradead.org

Improve the comment.
Suggested-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221022114424.515572025%40infradead.org

Provide a native implementation of set_memory_rox(), avoiding the
double set_memory_ro();set_memory_x(); calls.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>

Because endlessly repeating:

	set_memory_ro()
	set_memory_x()

is getting tedious.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/Y1jek64pXOsougmz@hirez.programming.kicks-ass.net

Straight up revert of commit:

  a970174d ("x86/mm: Do not verify W^X at boot up")

now that the root cause has been fixed.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221025201058.011279208@infradead.org

Now that text_poke is available before ftrace, remove the
SYSTEM_BOOTING exceptions.

Specifically, this cures a W+X case during boot.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221025201057.945960823@infradead.org

Move poking_init() up a bunch; specifically move it right after
mm_init() which is right before ftrace_init().

This will allow simplifying ftrace text poking which currently has
a bunch of exceptions for early boot.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221025201057.881703081@infradead.org

Instead of duplicating init_mm, allocate a fresh mm. The advantage is
that mm_alloc() has much simpler dependencies. Additionally it makes
more conceptual sense, init_mm has no (and must not have) user state
to duplicate.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221025201057.816175235@infradead.org

In order to allow using mm_alloc() much earlier, move initializing
mm_cachep into mm_init().
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20221025201057.751153381@infradead.org

Seth found that the CPU-entry-area; the piece of per-cpu data that is
mapped into the userspace page-tables for kPTI is not subject to any
randomization -- irrespective of kASLR settings.

On x86_64 a whole P4D (512 GB) of virtual address space is reserved for
this structure, which is plenty large enough to randomize things a
little.

As such, use a straight forward randomization scheme that avoids
duplicates to spread the existing CPUs over the available space.

  [ bp: Fix le build. ]
Reported-by: Seth Jenkins <sethjenkins@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>

KASAN maps shadow for the entire CPU-entry-area:
  [CPU_ENTRY_AREA_BASE, CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE]

This will explode once the per-cpu entry areas are randomized since it
will increase CPU_ENTRY_AREA_MAP_SIZE to 512 GB and KASAN fails to
allocate shadow for such big area.

Fix this by allocating KASAN shadow only for really used cpu entry area
addresses mapped by cea_map_percpu_pages()

Thanks to the 0day folks for finding and reporting this to be an issue.

[ dhansen: tweak changelog since this will get committed before peterz's
	   actual cpu-entry-area randomization ]
Signed-off-by: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Tested-by: Yujie Liu <yujie.liu@intel.com>
Cc: kernel test robot <yujie.liu@intel.com>
Link: https://lore.kernel.org/r/202210241508.2e203c3d-yujie.liu@intel.com

Pull fbdev fixes from Helge Deller:
 "A use-after-free bugfix in the smscufx driver and various minor error
  path fixes, smaller build fixes, sysfs fixes and typos in comments in
  the stifb, sisfb, da8xxfb, xilinxfb, sm501fb, gbefb and cyber2000fb
  drivers"

* tag 'fbdev-for-6.1-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev:
  fbdev: cyber2000fb: fix missing pci_disable_device()
  fbdev: sisfb: use explicitly signed char
  fbdev: smscufx: Fix several use-after-free bugs
  fbdev: xilinxfb: Make xilinxfb_release() return void
  fbdev: sisfb: fix repeated word in comment
  fbdev: gbefb: Convert sysfs snprintf to sysfs_emit
  fbdev: sm501fb: Convert sysfs snprintf to sysfs_emit
  fbdev: stifb: Fall back to cfb_fillrect() on 32-bit HCRX cards
  fbdev: da8xx-fb: Fix error handling in .remove()
  fbdev: MIPS supports iomem addresses

Pull char/misc fixes from Greg KH:
 "Some small driver fixes for 6.1-rc3.  They include:

   - iio driver bugfixes

   - counter driver bugfixes

   - coresight bugfixes, including a revert and then a second fix to get
     it right.

  All of these have been in linux-next with no reported problems"

* tag 'char-misc-6.1-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc: (21 commits)
  misc: sgi-gru: use explicitly signed char
  coresight: cti: Fix hang in cti_disable_hw()
  Revert "coresight: cti: Fix hang in cti_disable_hw()"
  counter: 104-quad-8: Fix race getting function mode and direction
  counter: microchip-tcb-capture: Handle Signal1 read and Synapse
  coresight: cti: Fix hang in cti_disable_hw()
  coresight: Fix possible deadlock with lock dependency
  counter: ti-ecap-capture: fix IS_ERR() vs NULL check
  counter: Reduce DEFINE_COUNTER_ARRAY_POLARITY() to defining counter_array
  iio: bmc150-accel-core: Fix unsafe buffer attributes
  iio: adxl367: Fix unsafe buffer attributes
  iio: adxl372: Fix unsafe buffer attributes
  iio: at91-sama5d2_adc: Fix unsafe buffer attributes
  iio: temperature: ltc2983: allocate iio channels once
  tools: iio: iio_utils: fix digit calculation
  iio: adc: stm32-adc: fix channel sampling time init
  iio: adc: mcp3911: mask out device ID in debug prints
  iio: adc: mcp3911: use correct id bits
  iio: adc: mcp3911: return proper error code on failure to allocate trigger
  iio: adc: mcp3911: fix sizeof() vs ARRAY_SIZE() bug
  ...

Pull USB fixes from Greg KH:
 "A few small USB fixes for 6.1-rc3. Include in here are:

   - MAINTAINERS update, including a big one for the USB gadget
     subsystem. Many thanks to Felipe for all of the years of hard work
     he has done on this codebase, it was greatly appreciated.

   - dwc3 driver fixes for reported problems.

   - xhci driver fixes for reported problems.

   - typec driver fixes for minor issues

   - uvc gadget driver change, and then revert as it wasn't relevant for
     6.1-final, as it is a new feature and people are still reviewing
     and modifying it.

  All of these have been in the linux-next tree with no reported issues"

* tag 'usb-6.1-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb:
  usb: dwc3: gadget: Don't set IMI for no_interrupt
  usb: dwc3: gadget: Stop processing more requests on IMI
  Revert "usb: gadget: uvc: limit isoc_sg to super speed gadgets"
  xhci: Remove device endpoints from bandwidth list when freeing the device
  xhci-pci: Set runtime PM as default policy on all xHC 1.2 or later devices
  xhci: Add quirk to reset host back to default state at shutdown
  usb: xhci: add XHCI_SPURIOUS_SUCCESS to ASM1042 despite being a V0.96 controller
  usb: dwc3: st: Rely on child's compatible instead of name
  usb: gadget: uvc: limit isoc_sg to super speed gadgets
  usb: bdc: change state when port disconnected
  usb: typec: ucsi: acpi: Implement resume callback
  usb: typec: ucsi: Check the connection on resume
  usb: gadget: aspeed: Fix probe regression
  usb: gadget: uvc: fix sg handling during video encode
  usb: gadget: uvc: fix sg handling in error case
  usb: gadget: uvc: fix dropped frame after missed isoc
  usb: dwc3: gadget: Don't delay End Transfer on delayed_status
  usb: dwc3: Don't switch OTG -> peripheral if extcon is present
  MAINTAINERS: Update maintainers for broadcom USB
  MAINTAINERS: move USB gadget and phy entries under the main USB entry

Pull gpio fixes from Bartosz Golaszewski:

 - convert gpio-tegra to using an immutable irqchip

 - MAINTAINERS update

* tag 'gpio-fixes-for-v6.1-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/brgl/linux:
  MAINTAINERS: Change myself to a maintainer
  gpio: tegra: Convert to immutable irq chip

Pull perf fixes from Borislav Petkov:

 - Rename a perf memory level event define to denote it is of CXL type

 - Add Alder and Raptor Lakes support to RAPL

 - Make sure raw sample data is output with tracepoints

* tag 'perf_urgent_for_v6.1_rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  perf/mem: Rename PERF_MEM_LVLNUM_EXTN_MEM to PERF_MEM_LVLNUM_CXL
  perf/x86/rapl: Add support for Intel Raptor Lake
  perf/x86/rapl: Add support for Intel AlderLake-N
  perf: Fix missing raw data on tracepoint events

Merge tag 'loongarch-fixes-6.1-1' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson

Pull LoongArch fixes from Huacai Chen:
 "Remove unused kernel stack padding, fix some build errors/warnings and
  two bugs in laptop platform driver"

* tag 'loongarch-fixes-6.1-1' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson:
  platform/loongarch: laptop: Fix possible UAF and simplify generic_acpi_laptop_init()
  platform/loongarch: laptop: Adjust resume order for loongson_hotkey_resume()
  LoongArch: BPF: Avoid declare variables in switch-case
  LoongArch: Use flexible-array member instead of zero-length array
  LoongArch: Remove unused kernel stack padding

Pull cifs fixes from Steve French:

 - use after free fix for reconnect race

 - two memory leak fixes

* tag '6.1-rc2-smb3-fixes' of git://git.samba.org/sfrench/cifs-2.6:
  cifs: fix use-after-free caused by invalid pointer `hostname`
  cifs: Fix pages leak when writedata alloc failed in cifs_write_from_iter()
  cifs: Fix pages array leak when writedata alloc failed in cifs_writedata_alloc()

Pull random number generator fix from Jason Donenfeld:
 "One fix from Jean-Philippe Brucker, addressing a regression in which
  early boot code on ARM64 would use the non-_early variant of the
  arch_get_random family of functions, resulting in the architectural
  random number generator appearing unavailable during that early phase
  of boot.

  The fix simply changes arch_get_random*() to arch_get_random*_early().

  This distinction between these two functions is a bit of an old wart
  I'm not a fan of, and for 6.2 I'll see if I can make obsolete the
  _early variant, so that one function does the right thing in all
  contexts without overhead"

* tag 'random-6.1-rc3-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/crng/random:
  random: use arch_get_random*_early() in random_init()

Pull SCSI fixes from James Bottomley:
 "Varions small  fixes, all  in drivers.

  Some of these arrived during the merge window and got held over to
  make sure of testing on the -rc tree.

  The biggest change is for standards conformance in the target driver,
  closely followed by a set of bug fixes in megaraid_sas"

* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (21 commits)
  scsi: ufs: core: Fix typo in comment
  scsi: mpi3mr: Select CONFIG_SCSI_SAS_ATTRS
  scsi: ufs: core: Fix typo for register name in comments
  scsi: pm80xx: Display proc_name in sysfs
  scsi: ufs: core: Fix the error log in ufshcd_query_flag_retry()
  scsi: ufs: core: Remove unneeded casts from void *
  scsi: lpfc: Fix spelling mistake "unsolicted" -> "unsolicited"
  scsi: qla2xxx: Use transport-defined speed mask for supported_speeds
  scsi: target: iblock: Fold iblock_emulate_read_cap_with_block_size() into iblock_get_blocks()
  scsi: qla2xxx: Fix serialization of DCBX TLV data request
  scsi: ufs: qcom: Remove redundant dev_err() call
  scsi: megaraid_sas: Move megasas_dbg_lvl init to megasas_init()
  scsi: megaraid_sas: Remove unnecessary memset()
  scsi: megaraid_sas: Simplify megasas_update_device_list
  scsi: megaraid_sas: Correct an error message
  scsi: megaraid_sas: Correct value passed to scsi_device_lookup()
  scsi: target: core: UA on all LUNs after reset
  scsi: target: core: New key must be used for moved PR
  scsi: target: core: Abort all preempted regs if requested
  scsi: target: core: Fix memory leak in preempt_and_abort
  ...

Pull block fixes from Jens Axboe:

 - NVMe pull request via Christoph:
      - make the multipath dma alignment match the non-multipath one
        (Keith Busch)
      - fix a bogus use of sg_init_marker() (Nam Cao)
      - fix circulr locking in nvme-tcp (Sagi Grimberg)

 - Initialization fix for requests allocated via the special hw queue
   allocator (John)

 - Fix for a regression added in this release with the batched
   completions of end_io backed requests (Ming)

 - Error handling leak fix for rbd (Yang)

 - Error handling leak fix for add_disk() failure (Yu)

* tag 'block-6.1-2022-10-28' of git://git.kernel.dk/linux:
  blk-mq: Properly init requests from blk_mq_alloc_request_hctx()
  blk-mq: don't add non-pt request with ->end_io to batch
  rbd: fix possible memory leak in rbd_sysfs_init()
  nvme-multipath: set queue dma alignment to 3
  nvme-tcp: fix possible circular locking when deleting a controller under memory pressure
  nvme-tcp: replace sg_init_marker() with sg_init_table()
  block: fix memory leak for elevator on add_disk failure

Pull io_uring fix from Jens Axboe:
 "Just a fix for a locking regression introduced with the deferred
  task_work running from this merge window"

* tag 'io_uring-6.1-2022-10-28' of git://git.kernel.dk/linux:
  io_uring: unlock if __io_run_local_work locked inside
  io_uring: use io_run_local_work_locked helper

Pull misc hotfixes from Andrew Morton:
 "Eight fix pre-6.0 bugs and the remainder address issues which were
  introduced in the 6.1-rc merge cycle, or address issues which aren't
  considered sufficiently serious to warrant a -stable backport"

* tag 'mm-hotfixes-stable-2022-10-28' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (23 commits)
  mm: multi-gen LRU: move lru_gen_add_mm() out of IRQ-off region
  lib: maple_tree: remove unneeded initialization in mtree_range_walk()
  mmap: fix remap_file_pages() regression
  mm/shmem: ensure proper fallback if page faults
  mm/userfaultfd: replace kmap/kmap_atomic() with kmap_local_page()
  x86: fortify: kmsan: fix KMSAN fortify builds
  x86: asm: make sure __put_user_size() evaluates pointer once
  Kconfig.debug: disable CONFIG_FRAME_WARN for KMSAN by default
  x86/purgatory: disable KMSAN instrumentation
  mm: kmsan: export kmsan_copy_page_meta()
  mm: migrate: fix return value if all subpages of THPs are migrated successfully
  mm/uffd: fix vma check on userfault for wp
  mm: prep_compound_tail() clear page->private
  mm,madvise,hugetlb: fix unexpected data loss with MADV_DONTNEED on hugetlbfs
  mm/page_isolation: fix clang deadcode warning
  fs/ext4/super.c: remove unused `deprecated_msg'
  ipc/msg.c: fix percpu_counter use after free
  memory tier, sysfs: rename attribute "nodes" to "nodelist"
  MAINTAINERS: git://github.com -> https://github.com for nilfs2
  mm/kmemleak: prevent soft lockup in kmemleak_scan()'s object iteration loops
  ...

Pull powerpc fixes from Michael Ellerman:

 - Fix a case of rescheduling with user access unlocked, when preempt is
   enabled.

 - A follow-up fix for a recent fix, which could lead to IRQ state
   assertions firing incorrectly.

 - Two fixes for lockdep warnings seen when using kfence with the Hash
   MMU.

 - Two fixes for preempt warnings seen when using the Hash MMU.

 - Two fixes for the VAS coprocessor mechanism used on pseries.

 - Prevent building some of our older KVM backends when
   CONTEXT_TRACKING_USER is enabled, as it's known to cause crashes.

 - A couple of fixes for issues seen with PMU NMIs.

Thanks to Nicholas Piggin, Guenter Roeck, Frederic Barrat Haren Myneni,
Sachin Sant, and Samuel Holland.

* tag 'powerpc-6.1-3' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux:
  powerpc/64s/interrupt: Fix clear of PACA_IRQS_HARD_DIS when returning to soft-masked context
  powerpc/64s/interrupt: Perf NMI should not take normal exit path
  powerpc/64/interrupt: Prevent NMI PMI causing a dangerous warning
  KVM: PPC: BookS PR-KVM and BookE do not support context tracking
  powerpc: Fix reschedule bug in KUAP-unlocked user copy
  powerpc/64s: Fix hash__change_memory_range preemption warning
  powerpc/64s: Disable preemption in hash lazy mmu mode
  powerpc/64s: make linear_map_hash_lock a raw spinlock
  powerpc/64s: make HPTE lock and native_tlbie_lock irq-safe
  powerpc/64s: Add lockdep for HPTE lock
  powerpc/pseries: Use lparcfg to reconfig VAS windows for DLPAR CPU
  powerpc/pseries/vas: Add VAS IRQ primary handler

Currently the return value of 'sub_driver->init' is not checked. If
sparse_keymap_setup() called in the init function fails, 'generic_
inputdev' is freed, then it will lead a UAF when using it in generic_
acpi_laptop_init(). Fix it by checking the return value and setting
generic_inputdev to NULL after free, so as to avoid double free it.

The error code in generic_subdriver_init() is always negative, so the
return of generic_subdriver_init() can be simplified.

Fixes: 6246ed09 ("LoongArch: Add ACPI-based generic laptop driver")
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

Some laptops don't support SW_LID, but still have backlight control,
move backlight resuming before SW_LID event handling so as to avoid
backlight mistake due to early return.
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

Not all compilers support declare variables in switch-case, so move
declarations to the beginning of a function. Otherwise we may get such
build errors:

arch/loongarch/net/bpf_jit.c: In function ‘emit_atomic’:
arch/loongarch/net/bpf_jit.c:362:3: error: a label can only be part of a statement and a declaration is not a statement
   u8 r0 = regmap[BPF_REG_0];
   ^~
arch/loongarch/net/bpf_jit.c: In function ‘build_insn’:
arch/loongarch/net/bpf_jit.c:727:3: error: a label can only be part of a statement and a declaration is not a statement
   u8 t7 = -1;
   ^~
arch/loongarch/net/bpf_jit.c:778:3: error: a label can only be part of a statement and a declaration is not a statement
   int ret;
   ^~~
arch/loongarch/net/bpf_jit.c:779:3: error: expected expression before ‘u64’
   u64 func_addr;
   ^~~
arch/loongarch/net/bpf_jit.c:780:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
   bool func_addr_fixed;
   ^~~~
arch/loongarch/net/bpf_jit.c:784:11: error: ‘func_addr’ undeclared (first use in this function); did you mean ‘in_addr’?
          &func_addr, &func_addr_fixed);
           ^~~~~~~~~
           in_addr
arch/loongarch/net/bpf_jit.c:784:11: note: each undeclared identifier is reported only once for each function it appears in
arch/loongarch/net/bpf_jit.c:814:3: error: a label can only be part of a statement and a declaration is not a statement
   u64 imm64 = (u64)(insn + 1)->imm << 32 | (u32)insn->imm;
   ^~~
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>