- 02 Feb, 2023 1 commit
-
-
Palmer Dabbelt authored
Jisheng Zhang <jszhang@kernel.org> says: Generally, riscv ISA extensions are fixed for any specific hardware platform, so a hart's features won't change after booting, this chacteristic makes it straightforward to use a static branch to check a specific ISA extension is supported or not to optimize performance. However, some ISA extensions such as SVPBMT and ZICBOM are handled via. the alternative sequences. Basically, for ease of maintenance, we prefer to use static branches in C code, but recently, Samuel found that the static branch usage in cpu_relax() breaks building with CONFIG_CC_OPTIMIZE_FOR_SIZE[1]. As Samuel pointed out, "Having a static branch in cpu_relax() is problematic because that function is widely inlined, including in some quite complex functions like in the VDSO. A quick measurement shows this static branch is responsible by itself for around 40% of the jump table." Samuel's findings pointed out one of a few downsides of static branches usage in C code to handle ISA extensions detected at boot time: static branch's metadata in the __jump_table section, which is not discarded after ISA extensions are finalized, wastes some space. I want to try to solve the issue for all possible dynamic handling of ISA extensions at boot time. Inspired by Mark[2], this patch introduces riscv_has_extension_*() helpers, which work like static branches but are patched using alternatives, thus the metadata can be freed after patching. [1]https://lore.kernel.org/linux-riscv/20220922060958.44203-1-samuel@sholland.org/ [2]https://lore.kernel.org/linux-arm-kernel/20220912162210.3626215-8-mark.rutland@arm.com/ [3]https://lore.kernel.org/linux-riscv/20221130225614.1594256-1-heiko@sntech.de/ * b4-shazam-merge: riscv: remove riscv_isa_ext_keys[] array and related usage riscv: KVM: Switch has_svinval() to riscv_has_extension_unlikely() riscv: cpu_relax: switch to riscv_has_extension_likely() riscv: alternative: patch alternatives in the vDSO riscv: switch to relative alternative entries riscv: module: Add ADD16 and SUB16 rela types riscv: module: move find_section to module.h riscv: fpu: switch has_fpu() to riscv_has_extension_likely() riscv: introduce riscv_has_extension_[un]likely() riscv: cpufeature: extend riscv_cpufeature_patch_func to all ISA extensions riscv: hwcap: make ISA extension ids can be used in asm riscv: cpufeature: detect RISCV_ALTERNATIVES_EARLY_BOOT earlier riscv: move riscv_noncoherent_supported() out of ZICBOM probe Link: https://lore.kernel.org/r/20230128172856.3814-1-jszhang@kernel.orgSigned-off-by:
Palmer Dabbelt <palmer@rivosinc.com>
-
- 01 Feb, 2023 13 commits
-
-
Jisheng Zhang authored
All users have switched to riscv_has_extension_*, remove unused definitions, vars and related setting code. Signed-off-by:
Jisheng Zhang <jszhang@kernel.org> Reviewed-by:
Andrew Jones <ajones@ventanamicro.com> Reviewed-by:
Heiko Stuebner <heiko@sntech.de> Reviewed-by:
Conor Dooley <conor.dooley@microchip.com> Reviewed-by:
Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230128172856.3814-14-jszhang@kernel.orgSigned-off-by:
-
Andrew Jones authored
Switch has_svinval() from static branch to the new helper riscv_has_extension_unlikely(). Signed-off-by:
Anup Patel <anup@brainfault.org> Link: https://lore.kernel.org/r/20230128172856.3814-13-jszhang@kernel.orgSigned-off-by:
-
Jisheng Zhang authored
Switch cpu_relax() from static branch to the new helper riscv_has_extension_likely() Signed-off-by:
-
Jisheng Zhang authored
Make it possible to use alternatives in the vDSO, so that better implementations can be used if possible. Signed-off-by:
-
Jisheng Zhang authored
Instead of using absolute addresses for both the old instrucions and the alternative instructions, use offsets relative to the alt_entry values. So this not only cuts the size of the alternative entry, but also meets the prerequisite for patching alternatives in the vDSO, since absolute alternative entries are subject to dynamic relocation, which is incompatible with the vDSO building. Signed-off-by:
-
Andrew Jones authored
To prepare for 16-bit relocation types to be emitted in alternatives add support for ADD16 and SUB16. Signed-off-by:
-
Jisheng Zhang authored
Move find_section() to module.h so that the implementation can be shared by the alternatives code. This will allow us to use alternatives in the vdso. Reviewed-by:
-
Jisheng Zhang authored
Switch has_fpu() from static branch to the new helper riscv_has_extension_likely(). Signed-off-by:
-
Jisheng Zhang authored
Generally, riscv ISA extensions are fixed for any specific hardware platform, so a hart's features won't change after booting. This chacteristic makes it straightforward to use a static branch to check if a specific ISA extension is supported or not to optimize performance. However, some ISA extensions such as SVPBMT and ZICBOM are handled via. the alternative sequences. Basically, for ease of maintenance, we prefer to use static branches in C code, but recently, Samuel found that the static branch usage in cpu_relax() breaks building with CONFIG_CC_OPTIMIZE_FOR_SIZE[1]. As Samuel pointed out, "Having a static branch in cpu_relax() is problematic because that function is widely inlined, including in some quite complex functions like in the VDSO. A quick measurement shows this static branch is responsible by itself for around 40% of the jump table." Samuel's findings pointed out one of a few downsides of static branches usage in C code to handle ISA extensions detected at boot time: static branch's metadata in the __jump_table section, which is not discarded after ISA extensions are finalized, wastes some space. I want to try to solve the issue for all possible dynamic handling of ISA extensions at boot time. Inspired by Mark[2], this patch introduces riscv_has_extension_*() helpers, which work like static branches but are patched using alternatives, thus the metadata can be freed after patching. Link: https://lore.kernel.org/linux-riscv/20220922060958.44203-1-samuel@sholland.org/ [1] Link: https://lore.kernel.org/linux-arm-kernel/20220912162210.3626215-8-mark.rutland@arm.com/ [2] Signed-off-by:
-
Jisheng Zhang authored
riscv_cpufeature_patch_func() currently only scans a limited set of cpufeatures, explicitly defined with macros. Extend it to probe for all ISA extensions. Signed-off-by:
-
Jisheng Zhang authored
So that ISA extensions can be used in assembly files, convert the multi-letter RISC-V ISA extension IDs enums to macros. In order to make them visible, move the #ifndef __ASSEMBLY__ guard to a later point in the header Signed-off-by:
-
Jisheng Zhang authored
Currently riscv_cpufeature_patch_func() does nothing at the RISCV_ALTERNATIVES_EARLY_BOOT stage. Add a check to detect whether we are in this stage and exit early. This will allow us to use riscv_cpufeature_patch_func() for scanning of all ISA extensions. Signed-off-by:
-
Jisheng Zhang authored
It's a bit weird to call riscv_noncoherent_supported() each time when insmoding a module. Move the calling out of feature patch func. Signed-off-by:
-
- 31 Jan, 2023 3 commits
-
-
Palmer Dabbelt authored
Heiko Stuebner <heiko@sntech.de> says: From: Heiko Stuebner <heiko.stuebner@vrull.eu> This series still tries to allow optimized string functions for specific extensions. The last approach of using an inline base function to hold the alternative calls did cause some issues in a number of places So instead of that we're now just using an alternative j at the beginning of the generic function to jump to a separate place inside the function itself. * b4-shazam-merge: RISC-V: add zbb support to string functions RISC-V: add infrastructure to allow different str* implementations Link: https://lore.kernel.org/r/20230113212301.3534711-1-heiko@sntech.deSigned-off-by:
-
Heiko Stuebner authored
Add handling for ZBB extension and add support for using it as a variant for optimized string functions. Support for the Zbb-str-variants is limited to the GNU-assembler for now, as LLVM has not yet acquired the functionality to selectively change the arch option in assembler code. This is still under review at https://reviews.llvm.org/D123515Co-developed-by:Christoph Muellner <christoph.muellner@vrull.eu> Signed-off-by:
Heiko Stuebner <heiko.stuebner@vrull.eu> Reviewed-by:
-
Heiko Stuebner authored
Depending on supported extensions on specific RISC-V cores, optimized str* functions might make sense. This adds basic infrastructure to allow patching the function calls via alternatives later on. The Linux kernel provides standard implementations for string functions but when architectures want to extend them, they need to provide their own. The added generic string functions are done in assembler (taken from disassembling the main-kernel functions for now) to allow us to control the used registers and extend them with optimized variants. This doesn't override the compiler's use of builtin replacements. So still first of all the compiler will select if a builtin will be better suitable i.e. for known strings. For all regular cases we will want to later select possible optimized variants and in the worst case fall back to the generic implemention added with this change. Reviewed-by:
-
- 25 Jan, 2023 4 commits
-
-
Heiko Stuebner authored
Commit b0f4c74e ("RISC-V: Fix unannoted hardirqs-on in return to userspace slow-path") renamed the do_notify_resume function to do_work_pending but did not change the prototype in signal.h Do that now, as the original function does not exist anymore. Fixes: b0f4c74e ("RISC-V: Fix unannoted hardirqs-on in return to userspace slow-path") Signed-off-by:
-
Jisheng Zhang authored
Alternatives live in a different section, so offsets used by jal instruction will point to wrong locations after the patch got applied. Similar to arm64, adjust the location to consider that offset. Co-developed-by:
-
Geert Uytterhoeven authored
Remove trailing whitespace that hurts my eyes. Signed-off-by:
Geert Uytterhoeven <geert+renesas@glider.be> Acked-by:
-
Guo Ren authored
KERN_VIRT_SIZE is 1/4 of the entries of the page global directory, not half. Fixes: f7ae0233 ("riscv: Move KASAN mapping next to the kernel mapping") Signed-off-by:
Guo Ren <guoren@linux.alibaba.com> Signed-off-by:
Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20230110080419.931185-1-guoren@kernel.orgSigned-off-by:
-
- 20 Jan, 2023 1 commit
-
-
Palmer Dabbelt authored
This cleans up the ISA string handling to more closely match a version of the ISA spec. This is visible in /proc/cpuinfo and the ordering changes may break something in userspace, but these orderings have changed before without issues so with any luck that's still the case. This also adds documentation so userspace has a better idea of what is intended when it comes to compatibility for /proc/cpuinfo, which should help everyone as this will likely keep changing. * b4-shazam-merge: Documentation: riscv: add a section about ISA string ordering in /proc/cpuinfo RISC-V: resort all extensions in consistent orders RISC-V: clarify ISA string ordering rules in cpu.c Link: https://lore.kernel.org/r/20221205144525.2148448-1-conor.dooley@microchip.comSigned-off-by:
-
- 18 Jan, 2023 5 commits
-
-
Conor Dooley authored
The RISC-V specs are permissive in what they allow as the ISA string, but how we output this to userspace in /proc/cpuinfo is quasi uABI. Formalise this as part of the uABI, by documenting the list of rules we use at this point in time. Signed-off-by:
-
Conor Dooley authored
Ordering between each and every list of extensions is wildly inconsistent. Per discussion on the lists pick the following policy: - The array defining order in /proc/cpuinfo follows a narrow interpretation of the ISA specifications, described in a comment immediately presiding it. - All other lists of extensions are sorted alphabetically. This will hopefully allow for easier review & future additions, and reduce conflicts between patchsets as the number of extensions grows. Link: https://lore.kernel.org/all/20221129144742.2935581-2-conor.dooley@microchip.com/Suggested-by:
-
Conor Dooley authored
While the current list of rules may have been accurate when created it now lacks some clarity in the face of isa-manual updates. Instead of trying to continuously align this rule-set with the one in the specifications, change the role of this comment. This particular comment is important, as the array it "decorates" defines the order in which the ISA string appears to userspace in /proc/cpuinfo. Re-jig and strengthen the wording to provide contributors with a set order in which to add entries & note why this particular struct needs more attention than others. While in the area, add some whitespace and tweak some wording for readability's sake. Suggested-by:
-
Palmer Dabbelt authored
This fixes an issue with the K210 Kconfig cleanups, I'm just merging it from that to avoid crossing the streams. * b4-shazam-merge: RISC-V: fix incorrect type of ARCH_CANAAN_K210_DTB_SOURCE Link: https://lore.kernel.org/r/20230111104848.2088516-1-conor.dooley@microchip.comSigned-off-by:
-
Conor Dooley authored
When adding the ARCH_ symbol for the builtin K210 dtb, I erroneously used the bool type for something that is quite obviously a string. Unfortunately, there is no such thing as "def_string", but in this case we can use "default" to propagate the value of SOC_CANAAN_K210_DTB_SOURCE to ARCH_CANAAN_K210_DTB_SOURCE. ARCH_CANAAN_K210_DTB_SOURCE is not user selectable, so using olddefconfig etc will update ARCH_CANAAN_K210_DTB_SOURCE to reflect any changes made to SOC_CANAAN_K210_DTB_SOURCE. Fixes: fc432119 ("RISC-V: kconfig.socs: convert usage of SOC_CANAAN to ARCH_CANAAN") Reported-by:
-
- 29 Dec, 2022 13 commits
-
-
Palmer Dabbelt authored
Heiko Stuebner <heiko@sntech.de> says: From: Heiko Stuebner <heiko.stuebner@vrull.eu> This series is split out of my work on optimizing string functions and provides the basics to: - actually allowing calls in alternatives Function calls use auipc + jalr to reach those 32bit relative addresses but when they're compiled the offset will be wrong as alternatives live in a different section. So when the patch gets applied the address will point to the wrong location. So similar to arm64 the target addresses need to be updated. This is probably also helpful for other things needing more complex code in alternatives. For v2 I got into some sort of cleanup spree for the general instruction parsing that already existed. A number of places do their own instruction parsing and I tried consolidating some of them. Noteable, the kvm parts still do, but I had to stop somewhere :-) * b4-shazam-merge: RISC-V: fix auipc-jalr addresses in patched alternatives RISC-V: add helpers for handling immediates in U-type and I-type pairs RISC-V: add rd reg parsing to insn.h header RISC-V: add U-type imm parsing to insn.h header RISC-V: kprobes: use central defined funct3 constants RISC-V: rename parse_asm.h to insn.h RISC-V: Move riscv_insn_is_* macros into a common header RISC-V: add auipc elements to parse_asm header RISC-V: add ebreak instructions to definitions RISC-V: detach funct-values from their offset RISC-V: add prefix to all constants/macros in parse_asm.h RISC-V: fix funct4 definition for c.jalr in parse_asm.h Link: https://lore.kernel.org/r/20221223221332.4127602-1-heiko@sntech.deSigned-off-by:
-
Heiko Stuebner authored
Alternatives live in a different section, so addresses used by call functions will point to wrong locations after the patch got applied. Similar to arm64, adjust the location to consider that offset. Reviewed-by:
Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> Tested-by:
-
Heiko Stuebner authored
Used together U-type and I-type instructions can for example be used to generate bigger jumps (i.e. in auipc+jalr pairs) by splitting the value into an upper immediate (i.e. auipc) and a 12bit immediate (i.e. jalr). Due to both immediates being considered signed this creates some corner cases, so add some helper to prevent this from getting duplicated in different places. Reviewed-by:
-
Heiko Stuebner authored
Add a macro to allow parsing of the rd register from an instruction. Reviewed-by:
-
Heiko Stuebner authored
Similar to other existing types, allow extracting the immediate for a U-type instruction. U-type immediates are special in that regard, that the value in the instruction in bits [31:12] already represents the same bits of the immediate, so no shifting is required. U-type immediates are for example used in the auipc instruction, so these constants make it easier to parse such instructions. Reviewed-by:
-
Heiko Stuebner authored
Don't redefine values that are already available in the central header asm/insn.h . Use the values from there instead. Reviewed-by:
-
Heiko Stuebner authored
The current parse_asm header should become a more centralized place for everything concerning parsing and constructing instructions. We already have a header insn-def.h similar to aarch64, so rename parse_asm.h to insn.h (again similar to aarch64) to show that it's meant for more than simple instruction parsing. Reviewed-by:
-
Heiko Stuebner authored
Right now the riscv kernel has (at least) two independent sets of functions to check if an encoded instruction is of a specific type. One in kgdb and one kprobes simulate-insn code. More parts of the kernel will probably need this in the future, so instead of allowing this duplication to go on further, move macros that do the function declaration in a common header, similar to at least aarch64. Reviewed-by:
-
Heiko Stuebner authored
Instruction parsing should not be done in individual code, but instead supported by central Right now kgdb and kprobes parse instructions and at least kprobes (and the upcoming auipc+jalr alternative fixer-function) need the auipc instruction. Reviewed-by:
-
Heiko Stuebner authored
kprobes need to match ebreak instructions, so add the necessary data to enable us to centralize that functionality. Reviewed-by:
-
Heiko Stuebner authored
Rather than defining funct3, funct4, etc values pre-shifted to their target-position in an instruction, define the values themselves and only shift them where needed. This allows using these funct-values in other places as well, for example when decoding functions. At the same time also reduces the use of magic numbers, one would need a spec manual to understand. Reviewed-by:
-
Heiko Stuebner authored
Some of the constants and macros already have suitable RV_, RVG_ or RVC_ prefixes. Extend this to the rest of the file as well, as we want to use these things in a broader scope soon. Reviewed-by:
-
Heiko Stuebner authored
The opcode definition for c.jalr is c.jalr c_rs1_n0 1..0=2 15..13=4 12=1 6..2=0 This means funct4 consisting of bit [15:12] is 1001b, so the value is 0x9. Fixes: edde5584 ("riscv: Add SW single-step support for KDB") Reported-by:
-
