- 16 Feb, 2024 38 commits
-
-
Heiko Carstens authored
Provide various vector instruction inline assemblies for crc32 calculations. This is just preparation to keep the conversion of the existing crc32 implementations from assembly to C small. Signed-off-by:Heiko Carstens <hca@linux.ibm.com>
-
Heiko Carstens authored
Provide several one instruction fpu inline assemebles and use them to implement the bogomips calculation in C like style. This is more for illustration purposes on how kernel fpu code can be written in C. This has the advantage that the author only has to take care of the floating point instructions, but doesn't need to take care of general purpose register allocation (if needed), and the semantics of all other instructions not related to fpu. Signed-off-by: -
Heiko Carstens authored
Move the s390 specific raid6 inline assemblies, make them generic, and reuse them to implement the raid6 gen/xor implementation. Signed-off-by: -
Heiko Carstens authored
With csum_partial(), which reads all bytes into registers it is easy to also implement csum_partial_copy_nocheck() which copies the buffer while calculating its checksum. For a 512 byte buffer this reduces the runtime by 19%. Compared to the old generic variant (memcpy() + cksm instruction) runtime is reduced by 42%). Signed-off-by: -
Heiko Carstens authored
Provide a faster variant of csum_partial() which uses vector registers instead of the cksm instruction. Signed-off-by: -
Heiko Carstens authored
Convert those callers of csum_partial() to use the cksm instruction, which are either very early or in critical paths, like panic/dump, so they don't have to rely on a working kernel infrastructure, which will be introduced with a subsequent patch. Signed-off-by: -
Heiko Carstens authored
Call instrument_read() from csum_partial() instead of kasan_check_read(). instrument_read() covers all memory access instrumentation methods. Signed-off-by: -
Heiko Carstens authored
TIF_FPU is unused - remove it. Signed-off-by: -
Heiko Carstens authored
The first invocation of kernel_fpu_begin() after switching from user to kernel context will save all vector registers, even if only parts of the vector registers are used within the kernel fpu context. Given that save and restore of all vector registers is quite expensive change the current approach in several ways: - Instead of saving and restoring all user registers limit this to those registers which are actually used within an kernel fpu context. - On context switch save all remaining user fpu registers, so they can be restored when the task is rescheduled. - Saving user registers within kernel_fpu_begin() is done without disabling and enabling interrupts - which also slightly reduces runtime. In worst case (e.g. interrupt context uses the same registers) this may lead to the situation that registers are saved several times, however the assumption is that this will not happen frequently, so that the new method is faster in nearly all cases. - save_user_fpu_regs() can still be called from all contexts and saves all (or all remaining) user registers to a tasks ufpu user fpu save area. Overall this reduces the time required to save and restore the user fpu context for nearly all cases. Signed-off-by: -
Heiko Carstens authored
The kernel_fpu structure has a quite large size of 520 bytes. In order to reduce stack footprint introduce several kernel fpu structures with different and also smaller sizes. This way every kernel fpu user must use the correct variant. A compile time check verifies that the correct variant is used. There are several users which use only 16 instead of all 32 vector registers. For those users the new kernel_fpu_16 structure with a size of only 266 bytes can be used. Signed-off-by: -
Heiko Carstens authored
Let fpu_vlm() and fpu_vstm() macros return the number of registers saved / loaded. This is helpful to read easy to read code in case there are several subsequent fpu_vlm() or fpu_vstm() calls: __vector128 *vxrs = .... vxrs += fpu_vstm(0, 15, vxrs); vxrs += fpu_vstm(16, 31, vxrs); Reviewed-by:
Claudio Imbrenda <imbrenda@linux.ibm.com> Signed-off-by:
-
Heiko Carstens authored
The anonymous union within struct fpu contains a floating point register array and a vector register array. Given that the vector register is always present remove the floating point register array. For configurations without vector registers save the floating point register contents within their corresponding vector register location. This allows to remove the union, and also to simplify ptrace and perf code. Signed-off-by: -
Heiko Carstens authored
KVM was the only user which modified the regs pointer in struct fpu. Remove the pointer and convert the rest of the core fpu code to directly access the save area embedded within struct fpu. Reviewed-by:
-
Heiko Carstens authored
KVM modifies the kernel fpu's regs pointer to its own area to implement its custom version of preemtible kernel fpu context. With general support for preemptible kernel fpu context there is no need for the extra complexity in KVM code anymore. Therefore convert KVM to a regular kernel fpu user. In particular this means that all TIF_FPU checks can be removed, since the fpu register context will never be changed by other kernel fpu users, and also the fpu register context will be restored if a thread is preempted. Reviewed-by:
-
Heiko Carstens authored
Make the kernel fpu context preemptible. Add another fpu structure to the thread_struct, and use it to save and restore the kernel fpu context if its task uses fpu registers when it is preempted. Reviewed-by:
-
Heiko Carstens authored
Change type of fpu mask consistently from u32 to int. This is a prerequisite to make the kernel fpu usage preemptible. Upcoming code uses __atomic* ops which work with int pointers. Reviewed-by:
-
Heiko Carstens authored
Rename save_fpu_regs(), load_fpu_regs(), and struct thread_struct's fpu member to save_user_fpu_regs(), load_user_fpu_regs(), and ufpu. This way the function and variable names reflect for which context they are supposed to be used. This large and trivial conversion is a prerequisite for making the kernel fpu usage preemptible. Reviewed-by:
-
Heiko Carstens authored
The FPU state, as represented by the CIF_FPU flag reflects the FPU state of a task, not the CPU it is running on. Therefore convert the flag to a regular TIF flag. This removes the magic in switch_to() where a save_fpu_regs() call for the currently (previous) running task sets the per-cpu CIF_FPU flag, which is required to restore FPU register contents of the next task, when it returns to user space. Signed-off-by: -
Heiko Carstens authored
Convert the rather large __kernel_fpu_begin()/__kernel_fpu_end() inline assemblies to C. The C variant is much more readable, and this also allows to get rid of the non-obvious usage of KERNEL_VXR_* constants within the inline assemblies. E.g. "tmll %[m],6" correlates with the two bits set in KERNEL_VXR_LOW. If the corresponding defines would be changed, the inline assembles would break in a subtle way. Therefore convert to C, use the proper defines, and allow the compiler to generate code using the (hopefully) most efficient instructions. Signed-off-by: -
Heiko Carstens authored
Instead of open-coding vlm and vstm inline assemblies at several locations, provide an fpu_* function for each instruction, and use them in the new save_vx_regs() and load_vx_regs() helper functions. Note that "O" and "R" inline assembly operand modifiers are used in order to pass the displacement and base register of the memory operands to the existing VLM and VSTM macros. The two operand modifiers are not available for clang. Therefore provide two variants of each inline assembly. The clang variant always uses and clobbers general purpose register 1, like in the previous inline assemblies, so it can be used as base register with a zero displacement. This generates slightly less efficient code, but can be removed as soon as clang has support for the used operand modifiers. Reviewed-by:
-
Heiko Carstens authored
Instead of open-coding lfpc, sfpc, and stfpc inline assemblies at several locations, provide an fpu_* function for each instruction and use the function instead. Reviewed-by:
-
Heiko Carstens authored
Deduplicate the 64 ld and std inline assemblies. Provide an fpu inline assembly for both instructions, and use them in the new save_fp_regs() and load_fp_regs() helper functions. Reviewed-by:
-
Heiko Carstens authored
The only user of sfpc_safe() needs to read the new fpc register value from memory before it is set with sfpc. Avoid this indirection and use lfpc, which reads the new value from memory. Also add the "fpu_" prefix to have a common name space for fpu related inline assemblies, and provide memory access instrumentation. Reviewed-by:
-
Heiko Carstens authored
Add documentation which describes what the fpu helper functions are good for, and why they should be used. Reviewed-by:
-
Heiko Carstens authored
Move, rename, and merge the fpu and vx header files. This way fpu header files have a consistent naming scheme (fpu*.h). Also get rid of the fpu subdirectory and move header files to asm directory, so that all fpu and vx header files can be found at the same location. Merge internal.h header file into other header files, since the internal helpers are used at many locations. so those helper functions are really not internal. Signed-off-by: -
Heiko Carstens authored
Address various checkpatch warnings, adjust whitespace, and try to increase readability. This is just preparation, in order to avoid that subsequent patches contain any distracting drive-by coding style changes. Reviewed-by:
-
Heiko Carstens authored
Use KERNEL_VXR_LOW instead of KERNEL_VXR_V0V7 for configurations without vector registers in order to decide if floating point registers need to be saved and restored. Kernel FPU areas which use floating point registers are supposed to use the KERNEL_FPR mask, however users may also open-code this and specify KERNEL_VXR_V0V7 and/or KERNEL_VXR_V8V15. If only KERNEL_VXR_V8V15 is specified floating point registers wouldn't be saved and restored. Improve this and check for both bits. There are currently no users where this would fix a bug. Reviewed-by:
-
Heiko Carstens authored
Remove the historic machine check handler code which validates registers. Registers are automatically validated as part of the machine check handling sequence (see Principles of Operation, Machine-Check Handling chapter, Validation). Signed-off-by: -
Heiko Carstens authored
The v1, v2, v3, and v4 parameters of the RXB macro are a bit misleading, since the reader can assume that the parameters always correlate with the instructions format fields V1, V2, V3, and V4 as defined in the Principles of Operation. This is not the case for a couple of instructions, therefore improve the description of the macro. Suggested by Jens Remus, who also provided the improved description. Suggested-by:
Jens Remus <jremus@linux.ibm.com> Reviewed-by:
-
Heiko Carstens authored
The VLGV macro generates the VLGV instruction and has a vr parameter which correlates to the V3 vector register field of the instruction (bits 12-15). Due to its position in the VRS-c instruction format of the VLGV instruction, this field correlates to the second bit of the RXB byte of the instruction (see Principles of Operation, Chapter "Vector Overview and Support Instructions"). Within the VLGV macro the MRXBOPC macro is used to generate the RXB field of the instruction. The usage of the MRXBOPC macro is incorrect, since the vector register number is passed as third parameter (which correlates to the first bit of the RXB byte), while it should be passed as fourth parameter (second bit of the RXB byte). In result an incorrect instruction would be generated if the VLGV macro would be used for vector register numbers larger than 15. Fix this and pass the vector register number as fourth parameter. Currently there are no users within the kernel which use the macro in a way that broken code would be generated. Reviewed-by:
Hendrik Brueckner <brueckner@linux.ibm.com> Signed-off-by:
-
Harald Freudenberger authored
This patch introduces dynamic debug hexdump invocation possibilities to be able to: - dump an CCA or EP11 CPRB request as early as possible when received via ioctl from userspace but after the ap message has been collected together. - dump an CCA or EP11 CPRB reply short before it is transferred via ioctl into userspace. Signed-off-by:
Harald Freudenberger <freude@linux.ibm.com> Reviewed-by:
Holger Dengler <dengler@linux.ibm.com> Signed-off-by:
-
Harald Freudenberger authored
This patch introduces two dynamic debug hexdump invocation possibilities to be able to a) dump an AP message immediately before it goes into the firmware queue and b) dump a fresh from the firmware queue received AP message. Signed-off-by:
-
Harald Freudenberger authored
This patch replaces all the s390 debug feature calls with debug level by dynamic debug calls pr_debug. These calls are much more flexible and each single invocation can get enabled/disabled at runtime wheres the s390 debug feature debug calls have only one knob - enable or disable all in one bunch. This patch follows a similar change for the AP bus and zcrypt device driver code. All this code uses dynamic debugging with pr_debug and friends for emitting debug traces now. Signed-off-by:
-
Harald Freudenberger authored
Cleanup and harmonize the s390 debug feature calls and defines for the pkey module to be similar to the debug feature as it is used in the zcrypt device driver and AP bus. More or less only renaming but no functional changes. Signed-off-by:
-
Harald Freudenberger authored
This patch replaces all the s390 debug feature calls with debug level by dynamic debug calls pr_debug. These calls are much more flexible and each single invocation can get enabled/disabled at runtime wheres the s390 debug feature debug calls have only one knob - enable or disable all in one bunch. The benefit is especially significant with high frequency called functions like the AP bus scan. In most debugging scenarios you don't want and need them, but sometimes it is crucial to know exactly when and how long the AP bus scan took. Signed-off-by:
-
Harald Freudenberger authored
This patch harmonizes the calls and defines around the s390 debug feature as it is used in the AP bus and zcrypt device driver code. More or less cleanup and renaming, no functional changes. Signed-off-by:
-
Gerald Schaefer authored
Fix virtual vs physical address confusion. This does not fix a bug since virtual and physical address spaces are currently the same. /proc/iomem should report the physical address ranges, so use __pa_symbol() for resource registration, similar to other architectures. Signed-off-by:
Gerald Schaefer <gerald.schaefer@linux.ibm.com> Reviewed-by:
Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by:
-
Nathan Chancellor authored
When building 'ARCH=s390 defconfig compat.config' with GCC and LD=ld.lld, there is an error when attempting to link the compat vDSO: ld.lld: error: unknown emulation: elf_s390 make[4]: *** [arch/s390/kernel/vdso32/Makefile:48: arch/s390/kernel/vdso32/vdso32.so.dbg] Error 1 Much like clang, ld.lld only supports the 64-bit s390 emulation. Add a dependency on not using LLD to CONFIG_COMPAT to avoid breaking the build with this toolchain combination. Signed-off-by:
Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20240214-s390-compat-lld-dep-v1-1-abf1f4b5e514@kernel.orgSigned-off-by:
-
- 14 Feb, 2024 2 commits
-
-
Heiko Carstens authored
When linking vdso64.so.dbg with ld.lld, there is a warning about not finding _start for the starting address: ld.lld: warning: cannot find entry symbol _start; not setting start address Fix this by removing the unused ENTRY in both vdso linker scripts. See commit e2471728 ("powerpc/vdso: Remove unused ENTRY in linker scripts"), which solved the same problem for powerpc, for further details. Reviewed-by:
-
Heiko Carstens authored
Commit e21f8baf ("s390/bug: add entry size to the __bug_table section") changed the __EMIT_BUG() inline assembly to emit mergeable __bug_table entries. This is at least currently not needed, but causes problems with the upcoming s390 ld.lld support: ld.lld: error: drivers/nvme/host/fc.o:(__bug_table): writable SHF_MERGE section is not supported Therefore revert the change for now. Reported-by:
-
