- 13 Nov, 2019 6 commits
-
-
Vincent Guittot authored
CVE-2018-20784 This re-applies the commit reverted here: commit c40f7d74 ("sched/fair: Fix infinite loop in update_blocked_averages() by reverting a9e7f654") I.e. now that cfs_rq can be safely removed/added in the list, we can re-apply: commit a9e7f654 ("sched/fair: Fix O(nr_cgroups) in load balance path") Signed-off-by:
Vincent Guittot <vincent.guittot@linaro.org> Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: sargun@sargun.me Cc: tj@kernel.org Cc: xiexiuqi@huawei.com Cc: xiezhipeng1@huawei.com Link: https://lkml.kernel.org/r/1549469662-13614-3-git-send-email-vincent.guittot@linaro.orgSigned-off-by:
Ingo Molnar <mingo@kernel.org> (backported from commit 039ae8bc) [ Connor Kuehl: In 'cfs_rq_is_decayed' the data member 'runnable_load_sum' belongs to struct cfs_rq and not sched_avg, so update that. Some instances of 'for_each_leaf_cfs_rq' required manual updating to the new 'for_each_leaf_cfs_rq_safe' and the last hunk for 'update_blocked_averages' required manual placement. ] Signed-off-by:
Connor Kuehl <connor.kuehl@canonical.com> Acked-by:
Sultan Alsawaf <sultan.alsawaf@canonical.com> Acked-by:
Stefan Bader <stefan.bader@canonical.com> Signed-off-by:
Khalid Elmously <khalid.elmously@canonical.com>
-
Vincent Guittot authored
CVE-2018-20784 Removing a cfs_rq from rq->leaf_cfs_rq_list can break the parent/child ordering of the list when it will be added back. In order to remove an empty and fully decayed cfs_rq, we must remove its children too, so they will be added back in the right order next time. With a normal decay of PELT, a parent will be empty and fully decayed if all children are empty and fully decayed too. In such a case, we just have to ensure that the whole branch will be added when a new task is enqueued. This is default behavior since : commit f6783319 ("sched/fair: Fix insertion in rq->leaf_cfs_rq_list") In case of throttling, the PELT of throttled cfs_rq will not be updated whereas the parent will. This breaks the assumption made above unless we remove the children of a cfs_rq that is throttled. Then, they will be added back when unthrottled and a sched_entity will be enqueued. As throttled cfs_rq are now removed from the list, we can remove the associated test in update_blocked_averages(). Signed-off-by:
-
Vincent Guittot authored
CVE-2018-20784 Sargun reported a crash: "I picked up c40f7d74 sched/fair: Fix infinite loop in update_blocked_averages() by reverting a9e7f654 and put it on top of 4.19.13. In addition to this, I uninlined list_add_leaf_cfs_rq for debugging. This revealed a new bug that we didn't get to because we kept getting crashes from the previous issue. When we are running with cgroups that are rapidly changing, with CFS bandwidth control, and in addition using the cpusets cgroup, we see this crash. Specifically, it seems to occur with cgroups that are throttled and we change the allowed cpuset." The algorithm used to order cfs_rq in rq->leaf_cfs_rq_list assumes that it will walk down to root the 1st time a cfs_rq is used and we will finish to add either a cfs_rq without parent or a cfs_rq with a parent that is already on the list. But this is not always true in presence of throttling. Because a cfs_rq can be throttled even if it has never been used but other CPUs of the cgroup have already used all the bandwdith, we are not sure to go down to the root and add all cfs_rq in the list. Ensure that all cfs_rq will be added in the list even if they are throttled. [ mingo: Fix !CGROUPS build. ] Reported-by:
Sargun Dhillon <sargun@sargun.me> Signed-off-by:
-
Peter Zijlstra authored
CVE-2018-20784 The magic in list_add_leaf_cfs_rq() requires that at the end of enqueue_task_fair(): rq->tmp_alone_branch == &rq->lead_cfs_rq_list If this is violated, list integrity is compromised for list entries and the tmp_alone_branch pointer might dangle. Also, reflow list_add_leaf_cfs_rq() while there. This looses one indentation level and generates a form that's convenient for the next patch. Signed-off-by:
-
Vincent Guittot authored
CVE-2018-20784 Fix the insertion of cfs_rq in rq->leaf_cfs_rq_list to ensure that a child will always be called before its parent. The hierarchical order in shares update list has been introduced by commit: 67e86250 ("sched: Introduce hierarchal order on shares update list") With the current implementation a child can be still put after its parent. Lets take the example of: root \ b /\ c d* | e* with root -> b -> c already enqueued but not d -> e so the leaf_cfs_rq_list looks like: head -> c -> b -> root -> tail The branch d -> e will be added the first time that they are enqueued, starting with e then d. When e is added, its parents is not already on the list so e is put at the tail : head -> c -> b -> root -> e -> tail Then, d is added at the head because its parent is already on the list: head -> d -> c -> b -> root -> e -> tail e is not placed at the right position and will be called the last whereas it should be called at the beginning. Because it follows the bottom-up enqueue sequence, we are sure that we will finished to add either a cfs_rq without parent or a cfs_rq with a parent that is already on the list. We can use this event to detect when we have finished to add a new branch. For the others, whose parents are not already added, we have to ensure that they will be added after their children that have just been inserted the steps before, and after any potential parents that are already in the list. The easiest way is to put the cfs_rq just after the last inserted one and to keep track of it untl the branch is fully added. Signed-off-by:
Dietmar Eggemann <dietmar.eggemann@arm.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Morten.Rasmussen@arm.com Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: bsegall@google.com Cc: kernellwp@gmail.com Cc: pjt@google.com Cc: yuyang.du@intel.com Link: http://lkml.kernel.org/r/1478598827-32372-3-git-send-email-vincent.guittot@linaro.orgSigned-off-by:
-
Linus Torvalds authored
CVE-2018-20784 Zhipeng Xie, Xie XiuQi and Sargun Dhillon reported lockups in the scheduler under high loads, starting at around the v4.18 time frame, and Zhipeng Xie tracked it down to bugs in the rq->leaf_cfs_rq_list manipulation. Do a (manual) revert of: a9e7f654 ("sched/fair: Fix O(nr_cgroups) in load balance path") It turns out that the list_del_leaf_cfs_rq() introduced by this commit is a surprising property that was not considered in followup commits such as: 9c2791f9 ("sched/fair: Fix hierarchical order in rq->leaf_cfs_rq_list") As Vincent Guittot explains: "I think that there is a bigger problem with commit a9e7f654 and cfs_rq throttling: Let take the example of the following topology TG2 --> TG1 --> root: 1) The 1st time a task is enqueued, we will add TG2 cfs_rq then TG1 cfs_rq to leaf_cfs_rq_list and we are sure to do the whole branch in one path because it has never been used and can't be throttled so tmp_alone_branch will point to leaf_cfs_rq_list at the end. 2) Then TG1 is throttled 3) and we add TG3 as a new child of TG1. 4) The 1st enqueue of a task on TG3 will add TG3 cfs_rq just before TG1 cfs_rq and tmp_alone_branch will stay on rq->leaf_cfs_rq_list. With commit a9e7f654, we can del a cfs_rq from rq->leaf_cfs_rq_list. So if the load of TG1 cfs_rq becomes NULL before step 2) above, TG1 cfs_rq is removed from the list. Then at step 4), TG3 cfs_rq is added at the beginning of rq->leaf_cfs_rq_list but tmp_alone_branch still points to TG3 cfs_rq because its throttled parent can't be enqueued when the lock is released. tmp_alone_branch doesn't point to rq->leaf_cfs_rq_list whereas it should. So if TG3 cfs_rq is removed or destroyed before tmp_alone_branch points on another TG cfs_rq, the next TG cfs_rq that will be added, will be linked outside rq->leaf_cfs_rq_list - which is bad. In addition, we can break the ordering of the cfs_rq in rq->leaf_cfs_rq_list but this ordering is used to update and propagate the update from leaf down to root." Instead of trying to work through all these cases and trying to reproduce the very high loads that produced the lockup to begin with, simplify the code temporarily by reverting a9e7f654 - which change was clearly not thought through completely. This (hopefully) gives us a kernel that doesn't lock up so people can continue to enjoy their holidays without worrying about regressions. ;-) [ mingo: Wrote changelog, fixed weird spelling in code comment while at it. ] Analyzed-by:
Xie XiuQi <xiexiuqi@huawei.com> Analyzed-by:
Zhipeng Xie <xiezhipeng1@huawei.com> Reported-by:
Linus Torvalds <torvalds@linux-foundation.org> Acked-by:
-
- 12 Nov, 2019 34 commits
-
-
Connor Kuehl authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Ignore: yes Signed-off-by:
-
Connor Kuehl authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Ubuntu commit 6a1909898c14 ("ARM: 8479/2: add implementation for arm-smccc") updated selectable config options. Accept the defaults. Signed-off-by:
-
Greg Kroah-Hartman authored
BugLink: https://bugs.launchpad.net/bugs/1852110Signed-off-by:
-
zhangyi (F) authored
BugLink: https://bugs.launchpad.net/bugs/1852110 During backport 1e2e547a "do d_instantiate/unlock_new_inode combinations safely", there was a error instantiating sequence of attaching dentry to inode and calling security_d_instantiate(). Before commit ce23e640 "->getxattr(): pass dentry and inode as separate arguments" and b9680917 "security_d_instantiate(): move to the point prior to attaching dentry to inode", security_d_instantiate() should be called beind __d_instantiate(), otherwise it will trigger below problem when CONFIG_SECURITY_SMACK on ext4 was enabled because d_inode(dentry) used by ->getxattr() is NULL before __d_instantiate() instantiate inode. [ 31.858026] BUG: unable to handle kernel paging request at ffffffffffffff70 ... [ 31.882024] Call Trace: [ 31.882378] [<ffffffffa347f75c>] ext4_xattr_get+0x8c/0x3e0 [ 31.883195] [<ffffffffa3489454>] ext4_xattr_security_get+0x24/0x40 [ 31.884086] [<ffffffffa336a56b>] generic_getxattr+0x5b/0x90 [ 31.884907] [<ffffffffa3700514>] smk_fetch+0xb4/0x150 [ 31.885634] [<ffffffffa3700772>] smack_d_instantiate+0x1c2/0x550 [ 31.886508] [<ffffffffa36f9a5a>] security_d_instantiate+0x3a/0x80 [ 31.887389] [<ffffffffa3353b26>] d_instantiate_new+0x36/0x130 [ 31.888223] [<ffffffffa342b1ef>] ext4_mkdir+0x4af/0x6a0 [ 31.888928] [<ffffffffa3343470>] vfs_mkdir+0x100/0x280 [ 31.889536] [<ffffffffa334b086>] SyS_mkdir+0xb6/0x170 [ 31.890255] [<ffffffffa307c855>] ? trace_do_page_fault+0x95/0x2b0 [ 31.891134] [<ffffffffa3c5e078>] entry_SYSCALL_64_fastpath+0x18/0x73 Cc: <stable@vger.kernel.org> # 3.16, 4.4 Signed-off-by:
zhangyi (F) <yi.zhang@huawei.com> Signed-off-by:
Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by:
-
Petr Vorel authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Fix backport of commit f18ddc13 upstream. Update backport to change ENOTSUPP to EOPNOTSUPP in alarm_timer_{del,set}(), which were removed in f2c45807 in v4.13-rc1. Fixes: c22df8ea7c5831d6fdca2f6f136f0d32d7064ff9 Signed-off-by:
Petr Vorel <pvorel@suse.cz> Acked-by:
Thadeu Lima de Souza Cascardo <cascardo@canonical.com> Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit d6951f58 upstream. The intention in the previous patch was to only place the processor tables in the .rodata section if big.Little was being built and we wanted the branch target hardening, but instead (due to the way it was tested) it ended up always placing the tables into the .rodata section. Although harmless, let's correct this anyway. Fixes: 3a4d0c21 ("ARM: ensure that processor vtables is not lost after boot") Signed-off-by:
Russell King <rmk+kernel@armlinux.org.uk> Signed-off-by:
David A. Long <dave.long@linaro.org> Reviewed-by:
Julien Thierry <julien.thierry@arm.com> Signed-off-by:
Sasha Levin <sashal@kernel.org> Signed-off-by:
Ard Biesheuvel <ardb@kernel.org> Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 3a4d0c21 upstream. Marek Szyprowski reported problems with CPU hotplug in current kernels. This was tracked down to the processor vtables being located in an init section, and therefore discarded after kernel boot, despite being required after boot to properly initialise the non-boot CPUs. Arrange for these tables to end up in .rodata when required. Reported-by:
Marek Szyprowski <m.szyprowski@samsung.com> Tested-by:
Krzysztof Kozlowski <krzk@kernel.org> Fixes: 383fb3ee ("ARM: spectre-v2: per-CPU vtables to work around big.Little systems") Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 383fb3ee upstream. In big.Little systems, some CPUs require the Spectre workarounds in paths such as the context switch, but other CPUs do not. In order to handle these differences, we need per-CPU vtables. We are unable to use the kernel's per-CPU variables to support this as per-CPU is not initialised at times when we need access to the vtables, so we have to use an array indexed by logical CPU number. We use an array-of-pointers to avoid having function pointers in the kernel's read/write .data section. Note: Added include of linux/slab.h in arch/arm/smp.c. Reviewed-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit e209950f upstream. Allow the way we access members of the processor vtable to be changed at compile time. We will need to move to per-CPU vtables to fix the Spectre variant 2 issues on big.Little systems. However, we have a couple of calls that do not need the vtable treatment, and indeed cause a kernel warning due to the (later) use of smp_processor_id(), so also introduce the PROC_TABLE macro for these which always use CPU 0's function pointers. Reviewed-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 945aceb1 upstream. Call the per-processor type check_bugs() method in the same way as we do other per-processor functions - move the "processor." detail into proc-fns.h. Reviewed-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 65987a85 upstream. Split out the lookup of the processor type and associated error handling from the rest of setup_processor() - we will need to use this in the secondary CPU bringup path for big.Little Spectre variant 2 mitigation. Reviewed-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 899a42f8 upstream. Move lookup_processor_type() out of the __init section so it is callable from (eg) the secondary startup code during hotplug. Reviewed-by:
-
Julien Thierry authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 5df7a99b upstream. In vfp_preserve_user_clear_hwstate, ufp_exc->fpinst2 gets assigned to itself. It should actually be hwstate->fpinst2 that gets assigned to the ufp_exc field. Fixes commit 3aa2df6e ("ARM: 8791/1: vfp: use __copy_to_user() when saving VFP state"). Reported-by:
David Binderman <dcb314@hotmail.com> Signed-off-by:
-
Julien Thierry authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit afaf6838 upstream. Introduce C and asm helpers to sanitize user address, taking the address range they target into account. Use asm helper for existing sanitization in __copy_from_user(). Signed-off-by:
-
Julien Thierry authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit e3aa6243 upstream. When Spectre mitigation is required, __put_user() needs to include check_uaccess. This is already the case for put_user(), so just make __put_user() an alias of put_user(). Signed-off-by:
-
Julien Thierry authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 621afc67 upstream. A mispredicted conditional call to set_fs could result in the wrong addr_limit being forwarded under speculation to a subsequent access_ok check, potentially forming part of a spectre-v1 attack using uaccess routines. This patch prevents this forwarding from taking place, but putting heavy barriers in set_fs after writing the addr_limit. Porting commit c2f0ad4f ("arm64: uaccess: Prevent speculative use of the current addr_limit"). Signed-off-by:
-
Julien Thierry authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 18ea66bd upstream. With Spectre-v1.1 mitigations, __put_user_error is pointless. In an attempt to remove it, replace its references in frame setups with __put_user. Signed-off-by:
-
Julien Thierry authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 31950890 upstream. Copy events to user using __copy_to_user() rather than copy members of individually with __put_user_error(). This has the benefit of disabling/enabling PAN once per event intead of once per event member. Signed-off-by:
-
Julien Thierry authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 3aa2df6e upstream. Use __copy_to_user() rather than __put_user_error() for individual members when saving VFP state. This has the benefit of disabling/enabling PAN once per copied struct intead of once per write. Signed-off-by:
-
Julien Thierry authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 5ca451cf upstream. When saving the ARM integer registers, use __copy_to_user() to copy them into user signal frame, rather than __put_user_error(). This has the benefit of disabling/enabling PAN once for the whole copy intead of once per write. Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit a3c0f847 upstream. Spectre variant 1 attacks are about this sequence of pseudo-code: index = load(user-manipulated pointer); access(base + index * stride); In order for the cache side-channel to work, the access() must me made to memory which userspace can detect whether cache lines have been loaded. On 32-bit ARM, this must be either user accessible memory, or a kernel mapping of that same user accessible memory. The problem occurs when the load() speculatively loads privileged data, and the subsequent access() is made to user accessible memory. Any load() which makes use of a user-maniplated pointer is a potential problem if the data it has loaded is used in a subsequent access. This also applies for the access() if the data loaded by that access is used by a subsequent access. Harden the get_user() accessors against Spectre attacks by forcing out of bounds addresses to a NULL pointer. This prevents get_user() being used as the load() step above. As a side effect, put_user() will also be affected even though it isn't implicated. Also harden copy_from_user() by redoing the bounds check within the arm_copy_from_user() code, and NULLing the pointer if out of bounds. Acked-by:
Mark Rutland <mark.rutland@arm.com> Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit b1cd0a14 upstream. Fixing __get_user() for spectre variant 1 is not sane: we would have to add address space bounds checking in order to validate that the location should be accessed, and then zero the address if found to be invalid. Since __get_user() is supposed to avoid the bounds check, and this is exactly what get_user() does, there's no point having two different implementations that are doing the same thing. So, when the Spectre workarounds are required, make __get_user() an alias of get_user(). Acked-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit d09fbb32 upstream. Borrow the x86 implementation of __inttype() to use in get_user() to select an integer type suitable to temporarily hold the result value. This is necessary to avoid propagating the volatile nature of the result argument, which can cause the following warning: lib/iov_iter.c:413:5: warning: optimization may eliminate reads and/or writes to register variables [-Wvolatile-register-var] Acked-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 8c8484a1 upstream. __get_user_error() is used as a fast accessor to make copying structure members as efficient as possible. However, with software PAN and the recent Spectre variant 1, the efficiency is reduced as these are no longer fast accessors. In the case of software PAN, it has to switch the domain register around each access, and with Spectre variant 1, it would have to repeat the access_ok() check for each access. Rather than using __get_user_error() to copy each semops element member, copy each semops element in full using __copy_from_user(). Acked-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 42019fc5 upstream. __get_user_error() is used as a fast accessor to make copying structure members in the signal handling path as efficient as possible. However, with software PAN and the recent Spectre variant 1, the efficiency is reduced as these are no longer fast accessors. In the case of software PAN, it has to switch the domain register around each access, and with Spectre variant 1, it would have to repeat the access_ok() check for each access. Use __copy_from_user() rather than __get_user_err() for individual members when restoring VFP state. Acked-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit c32cd419 upstream. __get_user_error() is used as a fast accessor to make copying structure members in the signal handling path as efficient as possible. However, with software PAN and the recent Spectre variant 1, the efficiency is reduced as these are no longer fast accessors. In the case of software PAN, it has to switch the domain register around each access, and with Spectre variant 1, it would have to repeat the access_ok() check for each access. It becomes much more efficient to use __copy_from_user() instead, so let's use this for the ARM integer registers. Acked-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 10573ae5 upstream. Prevent speculation at the syscall table decoding by clamping the index used to zero on invalid system call numbers, and using the csdb speculative barrier. Signed-off-by:
Tony Lindgren <tony@atomide.com> Reviewed-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 1d4238c5 upstream. Add an implementation of the array_index_mask_nospec() function for mitigating Spectre variant 1 throughout the kernel. Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit a78d1565 upstream. Add assembly and C macros for the new CSDB instruction. Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit c44f366e upstream. Warn at error level if the context switching function is not what we are expecting. This can happen with big.Little systems, which we currently do not support. Signed-off-by:
Marc Zyngier <marc.zyngier@arm.com> Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 10115105 upstream. Add firmware based hardening for cores that require more complex handling in firmware. Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit f5fe12b1 upstream. In order to prevent aliasing attacks on the branch predictor, invalidate the BTB or instruction cache on CPUs that are known to be affected when taking an abort on a address that is outside of a user task limit: Cortex A8, A9, A12, A17, A73, A75: flush BTB. Cortex A15, Brahma B15: invalidate icache. If the IBE bit is not set, then there is little point to enabling the workaround. Signed-off-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit e388b802 upstream. When the branch predictor hardening is enabled, firmware must have set the IBE bit in the auxiliary control register. If this bit has not been set, the Spectre workarounds will not be functional. Add validation that this bit is set, and print a warning at alert level if this is not the case. Signed-off-by:
Florian Fainelli <f.fainelli@gmail.com> Boot-tested-by:
-
Russell King authored
BugLink: https://bugs.launchpad.net/bugs/1852110 Commit 06c23f5f upstream. Required manual merge of arch/arm/mm/proc-v7.S. Harden the branch predictor against Spectre v2 attacks on context switches for ARMv7 and later CPUs. We do this by: Cortex A9, A12, A17, A73, A75: invalidating the BTB. Cortex A15, Brahma B15: invalidating the instruction cache. Cortex A57 and Cortex A72 are not addressed in this patch. Cortex R7 and Cortex R8 are also not addressed as we do not enforce memory protection on these cores. Signed-off-by:
-
