- 22 May, 2014 2 commits
-
-
Peter Zijlstra authored
The only idle method for arm64 is WFI and it therefore unconditionally requires the reschedule interrupt when idle. Suggested-by:
Catalin Marinas <catalin.marinas@arm.com> Signed-off-by:
Peter Zijlstra <peterz@infradead.org> Acked-by:
Thomas Gleixner <tglx@linutronix.de> Signed-off-by:
Ingo Molnar <mingo@kernel.org>
-
James Hogan authored
The Meta idle function jumps into the interrupt handler which efficiently blocks waiting for the next interrupt when it reads the interrupt status register (TXSTATI). No other (polling) idle functions can be used, therefore TIF_POLLING_NRFLAG is unnecessary, so lets remove it. Peter Zijlstra said: > Most archs have (x86) hlt or (arm) wfi like idle instructions, and if > that is your only possible idle function, you'll require the interrupt > to wake up and there's really no point to having the POLLING bit. Signed-off-by:
James Hogan <james.hogan@imgtec.com> Signed-off-by:
-
- 08 May, 2014 9 commits
-
-
Rafael J. Wysocki authored
The only value ever returned by cpuidle_idle_call() is 0 and its only caller ignores that value anyway, so make it void. Signed-off-by:
Rafael J. Wysocki <rafael.j.wysocki@intel.com> Cc: Daniel Lezcano <daniel.lezcano@linaro.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by:
-
Peter Zijlstra authored
Apply goto to reduce lines and nesting levels. Signed-off-by:
Nicolas Pitre <nicolas.pitre@linaro.org> Cc: Daniel Lezcano <daniel.lezcano@linaro.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/n/tip-cc6vb0snt3sr7op6rlbfeqfh@git.kernel.orgSigned-off-by:
-
Peter Zijlstra authored
With the generic idle functions assuming !polling we should only clear the polling bit at the very last opportunity in order to avoid spurious IPIs. Ideally we'd flip the default to polling, but that means auditing all arch idle functions. Signed-off-by:
-
Peter Zijlstra authored
Because mwait_idle_with_hints() gets called from !idle context it must call current_clr_polling(). This however means that resched_task() is very likely to send an IPI even when we were polling: CPU0 CPU1 if (current_set_polling_and_test()) goto out; __monitor(&ti->flags); if (!need_resched()) __mwait(eax, ecx); set_tsk_need_resched(p); smp_mb(); out: current_clr_polling(); if (!tsk_is_polling(p)) smp_send_reschedule(cpu); So while it is correct (extra IPIs aren't a problem, whereas a missed IPI would be) it is a performance problem (for some). Avoid this issue by using fetch_or() to atomically set NEED_RESCHED and test if POLLING_NRFLAG is set. Since a CPU stuck in mwait is unlikely to modify the flags word, contention on the cmpxchg is unlikely and thus we should mostly succeed in a single go. Signed-off-by:Nicolas Pitre <nico@linaro.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/n/tip-kf5suce6njh5xf5d3od13rr0@git.kernel.orgSigned-off-by:
-
Peter Zijlstra authored
Now that there are no architectures left using it, kill the support for TS_POLLING. Signed-off-by:
-
Peter Zijlstra authored
Standardize the idle polling indicator to TIF_POLLING_NRFLAG such that both TIF_NEED_RESCHED and TIF_POLLING_NRFLAG are in the same word. This will allow us, using fetch_or(), to both set NEED_RESCHED and check for POLLING_NRFLAG in a single operation and avoid pointless wakeups. Changing from the non-atomic thread_info::status flags to the atomic thread_info::flags shouldn't be a big issue since most polling state changes were followed/preceded by a full memory barrier anyway. Also, fix up the apm_32 idle function, clearly that was forgotten in the last conversion. The default idle state is !POLLING so just kill the lot. Signed-off-by:
-
Peter Zijlstra authored
Standardize the idle polling indicator to TIF_POLLING_NRFLAG such that both TIF_NEED_RESCHED and TIF_POLLING_NRFLAG are in the same word. This will allow us, using fetch_or(), to both set NEED_RESCHED and check for POLLING_NRFLAG in a single operation and avoid pointless wakeups. Changing from the non-atomic thread_info::status flags to the atomic thread_info::flags shouldn't be a big issue since most polling state changes were followed/preceded by a full memory barrier anyway. Signed-off-by:
-
Peter Zijlstra authored
Standardize the idle polling indicator to TIF_POLLING_NRFLAG such that both TIF_NEED_RESCHED and TIF_POLLING_NRFLAG are in the same word. This will allow us, using fetch_or(), to both set NEED_RESCHED and check for POLLING_NRFLAG in a single operation and avoid pointless wakeups. Changing from the non-atomic thread_info::status flags to the atomic thread_info::flags shouldn't be a big issue since most polling state changes were followed/preceded by a full memory barrier anyway. Signed-off-by:
Chris Metcalf <cmetcalf@tilera.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Chris Metcalf <cmetcalf@tilera.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/n/tip-35zzwlvwr7cp8xj196y10yyx@git.kernel.orgSigned-off-by:
-
Peter Zijlstra authored
Standardize the idle polling indicator to TIF_POLLING_NRFLAG such that both TIF_NEED_RESCHED and TIF_POLLING_NRFLAG are in the same word. This will allow us, using fetch_or(), to both set NEED_RESCHED and check for POLLING_NRFLAG in a single operation and avoid pointless wakeups. Changing from the non-atomic thread_info::status flags to the atomic thread_info::flags shouldn't be a big issue since most polling state changes were followed/preceded by a full memory barrier anyway. Signed-off-by:
Richard Henderson <rth@twiddle.net> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Matt Turner <mattst88@gmail.com> Cc: Richard Henderson <rth@twiddle.net> Cc: 蔡正龙 <zhenglong.cai@cs2c.com.cn> Cc: linux-alpha@vger.kernel.org Link: http://lkml.kernel.org/n/tip-9tfzr196gs0n2afxv0ga8pc3@git.kernel.orgSigned-off-by:
-
- 07 May, 2014 18 commits
-
-
Jason Low authored
It was found that when running some workloads (such as AIM7) on large systems with many cores, CPUs do not remain idle for long. Thus, tasks can wake/get enqueued while doing idle balancing. In this patch, while traversing the domains in idle balance, in addition to checking for pulled_task, we add an extra check for this_rq->nr_running for determining if we should stop searching for tasks to pull. If there are runnable tasks on this rq, then we will stop traversing the domains. This reduces the chance that idle balance delays a task from running. This patch resulted in approximately a 6% performance improvement when running a Java Server workload on an 8 socket machine. Signed-off-by:
Jason Low <jason.low2@hp.com> Signed-off-by:
-
Vincent Guittot authored
Create a dedicated topology table for ARM which will create new level to differentiate CPUs that can or not powergate independantly from others. The patch gives an example of how to add domain that will take advantage of SD_SHARE_POWERDOMAIN. Signed-off-by:
Vincent Guittot <vincent.guittot@linaro.org> Reviewed-by:
Dietmar Eggemann <dietmar.eggemann@arm.com> Tested-by:
-
Vincent Guittot authored
A new flag SD_SHARE_POWERDOMAIN is created to reflect whether groups of CPUs in a sched_domain level can or not reach different power state. As an example, the flag should be cleared at CPU level if groups of cores can be power gated independently. This information can be used in the load balance decision or to add load balancing level between group of CPUs that can power gate independantly. This flag is part of the topology flags that can be set by arch. Reviewed-by:
-
Vincent Guittot authored
Create a dedicated topology table for handling asymetric feature of powerpc. Signed-off-by:
Preeti U Murthy <preeti@linux.vnet.ibm.com> Signed-off-by:
-
Vincent Guittot authored
BOOK level is only relevant for s390 so we create a dedicated topology table with BOOK level and remove it from default table. Signed-off-by:
-
Vincent Guittot authored
We replace the old way to configure the scheduler topology with a new method which enables a platform to declare additionnal level (if needed). We still have a default topology table definition that can be used by platform that don't want more level than the SMT, MC, CPU and NUMA ones. This table can be overwritten by an arch which either wants to add new level where a load balance make sense like BOOK or powergating level or wants to change the flags configuration of some levels. For each level, we need a function pointer that returns cpumask for each cpu, a function pointer that returns the flags for the level and a name. Only flags that describe topology, can be set by an architecture. The current topology flags are: SD_SHARE_CPUPOWER SD_SHARE_PKG_RESOURCES SD_NUMA SD_ASYM_PACKING Then, each level must be a subset on the next one. The build sequence of the sched_domain will take care of removing useless levels like those with 1 CPU and those with the same CPU span and no more relevant information for load balancing than its children. Signed-off-by:
-
Rik van Riel authored
Changing PTEs and PMDs to pte_numa & pmd_numa is done with the mmap_sem held for reading, which means a pmd can be instantiated and turned into a numa one while __handle_mm_fault() is examining the value of old_pmd. If that happens, __handle_mm_fault() should just return and let the page fault retry, instead of throwing an oops. This is handled by the test for pmd_trans_huge(*pmd) below. Signed-off-by:
Rik van Riel <riel@redhat.com> Reviewed-by:
Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Reported-by:
Sunil Pandey <sunil.k.pandey@intel.com> Signed-off-by:
-
Rik van Riel authored
Setting the numa_preferred_node for a task in task_numa_migrate does nothing on a 2-node system. Either we migrate to the node that already was our preferred node, or we stay where we were. On a 4-node system, it can slightly decrease overhead, by not calling the NUMA code as much. Since every node tends to be directly connected to every other node, running on the wrong node for a while does not do much damage. However, on an 8 node system, there are far more bad nodes than there are good ones, and pretending that a second choice is actually the preferred node can greatly delay, or even prevent, a workload from converging. The only time we can safely pretend that a second choice node is the preferred node is when the task is part of a workload that spans multiple NUMA nodes. Signed-off-by:
Vinod Chegu <chegu_vinod@hp.com> Acked-by:
Mel Gorman <mgorman@suse.de> Signed-off-by:
-
Rik van Riel authored
When tasks have not converged on their preferred nodes yet, we want to retry fairly often, to make sure we do not migrate a task's memory to an undesirable location, only to have to move it again later. This patch reduces the interval at which migration is retried, when the task's numa_scan_period is small. Signed-off-by:
-
Rik van Riel authored
The NUMA code is smart enough to distribute the memory of workloads that span multiple NUMA nodes across those NUMA nodes. However, it still has a pretty high scan rate for such workloads, because any memory that is left on a node other than the node of the CPU that faulted on the memory is counted as non-local, which causes the scan rate to go up. Counting the memory on any node where the task's numa group is actively running as local, allows the scan rate to slow down once the application is settled in. This should reduce the overhead of the automatic NUMA placement code, when a workload spans multiple NUMA nodes. Signed-off-by:
-
Ingo Molnar authored
Signed-off-by: -
Jason Low authored
Also initialize the per-sd variables for newidle load balancing in sd_numa_init(). Signed-off-by:
-
Jason Low authored
The following commit: e5fc6611 ("sched: Fix race in idle_balance()") can potentially cause rq->max_idle_balance_cost to not be updated, even when load_balance(NEWLY_IDLE) is attempted and the per-sd max cost value is updated. Preeti noticed a similar issue with updating rq->next_balance. In this patch, we fix this by making sure we still check/update those values even if a task gets enqueued while browsing the domains. Signed-off-by:
-
Peter Zijlstra authored
Tim wrote: "The current code will call pick_next_task_fair a second time in the slow path if we did not pull any task in our first try. This is really unnecessary as we already know no task can be pulled and it doubles the delay for the cpu to enter idle. We instrumented some network workloads and that saw that pick_next_task_fair is frequently called twice before a cpu enters idle. The call to pick_next_task_fair can add non trivial latency as it calls load_balance which runs find_busiest_group on an hierarchy of sched domains spanning the cpus for a large system. For some 4 socket systems, we saw almost 0.25 msec spent per call of pick_next_task_fair before a cpu can be idled." Optimize the second call away for the common case and document the dependency. Reported-by:
Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by:
-
Steven Rostedt (Red Hat) authored
The check at the beginning of cpupri_find() makes sure that the task_pri variable does not exceed the cp->pri_to_cpu array length. But that length is CPUPRI_NR_PRIORITIES not MAX_RT_PRIO, where it will miss the last two priorities in that array. As task_pri is computed from convert_prio() which should never be bigger than CPUPRI_NR_PRIORITIES, if the check should cause a panic if it is hit. Reported-by:
Mike Galbraith <umgwanakikbuti@gmail.com> Signed-off-by:
Steven Rostedt <rostedt@goodmis.org> Signed-off-by:
-
Li Zefan authored
Free cpudl->free_cpus allocated in cpudl_init(). Signed-off-by:
Li Zefan <lizefan@huawei.com> Acked-by:
Juri Lelli <juri.lelli@gmail.com> Signed-off-by:
-
Juri Lelli authored
yield_task_dl() is broken: o it forces current to be throttled setting its runtime to zero; o it sets current's dl_se->dl_new to one, expecting that dl_task_timer() will queue it back with proper parameters at replenish time. Unfortunately, dl_task_timer() has this check at the very beginning: if (!dl_task(p) || dl_se->dl_new) goto unlock; So, it just bails out and the task is never replenished. It actually yielded forever. To fix this, introduce a new flag indicating that the task properly yielded the CPU before its current runtime expired. While this is a little overdoing at the moment, the flag would be useful in the future to discriminate between "good" jobs (of which remaining runtime could be reclaimed, i.e. recycled) and "bad" jobs (for which dl_throttled task has been set) that needed to be stopped. Reported-by:
yjay.kim <yjay.kim@lge.com> Signed-off-by:
-
Thomas Gleixner authored
Russell reported, that irqtime_account_idle_ticks() takes ages due to: for (i = 0; i < ticks; i++) irqtime_account_process_tick(current, 0, rq); It's sad, that this code was written way _AFTER_ the NOHZ idle functionality was available. I charge myself guitly for not paying attention when that crap got merged with commit abb74cef ("sched: Export ns irqtimes through /proc/stat") So instead of looping nr_ticks times just apply the whole thing at once. As a side note: The whole cputime_t vs. u64 business in that context wants to be cleaned up as well. There is no point in having all these back and forth conversions. Lets standardise on u64 nsec for all kernel internal accounting and be done with it. Everything else does not make sense at all for fine grained accounting. Frederic, can you please take care of that? Reported-by:Russell King <rmk+kernel@arm.linux.org.uk> Signed-off-by:
Paul E. McKenney <paulmck@linux.vnet.ibm.com> Signed-off-by:
-
- 24 Apr, 2014 1 commit
-
-
Masanari Iida authored
When 'flags' argument to sched_{set,get}attr() syscalls were added in: 6d35ab48 ("sched: Add 'flags' argument to sched_{set,get}attr() syscalls") no description for 'flags' was added. It causes the following warnings on "make htmldocs": Warning(/kernel/sched/core.c:3645): No description found for parameter 'flags' Warning(/kernel/sched/core.c:3789): No description found for parameter 'flags' Signed-off-by:Masanari Iida <standby24x7@gmail.com> Cc: peterz@infradead.org Link: http://lkml.kernel.org/r/1397753955-2914-1-git-send-email-standby24x7@gmail.comSigned-off-by:
-
- 22 Apr, 2014 2 commits
-
-
git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-gpioLinus Torvalds authored
Pull gpio fixes from Linus Walleij: "A small batch of GPIO fixes for the v3.15 series. I expect more to come in but I'm a bit behind on mail, might as well get these to you right now: - Change a crucial semantic ordering in the GPIO irqchip helpers - Fix two nasty regressions in the ACPI gpiolib extensions" * tag 'gpio-v3.15-2' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-gpio: gpio / ACPI: Prevent potential wrap of GPIO value on OpRegion read gpio / ACPI: Don't crash on NULL chip->dev gpio: set data first, then chip and handler
-
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds authored
Pull x86 vdso fix from Peter Anvin: "This is a single build fix for building with gold as opposed to GNU ld. It got queued up separately and was expected to be pushed during the merge window, but it got left behind" * 'x86-vdso-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86, vdso: Make the vdso linker script compatible with Gold
-
- 21 Apr, 2014 2 commits
-
-
git://git.kernel.org/pub/scm/linux/kernel/git/rw/umlLinus Torvalds authored
Pull UML fixes from Richard Weinberger: "Assorted fixes for UML" * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml: um: Memory corruption on startup um: Missing pipe handling uml: Simplify tempdir logic.
-
git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4Linus Torvalds authored
Pull ext4 fixes from Ted Ts'o: "These are regression and bug fixes for ext4. We had a number of new features in ext4 during this merge window (ZERO_RANGE and COLLAPSE_RANGE fallocate modes, renameat, etc.) so there were many more regression and bug fixes this time around. It didn't help that xfstests hadn't been fully updated to fully stress test COLLAPSE_RANGE until after -rc1" * tag 'ext4_for_linus_stable' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4: (31 commits) ext4: disable COLLAPSE_RANGE for bigalloc ext4: fix COLLAPSE_RANGE failure with 1KB block size ext4: use EINVAL if not a regular file in ext4_collapse_range() ext4: enforce we are operating on a regular file in ext4_zero_range() ext4: fix extent merging in ext4_ext_shift_path_extents() ext4: discard preallocations after removing space ext4: no need to truncate pagecache twice in collapse range ext4: fix removing status extents in ext4_collapse_range() ext4: use filemap_write_and_wait_range() correctly in collapse range ext4: use truncate_pagecache() in collapse range ext4: remove temporary shim used to merge COLLAPSE_RANGE and ZERO_RANGE ext4: fix ext4_count_free_clusters() with EXT4FS_DEBUG and bigalloc enabled ext4: always check ext4_ext_find_extent result ext4: fix error handling in ext4_ext_shift_extents ext4: silence sparse check warning for function ext4_trim_extent ext4: COLLAPSE_RANGE only works on extent-based files ext4: fix byte order problems introduced by the COLLAPSE_RANGE patches ext4: use i_size_read in ext4_unaligned_aio() fs: disallow all fallocate operation on active swapfile fs: move falloc collapse range check into the filesystem methods ...
-
- 20 Apr, 2014 6 commits
-
-
Anton Ivanov authored
The reverse case of this race (you must msync before read) is well known. This is the not so common one. It can be triggered only on systems which do a lot of task switching and only at UML startup. If you are starting 200+ UMLs ~ 0.5% will always die without this fix. Signed-off-by:
Anton Ivanov <antivano@cisco.com> [rw: minor whitespace fixes] Signed-off-by:
Richard Weinberger <richard@nod.at>
-
Anton Ivanov authored
UML does not handle sigpipe. As a result when running it under expect or redirecting the IO from the console to an external program it will crash if the program stops or exits. Signed-off-by:
-
Tristan Schmelcher authored
Inferring the mount hierarchy correctly from /proc/mounts is hard when MS_MOVE may have been used, and the previous code did it wrongly. This change simplifies the logic to only require that /dev/shm be _on_ tmpfs (which can be checked trivially with statfs) rather than that it be a _mountpoint_ of tmpfs, since there isn't a compelling reason to be that strict. We also now check for tmpfs on whatever directory we ultimately use so that the user is better informed. This change also moves the more standard TMPDIR environment variable check ahead of the others. Applies to 3.12. Signed-off-by:
Tristan Schmelcher <tschmelcher@google.com> Signed-off-by:
-
Linus Torvalds authored
-
git://git.infradead.org/users/vkoul/slave-dmaLinus Torvalds authored
Pull slave-dmaengine fixes from Vinod Koul: "Back from long weekend here in India and now the time to send fixes for slave dmaengine. - Dan's fix of sirf xlate code - Jean's fix for timberland - edma fixes by Sekhar for SG handling and Yuan for changing init call" * 'fixes' of git://git.infradead.org/users/vkoul/slave-dma: dma: fix eDMA driver as a subsys_initcall dmaengine: sirf: off by one in of_dma_sirfsoc_xlate() platform: Fix timberdale dependencies dma: edma: fix incorrect SG list handling -
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommuLinus Torvalds authored
Pull iommu fixes from Joerg Roedel: "Fixes for regressions: - fix wrong IOMMU enumeration causing some SCSI device drivers initialization failures - ARM-SMMU fixes for a panic condition and a wrong return value" * tag 'iommu-fixes-v3.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: iommu/arm-smmu: fix panic in arm_smmu_alloc_init_pte iommu/arm-smmu: Return 0 on unmap failure iommu/vt-d: fix bug in matching PCI devices with DRHD/RMRR descriptors iommu/vt-d: Fix get_domain_for_dev() handling of upstream PCIe bridges iommu/vt-d: fix memory leakage caused by commit ea8ea460
-
