- 01 Feb, 2006 40 commits
-
-
Pekka Enberg authored
Fix kzalloc() and kstrdup() caller report for CONFIG_DEBUG_SLAB. We must pass the caller to __cache_alloc() instead of directly doing __builtin_return_address(0) there; otherwise kzalloc() and kstrdup() are reported as the allocation site instead of the real one. Thanks to Valdis Kletnieks for reporting the problem and Steven Rostedt for the original idea. Signed-off-by:
Pekka Enberg <penberg@cs.helsinki.fi> Signed-off-by:
Andrew Morton <akpm@osdl.org> Signed-off-by:
Linus Torvalds <torvalds@osdl.org>
-
Andrew Morton authored
Sometimes it's nice to know who's calling. Signed-off-by:
-
Pekka Enberg authored
Replace uses of kmem_cache_t with proper struct kmem_cache in mm/slab.c. Signed-off-by:
-
Pekka Enberg authored
Rename the ac_data() function to more descriptive cpu_cache_get(). Acked-by:
Manfred Spraul <manfred@colorfullife.com> Signed-off-by:
-
Pekka Enberg authored
Introduce virt_to_cache() and virt_to_slab() functions to reduce duplicate code and introduce a proper abstraction should we want to support other kind of mapping for address to slab and cache (eg. for vmalloc() or I/O memory). Acked-by:
-
Pekka Enberg authored
From: Manfred Spraul <manfred@colorfullife.com> Reduce the amount of inline functions in slab to the functions that are used in the hot path: - no inline for debug functions - no __always_inline, inline is already __always_inline - remove inline from a few numa support functions. Before: text data bss dec hex filename 13588 752 48 14388 3834 mm/slab.o (defconfig) 16671 2492 48 19211 4b0b mm/slab.o (numa) After: text data bss dec hex filename 13366 752 48 14166 3756 mm/slab.o (defconfig) 16230 2492 48 18770 4952 mm/slab.o (numa) Signed-off-by:
-
Matthew Dobson authored
Create two helper functions slab_get_obj() and slab_put_obj() to replace duplicated code in mm/slab.c Signed-off-by:
Matthew Dobson <colpatch@us.ibm.com> Acked-by:
-
Matthew Dobson authored
Create a helper function, slab_destroy_objs() which called from slab_destroy(). This makes slab_destroy() smaller and more readable, and moves ifdefs outside the function body. Signed-off-by:
-
Steven Rostedt authored
Clean up cache_estimate() in mm/slab.c and improves the algorithm from O(n) to O(1). We first calculate the maximum number of objects a slab can hold after struct slab and kmem_bufctl_t for each object has been given enough space. After that, to respect alignment rules, we decrease the number of objects if necessary. As required padding is at most align-1 and memory of obj_size is at least align, it is always enough to decrease number of objects by one. The optimization was originally made by Balbir Singh with more improvements from Steven Rostedt. Manfred Spraul provider further modifications: no loop at all for the off-slab case and added comments to explain the background. Acked-by:
Balbir Singh <bsingharora@gmail.com> Signed-off-by:
Steven Rostedt <rostedt@goodmis.org> Signed-off-by:
-
Steven Rostedt authored
I noticed the code for index_of is a creative way of finding the cache index using the compiler to optimize to a single hard coded number. But I couldn't help noticing that it uses two methods to let you know that someone used it wrong. One is at compile time (the correct way), and the other is at run time (not good). Signed-off-by:
-
Christoph Lameter authored
Clean up kmem_cache_alloc_node a bit. Signed-off-by:
Christoph Lameter <clameter@sgi.com> Acked-by:
-
Manfred Spraul authored
An object cache has two different object lengths: - the amount of memory available for the user (object size) - the amount of memory allocated internally (buffer size) This patch does some renames to make the code reflect that better. Signed-off-by:
-
Christoph Lameter authored
Migrate a page with buffers without requiring writeback This introduces a new address space operation migratepage() that may be used by a filesystem to implement its own version of page migration. A version is provided that migrates buffers attached to pages. Some filesystems (ext2, ext3, xfs) are modified to utilize this feature. The swapper address space operation are modified so that a regular migrate_page() will occur for anonymous pages without writeback (migrate_pages forces every anonymous page to have a swap entry). Signed-off-by:
Mike Kravetz <kravetz@us.ibm.com> Signed-off-by:
-
Christoph Lameter authored
Modify policy layer to support direct page migration - Add migrate_pages_to() allowing the migration of a list of pages to a a specified node or to vma with a specific allocation policy in sets of MIGRATE_CHUNK_SIZE pages - Modify do_migrate_pages() to do a staged move of pages from the source nodes to the target nodes. Signed-off-by:
Paul Jackson <pj@sgi.com> Signed-off-by:
-
Christoph Lameter authored
Add remove_from_swap remove_from_swap() allows the restoration of the pte entries that existed before page migration occurred for anonymous pages by walking the reverse maps. This reduces swap use and establishes regular pte's without the need for page faults. Signed-off-by:
-
Christoph Lameter authored
Add direct migration support with fall back to swap. Direct migration support on top of the swap based page migration facility. This allows the direct migration of anonymous pages and the migration of file backed pages by dropping the associated buffers (requires writeout). Fall back to swap out if necessary. The patch is based on lots of patches from the hotplug project but the code was restructured, documented and simplified as much as possible. Note that an additional patch that defines the migrate_page() method for filesystems is necessary in order to avoid writeback for anonymous and file backed pages. Signed-off-by:
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Signed-off-by:
Alexey Dobriyan <adobriyan@gmail.com> Signed-off-by:
-
Christoph Lameter authored
Check for PageSwapCache after looking up and locking a swap page. The page migration code may change a swap pte to point to a different page under lock_page(). If that happens then the vm must retry the lookup operation in the swap space to find the correct page number. There are a couple of locations in the VM where a lock_page() is done on a swap page. In these locations we need to check afterwards if the page was migrated. If the page was migrated then the old page that was looked up before was freed and no longer has the PageSwapCache bit set. Signed-off-by:
Hirokazu Takahashi <taka@valinux.co.jp> Signed-off-by:
Dave Hansen <haveblue@us.ibm.com> Signed-off-by: Christoph Lameter <clameter@@sgi.com> Signed-off-by:
-
Christoph Lameter authored
If large amounts of zone memory are used by empty slabs then zone_reclaim becomes uneffective. This patch shakes the slab a bit. The problem with this patch is that the slab reclaim is not containable to a zone. Thus slab reclaim may affect the whole system and be extremely slow. This also means that we cannot determine how many pages were freed in this zone. Thus we need to go off node for at least one allocation. The functionality is disabled by default. We could modify the shrinkers to take a zone parameter but that would be quite invasive. Better ideas are welcome. Signed-off-by:
-
Christoph Lameter authored
In some situations one may want zone_reclaim to behave differently. For example a process writing large amounts of memory will spew unto other nodes to cache the writes if many pages in a zone become dirty. This may impact the performance of processes running on other nodes. Allowing writes during reclaim puts a stop to that behavior and throttles the process by restricting the pages to the local zone. Similarly one may want to contain processes to local memory by enabling regular swap behavior during zone_reclaim. Off node memory allocation can then be controlled through memory policies and cpusets. Signed-off-by:
-
Christoph Lameter authored
Currently the zone_reclaim code has a fixed window of 30 seconds of off node allocations should a local zone have no unused pagecache pages left. Reclaim will be attempted again after this timeout period to avoid repeated useless scans for memory. This is also useful to established sufficiently large off node allocation chunks to relieve the local node. It may be beneficial to adjust that time period for some special situations. For example if memory use was exceeding node capacity one may want to give up for longer periods of time. If memory spikes intermittendly then one may want to shorten the time period to reduce the number of off node allocations. This patch allows just that.... Signed-off-by:
-
Christoph Lameter authored
Instead of scanning all the pages in a zone, imitate real swap and scan only a portion of the pages and gradually scan more if we do not free up enough pages. This avoids a zone suddenly loosing all unused pagecache pages (we may after all access some of these again so they deserve another chance) but it still frees up large chunks of memory if a zone only contains unused pagecache pages. Signed-off-by:
-
Christoph Lameter authored
zone_reclaim should leave that to the real swapper. We are only interested in evicting unmapped pages. Signed-off-by:
-
Hugh Dickins authored
2.6.15's hugepage faulting introduced huge_pages_needed accounting into hugetlbfs: to count how many pages are already in cache, for spot check on how far a new mapping may be allowed to extend the file. But it's muddled: each hugepage found covers HPAGE_SIZE, not PAGE_SIZE. Once pages were already in cache, it would overshoot, wrap its hugepages count backwards, and so fail a harmless repeat mapping with -ENOMEM. Fixes the problem found by Don Dupuis. Signed-off-by:
Hugh Dickins <hugh@veritas.com> Acked-By:
Adam Litke <agl@us.ibm.com> Acked-by:
William Irwin <wli@holomorphy.com> Signed-off-by:
-
Benjamin LaHaise authored
Improve the performance of slab_put_obj(). Without the cast, gcc considers ptrdiff_t a 64 bit signed integer and ends up emitting code to use a full signed 128 bit divide on EM64T, which is substantially slower than a 32 bit unsigned divide. I noticed this when looking at the profile of a case where the slab balance is just on edge and thrashes back and forth freeing a block. Signed-off-by:
Benjamin LaHaise <benjamin.c.lahaise@intel.com> Signed-off-by:
-
Christoph Lameter authored
- If we only reclaim nr_pages then its okay to stay on node. Switch from > to >= for the comparison. - vm_table[] entry for zone_reclaim_mode is a bit screwed up. - Add empty lines around shrink_zone to show that this is the central function to be called. Signed-off-by:
-
Christoph Lameter authored
Make sc->may_writepage control the writeout behavior of shrink_list. Remove the laptop_mode trick from shrink_list and instead set may_writepage in try_to_free_pages properly. Signed-off-by:
-
Andy Whitcroft authored
GFP_ZONETYPES calculate from GFP_ZONEMASK GFP_ZONETYPES's value is directly related to the value of GFP_ZONEMASK. It takes one of two forms depending whether the top bit of GFP_ZONEMASK is a 'loner'. Supply both forms, enabling the loner. Signed-off-by:
Andy Whitcroft <apw@shadowen.org> Signed-off-by:
-
Andy Whitcroft authored
GFP_ZONETYPES define using GFP_ZONEMASK and add commentry Add commentry explaining the optimisation that we can apply to GFP_ZONETYPES when the leftmost bit is a 'loaner', it can only be set in isolation. Signed-off-by:
-
Christoph Lameter authored
Zone reclaim is usually only run on the local node. Headless nodes do not have any local processors. This patch checks for headless nodes and performs zone reclaim on them. Signed-off-by:
-
Christoph Lameter authored
Ensure that the performance of off node pages stays the same as before. Off node pagefault tests showed an 18% drop in performance without this patch. - Increase the timeout to 30 seconds to reduce the overhead. - Move all code possible out of the off node hot path for zone reclaim (Sorry Andrew, the struct initialization had to be sacrificed). The read_page_state() bit us there. - Check first for the timeout before any other checks. Signed-off-by:
-
Alexey Dobriyan authored
OpenBSD doesn't see "." correctly in directories created by Linux. Copying files over several KB will buy you infinite loop in __getblk_slow(). Copying files smaller than 1 KB seems to be OK. Sometimes files will be filled with zeros. Sometimes incorrectly copied file will reappear after next file with truncated size. Signed-off-by:
-
David Gibson authored
The flattened device tree is the only supported way of booting ARCH=powerpc kernels on non Open Firmware machines. The documentation for the flattened tree format and contents has been discussed on mailing lists and lately has been living in the dtc git tree. Really, it ought to go in the kernel's Documentation directory for maximum visibility. Signed-off-by:
David Gibson <dwg@au1.ibm.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by:
-
Rafael J. Wysocki authored
Prevent the kernel from setting the log level to 10 unconditionally during suspend/resume which was needed in the past for debugging, but generally is undesirable. Signed-off-by:
Rafael J. Wysocki <rjw@sisk.pl> Acked-by:
Pavel Machek <pavel@suse.cz> Signed-off-by:
-
Dave Jones authored
Whoops. kobject_register failed for hexium HV-PCI6/Orion (-13) [<c01d3eb6>] kobject_register+0x31/0x47 [<c023a996>] bus_add_driver+0x4a/0xfd [<c01de3c1>] __pci_register_driver+0x82/0xa4 [<d083400a>] hexium_init_module+0xa/0x47 [hexium_orion] [<c013bdae>] sys_init_module+0x167b/0x1822 [<c01633f7>] do_sync_read+0xb8/0xf3 [<c0133fa3>] autoremove_wake_function+0x0/0x2d [<c0145390>] audit_syscall_entry+0x118/0x13f [<c0106ae2>] do_syscall_trace+0x104/0x14a [<c0103d21>] syscall_call+0x7/0xb slashes in kobject names aren't allowed. Signed-off-by:
Dave Jones <davej@redhat.com> Cc: Mauro Carvalho Chehab <mchehab@infradead.org> Signed-off-by:
-
V. Ananda Krishnan authored
Scott Kilau <Scott_Kilau@digi.com> Digi serial port console doesn't work when baud rates are set higher than 38400. So the lookup table and code in jsm_neo.c has been modified and tested. Please let me have the feed-back. Signed-off-by:V.Ananda Krishnan <mansarov@us.ibm.com> Cc: Alan Cox <alan@lxorguk.ukuu.org.uk> Signed-off-by:
-
john stultz authored
Avoid lost tick compensation early in boot before the TSCs are synchronized. Currently timekeeping is enabled before the TSCs are synchronized, thus when the TSCs are synched (reset to zero), it appears that a number of lost ticks have occurred. This can cause premature expiry of timers and in extreme cases can cause the soft lockup detection to fire. This resolves issues reported by Andy Whitcroft as well as bug #5366 reported by Tim Mann. Signed-off-by:
John Stultz <johnstul@us.ibm.com> Acked-by:
-
Jack Steiner authored
Change sched_getaffinity() so that it returns a bitmap that indicates the legally schedulable cpus that a task is allowed to run on. Without this patch, if CONFIG_HOTPLUG_CPU is enabled, sched_getaffinity() unconditionally returns (at least on IA64) a mask with NR_CPUS bits set. This conveys no useful infornmation except for a kernel compile option. This fixes a breakage we obseved running recent kernels. We have MPI jobs that use sched_getaffinity() to determine where to place their threads. Placing them on non-existant cpus is problematic :-) Signed-off-by:
Jack Steiner <steiner@sgi.com> Acked-by:
Ingo Molnar <mingo@elte.hu> Cc: Nathan Lynch <ntl@pobox.com> Cc: Paul Jackson <pj@sgi.com> Signed-off-by:
-
Bryan O'Sullivan authored
This assembly version is measurably faster than the generic version in lib/iomap_copy.c. Signed-off-by:
Bryan O'Sullivan <bos@pathscale.com> Cc: Andi Kleen <ak@muc.de> Signed-off-by:
-
Bryan O'Sullivan authored
This arch-independent routine copies data to a memory-mapped I/O region, using 32-bit accesses. The naming is double-underscored to make it clear that it does not guarantee write ordering, nor does it perform a memory barrier afterwards; the kernel doc also explicitly states this. This style of access is required by some devices. This change also introduces include/linux/io.h, at Andrew's suggestion. It only has one occupant at the moment, but is a logical destination for oft-replicated contents of include/asm-*/{io,iomap}.h to migrate to. Signed-off-by: -
Bryan O'Sullivan authored
This can make the intent behind some arithmetic expressions clearer. Signed-off-by:
Adrian Bunk <bunk@stusta.de> Signed-off-by:
-
