[PATCH] get_unmapped_area() speedup

From: Ingo Molnar

Apparently our thread-creation performance has gone down the tubes again,
because the mm.free_area_cache search heuristic broke.

The initial, more naive hole-cache patch helped the testcode in the
beginning.  Then after some point glibc started creating a 'small hole' in
the vmas, which hole was _below_ the thread stacks, and which hole thus
prevented the intended operation of the cache.

The new code solves the problem by relaxing the 'smallest address hole cache'
rule a bit, the cache is now not re-set at every get_unmapped_area() time,
it's only re-set during unmaps.  It's also re-set if there are no allocatable
mappings at all - ie.  correctness is not affected.

mm/mmap.c

@@ -777,7 +777,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr,
 {
 	struct mm_struct *mm = current->mm;
 	struct vm_area_struct *vma;
-	int found_hole = 0;
+	unsigned long start_addr;
 
 	if (len > TASK_SIZE)
 		return -ENOMEM;
@@ -789,21 +789,29 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr,
 		    (!vma || addr + len <= vma->vm_start))
 			return addr;
 	}
-	addr = mm->free_area_cache;
+	start_addr = addr = mm->free_area_cache;
 
+full_search:
 	for (vma = find_vma(mm, addr); ; vma = vma->vm_next) {
 		/* At this point:  (!vma || addr < vma->vm_end). */
-		if (TASK_SIZE - len < addr)
-			return -ENOMEM;
+		if (TASK_SIZE - len < addr) {
 			/*
-		 * Record the first available hole.
+			 * Start a new search - just in case we missed
+			 * some holes.
 			 */
-		if (!found_hole && (!vma || addr < vma->vm_start)) {
-			mm->free_area_cache = addr;
-			found_hole = 1;
+			if (start_addr != TASK_UNMAPPED_BASE) {
+				start_addr = addr = TASK_UNMAPPED_BASE;
+				goto full_search;
+			}
+			return -ENOMEM;
 		}
-		if (!vma || addr + len <= vma->vm_start)
+		if (!vma || addr + len <= vma->vm_start) {
+			/*
+			 * Remember the place where we stopped the search:
+			 */
+			mm->free_area_cache = addr + len;
 			return addr;
+		}
 		addr = vma->vm_end;
 	}
 }