1. 27 Jan, 2009 9 commits
  2. 26 Jan, 2009 1 commit
  3. 23 Jan, 2009 7 commits
  4. 21 Jan, 2009 18 commits
    • Nick Piggin's avatar
      x86: make UV support configurable · 03b48632
      Nick Piggin authored
      Make X86 SGI Ultraviolet support configurable. Saves about 13K of text size
      on my modest config.
      
         text    data     bss     dec     hex filename
      6770537 1158680  694356 8623573  8395d5 vmlinux
      6757492 1157664  694228 8609384  835e68 vmlinux.nouv
      Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      03b48632
    • Ingo Molnar's avatar
      x86: uv cleanup, build fix #2 · 5b221278
      Ingo Molnar authored
      Fix more build-failure fallout from the UV cleanup - the UV drivers
      were not updated to include <asm/uv/uv.h>.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      5b221278
    • Ingo Molnar's avatar
      x86: make x86_32 use tlb_64.c, build fix, clean up X86_L1_CACHE_BYTES · ace6c6c8
      Ingo Molnar authored
      Fix:
      
        arch/x86/mm/tlb.c:47: error: ‘CONFIG_X86_INTERNODE_CACHE_BYTES’ undeclared here (not in a function)
      
      The CONFIG_X86_INTERNODE_CACHE_BYTES symbol is only defined on 64-bit,
      because vsmp support is 64-bit only. Define it on 32-bit too - where it
      will always be equal to X86_L1_CACHE_BYTES.
      
      Also move the default of X86_L1_CACHE_BYTES (which is separate from the
      more commonly used L1_CACHE_SHIFT kconfig symbol) from 128 bytes to
      64 bytes.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      ace6c6c8
    • Ingo Molnar's avatar
      Merge branch 'x86/mm' into core/percpu · 19803078
      Ingo Molnar authored
      Conflicts:
      	arch/x86/mm/fault.c
      19803078
    • Ingo Molnar's avatar
      x86: uv cleanup, build fix · 4ec71fa2
      Ingo Molnar authored
      Fix:
      
       arch/x86/mm/srat_64.c: In function ‘acpi_numa_processor_affinity_init’:
       arch/x86/mm/srat_64.c:141: error: implicit declaration of function ‘get_uv_system_type’
       arch/x86/mm/srat_64.c:141: error: ‘UV_X2APIC’ undeclared (first use in this function)
       arch/x86/mm/srat_64.c:141: error: (Each undeclared identifier is reported only once
       arch/x86/mm/srat_64.c:141: error: for each function it appears in.)
      
      A couple of UV definitions were moved to asm/uv/uv.h, but srat_64.c did
      not include that header. Add it.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      4ec71fa2
    • Ingo Molnar's avatar
      x86, mm: move tlb.c to arch/x86/mm/ · 55f4949f
      Ingo Molnar authored
      Impact: cleanup
      
      Now that it's unified, move the (SMP) TLB flushing code from arch/x86/kernel/
      to arch/x86/mm/, where it belongs logically.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      55f4949f
    • Ingo Molnar's avatar
      Merge branch 'cpus4096' into core/percpu · 3eb3963f
      Ingo Molnar authored
      Conflicts:
      	arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.c
      	arch/x86/kernel/tlb_32.c
      
      Merge it here because both the cpumask changes and the ongoing percpu
      work is touching the TLB code. The percpu changes take precedence, as
      they eliminate tlb_32.c altogether.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      3eb3963f
    • Ingo Molnar's avatar
    • Tejun Heo's avatar
      x86: rename tlb_64.c to tlb.c · 16c2d3f8
      Tejun Heo authored
      Impact: file rename
      
      tlb_64.c is now the tlb code for both 32 and 64.  Rename it to tlb.c.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      16c2d3f8
    • Tejun Heo's avatar
      x86: make x86_32 use tlb_64.c · 02cf94c3
      Tejun Heo authored
      Impact: less contention when issuing invalidate IPI, cleanup
      
      Make x86_32 use the same tlb code as 64bit.  The 64bit code uses
      multiple IPI vectors for tlb shootdown to reduce contention.  This
      patch makes x86_32 allocate the same 8 IPIs as x86_64 and share the
      code paths.
      
      Note that the usage of asmlinkage is inconsistent for x86_32 and 64
      and calls for further cleanup.  This has been noted with a FIXME
      comment in tlb_64.c.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      02cf94c3
    • Tejun Heo's avatar
      x86: prepare for tlb merge · 6dd01bed
      Tejun Heo authored
      Impact: clean up, ipi vector number reordering for x86_32
      
      Make the following changes to prepare for tlb merge.
      
      * reorder x86_32 ip vectors
      
      * adjust tlb_32.c and tlb_64.c such that their logics coincide exactly
      	- on spurious invalidate ipi, tlb_32 acks the irq
      	- tlb_64 now has proper memory barriers around clearing
                flush_cpumask (no change in generated code)
      
      * unexport flush_tlb_page from tlb_32.c, there's no user
      
      * use unsigned int for cpu id
      
      * drop unnecessary includes from tlb_64.c
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      6dd01bed
    • Tejun Heo's avatar
      x86: uv cleanup · bdbcdd48
      Tejun Heo authored
      Impact: cleanup
      
      Make the following uv related cleanups.
      
      * collect visible uv related definitions and interfaces into uv/uv.h
        and use it.  this cleans up the messy situation where on 64bit, uv
        is defined properly, on 32bit generic it's dummy and on the rest
        undefined.  after this clean up, uv is defined on 64 and dummy on
        32.
      
      * update uv_flush_tlb_others() such that it takes cpumask of
        to-be-flushed cpus as argument, instead of that minus self, and
        returns yet-to-be-flushed cpumask, instead of modifying the passed
        in parameter.  this interface change will ease dummy implementation
        of uv_flush_tlb_others() and makes uv tlb flush related stuff
        defined in tlb_uv proper.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      bdbcdd48
    • Brian Gerst's avatar
      x86: merge irq_regs.h · d650a514
      Brian Gerst authored
      Impact: cleanup, better irq_regs code generation for x86_64
      
      Make 64-bit use the same optimizations as 32-bit.
      Signed-off-by: default avatarBrian Gerst <brgerst@gmail.com>
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      d650a514
    • Brian Gerst's avatar
      x86: merge mmu_context.h · 6826c8ff
      Brian Gerst authored
      Impact: cleanup
      
      tj: * changed cpu to unsigned as was done on mmu_context_64.h as cpu
            id is officially unsigned int
          * added missing ';' to 32bit version of deactivate_mm()
      Signed-off-by: default avatarBrian Gerst <brgerst@gmail.com>
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      6826c8ff
    • Brian Gerst's avatar
      x86: set %fs to __KERNEL_PERCPU unconditionally for x86_32 · 0dd76d73
      Brian Gerst authored
      Impact: cleanup
      
      %fs is currently set to __KERNEL_DS at boot, and conditionally
      switched to __KERNEL_PERCPU for secondary cpus.  Instead, initialize
      GDT_ENTRY_PERCPU to the same attributes as GDT_ENTRY_KERNEL_DS and
      set %fs to __KERNEL_PERCPU unconditionally.
      Signed-off-by: default avatarBrian Gerst <brgerst@gmail.com>
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      0dd76d73
    • Brian Gerst's avatar
      x86: fix percpu_write with 64-bit constants · 299e2699
      Brian Gerst authored
      Impact: slightly better code generation for percpu_to_op()
      
      The processor will sign-extend 32-bit immediate values in 64-bit
      operations.  Use the 'e' constraint ("32-bit signed integer constant,
      or a symbolic reference known to fit that range") for 64-bit constants.
      Signed-off-by: default avatarBrian Gerst <brgerst@gmail.com>
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      299e2699
    • Brian Gerst's avatar
      x86: clean up gdt_page definition · 06deef89
      Brian Gerst authored
      Impact: cleanup && more compact percpu area layout with future changes
      
      Move 64-bit GDT to page-aligned section and clean up comment
      formatting.
      Signed-off-by: default avatarBrian Gerst <brgerst@gmail.com>
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      06deef89
    • Tejun Heo's avatar
      x86: update canary handling during switch · 67e68bde
      Tejun Heo authored
      Impact: cleanup
      
      In switch_to(), instead of taking offset to irq_stack_union.stack,
      make it a proper percpu access using __percpu_arg() and per_cpu_var().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      67e68bde
  5. 20 Jan, 2009 5 commits
    • Nick Piggin's avatar
      x86: optimise x86's do_page_fault (C entry point for the page fault path) · 92181f19
      Nick Piggin authored
      Impact: cleanup, restructure code to improve assembly
      
      gcc isn't _all_ that smart about spilling registers to stack or reusing
      stack slots, even with branch annotations. do_page_fault contained a lot
      of functionality, so split unlikely paths into their own functions, and
      mark them as noinline just to be sure. I consider this actually to be
      somewhat of a cleanup too: the main function now contains about half
      the number of lines so the normal path is easier to read, while the error
      cases are also nicely split away.
      
      Also, ensure the order of arguments to functions is always the same: regs,
      addr, error_code. This can reduce code size a tiny bit, and just looks neater
      too.
      
      And add a couple of branch annotations.
      
      Before:
        do_page_fault:
                subq    $360, %rsp      #,
      
      After:
        do_page_fault:
                subq    $56, %rsp       #,
      
      bloat-o-meter:
        add/remove: 8/0 grow/shrink: 0/1 up/down: 2222/-1680 (542)
        function                                     old     new   delta
        __bad_area_nosemaphore                         -     506    +506
        no_context                                     -     474    +474
        vmalloc_fault                                  -     424    +424
        spurious_fault                                 -     358    +358
        mm_fault_error                                 -     272    +272
        bad_area_access_error                          -      89     +89
        bad_area                                       -      89     +89
        bad_area_nosemaphore                           -      10     +10
        do_page_fault                               2464     784   -1680
      
      Yes, the total size increases by 542 bytes, due to the extra function calls.
      But these will very rarely be called (except for vmalloc_fault) in a normal
      workload. Importantly, do_page_fault is less than 1/3rd it's original size,
      and touches far less stack.
      
      Existing gotos and branch hints did move a lot of the infrequently used text
      out of the fastpath, but that's even further improved after this patch.
      Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
      Acked-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      92181f19
    • Ingo Molnar's avatar
      Merge commit 'v2.6.29-rc2' into x86/mm · 0ce1c383
      Ingo Molnar authored
      0ce1c383
    • Ingo Molnar's avatar
      x86, cpumask: fix tlb flush race · 5766b842
      Ingo Molnar authored
      Impact: fix bootup crash
      
      The cpumask is now passed in as a reference to mm->cpu_vm_mask, not on
      the stack - hence it is not constant anymore during the TLB flush.
      
      That way it could race and some static sanity checks would trigger:
      
      [  238.154287] ------------[ cut here ]------------
      [  238.156039] kernel BUG at arch/x86/kernel/tlb_32.c:130!
      [  238.156039] invalid opcode: 0000 [#1] SMP
      [  238.156039] last sysfs file: /sys/class/net/eth2/address
      [  238.156039] Modules linked in:
      [  238.156039]
      [  238.156039] Pid: 6493, comm: ifup-eth Not tainted (2.6.29-rc2-tip #1) P4DC6
      [  238.156039] EIP: 0060:[<c0118f87>] EFLAGS: 00010202 CPU: 2
      [  238.156039] EIP is at native_flush_tlb_others+0x35/0x158
      [  238.156039] EAX: c0ef972c EBX: f6143301 ECX: 00000000 EDX: 00000000
      [  238.156039] ESI: f61433a8 EDI: f6143200 EBP: f34f3e00 ESP: f34f3df0
      [  238.156039]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
      [  238.156039] Process ifup-eth (pid: 6493, ti=f34f2000 task=f399ab00 task.ti=f34f2000)
      [  238.156039] Stack:
      [  238.156039]  ffffffff f61433a8 ffffffff f6143200 f34f3e18 c0118e9c 00000000 f6143200
      [  238.156039]  f61433a8 f5bec738 f34f3e28 c0119435 c2b5b830 f6143200 f34f3e34 c01c2dc3
      [  238.156039]  bffd9000 f34f3e60 c01c3051 00000000 ffffffff f34f3e4c 00000000 00000071
      [  238.156039] Call Trace:
      [  238.156039]  [<c0118e9c>] ? flush_tlb_others+0x52/0x5b
      [  238.156039]  [<c0119435>] ? flush_tlb_mm+0x7f/0x8b
      [  238.156039]  [<c01c2dc3>] ? tlb_finish_mmu+0x2d/0x55
      [  238.156039]  [<c01c3051>] ? exit_mmap+0x124/0x170
      [  238.156039]  [<c013e965>] ? mmput+0x40/0xf5
      [  238.156039]  [<c01e4788>] ? flush_old_exec+0x640/0x94b
      [  238.156039]  [<c01ddb4e>] ? fsnotify_access+0x37/0x39
      [  238.156039]  [<c01e3435>] ? kernel_read+0x39/0x4b
      [  238.156039]  [<c021bc8a>] ? load_elf_binary+0x4a1/0x11bb
      [  238.156039]  [<c01c0af9>] ? might_fault+0x51/0x9c
      [  238.156039]  [<c010a2cc>] ? paravirt_read_tsc+0x20/0x4f
      [  238.156039]  [<c010a406>] ? native_sched_clock+0x5d/0x60
      [  238.156039]  [<c01e2fda>] ? search_binary_handler+0xab/0x2c4
      [  238.156039]  [<c021b7e9>] ? load_elf_binary+0x0/0x11bb
      [  238.156039]  [<c04ae9a5>] ? _raw_read_unlock+0x21/0x46
      [  238.156039]  [<c021b7e9>] ? load_elf_binary+0x0/0x11bb
      [  238.156039]  [<c01e2fe1>] ? search_binary_handler+0xb2/0x2c4
      [  238.156039]  [<c01e4076>] ? do_execve+0x21c/0x2ee
      [  238.156039]  [<c01029b7>] ? sys_execve+0x51/0x8c
      [  238.156039]  [<c0103eaf>] ? sysenter_do_call+0x12/0x43
      
      Fix it by not assuming that the cpumask is constant.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      5766b842
    • Ingo Molnar's avatar
    • Tejun Heo's avatar
      linker script: kill PERCPU_VADDR_PREALLOC() · 6b7c38d5
      Tejun Heo authored
      Impact: cleanup
      
      With .data.percpu.first in place, PERCPU_VADDR_PREALLOC() is no longer
      necessary.  Kill it.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      6b7c38d5