• Sourabh Jain's avatar
    powerpc: make fadump resilient with memory add/remove events · c6c5b14d
    Sourabh Jain authored
    Due to changes in memory resources caused by either memory hotplug or
    online/offline events, the elfcorehdr, which describes the CPUs and
    memory of the crashed kernel to the kernel that collects the dump (known
    as second/fadump kernel), becomes outdated. Consequently, attempting
    dump collection with an outdated elfcorehdr can lead to failed or
    inaccurate dump collection.
    
    Memory hotplug or online/offline events is referred as memory add/remove
    events in reset of the commit message.
    
    The current solution to address the aforementioned issue is as follows:
    Monitor memory add/remove events in userspace using udev rules, and
    re-register fadump whenever there are changes in memory resources. This
    leads to the creation of a new elfcorehdr with updated system memory
    information.
    
    There are several notable issues associated with re-registering fadump
    for every memory add/remove events.
    
    1. Bulk memory add/remove events with udev-based fadump re-registration
       can lead to race conditions and, more importantly, it creates a wide
       window during which fadump is inactive until all memory add/remove
       events are settled.
    2. Re-registering fadump for every memory add/remove event is
       inefficient.
    3. The memory for elfcorehdr is allocated based on the memblock regions
       available during early boot and remains fixed thereafter. However, if
       elfcorehdr is later recreated with additional memblock regions, its
       size will increase, potentially leading to memory corruption.
    
    Address the aforementioned challenges by shifting the creation of
    elfcorehdr from the first kernel (also referred as the crashed kernel),
    where it was created and frequently recreated for every memory
    add/remove event, to the fadump kernel. As a result, the elfcorehdr only
    needs to be created once, thus eliminating the necessity to re-register
    fadump during memory add/remove events.
    
    At present, the first kernel prepares fadump header and stores it in the
    fadump reserved area. The fadump header includes the start address of
    the elfcorehdr, crashing CPU details, and other relevant information. In
    the event of a crash in the first kernel, the second/fadump boots and
    accesses the fadump header prepared by the first kernel. It then
    performs the following steps in a platform-specific function
    [rtas|opal]_fadump_process:
    
    1. Sanity check for fadump header
    2. Update CPU notes in elfcorehdr
    
    Along with the above, update the setup_fadump()/fadump.c to create
    elfcorehdr and set its address to the global variable elfcorehdr_addr
    for the vmcore module to process it in the second/fadump kernel.
    
    Section below outlines the information required to create the elfcorehdr
    and the changes made to make it available to the fadump kernel if it's
    not already.
    
    To create elfcorehdr, the following crashed kernel information is
    required: CPU notes, vmcoreinfo, and memory ranges.
    
    At present, the CPU notes are already prepared in the fadump kernel, so
    no changes are needed in that regard. The fadump kernel has access to
    all crashed kernel memory regions, including boot memory regions that
    are relocated by firmware to fadump reserved areas, so no changes for
    that either. However, it is necessary to add new members to the fadump
    header, i.e., the 'fadump_crash_info_header' structure, in order to pass
    the crashed kernel's vmcoreinfo address and its size to fadump kernel.
    
    In addition to the vmcoreinfo address and size, there are a few other
    attributes also added to the fadump_crash_info_header structure.
    
    1. version:
       It stores the fadump header version, which is currently set to 1.
       This provides flexibility to update the fadump crash info header in
       the future without changing the magic number. For each change in the
       fadump header, the version will be increased. This will help the
       updated kernel determine how to handle kernel dumps from older
       kernels. The magic number remains relevant for checking fadump header
       corruption.
    
    2. pt_regs_sz/cpu_mask_sz:
       Store size of pt_regs and cpu_mask structure of first kernel. These
       attributes are used to prevent dump processing if the sizes of
       pt_regs or cpu_mask structure differ between the first and fadump
       kernels.
    
    Note: if either first/crashed kernel or second/fadump kernel do not have
    the changes introduced here then kernel fail to collect the dump and
    prints relevant error message on the console.
    Signed-off-by: default avatarSourabh Jain <sourabhjain@linux.ibm.com>
    Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
    Link: https://msgid.link/20240422195932.1583833-2-sourabhjain@linux.ibm.com
    c6c5b14d
rtas-fadump.c 15.5 KB