• Michael Kelley's avatar
    swiotlb: fix swiotlb_bounce() to do partial sync's correctly · e8068f2d
    Michael Kelley authored
    In current code, swiotlb_bounce() may do partial sync's correctly in
    some circumstances, but may incorrectly fail in other circumstances.
    The failure cases require both of these to be true:
    
    1) swiotlb_align_offset() returns a non-zero "offset" value
    2) the tlb_addr of the partial sync area points into the first
    "offset" bytes of the _second_ or subsequent swiotlb slot allocated
    for the mapping
    
    Code added in commit 868c9ddc ("swiotlb: add overflow checks
    to swiotlb_bounce") attempts to WARN on the invalid case where
    tlb_addr points into the first "offset" bytes of the _first_
    allocated slot. But there's no way for swiotlb_bounce() to distinguish
    the first slot from the second and subsequent slots, so the WARN
    can be triggered incorrectly when #2 above is true.
    
    Related, current code calculates an adjustment to the orig_addr stored
    in the swiotlb slot. The adjustment compensates for the difference
    in the tlb_addr used for the partial sync vs. the tlb_addr for the full
    mapping. The adjustment is stored in the local variable tlb_offset.
    But when #1 and #2 above are true, it's valid for this adjustment to
    be negative. In such case the arithmetic to adjust orig_addr produces
    the wrong result due to tlb_offset being declared as unsigned.
    
    Fix these problems by removing the over-constraining validations added
    in 868c9ddc. Change the declaration of tlb_offset to be signed
    instead of unsigned so the adjustment arithmetic works correctly.
    
    Tested with a test-only hack to how swiotlb_tbl_map_single() calls
    swiotlb_bounce(). Instead of calling swiotlb_bounce() just once
    for the entire mapped area, do a loop with each iteration doing
    only a 128 byte partial sync until the entire mapped area is
    sync'ed. Then with swiotlb=force on the kernel boot line, run a
    variety of raw disk writes followed by read and verification of
    all bytes of the written data. The storage device has DMA
    min_align_mask set, and the writes are done with a variety of
    original buffer memory address alignments and overall buffer
    sizes. For many of the combinations, current code triggers the
    WARN statements, or the data verification fails. With the fixes,
    no WARNs occur and all verifications pass.
    
    Fixes: 5f89468e ("swiotlb: manipulate orig_addr when tlb_addr has offset")
    Fixes: 868c9ddc ("swiotlb: add overflow checks to swiotlb_bounce")
    Signed-off-by: default avatarMichael Kelley <mhklinux@outlook.com>
    Dominique Martinet <dominique.martinet@atmark-techno.com>
    Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
    e8068f2d
swiotlb.c 51 KB