• Gleb Shchepa's avatar
    Bug #40761: Assert on sum function on · ffe23f0e
    Gleb Shchepa authored
                IF(..., CAST(longtext AS UNSIGNED), signed_val)
                (was: LEFT JOIN on inline view crashes server)
    
    Select from a LONGTEXT column wrapped with an expression
    like "IF(..., CAST(longtext_column AS UNSIGNED), smth_signed)"
    failed an assertion or crashed the server. IFNULL function was
    affected too.
    
    LONGTEXT column item has a maximum length of 32^2-1 bytes,
    at the same time this is a maximum possible length of any
    MySQL item. CAST(longtext_column AS UNSIGNED) returns some
    unsigned numeric result of length 32^2-1, so the result of
    IF/IFNULL function of this number and some other signed number
    will have text length of (32^2-1)+1=32^2 (one byte for the
    minus sign) - there is integer overflow, and the length is
    equal to zero. That caused assert/crash.
    
    The bug has been fixed by the same solution as in the CASE
    function implementation.
    
    
    mysql-test/r/func_if.result:
      Added test case for bug #40761.
    mysql-test/t/func_if.test:
      Added test case for bug #40761.
    sql/item_cmpfunc.cc:
      Bug #40761: Assert on sum function on
                  IF(..., CAST(longtext AS UNSIGNED), signed_val)
      
      1. Item_func_case::agg_str_lengths method has been moved
         to the Item_func superclass.
      2. Item_func_ifnull/Item_func_if::fix_length_and_dec methods
         have been updated to calculate max_length, decimals and
         unsigned flag like Item_func_case.
    sql/item_cmpfunc.h:
      Bug #40761: Assert on sum function on
                  IF(..., CAST(longtext AS UNSIGNED), signed_val)
      
      Item_func_case::agg_str_lengths method has been moved to
      the Item_func superclass.
    sql/item_func.cc:
      Bug #40761: Assert on sum function on
                  IF(..., CAST(longtext AS UNSIGNED), signed_val)
      
      Item_func_case::agg_str_lengths method has been moved to
      the Item_func superclass.
    sql/item_func.h:
      Bug #40761: Assert on sum function on
                  IF(..., CAST(longtext AS UNSIGNED), signed_val)
      
      Item_func_case::agg_str_lengths method has been moved to
      the Item_func superclass.
    ffe23f0e
item_func.h 44.7 KB