drm/amd/display: Use u16 for drm_bpp in DSC calculations
[Why] DSC calculations fail because the u16 bits_per_pixel from the DRM struct is being casted to the u8 drm_bpp parameters and locals. Integer wraparound is happening because this value is greater than 255. [How] Use u16 to match what's in the structure instead of u8. Signed-off-by: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com> Reviewed-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com> Acked-by: Qingqing Zhuo <qingqing.zhuo@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Showing
Please register or sign in to comment