drm/amd/display: fix shift-out-of-bounds in CalculateVMAndRowBytes
[WHY] When PTEBufferSizeInRequests is zero, UBSAN reports the following warning because dml_log2 returns an unexpected negative value: shift exponent 4294966273 is too large for 32-bit type 'int' [HOW] In the case PTEBufferSizeInRequests is zero, skip the dml_log2() and assign the result directly. Reviewed-by:Jun Lei <Jun.Lei@amd.com> Acked-by:
Qingqing Zhuo <qingqing.zhuo@amd.com> Signed-off-by:
Alex Hung <alex.hung@amd.com> Tested-by:
Daniel Wheeler <daniel.wheeler@amd.com> Signed-off-by:
Alex Deucher <alexander.deucher@amd.com>
Showing
Please register or sign in to comment