drm/format-helper: Make conversion_buf_size() support sub-byte pixel fmts

There are DRM fourcc formats that have pixels smaller than a byte, but the
conversion_buf_size() function assumes that pixels are a multiple of bytes
and use the struct drm_format_info .cpp field to calculate the dst_pitch.

Instead, calculate it by using the bits per pixel (bpp) and divide it by 8
to account for formats that have sub-byte pixels.
Signed-off-by: default avatarJavier Martinez Canillas <javierm@redhat.com>
Reviewed-by: default avatarArthur Grillo <arthurgrillo@riseup.net>
Reviewed-by: default avatarMaíra Canal <mcanal@igalia.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230307215039.346863-1-javierm@redhat.com
parent 96c25b03
......@@ -409,12 +409,15 @@ static size_t conversion_buf_size(u32 dst_format, unsigned int dst_pitch,
const struct drm_rect *clip)
{
const struct drm_format_info *dst_fi = drm_format_info(dst_format);
unsigned int bpp;
if (!dst_fi)
return -EINVAL;
if (!dst_pitch)
dst_pitch = drm_rect_width(clip) * dst_fi->cpp[0];
if (!dst_pitch) {
bpp = drm_format_info_bpp(dst_fi, 0);
dst_pitch = DIV_ROUND_UP(drm_rect_width(clip) * bpp, 8);
}
return dst_pitch * drm_rect_height(clip);
}
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment