Commit 2b620729 authored by Dan Carpenter's avatar Dan Carpenter Committed by Daniel Vetter

drm/dumb-buffers: Integer overflow in drm_mode_create_ioctl()

There is a comment here which says that DIV_ROUND_UP() and that's where
the problem comes from.  Say you pick:

	args->bpp = UINT_MAX - 7;
	args->width = 4;
	args->height = 1;

The integer overflow in DIV_ROUND_UP() means "cpp" is UINT_MAX / 8 and
because of how we picked args->width that means cpp < UINT_MAX / 4.

I've fixed it by preventing the integer overflow in DIV_ROUND_UP().  I
removed the check for !cpp because it's not possible after this change.
I also changed all the 0xffffffffU references to U32_MAX.
Signed-off-by: default avatarDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: default avatarDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20180516140026.GA19340@mwanda
parent 72cb0d89
...@@ -65,12 +65,13 @@ int drm_mode_create_dumb_ioctl(struct drm_device *dev, ...@@ -65,12 +65,13 @@ int drm_mode_create_dumb_ioctl(struct drm_device *dev,
return -EINVAL; return -EINVAL;
/* overflow checks for 32bit size calculations */ /* overflow checks for 32bit size calculations */
/* NOTE: DIV_ROUND_UP() can overflow */ if (args->bpp > U32_MAX - 8)
return -EINVAL;
cpp = DIV_ROUND_UP(args->bpp, 8); cpp = DIV_ROUND_UP(args->bpp, 8);
if (!cpp || cpp > 0xffffffffU / args->width) if (cpp > U32_MAX / args->width)
return -EINVAL; return -EINVAL;
stride = cpp * args->width; stride = cpp * args->width;
if (args->height > 0xffffffffU / stride) if (args->height > U32_MAX / stride)
return -EINVAL; return -EINVAL;
/* test for wrap-around */ /* test for wrap-around */
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment