• Helen Koike's avatar
    media: v4l2-rect.h: fix v4l2_rect_map_inside() top/left adjustments · f51e50db
    Helen Koike authored
    boundary->width and boundary->height are sizes relative to
    boundary->left and boundary->top coordinates, but they were not being
    taken into consideration to adjust r->left and r->top, leading to the
    following error:
    
    Consider the follow as initial values for boundary and r:
    
    struct v4l2_rect boundary = {
    	.left = 100,
    	.top = 100,
    	.width = 800,
    	.height = 600,
    }
    
    struct v4l2_rect r = {
    	.left = 0,
    	.top = 0,
    	.width = 1920,
    	.height = 960,
    }
    
    calling v4l2_rect_map_inside(&r, &boundary) was modifying r to:
    
    r = {
    	.left = 0,
    	.top = 0,
    	.width = 800,
    	.height = 600,
    }
    
    Which is wrongly outside the boundary rectangle, because:
    
    	v4l2_rect_set_max_size(r, boundary); // r->width = 800, r->height = 600
    	...
    	if (r->left + r->width > boundary->width) // true
    		r->left = boundary->width - r->width; // r->left = 800 - 800
    	if (r->top + r->height > boundary->height) // true
    		r->top = boundary->height - r->height; // r->height = 600 - 600
    
    Fix this by considering top/left coordinates from boundary.
    
    Fixes: ac49de8c ("[media] v4l2-rect.h: new header with struct v4l2_rect helper functions")
    Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
    Cc: <stable@vger.kernel.org>      # for v4.7 and up
    Signed-off-by: default avatarHans Verkuil <hverkuil-cisco@xs4all.nl>
    Signed-off-by: default avatarMauro Carvalho Chehab <mchehab+huawei@kernel.org>
    f51e50db
v4l2-rect.h 5.28 KB