Commit daa3fc44 authored by Hans Verkuil's avatar Hans Verkuil Committed by Mauro Carvalho Chehab

media: vicodec: set state resolution from raw format

The state structure contains the resolution expected by the decoder
and encoder. For an encoder that resolution should be taken from the
OUTPUT format, and for a decoder from the CAPTURE format.

If the wrong format is picked, a buffer overrun can occur if there is
a mismatch between the CAPTURE and OUTPUT formats.

The real fix would be to correctly implement the stateful codec
specification, but that will take more time. For now just prevent the
buffer overrun.
Signed-off-by: default avatarHans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: default avatarMauro Carvalho Chehab <mchehab+samsung@kernel.org>
parent e714c92f
......@@ -997,11 +997,18 @@ static int vicodec_start_streaming(struct vb2_queue *q,
q_data->sequence = 0;
if (!V4L2_TYPE_IS_OUTPUT(q->type))
if (!V4L2_TYPE_IS_OUTPUT(q->type)) {
if (!ctx->is_enc) {
state->width = q_data->width;
state->height = q_data->height;
}
return 0;
}
state->width = q_data->width;
state->height = q_data->height;
if (ctx->is_enc) {
state->width = q_data->width;
state->height = q_data->height;
}
state->ref_frame.width = state->ref_frame.height = 0;
state->ref_frame.luma = kvmalloc(size + 2 * size / chroma_div,
GFP_KERNEL);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment