Commit 996680d4 authored by Linus Torvalds's avatar Linus Torvalds

Merge tag 'media/v4.20-7' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media

Pull more media updates from Mauro Carvalho Chehab:
 "The Intel IPU3 camera driver"

* tag 'media/v4.20-7' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media: (23 commits)
  media: staging/ipu3-imgu: Add MAINTAINERS entry
  media: staging/ipu3-imgu: Address documentation comments
  media: v4l: Add Intel IPU3 meta buffer formats
  media: doc-rst: Add Intel IPU3 documentation
  media: ipu3-imgu: Fix firmware binary location
  media: ipu3-imgu: Fix compiler warnings
  media: staging/intel-ipu3: Add dual pipe support
  media: staging/intel-ipu3: Add Intel IPU3 meta data uAPI
  media: staging/intel-ipu3: Add imgu top level pci device driver
  media: staging/intel-ipu3: Add v4l2 driver based on media framework
  media: staging/intel-ipu3: Add css pipeline programming
  media: staging/intel-ipu3: css: Initialize css hardware
  media: staging/intel-ipu3: css: Compute and program ccs
  media: staging/intel-ipu3: css: Add static settings for image pipeline
  media: staging/intel-ipu3: css: Add support for firmware management
  media: staging/intel-ipu3: css: Add dma buff pool utility functions
  media: staging/intel-ipu3: Implement DMA mapping functions
  media: staging/intel-ipu3: mmu: Implement driver
  media: staging/intel-ipu3: abi: Add structs
  media: staging/intel-ipu3: abi: Add register definitions and enum
  ...
parents 5813540b 38b11beb
...@@ -472,6 +472,9 @@ enum v4l2_buf_type ...@@ -472,6 +472,9 @@ enum v4l2_buf_type
* - ``V4L2_BUF_TYPE_META_CAPTURE`` * - ``V4L2_BUF_TYPE_META_CAPTURE``
- 13 - 13
- Buffer for metadata capture, see :ref:`metadata`. - Buffer for metadata capture, see :ref:`metadata`.
* - ``V4L2_BUF_TYPE_META_OUTPUT``
- 14
- Buffer for metadata output, see :ref:`metadata`.
......
...@@ -14,21 +14,27 @@ Metadata Interface ...@@ -14,21 +14,27 @@ Metadata Interface
****************** ******************
Metadata refers to any non-image data that supplements video frames with Metadata refers to any non-image data that supplements video frames with
additional information. This may include statistics computed over the image additional information. This may include statistics computed over the image,
or frame capture parameters supplied by the image source. This interface is frame capture parameters supplied by the image source or device specific
intended for transfer of metadata to userspace and control of that operation. parameters for specifying how the device processes images. This interface is
intended for transfer of metadata between the userspace and the hardware and
control of that operation.
The metadata interface is implemented on video capture device nodes. The device The metadata interface is implemented on video device nodes. The device can be
can be dedicated to metadata or can implement both video and metadata capture dedicated to metadata or can support both video and metadata as specified in its
as specified in its reported capabilities. reported capabilities.
Querying Capabilities Querying Capabilities
===================== =====================
Device nodes supporting the metadata interface set the ``V4L2_CAP_META_CAPTURE`` Device nodes supporting the metadata capture interface set the
flag in the ``device_caps`` field of the ``V4L2_CAP_META_CAPTURE`` flag in the ``device_caps`` field of the
:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP` :c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP`
ioctl. That flag means the device can capture metadata to memory. ioctl. That flag means the device can capture metadata to memory. Similarly,
device nodes supporting metadata output interface set the
``V4L2_CAP_META_OUTPUT`` flag in the ``device_caps`` field of
:c:type:`v4l2_capability` structure. That flag means the device can read
metadata from memory.
At least one of the read/write or streaming I/O methods must be supported. At least one of the read/write or streaming I/O methods must be supported.
...@@ -42,10 +48,11 @@ to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be ...@@ -42,10 +48,11 @@ to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be
supported as well. supported as well.
To use the :ref:`format` ioctls applications set the ``type`` field of the To use the :ref:`format` ioctls applications set the ``type`` field of the
:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_META_CAPTURE`` and use the :c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_META_CAPTURE`` or to
:c:type:`v4l2_meta_format` ``meta`` member of the ``fmt`` union as needed per ``V4L2_BUF_TYPE_META_OUTPUT`` and use the :c:type:`v4l2_meta_format` ``meta``
the desired operation. Both drivers and applications must set the remainder of member of the ``fmt`` union as needed per the desired operation. Both drivers
the :c:type:`v4l2_format` structure to 0. and applications must set the remainder of the :c:type:`v4l2_format` structure
to 0.
.. c:type:: v4l2_meta_format .. c:type:: v4l2_meta_format
......
...@@ -19,6 +19,7 @@ These formats are used for the :ref:`metadata` interface only. ...@@ -19,6 +19,7 @@ These formats are used for the :ref:`metadata` interface only.
.. toctree:: .. toctree::
:maxdepth: 1 :maxdepth: 1
pixfmt-meta-intel-ipu3
pixfmt-meta-d4xx pixfmt-meta-d4xx
pixfmt-meta-uvc pixfmt-meta-uvc
pixfmt-meta-vsp1-hgo pixfmt-meta-vsp1-hgo
......
.. -*- coding: utf-8; mode: rst -*-
.. _v4l2-meta-fmt-params:
.. _v4l2-meta-fmt-stat-3a:
******************************************************************
V4L2_META_FMT_IPU3_PARAMS ('ip3p'), V4L2_META_FMT_IPU3_3A ('ip3s')
******************************************************************
.. c:type:: ipu3_uapi_stats_3a
3A statistics
=============
For IPU3 ImgU, the 3A statistics accelerators collect different statistics over
an input bayer frame. Those statistics, defined in data struct :c:type:`ipu3_uapi_stats_3a`,
are obtained from "ipu3-imgu 3a stat" metadata capture video node, which are then
passed to user space for statistics analysis using :c:type:`v4l2_meta_format` interface.
The statistics collected are AWB (Auto-white balance) RGBS (Red, Green, Blue and
Saturation measure) cells, AWB filter response, AF (Auto-focus) filter response,
and AE (Auto-exposure) histogram.
struct :c:type:`ipu3_uapi_4a_config` saves configurable parameters for all above.
.. code-block:: c
struct ipu3_uapi_stats_3a {
struct ipu3_uapi_awb_raw_buffer awb_raw_buffer;
struct ipu3_uapi_ae_raw_buffer_aligned ae_raw_buffer[IPU3_UAPI_MAX_STRIPES];
struct ipu3_uapi_af_raw_buffer af_raw_buffer;
struct ipu3_uapi_awb_fr_raw_buffer awb_fr_raw_buffer;
struct ipu3_uapi_4a_config stats_4a_config;
__u32 ae_join_buffers;
__u8 padding[28];
struct ipu3_uapi_stats_3a_bubble_info_per_stripe stats_3a_bubble_per_stripe;
struct ipu3_uapi_ff_status stats_3a_status;
};
.. c:type:: ipu3_uapi_params
Pipeline parameters
===================
IPU3 pipeline has a number of image processing stages, each of which takes a
set of parameters as input. The major stages of pipelines are shown here:
Raw pixels -> Bayer Downscaling -> Optical Black Correction ->
Linearization -> Lens Shading Correction -> White Balance / Exposure /
Focus Apply -> Bayer Noise Reduction -> ANR -> Demosaicing -> Color
Correction Matrix -> Gamma correction -> Color Space Conversion ->
Chroma Down Scaling -> Chromatic Noise Reduction -> Total Color
Correction -> XNR3 -> TNR -> DDR
The table below presents a description of the above algorithms.
======================== =======================================================
Name Description
======================== =======================================================
Optical Black Correction Optical Black Correction block subtracts a pre-defined
value from the respective pixel values to obtain better
image quality.
Defined in :c:type:`ipu3_uapi_obgrid_param`.
Linearization This algo block uses linearization parameters to
address non-linearity sensor effects. The Lookup table
table is defined in
:c:type:`ipu3_uapi_isp_lin_vmem_params`.
SHD Lens shading correction is used to correct spatial
non-uniformity of the pixel response due to optical
lens shading. This is done by applying a different gain
for each pixel. The gain, black level etc are
configured in :c:type:`ipu3_uapi_shd_config_static`.
BNR Bayer noise reduction block removes image noise by
applying a bilateral filter.
See :c:type:`ipu3_uapi_bnr_static_config` for details.
ANR Advanced Noise Reduction is a block based algorithm
that performs noise reduction in the Bayer domain. The
convolution matrix etc can be found in
:c:type:`ipu3_uapi_anr_config`.
Demosaicing Demosaicing converts raw sensor data in Bayer format
into RGB (Red, Green, Blue) presentation. Then add
outputs of estimation of Y channel for following stream
processing by Firmware. The struct is defined as
:c:type:`ipu3_uapi_dm_config`. (TODO)
Color Correction Color Correction algo transforms sensor specific color
space to the standard "sRGB" color space. This is done
by applying 3x3 matrix defined in
:c:type:`ipu3_uapi_ccm_mat_config`.
Gamma correction Gamma correction :c:type:`ipu3_uapi_gamma_config` is a
basic non-linear tone mapping correction that is
applied per pixel for each pixel component.
CSC Color space conversion transforms each pixel from the
RGB primary presentation to YUV (Y: brightness,
UV: Luminance) presentation. This is done by applying
a 3x3 matrix defined in
:c:type:`ipu3_uapi_csc_mat_config`
CDS Chroma down sampling
After the CSC is performed, the Chroma Down Sampling
is applied for a UV plane down sampling by a factor
of 2 in each direction for YUV 4:2:0 using a 4x2
configurable filter :c:type:`ipu3_uapi_cds_params`.
CHNR Chroma noise reduction
This block processes only the chrominance pixels and
performs noise reduction by cleaning the high
frequency noise.
See struct :c:type:`ipu3_uapi_yuvp1_chnr_config`.
TCC Total color correction as defined in struct
:c:type:`ipu3_uapi_yuvp2_tcc_static_config`.
XNR3 eXtreme Noise Reduction V3 is the third revision of
noise reduction algorithm used to improve image
quality. This removes the low frequency noise in the
captured image. Two related structs are being defined,
:c:type:`ipu3_uapi_isp_xnr3_params` for ISP data memory
and :c:type:`ipu3_uapi_isp_xnr3_vmem_params` for vector
memory.
TNR Temporal Noise Reduction block compares successive
frames in time to remove anomalies / noise in pixel
values. :c:type:`ipu3_uapi_isp_tnr3_vmem_params` and
:c:type:`ipu3_uapi_isp_tnr3_params` are defined for ISP
vector and data memory respectively.
======================== =======================================================
A few stages of the pipeline will be executed by firmware running on the ISP
processor, while many others will use a set of fixed hardware blocks also
called accelerator cluster (ACC) to crunch pixel data and produce statistics.
ACC parameters of individual algorithms, as defined by
:c:type:`ipu3_uapi_acc_param`, can be chosen to be applied by the user
space through struct :c:type:`ipu3_uapi_flags` embedded in
:c:type:`ipu3_uapi_params` structure. For parameters that are configured as
not enabled by the user space, the corresponding structs are ignored by the
driver, in which case the existing configuration of the algorithm will be
preserved.
Both 3A statistics and pipeline parameters described here are closely tied to
the underlying camera sub-system (CSS) APIs. They are usually consumed and
produced by dedicated user space libraries that comprise the important tuning
tools, thus freeing the developers from being bothered with the low level
hardware and algorithm details.
It should be noted that IPU3 DMA operations require the addresses of all data
structures (that includes both input and output) to be aligned on 32 byte
boundaries.
The meta data :c:type:`ipu3_uapi_params` will be sent to "ipu3-imgu parameters"
video node in ``V4L2_BUF_TYPE_META_CAPTURE`` format.
.. code-block:: c
struct ipu3_uapi_params {
/* Flags which of the settings below are to be applied */
struct ipu3_uapi_flags use;
/* Accelerator cluster parameters */
struct ipu3_uapi_acc_param acc_param;
/* ISP vector address space parameters */
struct ipu3_uapi_isp_lin_vmem_params lin_vmem_params;
struct ipu3_uapi_isp_tnr3_vmem_params tnr3_vmem_params;
struct ipu3_uapi_isp_xnr3_vmem_params xnr3_vmem_params;
/* ISP data memory (DMEM) parameters */
struct ipu3_uapi_isp_tnr3_params tnr3_dmem_params;
struct ipu3_uapi_isp_xnr3_params xnr3_dmem_params;
/* Optical black level compensation */
struct ipu3_uapi_obgrid_param obgrid_param;
};
Intel IPU3 ImgU uAPI data types
===============================
.. kernel-doc:: drivers/staging/media/ipu3/include/intel-ipu3.h
...@@ -258,6 +258,9 @@ specification the ioctl returns an ``EINVAL`` error code. ...@@ -258,6 +258,9 @@ specification the ioctl returns an ``EINVAL`` error code.
* - ``V4L2_CAP_STREAMING`` * - ``V4L2_CAP_STREAMING``
- 0x04000000 - 0x04000000
- The device supports the :ref:`streaming <mmap>` I/O method. - The device supports the :ref:`streaming <mmap>` I/O method.
* - ``V4L2_CAP_META_OUTPUT``
- 0x08000000
- The device supports the :ref:`metadata` output interface.
* - ``V4L2_CAP_TOUCH`` * - ``V4L2_CAP_TOUCH``
- 0x10000000 - 0x10000000
- This is a touch device. - This is a touch device.
......
...@@ -44,6 +44,7 @@ For more details see the file COPYING in the source distribution of Linux. ...@@ -44,6 +44,7 @@ For more details see the file COPYING in the source distribution of Linux.
davinci-vpbe davinci-vpbe
fimc fimc
imx imx
ipu3
ivtv ivtv
max2175 max2175
meye meye
......
This diff is collapsed.
...@@ -30,6 +30,7 @@ replace symbol V4L2_FIELD_TOP :c:type:`v4l2_field` ...@@ -30,6 +30,7 @@ replace symbol V4L2_FIELD_TOP :c:type:`v4l2_field`
# Documented enum v4l2_buf_type # Documented enum v4l2_buf_type
replace symbol V4L2_BUF_TYPE_META_CAPTURE :c:type:`v4l2_buf_type` replace symbol V4L2_BUF_TYPE_META_CAPTURE :c:type:`v4l2_buf_type`
replace symbol V4L2_BUF_TYPE_META_OUTPUT :c:type:`v4l2_buf_type`
replace symbol V4L2_BUF_TYPE_SDR_CAPTURE :c:type:`v4l2_buf_type` replace symbol V4L2_BUF_TYPE_SDR_CAPTURE :c:type:`v4l2_buf_type`
replace symbol V4L2_BUF_TYPE_SDR_OUTPUT :c:type:`v4l2_buf_type` replace symbol V4L2_BUF_TYPE_SDR_OUTPUT :c:type:`v4l2_buf_type`
replace symbol V4L2_BUF_TYPE_SLICED_VBI_CAPTURE :c:type:`v4l2_buf_type` replace symbol V4L2_BUF_TYPE_SLICED_VBI_CAPTURE :c:type:`v4l2_buf_type`
...@@ -163,6 +164,7 @@ replace define V4L2_CAP_META_CAPTURE device-capabilities ...@@ -163,6 +164,7 @@ replace define V4L2_CAP_META_CAPTURE device-capabilities
replace define V4L2_CAP_READWRITE device-capabilities replace define V4L2_CAP_READWRITE device-capabilities
replace define V4L2_CAP_ASYNCIO device-capabilities replace define V4L2_CAP_ASYNCIO device-capabilities
replace define V4L2_CAP_STREAMING device-capabilities replace define V4L2_CAP_STREAMING device-capabilities
replace define V4L2_CAP_META_OUTPUT device-capabilities
replace define V4L2_CAP_DEVICE_CAPS device-capabilities replace define V4L2_CAP_DEVICE_CAPS device-capabilities
replace define V4L2_CAP_TOUCH device-capabilities replace define V4L2_CAP_TOUCH device-capabilities
......
...@@ -7645,6 +7645,14 @@ S: Maintained ...@@ -7645,6 +7645,14 @@ S: Maintained
F: drivers/media/pci/intel/ipu3/ F: drivers/media/pci/intel/ipu3/
F: Documentation/media/uapi/v4l/pixfmt-srggb10-ipu3.rst F: Documentation/media/uapi/v4l/pixfmt-srggb10-ipu3.rst
INTEL IPU3 CSI-2 IMGU DRIVER
M: Sakari Ailus <sakari.ailus@linux.intel.com>
L: linux-media@vger.kernel.org
S: Maintained
F: drivers/staging/media/ipu3/
F: Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
F: Documentation/media/v4l-drivers/ipu3.rst
INTEL IXP4XX QMGR, NPE, ETHERNET and HSS SUPPORT INTEL IXP4XX QMGR, NPE, ETHERNET and HSS SUPPORT
M: Krzysztof Halasa <khalasa@piap.pl> M: Krzysztof Halasa <khalasa@piap.pl>
S: Maintained S: Maintained
......
...@@ -709,6 +709,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create) ...@@ -709,6 +709,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
requested_sizes[0] = f->fmt.sdr.buffersize; requested_sizes[0] = f->fmt.sdr.buffersize;
break; break;
case V4L2_BUF_TYPE_META_CAPTURE: case V4L2_BUF_TYPE_META_CAPTURE:
case V4L2_BUF_TYPE_META_OUTPUT:
requested_sizes[0] = f->fmt.meta.buffersize; requested_sizes[0] = f->fmt.meta.buffersize;
break; break;
default: default:
......
...@@ -323,6 +323,7 @@ static int __get_v4l2_format32(struct v4l2_format __user *p64, ...@@ -323,6 +323,7 @@ static int __get_v4l2_format32(struct v4l2_format __user *p64,
return copy_in_user(&p64->fmt.sdr, &p32->fmt.sdr, return copy_in_user(&p64->fmt.sdr, &p32->fmt.sdr,
sizeof(p64->fmt.sdr)) ? -EFAULT : 0; sizeof(p64->fmt.sdr)) ? -EFAULT : 0;
case V4L2_BUF_TYPE_META_CAPTURE: case V4L2_BUF_TYPE_META_CAPTURE:
case V4L2_BUF_TYPE_META_OUTPUT:
return copy_in_user(&p64->fmt.meta, &p32->fmt.meta, return copy_in_user(&p64->fmt.meta, &p32->fmt.meta,
sizeof(p64->fmt.meta)) ? -EFAULT : 0; sizeof(p64->fmt.meta)) ? -EFAULT : 0;
default: default:
...@@ -392,6 +393,7 @@ static int __put_v4l2_format32(struct v4l2_format __user *p64, ...@@ -392,6 +393,7 @@ static int __put_v4l2_format32(struct v4l2_format __user *p64,
return copy_in_user(&p32->fmt.sdr, &p64->fmt.sdr, return copy_in_user(&p32->fmt.sdr, &p64->fmt.sdr,
sizeof(p64->fmt.sdr)) ? -EFAULT : 0; sizeof(p64->fmt.sdr)) ? -EFAULT : 0;
case V4L2_BUF_TYPE_META_CAPTURE: case V4L2_BUF_TYPE_META_CAPTURE:
case V4L2_BUF_TYPE_META_OUTPUT:
return copy_in_user(&p32->fmt.meta, &p64->fmt.meta, return copy_in_user(&p32->fmt.meta, &p64->fmt.meta,
sizeof(p64->fmt.meta)) ? -EFAULT : 0; sizeof(p64->fmt.meta)) ? -EFAULT : 0;
default: default:
......
...@@ -597,7 +597,8 @@ static void determine_valid_ioctls(struct video_device *vdev) ...@@ -597,7 +597,8 @@ static void determine_valid_ioctls(struct video_device *vdev)
ops->vidioc_enum_fmt_vid_overlay || ops->vidioc_enum_fmt_vid_overlay ||
ops->vidioc_enum_fmt_meta_cap)) || ops->vidioc_enum_fmt_meta_cap)) ||
(is_tx && (ops->vidioc_enum_fmt_vid_out || (is_tx && (ops->vidioc_enum_fmt_vid_out ||
ops->vidioc_enum_fmt_vid_out_mplane))) ops->vidioc_enum_fmt_vid_out_mplane ||
ops->vidioc_enum_fmt_meta_out)))
set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls);
if ((is_rx && (ops->vidioc_g_fmt_vid_cap || if ((is_rx && (ops->vidioc_g_fmt_vid_cap ||
ops->vidioc_g_fmt_vid_cap_mplane || ops->vidioc_g_fmt_vid_cap_mplane ||
...@@ -605,7 +606,8 @@ static void determine_valid_ioctls(struct video_device *vdev) ...@@ -605,7 +606,8 @@ static void determine_valid_ioctls(struct video_device *vdev)
ops->vidioc_g_fmt_meta_cap)) || ops->vidioc_g_fmt_meta_cap)) ||
(is_tx && (ops->vidioc_g_fmt_vid_out || (is_tx && (ops->vidioc_g_fmt_vid_out ||
ops->vidioc_g_fmt_vid_out_mplane || ops->vidioc_g_fmt_vid_out_mplane ||
ops->vidioc_g_fmt_vid_out_overlay))) ops->vidioc_g_fmt_vid_out_overlay ||
ops->vidioc_g_fmt_meta_out)))
set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls);
if ((is_rx && (ops->vidioc_s_fmt_vid_cap || if ((is_rx && (ops->vidioc_s_fmt_vid_cap ||
ops->vidioc_s_fmt_vid_cap_mplane || ops->vidioc_s_fmt_vid_cap_mplane ||
...@@ -613,7 +615,8 @@ static void determine_valid_ioctls(struct video_device *vdev) ...@@ -613,7 +615,8 @@ static void determine_valid_ioctls(struct video_device *vdev)
ops->vidioc_s_fmt_meta_cap)) || ops->vidioc_s_fmt_meta_cap)) ||
(is_tx && (ops->vidioc_s_fmt_vid_out || (is_tx && (ops->vidioc_s_fmt_vid_out ||
ops->vidioc_s_fmt_vid_out_mplane || ops->vidioc_s_fmt_vid_out_mplane ||
ops->vidioc_s_fmt_vid_out_overlay))) ops->vidioc_s_fmt_vid_out_overlay ||
ops->vidioc_s_fmt_meta_out)))
set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls);
if ((is_rx && (ops->vidioc_try_fmt_vid_cap || if ((is_rx && (ops->vidioc_try_fmt_vid_cap ||
ops->vidioc_try_fmt_vid_cap_mplane || ops->vidioc_try_fmt_vid_cap_mplane ||
...@@ -621,7 +624,8 @@ static void determine_valid_ioctls(struct video_device *vdev) ...@@ -621,7 +624,8 @@ static void determine_valid_ioctls(struct video_device *vdev)
ops->vidioc_try_fmt_meta_cap)) || ops->vidioc_try_fmt_meta_cap)) ||
(is_tx && (ops->vidioc_try_fmt_vid_out || (is_tx && (ops->vidioc_try_fmt_vid_out ||
ops->vidioc_try_fmt_vid_out_mplane || ops->vidioc_try_fmt_vid_out_mplane ||
ops->vidioc_try_fmt_vid_out_overlay))) ops->vidioc_try_fmt_vid_out_overlay ||
ops->vidioc_try_fmt_meta_out)))
set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls);
SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay);
SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf);
......
...@@ -310,8 +310,8 @@ v4l2_fwnode_endpoint_parse_parallel_bus(struct fwnode_handle *fwnode, ...@@ -310,8 +310,8 @@ v4l2_fwnode_endpoint_parse_parallel_bus(struct fwnode_handle *fwnode,
} }
if (!fwnode_property_read_u32(fwnode, "data-active", &v)) { if (!fwnode_property_read_u32(fwnode, "data-active", &v)) {
flags &= ~(V4L2_MBUS_PCLK_SAMPLE_RISING | flags &= ~(V4L2_MBUS_DATA_ACTIVE_HIGH |
V4L2_MBUS_PCLK_SAMPLE_FALLING); V4L2_MBUS_DATA_ACTIVE_LOW);
flags |= v ? V4L2_MBUS_DATA_ACTIVE_HIGH : flags |= v ? V4L2_MBUS_DATA_ACTIVE_HIGH :
V4L2_MBUS_DATA_ACTIVE_LOW; V4L2_MBUS_DATA_ACTIVE_LOW;
pr_debug("data-active %s\n", v ? "high" : "low"); pr_debug("data-active %s\n", v ? "high" : "low");
......
...@@ -194,6 +194,7 @@ const char *v4l2_type_names[] = { ...@@ -194,6 +194,7 @@ const char *v4l2_type_names[] = {
[V4L2_BUF_TYPE_SDR_CAPTURE] = "sdr-cap", [V4L2_BUF_TYPE_SDR_CAPTURE] = "sdr-cap",
[V4L2_BUF_TYPE_SDR_OUTPUT] = "sdr-out", [V4L2_BUF_TYPE_SDR_OUTPUT] = "sdr-out",
[V4L2_BUF_TYPE_META_CAPTURE] = "meta-cap", [V4L2_BUF_TYPE_META_CAPTURE] = "meta-cap",
[V4L2_BUF_TYPE_META_OUTPUT] = "meta-out",
}; };
EXPORT_SYMBOL(v4l2_type_names); EXPORT_SYMBOL(v4l2_type_names);
...@@ -366,6 +367,7 @@ static void v4l_print_format(const void *arg, bool write_only) ...@@ -366,6 +367,7 @@ static void v4l_print_format(const void *arg, bool write_only)
(sdr->pixelformat >> 24) & 0xff); (sdr->pixelformat >> 24) & 0xff);
break; break;
case V4L2_BUF_TYPE_META_CAPTURE: case V4L2_BUF_TYPE_META_CAPTURE:
case V4L2_BUF_TYPE_META_OUTPUT:
meta = &p->fmt.meta; meta = &p->fmt.meta;
pr_cont(", dataformat=%c%c%c%c, buffersize=%u\n", pr_cont(", dataformat=%c%c%c%c, buffersize=%u\n",
(meta->dataformat >> 0) & 0xff, (meta->dataformat >> 0) & 0xff,
...@@ -999,6 +1001,10 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) ...@@ -999,6 +1001,10 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type)
if (is_vid && is_rx && ops->vidioc_g_fmt_meta_cap) if (is_vid && is_rx && ops->vidioc_g_fmt_meta_cap)
return 0; return 0;
break; break;
case V4L2_BUF_TYPE_META_OUTPUT:
if (is_vid && is_tx && ops->vidioc_g_fmt_meta_out)
return 0;
break;
default: default:
break; break;
} }
...@@ -1410,6 +1416,11 @@ static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops, ...@@ -1410,6 +1416,11 @@ static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops,
break; break;
ret = ops->vidioc_enum_fmt_meta_cap(file, fh, arg); ret = ops->vidioc_enum_fmt_meta_cap(file, fh, arg);
break; break;
case V4L2_BUF_TYPE_META_OUTPUT:
if (unlikely(!ops->vidioc_enum_fmt_meta_out))
break;
ret = ops->vidioc_enum_fmt_meta_out(file, fh, arg);
break;
} }
if (ret == 0) if (ret == 0)
v4l_fill_fmtdesc(p); v4l_fill_fmtdesc(p);
...@@ -1488,6 +1499,8 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, ...@@ -1488,6 +1499,8 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
return ops->vidioc_g_fmt_sdr_out(file, fh, arg); return ops->vidioc_g_fmt_sdr_out(file, fh, arg);
case V4L2_BUF_TYPE_META_CAPTURE: case V4L2_BUF_TYPE_META_CAPTURE:
return ops->vidioc_g_fmt_meta_cap(file, fh, arg); return ops->vidioc_g_fmt_meta_cap(file, fh, arg);
case V4L2_BUF_TYPE_META_OUTPUT:
return ops->vidioc_g_fmt_meta_out(file, fh, arg);
} }
return -EINVAL; return -EINVAL;
} }
...@@ -1601,6 +1614,11 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, ...@@ -1601,6 +1614,11 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops,
break; break;
CLEAR_AFTER_FIELD(p, fmt.meta); CLEAR_AFTER_FIELD(p, fmt.meta);
return ops->vidioc_s_fmt_meta_cap(file, fh, arg); return ops->vidioc_s_fmt_meta_cap(file, fh, arg);
case V4L2_BUF_TYPE_META_OUTPUT:
if (unlikely(!ops->vidioc_s_fmt_meta_out))
break;
CLEAR_AFTER_FIELD(p, fmt.meta);
return ops->vidioc_s_fmt_meta_out(file, fh, arg);
} }
return -EINVAL; return -EINVAL;
} }
...@@ -1693,6 +1711,11 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, ...@@ -1693,6 +1711,11 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops,
break; break;
CLEAR_AFTER_FIELD(p, fmt.meta); CLEAR_AFTER_FIELD(p, fmt.meta);
return ops->vidioc_try_fmt_meta_cap(file, fh, arg); return ops->vidioc_try_fmt_meta_cap(file, fh, arg);
case V4L2_BUF_TYPE_META_OUTPUT:
if (unlikely(!ops->vidioc_try_fmt_meta_out))
break;
CLEAR_AFTER_FIELD(p, fmt.meta);
return ops->vidioc_try_fmt_meta_out(file, fh, arg);
} }
return -EINVAL; return -EINVAL;
} }
......
...@@ -39,4 +39,6 @@ source "drivers/staging/media/tegra-vde/Kconfig" ...@@ -39,4 +39,6 @@ source "drivers/staging/media/tegra-vde/Kconfig"
source "drivers/staging/media/zoran/Kconfig" source "drivers/staging/media/zoran/Kconfig"
source "drivers/staging/media/ipu3/Kconfig"
endif endif
...@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_SUNXI) += sunxi/ ...@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_SUNXI) += sunxi/
obj-$(CONFIG_TEGRA_VDE) += tegra-vde/ obj-$(CONFIG_TEGRA_VDE) += tegra-vde/
obj-$(CONFIG_VIDEO_ZORAN) += zoran/ obj-$(CONFIG_VIDEO_ZORAN) += zoran/
obj-$(CONFIG_VIDEO_ROCKCHIP_VPU) += rockchip/vpu/ obj-$(CONFIG_VIDEO_ROCKCHIP_VPU) += rockchip/vpu/
obj-$(CONFIG_VIDEO_IPU3_IMGU) += ipu3/
config VIDEO_IPU3_IMGU
tristate "Intel ipu3-imgu driver"
depends on PCI && VIDEO_V4L2
depends on MEDIA_CONTROLLER && VIDEO_V4L2_SUBDEV_API
depends on X86
select IOMMU_IOVA
select VIDEOBUF2_DMA_SG
---help---
This is the Video4Linux2 driver for Intel IPU3 image processing unit,
found in Intel Skylake and Kaby Lake SoCs and used for processing
images and video.
Say Y or M here if you have a Skylake/Kaby Lake SoC with a MIPI
camera. The module will be called ipu3-imgu.
#
# Makefile for the IPU3 ImgU drivers
#
ipu3-imgu-objs += \
ipu3-mmu.o ipu3-dmamap.o \
ipu3-tables.o ipu3-css-pool.o \
ipu3-css-fw.o ipu3-css-params.o \
ipu3-css.o ipu3-v4l2.o ipu3.o
obj-$(CONFIG_VIDEO_IPU3_IMGU) += ipu3-imgu.o
This is a list of things that need to be done to get this driver out of the
staging directory.
- Request API conversion. Remove of the dual pipeline and associate buffers
as well as formats and the binary used to a request. Remove the
opportunistic buffer management. (Sakari)
- Using ENABLED and IMMUTABLE link flags for the links where those are
relevant. (Sakari)
- Prefix imgu for all public APIs, i.e. change ipu3_v4l2_register() to
imgu_v4l2_register(). (Sakari)
- Use V4L2_CTRL_TYPE_MENU for dual-pipe mode control. (Sakari)
- IPU3 driver documentation (Laurent)
Add diagram in driver rst to describe output capability.
Comments on configuring v4l2 subdevs for CIO2 and ImgU.
- uAPI documentation:
Further clarification on some ambiguities such as data type conversion of
IEFD CU inputs. (Sakari)
Move acronyms to doc-rst file. (Mauro)
- Switch to yavta from v4l2n in driver docs.
- Elaborate the functionality of different selection rectangles in driver
documentation. This may require driver changes as well.
- More detailed documentation on calculating BDS, GCD etc. sizes needed.
- Document different operation modes, and which buffer queues are relevant
in each mode. To process an image, which queues require a buffer an in
which ones is it optional?
This diff is collapsed.
This diff is collapsed.
// SPDX-License-Identifier: GPL-2.0
// Copyright (C) 2018 Intel Corporation
#include <linux/device.h>
#include <linux/firmware.h>
#include <linux/mm.h>
#include <linux/slab.h>
#include "ipu3-css.h"
#include "ipu3-css-fw.h"
#include "ipu3-dmamap.h"
static void ipu3_css_fw_show_binary(struct device *dev, struct imgu_fw_info *bi,
const char *name)
{
unsigned int i;
dev_dbg(dev, "found firmware binary type %i size %i name %s\n",
bi->type, bi->blob.size, name);
if (bi->type != IMGU_FW_ISP_FIRMWARE)
return;
dev_dbg(dev, " id %i mode %i bds 0x%x veceven %i/%i out_pins %i\n",
bi->info.isp.sp.id, bi->info.isp.sp.pipeline.mode,
bi->info.isp.sp.bds.supported_bds_factors,
bi->info.isp.sp.enable.vf_veceven,
bi->info.isp.sp.vf_dec.is_variable,
bi->info.isp.num_output_pins);
dev_dbg(dev, " input (%i,%i)-(%i,%i) formats %s%s%s\n",
bi->info.isp.sp.input.min_width,
bi->info.isp.sp.input.min_height,
bi->info.isp.sp.input.max_width,
bi->info.isp.sp.input.max_height,
bi->info.isp.sp.enable.input_yuv ? "yuv420 " : "",
bi->info.isp.sp.enable.input_feeder ||
bi->info.isp.sp.enable.input_raw ? "raw8 raw10 " : "",
bi->info.isp.sp.enable.input_raw ? "raw12" : "");
dev_dbg(dev, " internal (%i,%i)\n",
bi->info.isp.sp.internal.max_width,
bi->info.isp.sp.internal.max_height);
dev_dbg(dev, " output (%i,%i)-(%i,%i) formats",
bi->info.isp.sp.output.min_width,
bi->info.isp.sp.output.min_height,
bi->info.isp.sp.output.max_width,
bi->info.isp.sp.output.max_height);
for (i = 0; i < bi->info.isp.num_output_formats; i++)
dev_dbg(dev, " %i", bi->info.isp.output_formats[i]);
dev_dbg(dev, " vf");
for (i = 0; i < bi->info.isp.num_vf_formats; i++)
dev_dbg(dev, " %i", bi->info.isp.vf_formats[i]);
dev_dbg(dev, "\n");
}
unsigned int ipu3_css_fw_obgrid_size(const struct imgu_fw_info *bi)
{
unsigned int width = DIV_ROUND_UP(bi->info.isp.sp.internal.max_width,
IMGU_OBGRID_TILE_SIZE * 2) + 1;
unsigned int height = DIV_ROUND_UP(bi->info.isp.sp.internal.max_height,
IMGU_OBGRID_TILE_SIZE * 2) + 1;
unsigned int obgrid_size;
width = ALIGN(width, IPU3_UAPI_ISP_VEC_ELEMS / 4);
obgrid_size = PAGE_ALIGN(width * height *
sizeof(struct ipu3_uapi_obgrid_param)) *
bi->info.isp.sp.iterator.num_stripes;
return obgrid_size;
}
void *ipu3_css_fw_pipeline_params(struct ipu3_css *css, unsigned int pipe,
enum imgu_abi_param_class cls,
enum imgu_abi_memories mem,
struct imgu_fw_isp_parameter *par,
size_t par_size, void *binary_params)
{
struct imgu_fw_info *bi =
&css->fwp->binary_header[css->pipes[pipe].bindex];
if (par->offset + par->size >
bi->info.isp.sp.mem_initializers.params[cls][mem].size)
return NULL;
if (par->size != par_size)
pr_warn("parameter size doesn't match defined size\n");
if (par->size < par_size)
return NULL;
return binary_params + par->offset;
}
void ipu3_css_fw_cleanup(struct ipu3_css *css)
{
struct imgu_device *imgu = dev_get_drvdata(css->dev);
if (css->binary) {
unsigned int i;
for (i = 0; i < css->fwp->file_header.binary_nr; i++)
ipu3_dmamap_free(imgu, &css->binary[i]);
kfree(css->binary);
}
if (css->fw)
release_firmware(css->fw);
css->binary = NULL;
css->fw = NULL;
}
int ipu3_css_fw_init(struct ipu3_css *css)
{
static const u32 BLOCK_MAX = 65536;
struct imgu_device *imgu = dev_get_drvdata(css->dev);
struct device *dev = css->dev;
unsigned int i, j, binary_nr;
int r;
r = request_firmware(&css->fw, IMGU_FW_NAME, css->dev);
if (r)
return r;
/* Check and display fw header info */
css->fwp = (struct imgu_fw_header *)css->fw->data;
if (css->fw->size < sizeof(struct imgu_fw_header *) ||
css->fwp->file_header.h_size != sizeof(struct imgu_fw_bi_file_h))
goto bad_fw;
if (sizeof(struct imgu_fw_bi_file_h) +
css->fwp->file_header.binary_nr * sizeof(struct imgu_fw_info) >
css->fw->size)
goto bad_fw;
dev_info(dev, "loaded firmware version %.64s, %u binaries, %zu bytes\n",
css->fwp->file_header.version, css->fwp->file_header.binary_nr,
css->fw->size);
/* Validate and display info on fw binaries */
binary_nr = css->fwp->file_header.binary_nr;
css->fw_bl = -1;
css->fw_sp[0] = -1;
css->fw_sp[1] = -1;
for (i = 0; i < binary_nr; i++) {
struct imgu_fw_info *bi = &css->fwp->binary_header[i];
const char *name = (void *)css->fwp + bi->blob.prog_name_offset;
size_t len;
if (bi->blob.prog_name_offset >= css->fw->size)
goto bad_fw;
len = strnlen(name, css->fw->size - bi->blob.prog_name_offset);
if (len + 1 > css->fw->size - bi->blob.prog_name_offset ||
len + 1 >= IMGU_ABI_MAX_BINARY_NAME)
goto bad_fw;
if (bi->blob.size != bi->blob.text_size + bi->blob.icache_size
+ bi->blob.data_size + bi->blob.padding_size)
goto bad_fw;
if (bi->blob.offset + bi->blob.size > css->fw->size)
goto bad_fw;
if (bi->type == IMGU_FW_BOOTLOADER_FIRMWARE) {
css->fw_bl = i;
if (bi->info.bl.sw_state >= css->iomem_length ||
bi->info.bl.num_dma_cmds >= css->iomem_length ||
bi->info.bl.dma_cmd_list >= css->iomem_length)
goto bad_fw;
}
if (bi->type == IMGU_FW_SP_FIRMWARE ||
bi->type == IMGU_FW_SP1_FIRMWARE) {
css->fw_sp[bi->type == IMGU_FW_SP_FIRMWARE ? 0 : 1] = i;
if (bi->info.sp.per_frame_data >= css->iomem_length ||
bi->info.sp.init_dmem_data >= css->iomem_length ||
bi->info.sp.host_sp_queue >= css->iomem_length ||
bi->info.sp.isp_started >= css->iomem_length ||
bi->info.sp.sw_state >= css->iomem_length ||
bi->info.sp.sleep_mode >= css->iomem_length ||
bi->info.sp.invalidate_tlb >= css->iomem_length ||
bi->info.sp.host_sp_com >= css->iomem_length ||
bi->info.sp.output + 12 >= css->iomem_length ||
bi->info.sp.host_sp_queues_initialized >=
css->iomem_length)
goto bad_fw;
}
if (bi->type != IMGU_FW_ISP_FIRMWARE)
continue;
if (bi->info.isp.sp.pipeline.mode >= IPU3_CSS_PIPE_ID_NUM)
goto bad_fw;
if (bi->info.isp.sp.iterator.num_stripes >
IPU3_UAPI_MAX_STRIPES)
goto bad_fw;
if (bi->info.isp.num_vf_formats > IMGU_ABI_FRAME_FORMAT_NUM ||
bi->info.isp.num_output_formats > IMGU_ABI_FRAME_FORMAT_NUM)
goto bad_fw;
for (j = 0; j < bi->info.isp.num_output_formats; j++)
if (bi->info.isp.output_formats[j] < 0 ||
bi->info.isp.output_formats[j] >=
IMGU_ABI_FRAME_FORMAT_NUM)
goto bad_fw;
for (j = 0; j < bi->info.isp.num_vf_formats; j++)
if (bi->info.isp.vf_formats[j] < 0 ||
bi->info.isp.vf_formats[j] >=
IMGU_ABI_FRAME_FORMAT_NUM)
goto bad_fw;
if (bi->info.isp.sp.block.block_width <= 0 ||
bi->info.isp.sp.block.block_width > BLOCK_MAX ||
bi->info.isp.sp.block.output_block_height <= 0 ||
bi->info.isp.sp.block.output_block_height > BLOCK_MAX)
goto bad_fw;
if (bi->blob.memory_offsets.offsets[IMGU_ABI_PARAM_CLASS_PARAM]
+ sizeof(struct imgu_fw_param_memory_offsets) >
css->fw->size ||
bi->blob.memory_offsets.offsets[IMGU_ABI_PARAM_CLASS_CONFIG]
+ sizeof(struct imgu_fw_config_memory_offsets) >
css->fw->size ||
bi->blob.memory_offsets.offsets[IMGU_ABI_PARAM_CLASS_STATE]
+ sizeof(struct imgu_fw_state_memory_offsets) >
css->fw->size)
goto bad_fw;
ipu3_css_fw_show_binary(dev, bi, name);
}
if (css->fw_bl == -1 || css->fw_sp[0] == -1 || css->fw_sp[1] == -1)
goto bad_fw;
/* Allocate and map fw binaries into IMGU */
css->binary = kcalloc(binary_nr, sizeof(*css->binary), GFP_KERNEL);
if (!css->binary) {
r = -ENOMEM;
goto error_out;
}
for (i = 0; i < css->fwp->file_header.binary_nr; i++) {
struct imgu_fw_info *bi = &css->fwp->binary_header[i];
void *blob = (void *)css->fwp + bi->blob.offset;
size_t size = bi->blob.size;
if (!ipu3_dmamap_alloc(imgu, &css->binary[i], size)) {
r = -ENOMEM;
goto error_out;
}
memcpy(css->binary[i].vaddr, blob, size);
}
return 0;
bad_fw:
dev_err(dev, "invalid firmware binary, size %u\n", (int)css->fw->size);
r = -ENODEV;
error_out:
ipu3_css_fw_cleanup(css);
return r;
}
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright (C) 2018 Intel Corporation */
#ifndef __IPU3_CSS_FW_H
#define __IPU3_CSS_FW_H
/******************* Firmware file definitions *******************/
#define IMGU_FW_NAME "intel/ipu3-fw.bin"
typedef u32 imgu_fw_ptr;
enum imgu_fw_type {
IMGU_FW_SP_FIRMWARE, /* Firmware for the SP */
IMGU_FW_SP1_FIRMWARE, /* Firmware for the SP1 */
IMGU_FW_ISP_FIRMWARE, /* Firmware for the ISP */
IMGU_FW_BOOTLOADER_FIRMWARE, /* Firmware for the BootLoader */
IMGU_FW_ACC_FIRMWARE /* Firmware for accelerations */
};
enum imgu_fw_acc_type {
IMGU_FW_ACC_NONE, /* Normal binary */
IMGU_FW_ACC_OUTPUT, /* Accelerator stage on output frame */
IMGU_FW_ACC_VIEWFINDER, /* Accelerator stage on viewfinder frame */
IMGU_FW_ACC_STANDALONE, /* Stand-alone acceleration */
};
struct imgu_fw_isp_parameter {
u32 offset; /* Offset in isp_<mem> config, params, etc. */
u32 size; /* Disabled if 0 */
};
struct imgu_fw_param_memory_offsets {
struct {
struct imgu_fw_isp_parameter lin; /* lin_vmem_params */
struct imgu_fw_isp_parameter tnr3; /* tnr3_vmem_params */
struct imgu_fw_isp_parameter xnr3; /* xnr3_vmem_params */
} vmem;
struct {
struct imgu_fw_isp_parameter tnr;
struct imgu_fw_isp_parameter tnr3; /* tnr3_params */
struct imgu_fw_isp_parameter xnr3; /* xnr3_params */
struct imgu_fw_isp_parameter plane_io_config; /* 192 bytes */
struct imgu_fw_isp_parameter rgbir; /* rgbir_params */
} dmem;
};
struct imgu_fw_config_memory_offsets {
struct {
struct imgu_fw_isp_parameter iterator;
struct imgu_fw_isp_parameter dvs;
struct imgu_fw_isp_parameter output;
struct imgu_fw_isp_parameter raw;
struct imgu_fw_isp_parameter input_yuv;
struct imgu_fw_isp_parameter tnr;
struct imgu_fw_isp_parameter tnr3;
struct imgu_fw_isp_parameter ref;
} dmem;
};
struct imgu_fw_state_memory_offsets {
struct {
struct imgu_fw_isp_parameter tnr;
struct imgu_fw_isp_parameter tnr3;
struct imgu_fw_isp_parameter ref;
} dmem;
};
union imgu_fw_all_memory_offsets {
struct {
u64 imgu_fw_mem_offsets[3]; /* params, config, state */
} offsets;
struct {
u64 ptr;
} array[IMGU_ABI_PARAM_CLASS_NUM];
};
struct imgu_fw_binary_xinfo {
/* Part that is of interest to the SP. */
struct imgu_abi_binary_info sp;
/* Rest of the binary info, only interesting to the host. */
u32 type; /* enum imgu_fw_acc_type */
u32 num_output_formats __aligned(8);
u32 output_formats[IMGU_ABI_FRAME_FORMAT_NUM]; /* enum frame_format */
/* number of supported vf formats */
u32 num_vf_formats __aligned(8);
/* types of supported vf formats */
u32 vf_formats[IMGU_ABI_FRAME_FORMAT_NUM]; /* enum frame_format */
u8 num_output_pins;
imgu_fw_ptr xmem_addr;
u64 imgu_fw_blob_descr_ptr __aligned(8);
u32 blob_index __aligned(8);
union imgu_fw_all_memory_offsets mem_offsets __aligned(8);
struct imgu_fw_binary_xinfo *next __aligned(8);
};
struct imgu_fw_sp_info {
u32 init_dmem_data; /* data sect config, stored to dmem */
u32 per_frame_data; /* Per frame data, stored to dmem */
u32 group; /* Per pipeline data, loaded by dma */
u32 output; /* SP output data, loaded by dmem */
u32 host_sp_queue; /* Host <-> SP queues */
u32 host_sp_com; /* Host <-> SP commands */
u32 isp_started; /* P'ed from sensor thread, csim only */
u32 sw_state; /* Polled from css, enum imgu_abi_sp_swstate */
u32 host_sp_queues_initialized; /* Polled from the SP */
u32 sleep_mode; /* different mode to halt SP */
u32 invalidate_tlb; /* inform SP to invalidate mmu TLB */
u32 debug_buffer_ddr_address; /* the addr of DDR debug queue */
/* input system perf count array */
u32 perf_counter_input_system_error;
u32 threads_stack; /* sp thread's stack pointers */
u32 threads_stack_size; /* sp thread's stack sizes */
u32 curr_binary_id; /* current binary id */
u32 raw_copy_line_count; /* raw copy line counter */
u32 ddr_parameter_address; /* acc param ddrptr, sp dmem */
u32 ddr_parameter_size; /* acc param size, sp dmem */
/* Entry functions */
u32 sp_entry; /* The SP entry function */
u32 tagger_frames_addr; /* Base address of tagger state */
};
struct imgu_fw_bl_info {
u32 num_dma_cmds; /* Number of cmds sent by CSS */
u32 dma_cmd_list; /* Dma command list sent by CSS */
u32 sw_state; /* Polled from css, enum imgu_abi_bl_swstate */
/* Entry functions */
u32 bl_entry; /* The SP entry function */
};
struct imgu_fw_acc_info {
u32 per_frame_data; /* Dummy for now */
};
union imgu_fw_union {
struct imgu_fw_binary_xinfo isp; /* ISP info */
struct imgu_fw_sp_info sp; /* SP info */
struct imgu_fw_sp_info sp1; /* SP1 info */
struct imgu_fw_bl_info bl; /* Bootloader info */
struct imgu_fw_acc_info acc; /* Accelerator info */
};
struct imgu_fw_info {
size_t header_size; /* size of fw header */
u32 type __aligned(8); /* enum imgu_fw_type */
union imgu_fw_union info; /* Binary info */
struct imgu_abi_blob_info blob; /* Blob info */
/* Dynamic part */
u64 next;
u32 loaded __aligned(8); /* Firmware has been loaded */
const u64 isp_code __aligned(8); /* ISP pointer to code */
/* Firmware handle between user space and kernel */
u32 handle __aligned(8);
/* Sections to copy from/to ISP */
struct imgu_abi_isp_param_segments mem_initializers;
/* Initializer for local ISP memories */
};
struct imgu_fw_bi_file_h {
char version[64]; /* branch tag + week day + time */
int binary_nr; /* Number of binaries */
unsigned int h_size; /* sizeof(struct imgu_fw_bi_file_h) */
};
struct imgu_fw_header {
struct imgu_fw_bi_file_h file_header;
struct imgu_fw_info binary_header[1]; /* binary_nr items */
};
/******************* Firmware functions *******************/
int ipu3_css_fw_init(struct ipu3_css *css);
void ipu3_css_fw_cleanup(struct ipu3_css *css);
unsigned int ipu3_css_fw_obgrid_size(const struct imgu_fw_info *bi);
void *ipu3_css_fw_pipeline_params(struct ipu3_css *css, unsigned int pipe,
enum imgu_abi_param_class cls,
enum imgu_abi_memories mem,
struct imgu_fw_isp_parameter *par,
size_t par_size, void *binary_params);
#endif
This diff is collapsed.
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright (C) 2018 Intel Corporation */
#ifndef __IPU3_PARAMS_H
#define __IPU3_PARAMS_H
int ipu3_css_cfg_acc(struct ipu3_css *css, unsigned int pipe,
struct ipu3_uapi_flags *use,
struct imgu_abi_acc_param *acc,
struct imgu_abi_acc_param *acc_old,
struct ipu3_uapi_acc_param *acc_user);
int ipu3_css_cfg_vmem0(struct ipu3_css *css, unsigned int pipe,
struct ipu3_uapi_flags *use,
void *vmem0, void *vmem0_old,
struct ipu3_uapi_params *user);
int ipu3_css_cfg_dmem0(struct ipu3_css *css, unsigned int pipe,
struct ipu3_uapi_flags *use,
void *dmem0, void *dmem0_old,
struct ipu3_uapi_params *user);
void ipu3_css_cfg_gdc_table(struct imgu_abi_gdc_warp_param *gdc,
int frame_in_x, int frame_in_y,
int frame_out_x, int frame_out_y,
int env_w, int env_h);
#endif /*__IPU3_PARAMS_H */
// SPDX-License-Identifier: GPL-2.0
// Copyright (C) 2018 Intel Corporation
#include <linux/device.h>
#include "ipu3.h"
#include "ipu3-css-pool.h"
#include "ipu3-dmamap.h"
int ipu3_css_dma_buffer_resize(struct imgu_device *imgu,
struct ipu3_css_map *map, size_t size)
{
if (map->size < size && map->vaddr) {
dev_warn(&imgu->pci_dev->dev, "dma buf resized from %zu to %zu",
map->size, size);
ipu3_dmamap_free(imgu, map);
if (!ipu3_dmamap_alloc(imgu, map, size))
return -ENOMEM;
}
return 0;
}
void ipu3_css_pool_cleanup(struct imgu_device *imgu, struct ipu3_css_pool *pool)
{
unsigned int i;
for (i = 0; i < IPU3_CSS_POOL_SIZE; i++)
ipu3_dmamap_free(imgu, &pool->entry[i].param);
}
int ipu3_css_pool_init(struct imgu_device *imgu, struct ipu3_css_pool *pool,
size_t size)
{
unsigned int i;
for (i = 0; i < IPU3_CSS_POOL_SIZE; i++) {
pool->entry[i].valid = false;
if (size == 0) {
pool->entry[i].param.vaddr = NULL;
continue;
}
if (!ipu3_dmamap_alloc(imgu, &pool->entry[i].param, size))
goto fail;
}
pool->last = IPU3_CSS_POOL_SIZE;
return 0;
fail:
ipu3_css_pool_cleanup(imgu, pool);
return -ENOMEM;
}
/*
* Allocate a new parameter via recycling the oldest entry in the pool.
*/
void ipu3_css_pool_get(struct ipu3_css_pool *pool)
{
/* Get the oldest entry */
u32 n = (pool->last + 1) % IPU3_CSS_POOL_SIZE;
pool->entry[n].valid = true;
pool->last = n;
}
/*
* Undo, for all practical purposes, the effect of pool_get().
*/
void ipu3_css_pool_put(struct ipu3_css_pool *pool)
{
pool->entry[pool->last].valid = false;
pool->last = (pool->last + IPU3_CSS_POOL_SIZE - 1) % IPU3_CSS_POOL_SIZE;
}
/**
* ipu3_css_pool_last - Retrieve the nth pool entry from last
*
* @pool: a pointer to &struct ipu3_css_pool.
* @n: the distance to the last index.
*
* Returns:
* The nth entry from last or null map to indicate no frame stored.
*/
const struct ipu3_css_map *
ipu3_css_pool_last(struct ipu3_css_pool *pool, unsigned int n)
{
static const struct ipu3_css_map null_map = { 0 };
int i = (pool->last + IPU3_CSS_POOL_SIZE - n) % IPU3_CSS_POOL_SIZE;
WARN_ON(n >= IPU3_CSS_POOL_SIZE);
if (!pool->entry[i].valid)
return &null_map;
return &pool->entry[i].param;
}
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright (C) 2018 Intel Corporation */
#ifndef __IPU3_UTIL_H
#define __IPU3_UTIL_H
struct device;
struct imgu_device;
#define IPU3_CSS_POOL_SIZE 4
/**
* ipu3_css_map - store DMA mapping info for buffer
*
* @size: size of the buffer in bytes.
* @vaddr: kernel virtual address.
* @daddr: iova dma address to access IPU3.
* @vma: private, a pointer to &struct vm_struct,
* used for ipu3_dmamap_free.
*/
struct ipu3_css_map {
size_t size;
void *vaddr;
dma_addr_t daddr;
struct vm_struct *vma;
};
/**
* ipu3_css_pool - circular buffer pool definition
*
* @entry: array with IPU3_CSS_POOL_SIZE elements.
* @entry.param: a &struct ipu3_css_map for storing the mem mapping.
* @entry.valid: used to mark if the entry has valid data.
* @last: write pointer, initialized to IPU3_CSS_POOL_SIZE.
*/
struct ipu3_css_pool {
struct {
struct ipu3_css_map param;
bool valid;
} entry[IPU3_CSS_POOL_SIZE];
u32 last;
};
int ipu3_css_dma_buffer_resize(struct imgu_device *imgu,
struct ipu3_css_map *map, size_t size);
void ipu3_css_pool_cleanup(struct imgu_device *imgu,
struct ipu3_css_pool *pool);
int ipu3_css_pool_init(struct imgu_device *imgu, struct ipu3_css_pool *pool,
size_t size);
void ipu3_css_pool_get(struct ipu3_css_pool *pool);
void ipu3_css_pool_put(struct ipu3_css_pool *pool);
const struct ipu3_css_map *ipu3_css_pool_last(struct ipu3_css_pool *pool,
u32 last);
#endif
This diff is collapsed.
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright (C) 2018 Intel Corporation */
#ifndef __IPU3_CSS_H
#define __IPU3_CSS_H
#include <linux/videodev2.h>
#include <linux/types.h>
#include "ipu3-abi.h"
#include "ipu3-css-pool.h"
/* 2 stages for split isp pipeline, 1 for scaling */
#define IMGU_NUM_SP 2
#define IMGU_MAX_PIPELINE_NUM 20
#define IMGU_MAX_PIPE_NUM 2
/* For DVS etc., format FRAME_FMT_YUV420_16 */
#define IPU3_CSS_AUX_FRAME_REF 0
/* For temporal noise reduction DVS etc., format FRAME_FMT_YUV_LINE */
#define IPU3_CSS_AUX_FRAME_TNR 1
#define IPU3_CSS_AUX_FRAME_TYPES 2 /* REF and TNR */
#define IPU3_CSS_AUX_FRAMES 2 /* 2 for REF and 2 for TNR */
#define IPU3_CSS_QUEUE_IN 0
#define IPU3_CSS_QUEUE_PARAMS 1
#define IPU3_CSS_QUEUE_OUT 2
#define IPU3_CSS_QUEUE_VF 3
#define IPU3_CSS_QUEUE_STAT_3A 4
#define IPU3_CSS_QUEUES 5
#define IPU3_CSS_RECT_EFFECTIVE 0 /* Effective resolution */
#define IPU3_CSS_RECT_BDS 1 /* Resolution after BDS */
#define IPU3_CSS_RECT_ENVELOPE 2 /* DVS envelope size */
#define IPU3_CSS_RECT_GDC 3 /* gdc output res */
#define IPU3_CSS_RECTS 4 /* number of rects */
#define IA_CSS_BINARY_MODE_PRIMARY 2
#define IA_CSS_BINARY_MODE_VIDEO 3
#define IPU3_CSS_DEFAULT_BINARY 3 /* default binary index */
/*
* The pipe id type, distinguishes the kind of pipes that
* can be run in parallel.
*/
enum ipu3_css_pipe_id {
IPU3_CSS_PIPE_ID_PREVIEW,
IPU3_CSS_PIPE_ID_COPY,
IPU3_CSS_PIPE_ID_VIDEO,
IPU3_CSS_PIPE_ID_CAPTURE,
IPU3_CSS_PIPE_ID_YUVPP,
IPU3_CSS_PIPE_ID_ACC,
IPU3_CSS_PIPE_ID_NUM
};
struct ipu3_css_resolution {
u32 w;
u32 h;
};
enum ipu3_css_buffer_state {
IPU3_CSS_BUFFER_NEW, /* Not yet queued */
IPU3_CSS_BUFFER_QUEUED, /* Queued, waiting to be filled */
IPU3_CSS_BUFFER_DONE, /* Finished processing, removed from queue */
IPU3_CSS_BUFFER_FAILED, /* Was not processed, removed from queue */
};
struct ipu3_css_buffer {
/* Private fields: user doesn't touch */
dma_addr_t daddr;
unsigned int queue;
enum ipu3_css_buffer_state state;
struct list_head list;
u8 queue_pos;
unsigned int pipe;
};
struct ipu3_css_format {
u32 pixelformat;
enum v4l2_colorspace colorspace;
enum imgu_abi_frame_format frame_format;
enum imgu_abi_bayer_order bayer_order;
enum imgu_abi_osys_format osys_format;
enum imgu_abi_osys_tiling osys_tiling;
u32 bytesperpixel_num; /* Bytes per pixel in first plane * 50 */
u8 bit_depth; /* Effective bits per pixel */
u8 chroma_decim; /* Chroma plane decimation, 0=no chroma plane */
u8 width_align; /* Alignment requirement for width_pad */
u8 flags;
};
struct ipu3_css_queue {
union {
struct v4l2_pix_format_mplane mpix;
struct v4l2_meta_format meta;
} fmt;
const struct ipu3_css_format *css_fmt;
unsigned int width_pad;
struct list_head bufs;
};
struct ipu3_css_pipe {
enum ipu3_css_pipe_id pipe_id;
unsigned int bindex;
struct ipu3_css_queue queue[IPU3_CSS_QUEUES];
struct v4l2_rect rect[IPU3_CSS_RECTS];
bool vf_output_en;
/* Protect access to queue[IPU3_CSS_QUEUES] */
spinlock_t qlock;
/* Data structures shared with IMGU and driver, always allocated */
struct ipu3_css_map sp_ddr_ptrs;
struct ipu3_css_map xmem_sp_stage_ptrs[IPU3_CSS_PIPE_ID_NUM]
[IMGU_ABI_MAX_STAGES];
struct ipu3_css_map xmem_isp_stage_ptrs[IPU3_CSS_PIPE_ID_NUM]
[IMGU_ABI_MAX_STAGES];
/*
* Data structures shared with IMGU and driver, binary specific.
* PARAM_CLASS_CONFIG and PARAM_CLASS_STATE parameters.
*/
struct ipu3_css_map binary_params_cs[IMGU_ABI_PARAM_CLASS_NUM - 1]
[IMGU_ABI_NUM_MEMORIES];
struct {
struct ipu3_css_map mem[IPU3_CSS_AUX_FRAMES];
unsigned int width;
unsigned int height;
unsigned int bytesperline;
unsigned int bytesperpixel;
} aux_frames[IPU3_CSS_AUX_FRAME_TYPES];
struct {
struct ipu3_css_pool parameter_set_info;
struct ipu3_css_pool acc;
struct ipu3_css_pool gdc;
struct ipu3_css_pool obgrid;
/* PARAM_CLASS_PARAM parameters for binding while streaming */
struct ipu3_css_pool binary_params_p[IMGU_ABI_NUM_MEMORIES];
} pool;
struct ipu3_css_map abi_buffers[IPU3_CSS_QUEUES]
[IMGU_ABI_HOST2SP_BUFQ_SIZE];
};
/* IPU3 Camera Sub System structure */
struct ipu3_css {
struct device *dev;
void __iomem *base;
const struct firmware *fw;
struct imgu_fw_header *fwp;
int iomem_length;
int fw_bl, fw_sp[IMGU_NUM_SP]; /* Indices of bl and SP binaries */
struct ipu3_css_map *binary; /* fw binaries mapped to device */
bool streaming; /* true when streaming is enabled */
struct ipu3_css_pipe pipes[IMGU_MAX_PIPE_NUM];
struct ipu3_css_map xmem_sp_group_ptrs;
/* enabled pipe(s) */
DECLARE_BITMAP(enabled_pipes, IMGU_MAX_PIPE_NUM);
};
/******************* css v4l *******************/
int ipu3_css_init(struct device *dev, struct ipu3_css *css,
void __iomem *base, int length);
void ipu3_css_cleanup(struct ipu3_css *css);
int ipu3_css_fmt_try(struct ipu3_css *css,
struct v4l2_pix_format_mplane *fmts[IPU3_CSS_QUEUES],
struct v4l2_rect *rects[IPU3_CSS_RECTS],
unsigned int pipe);
int ipu3_css_fmt_set(struct ipu3_css *css,
struct v4l2_pix_format_mplane *fmts[IPU3_CSS_QUEUES],
struct v4l2_rect *rects[IPU3_CSS_RECTS],
unsigned int pipe);
int ipu3_css_meta_fmt_set(struct v4l2_meta_format *fmt);
int ipu3_css_buf_queue(struct ipu3_css *css, unsigned int pipe,
struct ipu3_css_buffer *b);
struct ipu3_css_buffer *ipu3_css_buf_dequeue(struct ipu3_css *css);
int ipu3_css_start_streaming(struct ipu3_css *css);
void ipu3_css_stop_streaming(struct ipu3_css *css);
bool ipu3_css_queue_empty(struct ipu3_css *css);
bool ipu3_css_is_streaming(struct ipu3_css *css);
bool ipu3_css_pipe_queue_empty(struct ipu3_css *css, unsigned int pipe);
/******************* css hw *******************/
int ipu3_css_set_powerup(struct device *dev, void __iomem *base);
void ipu3_css_set_powerdown(struct device *dev, void __iomem *base);
int ipu3_css_irq_ack(struct ipu3_css *css);
/******************* set parameters ************/
int ipu3_css_set_parameters(struct ipu3_css *css, unsigned int pipe,
struct ipu3_uapi_params *set_params);
/******************* auxiliary helpers *******************/
static inline enum ipu3_css_buffer_state
ipu3_css_buf_state(struct ipu3_css_buffer *b)
{
return b->state;
}
/* Initialize given buffer. May be called several times. */
static inline void ipu3_css_buf_init(struct ipu3_css_buffer *b,
unsigned int queue, dma_addr_t daddr)
{
b->state = IPU3_CSS_BUFFER_NEW;
b->queue = queue;
b->daddr = daddr;
}
#endif
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2018 Intel Corporation
* Copyright 2018 Google LLC.
*
* Author: Tomasz Figa <tfiga@chromium.org>
* Author: Yong Zhi <yong.zhi@intel.com>
*/
#include <linux/vmalloc.h>
#include "ipu3.h"
#include "ipu3-css-pool.h"
#include "ipu3-mmu.h"
/*
* Free a buffer allocated by ipu3_dmamap_alloc_buffer()
*/
static void ipu3_dmamap_free_buffer(struct page **pages,
size_t size)
{
int count = size >> PAGE_SHIFT;
while (count--)
__free_page(pages[count]);
kvfree(pages);
}
/*
* Based on the implementation of __iommu_dma_alloc_pages()
* defined in drivers/iommu/dma-iommu.c
*/
static struct page **ipu3_dmamap_alloc_buffer(size_t size,
unsigned long order_mask,
gfp_t gfp)
{
struct page **pages;
unsigned int i = 0, count = size >> PAGE_SHIFT;
const gfp_t high_order_gfp = __GFP_NOWARN | __GFP_NORETRY;
/* Allocate mem for array of page ptrs */
pages = kvmalloc_array(count, sizeof(*pages), GFP_KERNEL);
if (!pages)
return NULL;
order_mask &= (2U << MAX_ORDER) - 1;
if (!order_mask)
return NULL;
gfp |= __GFP_HIGHMEM | __GFP_ZERO;
while (count) {
struct page *page = NULL;
unsigned int order_size;
for (order_mask &= (2U << __fls(count)) - 1;
order_mask; order_mask &= ~order_size) {
unsigned int order = __fls(order_mask);
order_size = 1U << order;
page = alloc_pages((order_mask - order_size) ?
gfp | high_order_gfp : gfp, order);
if (!page)
continue;
if (!order)
break;
if (!PageCompound(page)) {
split_page(page, order);
break;
}
__free_pages(page, order);
}
if (!page) {
ipu3_dmamap_free_buffer(pages, i << PAGE_SHIFT);
return NULL;
}
count -= order_size;
while (order_size--)
pages[i++] = page++;
}
return pages;
}
/**
* ipu3_dmamap_alloc - allocate and map a buffer into KVA
* @imgu: struct device pointer
* @map: struct to store mapping variables
* @len: size required
*
* Returns:
* KVA on success
* %NULL on failure
*/
void *ipu3_dmamap_alloc(struct imgu_device *imgu, struct ipu3_css_map *map,
size_t len)
{
unsigned long shift = iova_shift(&imgu->iova_domain);
unsigned int alloc_sizes = imgu->mmu->pgsize_bitmap;
struct device *dev = &imgu->pci_dev->dev;
size_t size = PAGE_ALIGN(len);
struct page **pages;
dma_addr_t iovaddr;
struct iova *iova;
int i, rval;
dev_dbg(dev, "%s: allocating %zu\n", __func__, size);
iova = alloc_iova(&imgu->iova_domain, size >> shift,
imgu->mmu->aperture_end >> shift, 0);
if (!iova)
return NULL;
pages = ipu3_dmamap_alloc_buffer(size, alloc_sizes >> PAGE_SHIFT,
GFP_KERNEL);
if (!pages)
goto out_free_iova;
/* Call IOMMU driver to setup pgt */
iovaddr = iova_dma_addr(&imgu->iova_domain, iova);
for (i = 0; i < size / PAGE_SIZE; ++i) {
rval = ipu3_mmu_map(imgu->mmu, iovaddr,
page_to_phys(pages[i]), PAGE_SIZE);
if (rval)
goto out_unmap;
iovaddr += PAGE_SIZE;
}
/* Now grab a virtual region */
map->vma = __get_vm_area(size, VM_USERMAP, VMALLOC_START, VMALLOC_END);
if (!map->vma)
goto out_unmap;
map->vma->pages = pages;
/* And map it in KVA */
if (map_vm_area(map->vma, PAGE_KERNEL, pages))
goto out_vunmap;
map->size = size;
map->daddr = iova_dma_addr(&imgu->iova_domain, iova);
map->vaddr = map->vma->addr;
dev_dbg(dev, "%s: allocated %zu @ IOVA %pad @ VA %p\n", __func__,
size, &map->daddr, map->vma->addr);
return map->vma->addr;
out_vunmap:
vunmap(map->vma->addr);
out_unmap:
ipu3_dmamap_free_buffer(pages, size);
ipu3_mmu_unmap(imgu->mmu, iova_dma_addr(&imgu->iova_domain, iova),
i * PAGE_SIZE);
map->vma = NULL;
out_free_iova:
__free_iova(&imgu->iova_domain, iova);
return NULL;
}
void ipu3_dmamap_unmap(struct imgu_device *imgu, struct ipu3_css_map *map)
{
struct iova *iova;
iova = find_iova(&imgu->iova_domain,
iova_pfn(&imgu->iova_domain, map->daddr));
if (WARN_ON(!iova))
return;
ipu3_mmu_unmap(imgu->mmu, iova_dma_addr(&imgu->iova_domain, iova),
iova_size(iova) << iova_shift(&imgu->iova_domain));
__free_iova(&imgu->iova_domain, iova);
}
/*
* Counterpart of ipu3_dmamap_alloc
*/
void ipu3_dmamap_free(struct imgu_device *imgu, struct ipu3_css_map *map)
{
struct vm_struct *area = map->vma;
dev_dbg(&imgu->pci_dev->dev, "%s: freeing %zu @ IOVA %pad @ VA %p\n",
__func__, map->size, &map->daddr, map->vaddr);
if (!map->vaddr)
return;
ipu3_dmamap_unmap(imgu, map);
if (WARN_ON(!area) || WARN_ON(!area->pages))
return;
ipu3_dmamap_free_buffer(area->pages, map->size);
vunmap(map->vaddr);
map->vaddr = NULL;
}
int ipu3_dmamap_map_sg(struct imgu_device *imgu, struct scatterlist *sglist,
int nents, struct ipu3_css_map *map)
{
unsigned long shift = iova_shift(&imgu->iova_domain);
struct scatterlist *sg;
struct iova *iova;
size_t size = 0;
int i;
for_each_sg(sglist, sg, nents, i) {
if (sg->offset)
return -EINVAL;
if (i != nents - 1 && !PAGE_ALIGNED(sg->length))
return -EINVAL;
size += sg->length;
}
size = iova_align(&imgu->iova_domain, size);
dev_dbg(&imgu->pci_dev->dev, "dmamap: mapping sg %d entries, %zu pages\n",
nents, size >> shift);
iova = alloc_iova(&imgu->iova_domain, size >> shift,
imgu->mmu->aperture_end >> shift, 0);
if (!iova)
return -ENOMEM;
dev_dbg(&imgu->pci_dev->dev, "dmamap: iova low pfn %lu, high pfn %lu\n",
iova->pfn_lo, iova->pfn_hi);
if (ipu3_mmu_map_sg(imgu->mmu, iova_dma_addr(&imgu->iova_domain, iova),
sglist, nents) < size)
goto out_fail;
memset(map, 0, sizeof(*map));
map->daddr = iova_dma_addr(&imgu->iova_domain, iova);
map->size = size;
return 0;
out_fail:
__free_iova(&imgu->iova_domain, iova);
return -EFAULT;
}
int ipu3_dmamap_init(struct imgu_device *imgu)
{
unsigned long order, base_pfn;
int ret = iova_cache_get();
if (ret)
return ret;
order = __ffs(imgu->mmu->pgsize_bitmap);
base_pfn = max_t(unsigned long, 1, imgu->mmu->aperture_start >> order);
init_iova_domain(&imgu->iova_domain, 1UL << order, base_pfn);
return 0;
}
void ipu3_dmamap_exit(struct imgu_device *imgu)
{
put_iova_domain(&imgu->iova_domain);
iova_cache_put();
}
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright (C) 2018 Intel Corporation */
/* Copyright 2018 Google LLC. */
#ifndef __IPU3_DMAMAP_H
#define __IPU3_DMAMAP_H
struct imgu_device;
struct scatterlist;
void *ipu3_dmamap_alloc(struct imgu_device *imgu, struct ipu3_css_map *map,
size_t len);
void ipu3_dmamap_free(struct imgu_device *imgu, struct ipu3_css_map *map);
int ipu3_dmamap_map_sg(struct imgu_device *imgu, struct scatterlist *sglist,
int nents, struct ipu3_css_map *map);
void ipu3_dmamap_unmap(struct imgu_device *imgu, struct ipu3_css_map *map);
int ipu3_dmamap_init(struct imgu_device *imgu);
void ipu3_dmamap_exit(struct imgu_device *imgu);
#endif
This diff is collapsed.
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright (C) 2018 Intel Corporation */
/* Copyright 2018 Google LLC. */
#ifndef __IPU3_MMU_H
#define __IPU3_MMU_H
/**
* struct ipu3_mmu_info - Describes mmu geometry
*
* @aperture_start: First address that can be mapped
* @aperture_end: Last address that can be mapped
* @pgsize_bitmap: Bitmap of page sizes in use
*/
struct ipu3_mmu_info {
dma_addr_t aperture_start;
dma_addr_t aperture_end;
unsigned long pgsize_bitmap;
};
struct device;
struct scatterlist;
struct ipu3_mmu_info *ipu3_mmu_init(struct device *parent, void __iomem *base);
void ipu3_mmu_exit(struct ipu3_mmu_info *info);
void ipu3_mmu_suspend(struct ipu3_mmu_info *info);
void ipu3_mmu_resume(struct ipu3_mmu_info *info);
int ipu3_mmu_map(struct ipu3_mmu_info *info, unsigned long iova,
phys_addr_t paddr, size_t size);
size_t ipu3_mmu_unmap(struct ipu3_mmu_info *info, unsigned long iova,
size_t size);
size_t ipu3_mmu_map_sg(struct ipu3_mmu_info *info, unsigned long iova,
struct scatterlist *sg, unsigned int nents);
#endif
This diff is collapsed.
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright (C) 2018 Intel Corporation */
#ifndef __IPU3_TABLES_H
#define __IPU3_TABLES_H
#include "ipu3-abi.h"
#define IMGU_BDS_GRANULARITY 32 /* Downscaling granularity */
#define IMGU_BDS_MIN_SF_INV IMGU_BDS_GRANULARITY
#define IMGU_BDS_CONFIG_LEN 97
#define IMGU_SCALER_DOWNSCALE_4TAPS_LEN 128
#define IMGU_SCALER_DOWNSCALE_2TAPS_LEN 64
#define IMGU_SCALER_FP ((u32)1 << 31) /* 1.0 in fixed point */
#define IMGU_XNR3_VMEM_LUT_LEN 16
#define IMGU_GDC_LUT_UNIT 4
#define IMGU_GDC_LUT_LEN 256
struct ipu3_css_bds_config {
struct imgu_abi_bds_phase_arr hor_phase_arr;
struct imgu_abi_bds_phase_arr ver_phase_arr;
struct imgu_abi_bds_ptrn_arr ptrn_arr;
u16 sample_patrn_length;
u8 hor_ds_en;
u8 ver_ds_en;
};
struct ipu3_css_xnr3_vmem_defaults {
s16 x[IMGU_XNR3_VMEM_LUT_LEN];
s16 a[IMGU_XNR3_VMEM_LUT_LEN];
s16 b[IMGU_XNR3_VMEM_LUT_LEN];
s16 c[IMGU_XNR3_VMEM_LUT_LEN];
};
extern const struct ipu3_css_bds_config
ipu3_css_bds_configs[IMGU_BDS_CONFIG_LEN];
extern const s32 ipu3_css_downscale_4taps[IMGU_SCALER_DOWNSCALE_4TAPS_LEN];
extern const s32 ipu3_css_downscale_2taps[IMGU_SCALER_DOWNSCALE_2TAPS_LEN];
extern const s16 ipu3_css_gdc_lut[IMGU_GDC_LUT_UNIT][IMGU_GDC_LUT_LEN];
extern const struct ipu3_css_xnr3_vmem_defaults ipu3_css_xnr3_vmem_defaults;
extern const struct ipu3_uapi_bnr_static_config ipu3_css_bnr_defaults;
extern const struct ipu3_uapi_dm_config ipu3_css_dm_defaults;
extern const struct ipu3_uapi_ccm_mat_config ipu3_css_ccm_defaults;
extern const struct ipu3_uapi_gamma_corr_lut ipu3_css_gamma_lut;
extern const struct ipu3_uapi_csc_mat_config ipu3_css_csc_defaults;
extern const struct ipu3_uapi_cds_params ipu3_css_cds_defaults;
extern const struct ipu3_uapi_shd_config_static ipu3_css_shd_defaults;
extern const struct ipu3_uapi_yuvp1_iefd_config ipu3_css_iefd_defaults;
extern const struct ipu3_uapi_yuvp1_yds_config ipu3_css_yds_defaults;
extern const struct ipu3_uapi_yuvp1_chnr_config ipu3_css_chnr_defaults;
extern const struct ipu3_uapi_yuvp1_y_ee_nr_config ipu3_css_y_ee_nr_defaults;
extern const struct ipu3_uapi_yuvp2_tcc_gain_pcwl_lut_static_config
ipu3_css_tcc_gain_pcwl_lut;
extern const struct ipu3_uapi_yuvp2_tcc_r_sqr_lut_static_config
ipu3_css_tcc_r_sqr_lut;
extern const struct imgu_abi_anr_config ipu3_css_anr_defaults;
extern const struct ipu3_uapi_awb_fr_config_s ipu3_css_awb_fr_defaults;
extern const struct ipu3_uapi_ae_grid_config ipu3_css_ae_grid_defaults;
extern const struct ipu3_uapi_ae_ccm ipu3_css_ae_ccm_defaults;
extern const struct ipu3_uapi_af_config_s ipu3_css_af_defaults;
extern const struct ipu3_uapi_awb_config_s ipu3_css_awb_defaults;
#endif
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
...@@ -48,6 +48,9 @@ struct v4l2_fh; ...@@ -48,6 +48,9 @@ struct v4l2_fh;
* @vidioc_enum_fmt_meta_cap: pointer to the function that implements * @vidioc_enum_fmt_meta_cap: pointer to the function that implements
* :ref:`VIDIOC_ENUM_FMT <vidioc_enum_fmt>` ioctl logic * :ref:`VIDIOC_ENUM_FMT <vidioc_enum_fmt>` ioctl logic
* for metadata capture * for metadata capture
* @vidioc_enum_fmt_meta_out: pointer to the function that implements
* :ref:`VIDIOC_ENUM_FMT <vidioc_enum_fmt>` ioctl logic
* for metadata output
* @vidioc_g_fmt_vid_cap: pointer to the function that implements * @vidioc_g_fmt_vid_cap: pointer to the function that implements
* :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture
* in single plane mode * in single plane mode
...@@ -80,6 +83,8 @@ struct v4l2_fh; ...@@ -80,6 +83,8 @@ struct v4l2_fh;
* Radio output * Radio output
* @vidioc_g_fmt_meta_cap: pointer to the function that implements * @vidioc_g_fmt_meta_cap: pointer to the function that implements
* :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for metadata capture * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for metadata capture
* @vidioc_g_fmt_meta_out: pointer to the function that implements
* :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for metadata output
* @vidioc_s_fmt_vid_cap: pointer to the function that implements * @vidioc_s_fmt_vid_cap: pointer to the function that implements
* :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture
* in single plane mode * in single plane mode
...@@ -112,6 +117,8 @@ struct v4l2_fh; ...@@ -112,6 +117,8 @@ struct v4l2_fh;
* Radio output * Radio output
* @vidioc_s_fmt_meta_cap: pointer to the function that implements * @vidioc_s_fmt_meta_cap: pointer to the function that implements
* :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for metadata capture * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for metadata capture
* @vidioc_s_fmt_meta_out: pointer to the function that implements
* :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for metadata output
* @vidioc_try_fmt_vid_cap: pointer to the function that implements * @vidioc_try_fmt_vid_cap: pointer to the function that implements
* :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture
* in single plane mode * in single plane mode
...@@ -146,6 +153,8 @@ struct v4l2_fh; ...@@ -146,6 +153,8 @@ struct v4l2_fh;
* Radio output * Radio output
* @vidioc_try_fmt_meta_cap: pointer to the function that implements * @vidioc_try_fmt_meta_cap: pointer to the function that implements
* :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for metadata capture * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for metadata capture
* @vidioc_try_fmt_meta_out: pointer to the function that implements
* :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for metadata output
* @vidioc_reqbufs: pointer to the function that implements * @vidioc_reqbufs: pointer to the function that implements
* :ref:`VIDIOC_REQBUFS <vidioc_reqbufs>` ioctl * :ref:`VIDIOC_REQBUFS <vidioc_reqbufs>` ioctl
* @vidioc_querybuf: pointer to the function that implements * @vidioc_querybuf: pointer to the function that implements
...@@ -314,6 +323,8 @@ struct v4l2_ioctl_ops { ...@@ -314,6 +323,8 @@ struct v4l2_ioctl_ops {
struct v4l2_fmtdesc *f); struct v4l2_fmtdesc *f);
int (*vidioc_enum_fmt_meta_cap)(struct file *file, void *fh, int (*vidioc_enum_fmt_meta_cap)(struct file *file, void *fh,
struct v4l2_fmtdesc *f); struct v4l2_fmtdesc *f);
int (*vidioc_enum_fmt_meta_out)(struct file *file, void *fh,
struct v4l2_fmtdesc *f);
/* VIDIOC_G_FMT handlers */ /* VIDIOC_G_FMT handlers */
int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh,
...@@ -342,6 +353,8 @@ struct v4l2_ioctl_ops { ...@@ -342,6 +353,8 @@ struct v4l2_ioctl_ops {
struct v4l2_format *f); struct v4l2_format *f);
int (*vidioc_g_fmt_meta_cap)(struct file *file, void *fh, int (*vidioc_g_fmt_meta_cap)(struct file *file, void *fh,
struct v4l2_format *f); struct v4l2_format *f);
int (*vidioc_g_fmt_meta_out)(struct file *file, void *fh,
struct v4l2_format *f);
/* VIDIOC_S_FMT handlers */ /* VIDIOC_S_FMT handlers */
int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh,
...@@ -370,6 +383,8 @@ struct v4l2_ioctl_ops { ...@@ -370,6 +383,8 @@ struct v4l2_ioctl_ops {
struct v4l2_format *f); struct v4l2_format *f);
int (*vidioc_s_fmt_meta_cap)(struct file *file, void *fh, int (*vidioc_s_fmt_meta_cap)(struct file *file, void *fh,
struct v4l2_format *f); struct v4l2_format *f);
int (*vidioc_s_fmt_meta_out)(struct file *file, void *fh,
struct v4l2_format *f);
/* VIDIOC_TRY_FMT handlers */ /* VIDIOC_TRY_FMT handlers */
int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh,
...@@ -398,6 +413,8 @@ struct v4l2_ioctl_ops { ...@@ -398,6 +413,8 @@ struct v4l2_ioctl_ops {
struct v4l2_format *f); struct v4l2_format *f);
int (*vidioc_try_fmt_meta_cap)(struct file *file, void *fh, int (*vidioc_try_fmt_meta_cap)(struct file *file, void *fh,
struct v4l2_format *f); struct v4l2_format *f);
int (*vidioc_try_fmt_meta_out)(struct file *file, void *fh,
struct v4l2_format *f);
/* Buffer handlers */ /* Buffer handlers */
int (*vidioc_reqbufs)(struct file *file, void *fh, int (*vidioc_reqbufs)(struct file *file, void *fh,
......
...@@ -145,6 +145,7 @@ enum v4l2_buf_type { ...@@ -145,6 +145,7 @@ enum v4l2_buf_type {
V4L2_BUF_TYPE_SDR_CAPTURE = 11, V4L2_BUF_TYPE_SDR_CAPTURE = 11,
V4L2_BUF_TYPE_SDR_OUTPUT = 12, V4L2_BUF_TYPE_SDR_OUTPUT = 12,
V4L2_BUF_TYPE_META_CAPTURE = 13, V4L2_BUF_TYPE_META_CAPTURE = 13,
V4L2_BUF_TYPE_META_OUTPUT = 14,
/* Deprecated, do not use */ /* Deprecated, do not use */
V4L2_BUF_TYPE_PRIVATE = 0x80, V4L2_BUF_TYPE_PRIVATE = 0x80,
}; };
...@@ -469,6 +470,7 @@ struct v4l2_capability { ...@@ -469,6 +470,7 @@ struct v4l2_capability {
#define V4L2_CAP_READWRITE 0x01000000 /* read/write systemcalls */ #define V4L2_CAP_READWRITE 0x01000000 /* read/write systemcalls */
#define V4L2_CAP_ASYNCIO 0x02000000 /* async I/O */ #define V4L2_CAP_ASYNCIO 0x02000000 /* async I/O */
#define V4L2_CAP_STREAMING 0x04000000 /* streaming I/O ioctls */ #define V4L2_CAP_STREAMING 0x04000000 /* streaming I/O ioctls */
#define V4L2_CAP_META_OUTPUT 0x08000000 /* Is a metadata output device */
#define V4L2_CAP_TOUCH 0x10000000 /* Is a touch device */ #define V4L2_CAP_TOUCH 0x10000000 /* Is a touch device */
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment