Flush mechanism for DSPP blocks has changed in sc7280 family, it
allows individual sub blocks to be flushed in coordination with
master flush control.

Representation: master_flush && (PCC_flush | IGC_flush .. etc )

This change adds necessary support for the above design.

Changes in v1:
- Few nits (Doug, Dmitry)
- Restrict sub-block flush programming to dpu_hw_ctl file (Dmitry)

Changes in v2:
- Move the address offset to flush macro (Dmitry)
- Separate ops for the sub block flush (Dmitry)

Changes in v3:
- Reuse the DPU_DSPP_xx enum instead of a new one (Dmitry)

Changes in v4:
- Use shorter version for unsigned int (Stephen)

Changes in v5:
- Spurious patch please ignore.

Changes in v6:
- Add SOB tag (Doug, Dmitry)

Changes in v7:
- Cache flush mask per dspp (Dmitry)
- Few nits (Marijn)

Changes in v8:
- Few nits (Marijn)

Changes in v9:
- Use DSPP enum while accessing flush mask to make it readable (Dmitry)
- Few nits (Dmitry)

Changes in v10:
- Fix white spaces in a separate patch (Dmitry)

Changes in v11:
- Define a macro for dspp flush selection (Marijn)
- Few nits (Marijn)

Changes in v12:
- Minor comments (reorder macros and a condition) (Marijn)
Signed-off-by: Kalyan Thota <quic_kalyant@quicinc.com>
Tested-by: Douglas Anderson <dianders@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/520701/
Link: https://lore.kernel.org/r/1674814487-2112-1-git-send-email-quic_kalyant@quicinc.comSigned-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

Document the DPU hardware found on the Qualcomm SM8550 platform.
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/517511/
Link: https://lore.kernel.org/r/20230103-topic-sm8550-upstream-mdss-dsi-v3-2-660c3bcb127f@linaro.org
[DB: removed interconnect header inclusion]
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

Now that the TE setup has been moved to prepare_for_kickoff(),  we have
not prepare_commit() callbacks left. This makes dpu_encoder_prepare_commit()
do nothing. Remove prepare_commit() from DPU driver.

Changes in V3:
- Reworded commit message to be more clear
- Corrected spelling mistake in commit message

Changes in V4:
- Reworded commit message for clarity
Signed-off-by: Jessica Zhang <quic_jesszhan@quicinc.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Reviewed-by: Marijn Suijten <marijn.suijten@somainline.org>
Patchwork: https://patchwork.freedesktop.org/patch/523606/
Link: https://lore.kernel.org/r/20230221184256.1436-4-quic_jesszhan@quicinc.com
[DB: fixed merge conflict]
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

Currently, DPU will enable TE during prepare_commit(). However, this
will cause a crash and reboot to sahara when trying to read/write to
register in get_autorefresh_config(), because the core clock rates
aren't set at that time.

This used to work because phys_enc->hw_pp is only initialized in mode
set [1], so the first prepare_commit() will return before any register
read/write as hw_pp would be NULL.

However, when we try to implement support for INTF TE, we will run into
the clock issue described above as hw_intf will *not* be NULL on the
first prepare_commit(). This is because the initialization of
dpu_enc->hw_intf has been moved to dpu_encoder_setup() [2].

To avoid this issue, let's enable TE during prepare_for_kickoff()
instead as the core clock rates are guaranteed to be set then.

Depends on: "Implement tearcheck support on INTF block" [3]

Changes in V3:
- Added function prototypes
- Reordered function definitions to make change more legible
- Removed prepare_commit() function from dpu_encoder_phys_cmd

Changes in V4:
- Reworded commit message to be more specific
- Removed dpu_encoder_phys_cmd_is_ongoing_pptx() prototype

[1] https://gitlab.freedesktop.org/drm/msm/-/blob/msm-next/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c#L1109
[2] https://gitlab.freedesktop.org/drm/msm/-/blob/msm-next/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c#L2339
[3] https://patchwork.freedesktop.org/series/112332/Signed-off-by: Jessica Zhang <quic_jesszhan@quicinc.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Reviewed-by: Marijn Suijten <marijn.suijten@somainline.org>
Patchwork: https://patchwork.freedesktop.org/patch/523602/
Link: https://lore.kernel.org/r/20230221184256.1436-2-quic_jesszhan@quicinc.comSigned-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

Add a NULL check before calling prepare_commit() in
msm_atomic_commit_tail()
Signed-off-by: Jessica Zhang <quic_jesszhan@quicinc.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Reviewed-by: Marijn Suijten <marijn.suijten@somainline.org>
Patchwork: https://patchwork.freedesktop.org/patch/523604/
Link: https://lore.kernel.org/r/20230221184256.1436-3-quic_jesszhan@quicinc.comSigned-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

The adreno_load_gpu() path is guarded by an error check on
adreno_load_fw(). This function is responsible for loading
Qualcomm-only-signed binaries (e.g. SQE and GMU FW for A6XX), but it
does not take the vendor-signed ZAP blob into account.

By embedding the SQE (and GMU, if necessary) firmware into the
initrd/kernel, we can trigger and unfortunate path that would not bail
out early and proceed with gpu->hw_init(). That will fail, as the ZAP
loader path will not find the firmware and return back to
adreno_load_gpu().

This error path involves pm_runtime_put_sync() which then calls idle()
instead of suspend(). This is suboptimal, as it means that we're not
going through the clean shutdown sequence. With at least A619_holi, this
makes the GPU not wake up until it goes through at least one more
start-fail-stop cycle. The pm_runtime_put_sync that appears in the error
path actually does not guarantee that because of the earlier enabling of
runtime autosuspend.

Fix that by using pm_runtime_put_sync_suspend to force a clean shutdown.

Test cases:
1. All firmware baked into kernel
2. error loading ZAP fw in initrd -> load from rootfs at DE start

Both succeed on A619_holi (SM6375) and A630 (SDM845).

Fixes: 0d997f95 ("drm/msm/adreno: fix runtime PM imbalance at gpu load")
Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Reviewed-by: Johan Hovold <johan+linaro@kernel.org>
Patchwork: https://patchwork.freedesktop.org/patch/530001/
Link: https://lore.kernel.org/r/20230330231517.2747024-1-konrad.dybcio@linaro.orgSigned-off-by: Rob Clark <robdclark@chromium.org>

Recently I contributed the switch to OPP API for all Adreno generations.
I did however also skip over the fact that GPUs with a GMU don't specify
a core clock of any kind in the GPU node. While that didn't break
anything, it did introduce unwanted spam in the dmesg:

adreno 5000000.gpu: error -ENOENT: _opp_set_clknames: Couldn't find clock with name: core_clk

Guard the entire logic so that it's not used with GMU-equipped GPUs.

Fixes: 9f251f93 ("drm/msm/adreno: Use OPP for every GPU generation")
Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/530347/
Link: https://lore.kernel.org/r/20230223-topic-gmuwrapper-v6-1-2034115bb60c@linaro.orgSigned-off-by: Rob Clark <robdclark@chromium.org>

Add support for matching QFPROM fuse values to get the correct speed bin
on A650 (SM8250) GPUs.
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/530043/
Link: https://lore.kernel.org/r/20230331-topic-konahana_speedbin-v3-2-2dede22dd7f7@linaro.orgSigned-off-by: Rob Clark <robdclark@chromium.org>

Add support for matching QFPROM fuse values to get the correct speed bin
on A640 (SM8150) GPUs.
Reviewed-by: Akhil P Oommen <quic_akhilpo@quicinc.com>
Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/530042/
Link: https://lore.kernel.org/r/20230331-topic-konahana_speedbin-v3-1-2dede22dd7f7@linaro.orgSigned-off-by: Rob Clark <robdclark@chromium.org>

Use adreno_fault_handler() to implement a5xx_fault_handler(). This
enables devcoredump support on a5xx platforms, allowing one to capture
the crashed GPU state at the time of context fault.
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/522724/
Link: https://lore.kernel.org/r/20230214123504.3729522-4-dmitry.baryshkov@linaro.orgSigned-off-by: Rob Clark <robdclark@chromium.org>

Split the a6xx_fault_handler() into the generic adreno_fault_handler()
and platform-specific parts. The adreno_fault_handler() can further be
used by a5xx and hopefully by a4xx (at some point).
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/522722/
Link: https://lore.kernel.org/r/20230214123504.3729522-3-dmitry.baryshkov@linaro.orgSigned-off-by: Rob Clark <robdclark@chromium.org>

The commit e25e92e0 ("drm/msm: devcoredump iommu fault support")
enabled SMMU stalling to collect GPU state, but only for a6xx. It tied
enabling the stall with tha per-instance pagetables creation.

Since that commit SoCs with a5xx also gained support for
adreno-smmu-priv. Move stalling into generic code and add corresponding
resume_translation calls.
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/522720/
Link: https://lore.kernel.org/r/20230214123504.3729522-2-dmitry.baryshkov@linaro.orgSigned-off-by: Rob Clark <robdclark@chromium.org>

Drop our custom thing and switch to drm_crtc_next_vblank_start() for
calculating the time of the start of the next vblank period.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/525819/
Link: https://lore.kernel.org/r/20230308155322.344664-15-robdclark@gmail.com

Add a way for various userspace waits to signal urgency.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/525817/
Link: https://lore.kernel.org/r/20230308155322.344664-14-robdclark@gmail.com

Track the nearest deadline on a fence timeline and set a timer to expire
shortly before to trigger boost if the fence has not yet been signaled.

v2: rebase
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/525816/
Link: https://lore.kernel.org/r/20230308155322.344664-13-robdclark@gmail.com

This series adds a deadline hint to fences, so realtime deadlines
such as vblank can be communicated to the fence signaller for power/
frequency management decisions.

This is partially inspired by a trick i915 does, but implemented
via dma-fence for a couple of reasons:

1) To continue to be able to use the atomic helpers
2) To support cases where display and gpu are different drivers

See https://patchwork.freedesktop.org/series/93035/

This does not yet add any UAPI, although this will be needed in
a number of cases:

1) Workloads "ping-ponging" between CPU and GPU, where we don't
   want the GPU freq governor to interpret time stalled waiting
   for GPU as "idle" time
2) Cases where the compositor is waiting for fences to be signaled
   before issuing the atomic ioctl, for example to maintain 60fps
   cursor updates even when the GPU is not able to maintain that
   framerate.
Signed-off-by: Rob Clark <robdclark@chromium.org>

For an atomic commit updating a single CRTC (ie. a pageflip) calculate
the next vblank time, and inform the fence(s) of that deadline.

v2: Comment typo fix (danvet)
v3: If there are multiple CRTCs, consider the time of the soonest vblank
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>

Will be used in the next commit to set a deadline on fences that an
atomic update is waiting on.

v2: Calculate time at *start* of vblank period, not end
v3: Fix kbuild complaints
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Mario Kleiner <mario.kleiner.de@gmail.com>

As the finished fence is the one that is exposed to userspace, and
therefore the one that other operations, like atomic update, would
block on, we need to propagate the deadline from from the finished
fence to the actual hw fence.

v2: Split into drm_sched_fence_set_parent() (ckoenig)
v3: Ensure a thread calling drm_sched_fence_set_deadline_finished() sees
    fence->parent set before drm_sched_fence_set_parent() does this
    test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT).
Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Luben Tuikov <luben.tuikov@amd.com>

We had all of the internal driver APIs, but not the all important
userspace uABI, in the dma-buf doc.  Fix that.  And re-arrange the
comments slightly as otherwise the comments for the ioctl nr defines
would not show up.

v2: Fix docs build warning coming from newly including the uabi header
    in the docs build
Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Pekka Paalanen <pekka.paalanen@collabora.com>

Add a way to set a deadline on remaining resv fences according to the
requested usage.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Christian König <christian.koenig@amd.com>

Propagate the deadline to all the fences in the chain.

v2: Use dma_fence_chain_contained [Tvrtko]
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Christian König <christian.koenig@amd.com>

Move the one-time RPMh setup to a6xx_gmu_init().  To get rid of the hack
for one-time init vs start, add in an extra a6xx_rpmh_stop() at the end
of the init sequence.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527854/
Link: https://lore.kernel.org/r/20230320144356.803762-15-robdclark@gmail.com

These allocations are only done the first (successful) time through
hw_init() so they won't actually happen in the job_run() path.  But
lockdep doesn't know this.  So dis-entangle them from the hw_init()
path.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527852/
Link: https://lore.kernel.org/r/20230320144356.803762-14-robdclark@gmail.com

It is already a no-op, since we've already loaded the fw from
adreno_load_gpu(), so drop the redundant call.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527849/
Link: https://lore.kernel.org/r/20230320144356.803762-13-robdclark@gmail.com

Avoid allocation under idr_lock, to prevent deadlock against the
job_free() path (which runs on same thread as job_run(), which makes
it also part of the fence-signaling path.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527847/
Link: https://lore.kernel.org/r/20230320144356.803762-12-robdclark@gmail.com

Needed to idr_preload() which returns with preemption disabled.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527846/
Link: https://lore.kernel.org/r/20230320144356.803762-11-robdclark@gmail.com

Now that everything that controls which LRU an obj lives in *except* the
backing pages is protected by the LRU lock, add a special path to unpin
in the job_run() path, where we are assured that we already have backing
pages and will not be racing against eviction (because the GEM object's
dma_resv contains the fence that will be signaled when the submit/job
completes).
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527845/
Link: https://lore.kernel.org/r/20230320144356.803762-10-robdclark@gmail.com

Since the LRU lock is already acquired when moving an obj between LRUs,
we can use it to protect pin_count and madv, without any significant
change in locking (ie. it just expands the scope of the lock by a hand-
ful of instructions).  This prepares the way to decrement the pin_count
in the job_run() path without needing to hold the obj lock, to avoid a
potential deadlock (or rather stall) caused by the fence-signaling path
(job_run()) blocking on shrinker/reclaim.  (Only a stall because the
wait for fence signaling wait_for_idle() is not infinite.)
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527843/
Link: https://lore.kernel.org/r/20230320144356.803762-9-robdclark@gmail.com

Just code-motion.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527841/
Link: https://lore.kernel.org/r/20230320144356.803762-8-robdclark@gmail.com

Export the locked version or lru's move_tail().
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527835/
Link: https://lore.kernel.org/r/20230320144356.803762-7-robdclark@gmail.com

vmap'ing is just pinning in disguise.  So treat it as such and simplify
the LRU tracking.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527837/
Link: https://lore.kernel.org/r/20230320144356.803762-6-robdclark@gmail.com

We need to use the inuse count to track that a BO is pinned until
we have the hw_fence.  But we want to remove the obj lock from the
job_run() path as this could deadlock against reclaim/shrinker
(because it is blocking the hw_fence from eventually being signaled).
So split that tracking out into a per-vma lock with narrower scope.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527839/
Link: https://lore.kernel.org/r/20230320144356.803762-5-robdclark@gmail.com

Stop open coding VMA construction, which will be needed in the next
commit.  And since the VMA already has a ptr to the adress space, stop
passing that around everywhere.  (Also, an aspace always has an mmu so
we can drop a couple pointless NULL checks.)
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527833/
Link: https://lore.kernel.org/r/20230320144356.803762-4-robdclark@gmail.com

The flags are only accessed (1) when submit is constructed, before
enqueuing to gpu sched (ie. when still visible to only the task calling
the submit ioctl), (2) here, where we own a reference to the submit and
are serialized on the gpu sched thread, and (3) after the submit is
retired and last reference is dropped, which is serialized on the
submit's reference count.  Hence locking is unneeded here.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527830/
Link: https://lore.kernel.org/r/20230320144356.803762-3-robdclark@gmail.com

Avoid allocating memory in job_run() by pre-allocating the hw_fence.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527832/
Link: https://lore.kernel.org/r/20230320144356.803762-2-robdclark@gmail.com

Selecting CONFIG_PM_GENERIC_DOMAINS causes a build failure when CONFIG_PM
is not enabled:

WARNING: unmet direct dependencies detected for PM_GENERIC_DOMAINS
  Depends on [n]: PM [=n]
  Selected by [m]:
  - DRM_MSM [=m] && HAS_IOMEM [=y] && DRM [=m] && (ARCH_QCOM [=y] || SOC_IMX5 || COMPILE_TEST [=y]) && COMMON_CLK [=y] && IOMMU_SUPPORT [=y] && (QCOM_OCMEM [=y] || QCOM_OCMEM [=y]=n) && (QCOM_LLCC [=n] || QCOM_LLCC [=n]=n) && (QCOM_COMMAND_DB [=y] || QCOM_COMMAND_DB [=y]=n) && DEVFREQ_GOV_SIMPLE_ONDEMAND [=y]

drivers/base/power/domain.c:654:13: error: use of undeclared identifier 'pm_wq'
        queue_work(pm_wq, &genpd->power_off_work);
                   ^
drivers/base/power/domain.c:853:26: error: no member named 'ignore_children' in 'struct dev_pm_info'
                if (!dev || dev->power.ignore_children)
                            ~~~~~~~~~~ ^

Fixes: c11fa120 ("drm/msm/a6xx: Use genpd notifier to ensure cx-gdsc collapse")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/528597/
Link: https://lore.kernel.org/r/20230324095502.3289094-1-arnd@kernel.orgSigned-off-by: Rob Clark <robdclark@chromium.org>

Propagate the deadline to all the fences in the array.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Christian König <christian.koenig@amd.com>

Add a way to hint to the fence signaler of an upcoming deadline, such as
vblank, which the fence waiter would prefer not to miss.  This is to aid
the fence signaler in making power management decisions, like boosting
frequency as the deadline approaches and awareness of missing deadlines
so that can be factored in to the frequency scaling.

v2: Drop dma_fence::deadline and related logic to filter duplicate
    deadlines, to avoid increasing dma_fence size.  The fence-context
    implementation will need similar logic to track deadlines of all
    the fences on the same timeline.  [ckoenig]
v3: Clarify locking wrt. set_deadline callback
v4: Clarify in docs comment that this is a hint
v5: Drop DMA_FENCE_FLAG_HAS_DEADLINE_BIT.
v6: More docs
v7: Fix typo, clarify past deadlines
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com>

It's been a bit overdue.  Regen headers to pull in a2xx perfcntr
updates, etc.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527926/
Link: https://lore.kernel.org/r/20230320185416.938842-2-robdclark@gmail.com