Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

This will be required to support additional HW features.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

This is a simplification that'll be used to improve interlock handling.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

This will be required to support Volta, where window ID != head.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

There should be no code changes here, just shuffling stuff around.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Removes duplicated code from OR-specific functions.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

We don't support address-only transactions there.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

The code is about to be split up, and this matches dispnv04.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

This function is useful outside of DRM code.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Hasn't been required for a long time.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Fences attached to deferred client work items now originate from channels
belonging to the client, meaning we can be certain they've been signalled
before we destroy a client.

This closes a race that could happen if the dma_fence_wait_timeout() call
didn't succeed.  When the fence was later signalled, a use-after-free was
possible.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

As VMAs are per-client, unlike buffers, this allows us to avoid referencing
foreign fences (those that belong to another client/driver) from the client
deferred work handler, and prevent some not-fun race conditions that can be
triggered when a fence stalls.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

An upcoming patch will use these to fix issues related to the deferred
unmapping of GEM objects.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

We previously only did this for push buffers, but an upcoming patch will
need to attach fences to all VMAs to resolve another issue.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

These were missed the first time around due to the driver version I traced
using the older registers still.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

There are differences on GM200 and newer too, but we can't fix them there
as they come from firmware packages.

A request has been made to NVIDIA to release updated firmware.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Makes it easier to diff against RM traces.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

There's a number of places that require this data, so let's separate out
the calculations to ensure they remain consistent.

This is incorrect for GM200 and newer, but will produce the same results
as we did before.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

There's also a couple of hardcoded tables for a couple of very specific
configurations that NVGPU's algorithm didn't work for.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Required to support Volta.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

RM and NVGPU both have a variant of this, we probably should too.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>