roundup will create an error in 32-bit architectures as we use
64-bit variables.
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Fix warning of
"warning: ‘old_base’ may be used uninitialized in this function"
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Use bitmap_zalloc()/bitmap_free() instead of hand-writing them.

It is less verbose and it improves the semantic.
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

There were some defines that are unused in the current upstreamed
code.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

timestamp could be unset in both _hl_interrupt_wait_ioctl() and
_hl_interrupt_wait_ioctl_user_addr() so it is better to explicitly
initialize it to 0 when declaring it.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

tmp is declared in the scope of the function cs_do_release() and
inside a block inside that function.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Max power API is not supported in secured devices. Hence, we should
skip setting it during boot.
Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Otherwise, due to how we calculate it, we might fail in FIELD_PREP
checks.
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

glbl_sts_clr_val was set but never used
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

If we send a packet to the f/w, and that packet is unsupported, we
want to be able to identify this situation and possibly ignore this.

Therefore, if the f/w returned an error, we need to propagate it
to the callers in the result value, if those callers were interested
in it.

In addition, no point of printing the error code here because each
caller prints its own error with a specific message.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

We need this property for backward compatibility against the f/w.
Signed-off-by: Sagiv Ozeri <sozeri@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

User application should be able to get notification for any decoder
completion. Hence, we introduce a new interface in which a user
can wait for all current decoder pending interrupts.
Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Current naming convention can be misleading. Hence renaming some
variables and defines in order to be more explicit.
Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Currently we are not waiting for preboot ready after hard reset.
This leads to a race in which COMMs protocol begins but will get no
response from the f/w.
Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Correctable ECC events are not fatal, but as they accumulate, the f/w
can decide that a hard-rest is required. This indication is
propagated to the host using the existing ECC event interface.
Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Enable the Gaudi2 ASIC code in the pci probe callback of the driver so
the driver will handle Gaudi2 ASICs.

Add the PCI ID to the PCI table and add the ASIC enum value to all
relevant places.

Fixup the device parameters initialization for Gaudi2.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Gaudi2 has new MMU units. A PMMU for device->host accesses, and HMMU
for HBM accesses.

The page tables of both MMUs are located in the host's memory (referred
to in the code as host-resident pgt).
Signed-off-by: Moti Haimovski <mhaimovski@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

In Gaudi2 we moved to a different wait for command submission
completion model. Instead of receiving interrupt only on external
queues, we use the device's sync manager to notify us when the
entire command submission finishes.

This enables us to remove the categorization of queues to external
and internal, and treat each queue equally, without the need to parse
and patch any command buffer.

This change also requires refactoring to the IRQ handling of
CS completions.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Add the Gaudi2 code to initialize the ASIC's profiler. The profile
receives its initialization values from the user, same as in Gaudi2,
but the code to initialize is in the driver because the configuration
space of the device is not directly exposed to the user.
Signed-off-by: Benjamin Dotan <bdotan@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Use the generic security module to block all registers in the ASIC and
then open only those that are needed to be accessed by the user.
Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

As the ASICs become more complex and have many more registers, we need
a better way to configure the security properties.

As a reminder, we have two dedicated mechanisms for security:
Range Registers and Protection bits. Those mechanisms protect sensitive
memory and configuration areas inside the device.

The generic module handles the low-level part of the configuration,
because the configuration mechanism is identical in all ASICs. The
difference is the address ranges and register names.

Any ASIC that use this block should first block all the register
blocks in the ASIC. Then, it should open only the registers that
need to be accessed by the user (This is opposed to Goya and Gaudi,
where we blocked only what should not be accesses by the user).

The module contains several functions, to unblock single register,
multiple registers, entire blocks, ranges, ranges with mask.
Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

There are a couple of device variables that are used for testing
purposes and they are set to fixed values.

Remove the variables that are not relevant anymore and document the
remaining variables.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

New asic properties were added for Gaudi2. We want to initialize
and use them, when relevant, also for Goya and Gaudi.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

There are a number of new ASIC-specific functions that were added
for Gaudi2. To make the common code work, we need to define empty
implementations of those functions for Goya and Gaudi.

Some functions will return error if called with Goya/Gaudi.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Add the ASIC-specific code for Gaudi2. Supply (almost) all of the
function callbacks that the driver's common code need to initialize,
finalize and submit workloads to the Gaudi2 ASIC.

It also contains the code to initialize the F/W of the Gaudi2 ASIC
and to receive events from the F/W.

It contains new debugfs entry to dump razwi events. razwi is a case
where the device's engines create a transaction that reaches an
invalid destination.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Add the new defines for GAUDI2 uapi interface.

It includes the following:
1. Enums of engines and PLLs.
2. New information in the info IOCTL that is retrieved by the driver.
3. Update comments regarding the CB/CS/wait for CS ioctls.
4. New fields in the debug IOCTL for configuring the profiler for
   Gaudi2.

There is no new IOCTL.

Some of the changes are also relevant for Greco (which will be
upstreamed later this year). When ever it says "Greco and onwards",
it means it is also for Gaudi2.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Add the relevant GAUDI2 ASIC registers header files. These files are
generated automatically from a tool maintained by the VLSI engineers.

There are more files which are not upstreamed because only very few
defines from those files are used in the driver. For those files, I
copied the relevant defines into gaudi2_regs.h and gaudi2_masks.h, to
reduce the size of this patch.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Region structure is derived from region type, hence no need to pass
it as an argument.
Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

PCI bar size is resource_size_t so we should use %pa to make it work
correctly on all architectures.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

in gaudi_scrub_device_mem, replace call to hl_poll_timeout
with a while loop to avoid using dummy variables.
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Dafna Hirschfeld <dhirschfeld@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Because in future ASICs the driver will allow the user to set the
page size we need to make sure this data is propagated in all APIs.

In addition, since this is already an ASIC property we no longer need
ASIC function for it.
Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

free_device_memory() ends with if and else, each has a return statement,
followed by another return statement that can never be reached.
Restructure the function and remove this dead code.
Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

We want to receive an error interrupt in case the watchdog timer
expires on arbitration event in the queues.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

We dropped support for page sizes that are not power of 2.
Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

This is a pre-requisite patch for adding tracepoints to the DMA memory
operations (allocation/free) in the driver.

The main purpose is to be able to cross data with the map operations and
determine whether memory violation occurred, for example free DMA
allocation before unmapping it from device memory.

To achieve this the DMA alloc/free code flows were refactored so that a
single DMA tracepoint will catch many flows.
Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

Also beautify code by preferring single line wherever possible.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

This fixes a sparse warning of
"cast truncates bits from constant value"
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

packets are defined as LE so we need to convert before assigning
values to them.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

function name in comment didn't match actual function name.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>

The values in this enum are not used by h/w but are a contract
between userspace and the kernel driver so they must be defined
in the uapi file.
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>