Commit 1763e735 authored by Linus Torvalds's avatar Linus Torvalds

Merge branch 'for-linus' of git://git.infradead.org/users/vkoul/slave-dma

Pull slave-dmaengine updates from Vinod Koul:
 "This time we have dmatest improvements from Andy along with dw_dmac
  fixes.  He has also done support for acpi for dmanegine.

  Also we have bunch of fixes going in DT support for dmanegine for
  various folks.  Then Haswell and other ioat changes from Dave and
  SUDMAC support from Shimoda."

* 'for-linus' of git://git.infradead.org/users/vkoul/slave-dma: (53 commits)
  dma: tegra: implement suspend/resume callbacks
  dma:of: Use a mutex to protect the of_dma_list
  dma: of: Fix of_node reference leak
  dmaengine: sirf: move driver init from module_init to subsys_initcall
  sudmac: add support for SUDMAC
  dma: sh: add Kconfig
  at_hdmac: move to generic DMA binding
  ioatdma: ioat3_alloc_sed can be static
  ioatdma: Adding write back descriptor error status support for ioatdma 3.3
  ioatdma: S1200 platforms ioatdma channel 2 and 3 falsely advertise RAID cap
  ioatdma: Adding support for 16 src PQ ops and super extended descriptors
  ioatdma: Removing hw bug workaround for CB3.x .2 and earlier
  dw_dmac: add ACPI support
  dmaengine: call acpi_dma_request_slave_channel as well
  dma: acpi-dma: introduce ACPI DMA helpers
  dma: of: Remove unnecessary list_empty check
  DMA: OF: Check properties value before running be32_to_cpup() on it
  DMA: of: Constant names
  ioatdma: skip silicon bug workaround for pq_align for cb3.3
  ioatdma: Removing PQ val disable for cb3.3
  ...
parents b29bdba5 3065c194
...@@ -66,6 +66,83 @@ the ACPI device explicitly to acpi_platform_device_ids list defined in ...@@ -66,6 +66,83 @@ the ACPI device explicitly to acpi_platform_device_ids list defined in
drivers/acpi/acpi_platform.c. This limitation is only for the platform drivers/acpi/acpi_platform.c. This limitation is only for the platform
devices, SPI and I2C devices are created automatically as described below. devices, SPI and I2C devices are created automatically as described below.
DMA support
~~~~~~~~~~~
DMA controllers enumerated via ACPI should be registered in the system to
provide generic access to their resources. For example, a driver that would
like to be accessible to slave devices via generic API call
dma_request_slave_channel() must register itself at the end of the probe
function like this:
err = devm_acpi_dma_controller_register(dev, xlate_func, dw);
/* Handle the error if it's not a case of !CONFIG_ACPI */
and implement custom xlate function if needed (usually acpi_dma_simple_xlate()
is enough) which converts the FixedDMA resource provided by struct
acpi_dma_spec into the corresponding DMA channel. A piece of code for that case
could look like:
#ifdef CONFIG_ACPI
struct filter_args {
/* Provide necessary information for the filter_func */
...
};
static bool filter_func(struct dma_chan *chan, void *param)
{
/* Choose the proper channel */
...
}
static struct dma_chan *xlate_func(struct acpi_dma_spec *dma_spec,
struct acpi_dma *adma)
{
dma_cap_mask_t cap;
struct filter_args args;
/* Prepare arguments for filter_func */
...
return dma_request_channel(cap, filter_func, &args);
}
#else
static struct dma_chan *xlate_func(struct acpi_dma_spec *dma_spec,
struct acpi_dma *adma)
{
return NULL;
}
#endif
dma_request_slave_channel() will call xlate_func() for each registered DMA
controller. In the xlate function the proper channel must be chosen based on
information in struct acpi_dma_spec and the properties of the controller
provided by struct acpi_dma.
Clients must call dma_request_slave_channel() with the string parameter that
corresponds to a specific FixedDMA resource. By default "tx" means the first
entry of the FixedDMA resource array, "rx" means the second entry. The table
below shows a layout:
Device (I2C0)
{
...
Method (_CRS, 0, NotSerialized)
{
Name (DBUF, ResourceTemplate ()
{
FixedDMA (0x0018, 0x0004, Width32bit, _Y48)
FixedDMA (0x0019, 0x0005, Width32bit, )
})
...
}
}
So, the FixedDMA with request line 0x0018 is "tx" and next one is "rx" in
this example.
In robust cases the client unfortunately needs to call
acpi_dma_request_slave_chan_by_index() directly and therefore choose the
specific FixedDMA resource by its index.
SPI serial bus support SPI serial bus support
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
Slave devices behind SPI bus have SpiSerialBus resource attached to them. Slave devices behind SPI bus have SpiSerialBus resource attached to them.
......
* Atmel Direct Memory Access Controller (DMA) * Atmel Direct Memory Access Controller (DMA)
Required properties: Required properties:
- compatible: Should be "atmel,<chip>-dma" - compatible: Should be "atmel,<chip>-dma".
- reg: Should contain DMA registers location and length - reg: Should contain DMA registers location and length.
- interrupts: Should contain DMA interrupt - interrupts: Should contain DMA interrupt.
- #dma-cells: Must be <2>, used to represent the number of integer cells in
the dmas property of client devices.
Examples: Example:
dma@ffffec00 { dma0: dma@ffffec00 {
compatible = "atmel,at91sam9g45-dma"; compatible = "atmel,at91sam9g45-dma";
reg = <0xffffec00 0x200>; reg = <0xffffec00 0x200>;
interrupts = <21>; interrupts = <21>;
#dma-cells = <2>;
};
DMA clients connected to the Atmel DMA controller must use the format
described in the dma.txt file, using a three-cell specifier for each channel:
a phandle plus two interger cells.
The three cells in order are:
1. A phandle pointing to the DMA controller.
2. The memory interface (16 most significant bits), the peripheral interface
(16 less significant bits).
3. The peripheral identifier for the hardware handshaking interface. The
identifier can be different for tx and rx.
Example:
i2c0@i2c@f8010000 {
compatible = "atmel,at91sam9x5-i2c";
reg = <0xf8010000 0x100>;
interrupts = <9 4 6>;
dmas = <&dma0 1 7>,
<&dma0 1 8>;
dma-names = "tx", "rx";
}; };
DMA Test Guide
==============
Andy Shevchenko <andriy.shevchenko@linux.intel.com>
This small document introduces how to test DMA drivers using dmatest module.
Part 1 - How to build the test module
The menuconfig contains an option that could be found by following path:
Device Drivers -> DMA Engine support -> DMA Test client
In the configuration file the option called CONFIG_DMATEST. The dmatest could
be built as module or inside kernel. Let's consider those cases.
Part 2 - When dmatest is built as a module...
After mounting debugfs and loading the module, the /sys/kernel/debug/dmatest
folder with nodes will be created. They are the same as module parameters with
addition of the 'run' node that controls run and stop phases of the test.
Note that in this case test will not run on load automatically.
Example of usage:
% echo dma0chan0 > /sys/kernel/debug/dmatest/channel
% echo 2000 > /sys/kernel/debug/dmatest/timeout
% echo 1 > /sys/kernel/debug/dmatest/iterations
% echo 1 > /sys/kernel/debug/dmatest/run
Hint: available channel list could be extracted by running the following
command:
% ls -1 /sys/class/dma/
After a while you will start to get messages about current status or error like
in the original code.
Note that running a new test will stop any in progress test.
The following command should return actual state of the test.
% cat /sys/kernel/debug/dmatest/run
To wait for test done the user may perform a busy loop that checks the state.
% while [ $(cat /sys/kernel/debug/dmatest/run) = "Y" ]
> do
> echo -n "."
> sleep 1
> done
> echo
Part 3 - When built-in in the kernel...
The module parameters that is supplied to the kernel command line will be used
for the first performed test. After user gets a control, the test could be
interrupted or re-run with same or different parameters. For the details see
the above section "Part 2 - When dmatest is built as a module..."
In both cases the module parameters are used as initial values for the test case.
You always could check them at run-time by running
% grep -H . /sys/module/dmatest/parameters/*
Part 4 - Gathering the test results
The module provides a storage for the test results in the memory. The gathered
data could be used after test is done.
The special file 'results' in the debugfs represents gathered data of the in
progress test. The messages collected are printed to the kernel log as well.
Example of output:
% cat /sys/kernel/debug/dmatest/results
dma0chan0-copy0: #1: No errors with src_off=0x7bf dst_off=0x8ad len=0x3fea (0)
The message format is unified across the different types of errors. A number in
the parens represents additional information, e.g. error code, error counter,
or status.
Comparison between buffers is stored to the dedicated structure.
Note that the verify result is now accessible only via file 'results' in the
debugfs.
...@@ -28,6 +28,7 @@ ...@@ -28,6 +28,7 @@
#include <linux/init.h> #include <linux/init.h>
#include <linux/device.h> #include <linux/device.h>
#include <linux/dma-mapping.h> #include <linux/dma-mapping.h>
#include <linux/of.h>
#include <linux/omap-dma.h> #include <linux/omap-dma.h>
#include "soc.h" #include "soc.h"
...@@ -304,6 +305,9 @@ static int __init omap2_system_dma_init(void) ...@@ -304,6 +305,9 @@ static int __init omap2_system_dma_init(void)
if (res) if (res)
return res; return res;
if (of_have_populated_dt())
return res;
pdev = platform_device_register_full(&omap_dma_dev_info); pdev = platform_device_register_full(&omap_dma_dev_info);
if (IS_ERR(pdev)) if (IS_ERR(pdev))
return PTR_ERR(pdev); return PTR_ERR(pdev);
......
...@@ -63,8 +63,6 @@ config INTEL_IOATDMA ...@@ -63,8 +63,6 @@ config INTEL_IOATDMA
depends on PCI && X86 depends on PCI && X86
select DMA_ENGINE select DMA_ENGINE
select DCA select DCA
select ASYNC_TX_DISABLE_PQ_VAL_DMA
select ASYNC_TX_DISABLE_XOR_VAL_DMA
help help
Enable support for the Intel(R) I/OAT DMA engine present Enable support for the Intel(R) I/OAT DMA engine present
in recent Intel Xeon chipsets. in recent Intel Xeon chipsets.
...@@ -174,15 +172,7 @@ config TEGRA20_APB_DMA ...@@ -174,15 +172,7 @@ config TEGRA20_APB_DMA
This DMA controller transfers data from memory to peripheral fifo This DMA controller transfers data from memory to peripheral fifo
or vice versa. It does not support memory to memory data transfer. or vice versa. It does not support memory to memory data transfer.
source "drivers/dma/sh/Kconfig"
config SH_DMAE
tristate "Renesas SuperH DMAC support"
depends on (SUPERH && SH_DMA) || (ARM && ARCH_SHMOBILE)
depends on !SH_DMA_API
select DMA_ENGINE
help
Enable support for the Renesas SuperH DMA controllers.
config COH901318 config COH901318
bool "ST-Ericsson COH901318 DMA support" bool "ST-Ericsson COH901318 DMA support"
...@@ -328,6 +318,10 @@ config DMA_ENGINE ...@@ -328,6 +318,10 @@ config DMA_ENGINE
config DMA_VIRTUAL_CHANNELS config DMA_VIRTUAL_CHANNELS
tristate tristate
config DMA_ACPI
def_bool y
depends on ACPI
config DMA_OF config DMA_OF
def_bool y def_bool y
depends on OF depends on OF
......
...@@ -3,6 +3,7 @@ ccflags-$(CONFIG_DMADEVICES_VDEBUG) += -DVERBOSE_DEBUG ...@@ -3,6 +3,7 @@ ccflags-$(CONFIG_DMADEVICES_VDEBUG) += -DVERBOSE_DEBUG
obj-$(CONFIG_DMA_ENGINE) += dmaengine.o obj-$(CONFIG_DMA_ENGINE) += dmaengine.o
obj-$(CONFIG_DMA_VIRTUAL_CHANNELS) += virt-dma.o obj-$(CONFIG_DMA_VIRTUAL_CHANNELS) += virt-dma.o
obj-$(CONFIG_DMA_ACPI) += acpi-dma.o
obj-$(CONFIG_DMA_OF) += of-dma.o obj-$(CONFIG_DMA_OF) += of-dma.o
obj-$(CONFIG_NET_DMA) += iovlock.o obj-$(CONFIG_NET_DMA) += iovlock.o
...@@ -18,7 +19,7 @@ obj-$(CONFIG_DW_DMAC) += dw_dmac.o ...@@ -18,7 +19,7 @@ obj-$(CONFIG_DW_DMAC) += dw_dmac.o
obj-$(CONFIG_AT_HDMAC) += at_hdmac.o obj-$(CONFIG_AT_HDMAC) += at_hdmac.o
obj-$(CONFIG_MX3_IPU) += ipu/ obj-$(CONFIG_MX3_IPU) += ipu/
obj-$(CONFIG_TXX9_DMAC) += txx9dmac.o obj-$(CONFIG_TXX9_DMAC) += txx9dmac.o
obj-$(CONFIG_SH_DMAE) += sh/ obj-$(CONFIG_SH_DMAE_BASE) += sh/
obj-$(CONFIG_COH901318) += coh901318.o coh901318_lli.o obj-$(CONFIG_COH901318) += coh901318.o coh901318_lli.o
obj-$(CONFIG_AMCC_PPC440SPE_ADMA) += ppc4xx/ obj-$(CONFIG_AMCC_PPC440SPE_ADMA) += ppc4xx/
obj-$(CONFIG_IMX_SDMA) += imx-sdma.o obj-$(CONFIG_IMX_SDMA) += imx-sdma.o
......
/*
* ACPI helpers for DMA request / controller
*
* Based on of-dma.c
*
* Copyright (C) 2013, Intel Corporation
* Author: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
#include <linux/device.h>
#include <linux/module.h>
#include <linux/list.h>
#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/acpi.h>
#include <linux/acpi_dma.h>
static LIST_HEAD(acpi_dma_list);
static DEFINE_MUTEX(acpi_dma_lock);
/**
* acpi_dma_controller_register - Register a DMA controller to ACPI DMA helpers
* @dev: struct device of DMA controller
* @acpi_dma_xlate: translation function which converts a dma specifier
* into a dma_chan structure
* @data pointer to controller specific data to be used by
* translation function
*
* Returns 0 on success or appropriate errno value on error.
*
* Allocated memory should be freed with appropriate acpi_dma_controller_free()
* call.
*/
int acpi_dma_controller_register(struct device *dev,
struct dma_chan *(*acpi_dma_xlate)
(struct acpi_dma_spec *, struct acpi_dma *),
void *data)
{
struct acpi_device *adev;
struct acpi_dma *adma;
if (!dev || !acpi_dma_xlate)
return -EINVAL;
/* Check if the device was enumerated by ACPI */
if (!ACPI_HANDLE(dev))
return -EINVAL;
if (acpi_bus_get_device(ACPI_HANDLE(dev), &adev))
return -EINVAL;
adma = kzalloc(sizeof(*adma), GFP_KERNEL);
if (!adma)
return -ENOMEM;
adma->dev = dev;
adma->acpi_dma_xlate = acpi_dma_xlate;
adma->data = data;
/* Now queue acpi_dma controller structure in list */
mutex_lock(&acpi_dma_lock);
list_add_tail(&adma->dma_controllers, &acpi_dma_list);
mutex_unlock(&acpi_dma_lock);
return 0;
}
EXPORT_SYMBOL_GPL(acpi_dma_controller_register);
/**
* acpi_dma_controller_free - Remove a DMA controller from ACPI DMA helpers list
* @dev: struct device of DMA controller
*
* Memory allocated by acpi_dma_controller_register() is freed here.
*/
int acpi_dma_controller_free(struct device *dev)
{
struct acpi_dma *adma;
if (!dev)
return -EINVAL;
mutex_lock(&acpi_dma_lock);
list_for_each_entry(adma, &acpi_dma_list, dma_controllers)
if (adma->dev == dev) {
list_del(&adma->dma_controllers);
mutex_unlock(&acpi_dma_lock);
kfree(adma);
return 0;
}
mutex_unlock(&acpi_dma_lock);
return -ENODEV;
}
EXPORT_SYMBOL_GPL(acpi_dma_controller_free);
static void devm_acpi_dma_release(struct device *dev, void *res)
{
acpi_dma_controller_free(dev);
}
/**
* devm_acpi_dma_controller_register - resource managed acpi_dma_controller_register()
* @dev: device that is registering this DMA controller
* @acpi_dma_xlate: translation function
* @data pointer to controller specific data
*
* Managed acpi_dma_controller_register(). DMA controller registered by this
* function are automatically freed on driver detach. See
* acpi_dma_controller_register() for more information.
*/
int devm_acpi_dma_controller_register(struct device *dev,
struct dma_chan *(*acpi_dma_xlate)
(struct acpi_dma_spec *, struct acpi_dma *),
void *data)
{
void *res;
int ret;
res = devres_alloc(devm_acpi_dma_release, 0, GFP_KERNEL);
if (!res)
return -ENOMEM;
ret = acpi_dma_controller_register(dev, acpi_dma_xlate, data);
if (ret) {
devres_free(res);
return ret;
}
devres_add(dev, res);
return 0;
}
EXPORT_SYMBOL_GPL(devm_acpi_dma_controller_register);
/**
* devm_acpi_dma_controller_free - resource managed acpi_dma_controller_free()
*
* Unregister a DMA controller registered with
* devm_acpi_dma_controller_register(). Normally this function will not need to
* be called and the resource management code will ensure that the resource is
* freed.
*/
void devm_acpi_dma_controller_free(struct device *dev)
{
WARN_ON(devres_destroy(dev, devm_acpi_dma_release, NULL, NULL));
}
EXPORT_SYMBOL_GPL(devm_acpi_dma_controller_free);
struct acpi_dma_parser_data {
struct acpi_dma_spec dma_spec;
size_t index;
size_t n;
};
/**
* acpi_dma_parse_fixed_dma - Parse FixedDMA ACPI resources to a DMA specifier
* @res: struct acpi_resource to get FixedDMA resources from
* @data: pointer to a helper struct acpi_dma_parser_data
*/
static int acpi_dma_parse_fixed_dma(struct acpi_resource *res, void *data)
{
struct acpi_dma_parser_data *pdata = data;
if (res->type == ACPI_RESOURCE_TYPE_FIXED_DMA) {
struct acpi_resource_fixed_dma *dma = &res->data.fixed_dma;
if (pdata->n++ == pdata->index) {
pdata->dma_spec.chan_id = dma->channels;
pdata->dma_spec.slave_id = dma->request_lines;
}
}
/* Tell the ACPI core to skip this resource */
return 1;
}
/**
* acpi_dma_request_slave_chan_by_index - Get the DMA slave channel
* @dev: struct device to get DMA request from
* @index: index of FixedDMA descriptor for @dev
*
* Returns pointer to appropriate dma channel on success or NULL on error.
*/
struct dma_chan *acpi_dma_request_slave_chan_by_index(struct device *dev,
size_t index)
{
struct acpi_dma_parser_data pdata;
struct acpi_dma_spec *dma_spec = &pdata.dma_spec;
struct list_head resource_list;
struct acpi_device *adev;
struct acpi_dma *adma;
struct dma_chan *chan = NULL;
/* Check if the device was enumerated by ACPI */
if (!dev || !ACPI_HANDLE(dev))
return NULL;
if (acpi_bus_get_device(ACPI_HANDLE(dev), &adev))
return NULL;
memset(&pdata, 0, sizeof(pdata));
pdata.index = index;
/* Initial values for the request line and channel */
dma_spec->chan_id = -1;
dma_spec->slave_id = -1;
INIT_LIST_HEAD(&resource_list);
acpi_dev_get_resources(adev, &resource_list,
acpi_dma_parse_fixed_dma, &pdata);
acpi_dev_free_resource_list(&resource_list);
if (dma_spec->slave_id < 0 || dma_spec->chan_id < 0)
return NULL;
mutex_lock(&acpi_dma_lock);
list_for_each_entry(adma, &acpi_dma_list, dma_controllers) {
dma_spec->dev = adma->dev;
chan = adma->acpi_dma_xlate(dma_spec, adma);
if (chan)
break;
}
mutex_unlock(&acpi_dma_lock);
return chan;
}
EXPORT_SYMBOL_GPL(acpi_dma_request_slave_chan_by_index);
/**
* acpi_dma_request_slave_chan_by_name - Get the DMA slave channel
* @dev: struct device to get DMA request from
* @name: represents corresponding FixedDMA descriptor for @dev
*
* In order to support both Device Tree and ACPI in a single driver we
* translate the names "tx" and "rx" here based on the most common case where
* the first FixedDMA descriptor is TX and second is RX.
*
* Returns pointer to appropriate dma channel on success or NULL on error.
*/
struct dma_chan *acpi_dma_request_slave_chan_by_name(struct device *dev,
const char *name)
{
size_t index;
if (!strcmp(name, "tx"))
index = 0;
else if (!strcmp(name, "rx"))
index = 1;
else
return NULL;
return acpi_dma_request_slave_chan_by_index(dev, index);
}
EXPORT_SYMBOL_GPL(acpi_dma_request_slave_chan_by_name);
/**
* acpi_dma_simple_xlate - Simple ACPI DMA engine translation helper
* @dma_spec: pointer to ACPI DMA specifier
* @adma: pointer to ACPI DMA controller data
*
* A simple translation function for ACPI based devices. Passes &struct
* dma_spec to the DMA controller driver provided filter function. Returns
* pointer to the channel if found or %NULL otherwise.
*/
struct dma_chan *acpi_dma_simple_xlate(struct acpi_dma_spec *dma_spec,
struct acpi_dma *adma)
{
struct acpi_dma_filter_info *info = adma->data;
if (!info || !info->filter_fn)
return NULL;
return dma_request_channel(info->dma_cap, info->filter_fn, dma_spec);
}
EXPORT_SYMBOL_GPL(acpi_dma_simple_xlate);
...@@ -24,6 +24,7 @@ ...@@ -24,6 +24,7 @@
#include <linux/slab.h> #include <linux/slab.h>
#include <linux/of.h> #include <linux/of.h>
#include <linux/of_device.h> #include <linux/of_device.h>
#include <linux/of_dma.h>
#include "at_hdmac_regs.h" #include "at_hdmac_regs.h"
#include "dmaengine.h" #include "dmaengine.h"
...@@ -677,7 +678,7 @@ atc_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl, ...@@ -677,7 +678,7 @@ atc_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl,
ctrlb |= ATC_DST_ADDR_MODE_FIXED ctrlb |= ATC_DST_ADDR_MODE_FIXED
| ATC_SRC_ADDR_MODE_INCR | ATC_SRC_ADDR_MODE_INCR
| ATC_FC_MEM2PER | ATC_FC_MEM2PER
| ATC_SIF(AT_DMA_MEM_IF) | ATC_DIF(AT_DMA_PER_IF); | ATC_SIF(atchan->mem_if) | ATC_DIF(atchan->per_if);
reg = sconfig->dst_addr; reg = sconfig->dst_addr;
for_each_sg(sgl, sg, sg_len, i) { for_each_sg(sgl, sg, sg_len, i) {
struct at_desc *desc; struct at_desc *desc;
...@@ -716,7 +717,7 @@ atc_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl, ...@@ -716,7 +717,7 @@ atc_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl,
ctrlb |= ATC_DST_ADDR_MODE_INCR ctrlb |= ATC_DST_ADDR_MODE_INCR
| ATC_SRC_ADDR_MODE_FIXED | ATC_SRC_ADDR_MODE_FIXED
| ATC_FC_PER2MEM | ATC_FC_PER2MEM
| ATC_SIF(AT_DMA_PER_IF) | ATC_DIF(AT_DMA_MEM_IF); | ATC_SIF(atchan->per_if) | ATC_DIF(atchan->mem_if);
reg = sconfig->src_addr; reg = sconfig->src_addr;
for_each_sg(sgl, sg, sg_len, i) { for_each_sg(sgl, sg, sg_len, i) {
...@@ -822,8 +823,8 @@ atc_dma_cyclic_fill_desc(struct dma_chan *chan, struct at_desc *desc, ...@@ -822,8 +823,8 @@ atc_dma_cyclic_fill_desc(struct dma_chan *chan, struct at_desc *desc,
desc->lli.ctrlb = ATC_DST_ADDR_MODE_FIXED desc->lli.ctrlb = ATC_DST_ADDR_MODE_FIXED
| ATC_SRC_ADDR_MODE_INCR | ATC_SRC_ADDR_MODE_INCR
| ATC_FC_MEM2PER | ATC_FC_MEM2PER
| ATC_SIF(AT_DMA_MEM_IF) | ATC_SIF(atchan->mem_if)
| ATC_DIF(AT_DMA_PER_IF); | ATC_DIF(atchan->per_if);
break; break;
case DMA_DEV_TO_MEM: case DMA_DEV_TO_MEM:
...@@ -833,8 +834,8 @@ atc_dma_cyclic_fill_desc(struct dma_chan *chan, struct at_desc *desc, ...@@ -833,8 +834,8 @@ atc_dma_cyclic_fill_desc(struct dma_chan *chan, struct at_desc *desc,
desc->lli.ctrlb = ATC_DST_ADDR_MODE_INCR desc->lli.ctrlb = ATC_DST_ADDR_MODE_INCR
| ATC_SRC_ADDR_MODE_FIXED | ATC_SRC_ADDR_MODE_FIXED
| ATC_FC_PER2MEM | ATC_FC_PER2MEM
| ATC_SIF(AT_DMA_PER_IF) | ATC_SIF(atchan->per_if)
| ATC_DIF(AT_DMA_MEM_IF); | ATC_DIF(atchan->mem_if);
break; break;
default: default:
...@@ -1188,6 +1189,67 @@ static void atc_free_chan_resources(struct dma_chan *chan) ...@@ -1188,6 +1189,67 @@ static void atc_free_chan_resources(struct dma_chan *chan)
dev_vdbg(chan2dev(chan), "free_chan_resources: done\n"); dev_vdbg(chan2dev(chan), "free_chan_resources: done\n");
} }
#ifdef CONFIG_OF
static bool at_dma_filter(struct dma_chan *chan, void *slave)
{
struct at_dma_slave *atslave = slave;
if (atslave->dma_dev == chan->device->dev) {
chan->private = atslave;
return true;
} else {
return false;
}
}
static struct dma_chan *at_dma_xlate(struct of_phandle_args *dma_spec,
struct of_dma *of_dma)
{
struct dma_chan *chan;
struct at_dma_chan *atchan;
struct at_dma_slave *atslave;
dma_cap_mask_t mask;
unsigned int per_id;
struct platform_device *dmac_pdev;
if (dma_spec->args_count != 2)
return NULL;
dmac_pdev = of_find_device_by_node(dma_spec->np);
dma_cap_zero(mask);
dma_cap_set(DMA_SLAVE, mask);
atslave = devm_kzalloc(&dmac_pdev->dev, sizeof(*atslave), GFP_KERNEL);
if (!atslave)
return NULL;
/*
* We can fill both SRC_PER and DST_PER, one of these fields will be
* ignored depending on DMA transfer direction.
*/
per_id = dma_spec->args[1];
atslave->cfg = ATC_FIFOCFG_HALFFIFO | ATC_DST_H2SEL_HW
| ATC_SRC_H2SEL_HW | ATC_DST_PER(per_id)
| ATC_SRC_PER(per_id);
atslave->dma_dev = &dmac_pdev->dev;
chan = dma_request_channel(mask, at_dma_filter, atslave);
if (!chan)
return NULL;
atchan = to_at_dma_chan(chan);
atchan->per_if = dma_spec->args[0] & 0xff;
atchan->mem_if = (dma_spec->args[0] >> 16) & 0xff;
return chan;
}
#else
static struct dma_chan *at_dma_xlate(struct of_phandle_args *dma_spec,
struct of_dma *of_dma)
{
return NULL;
}
#endif
/*-- Module Management -----------------------------------------------*/ /*-- Module Management -----------------------------------------------*/
...@@ -1342,6 +1404,8 @@ static int __init at_dma_probe(struct platform_device *pdev) ...@@ -1342,6 +1404,8 @@ static int __init at_dma_probe(struct platform_device *pdev)
for (i = 0; i < plat_dat->nr_channels; i++) { for (i = 0; i < plat_dat->nr_channels; i++) {
struct at_dma_chan *atchan = &atdma->chan[i]; struct at_dma_chan *atchan = &atdma->chan[i];
atchan->mem_if = AT_DMA_MEM_IF;
atchan->per_if = AT_DMA_PER_IF;
atchan->chan_common.device = &atdma->dma_common; atchan->chan_common.device = &atdma->dma_common;
dma_cookie_init(&atchan->chan_common); dma_cookie_init(&atchan->chan_common);
list_add_tail(&atchan->chan_common.device_node, list_add_tail(&atchan->chan_common.device_node,
...@@ -1388,8 +1452,25 @@ static int __init at_dma_probe(struct platform_device *pdev) ...@@ -1388,8 +1452,25 @@ static int __init at_dma_probe(struct platform_device *pdev)
dma_async_device_register(&atdma->dma_common); dma_async_device_register(&atdma->dma_common);
/*
* Do not return an error if the dmac node is not present in order to
* not break the existing way of requesting channel with
* dma_request_channel().
*/
if (pdev->dev.of_node) {
err = of_dma_controller_register(pdev->dev.of_node,
at_dma_xlate, atdma);
if (err) {
dev_err(&pdev->dev, "could not register of_dma_controller\n");
goto err_of_dma_controller_register;
}
}
return 0; return 0;
err_of_dma_controller_register:
dma_async_device_unregister(&atdma->dma_common);
dma_pool_destroy(atdma->dma_desc_pool);
err_pool_create: err_pool_create:
platform_set_drvdata(pdev, NULL); platform_set_drvdata(pdev, NULL);
free_irq(platform_get_irq(pdev, 0), atdma); free_irq(platform_get_irq(pdev, 0), atdma);
...@@ -1406,7 +1487,7 @@ static int __init at_dma_probe(struct platform_device *pdev) ...@@ -1406,7 +1487,7 @@ static int __init at_dma_probe(struct platform_device *pdev)
return err; return err;
} }
static int __exit at_dma_remove(struct platform_device *pdev) static int at_dma_remove(struct platform_device *pdev)
{ {
struct at_dma *atdma = platform_get_drvdata(pdev); struct at_dma *atdma = platform_get_drvdata(pdev);
struct dma_chan *chan, *_chan; struct dma_chan *chan, *_chan;
...@@ -1564,7 +1645,7 @@ static const struct dev_pm_ops at_dma_dev_pm_ops = { ...@@ -1564,7 +1645,7 @@ static const struct dev_pm_ops at_dma_dev_pm_ops = {
}; };
static struct platform_driver at_dma_driver = { static struct platform_driver at_dma_driver = {
.remove = __exit_p(at_dma_remove), .remove = at_dma_remove,
.shutdown = at_dma_shutdown, .shutdown = at_dma_shutdown,
.id_table = atdma_devtypes, .id_table = atdma_devtypes,
.driver = { .driver = {
......
...@@ -220,6 +220,8 @@ enum atc_status { ...@@ -220,6 +220,8 @@ enum atc_status {
* @device: parent device * @device: parent device
* @ch_regs: memory mapped register base * @ch_regs: memory mapped register base
* @mask: channel index in a mask * @mask: channel index in a mask
* @per_if: peripheral interface
* @mem_if: memory interface
* @status: transmit status information from irq/prep* functions * @status: transmit status information from irq/prep* functions
* to tasklet (use atomic operations) * to tasklet (use atomic operations)
* @tasklet: bottom half to finish transaction work * @tasklet: bottom half to finish transaction work
...@@ -238,6 +240,8 @@ struct at_dma_chan { ...@@ -238,6 +240,8 @@ struct at_dma_chan {
struct at_dma *device; struct at_dma *device;
void __iomem *ch_regs; void __iomem *ch_regs;
u8 mask; u8 mask;
u8 per_if;
u8 mem_if;
unsigned long status; unsigned long status;
struct tasklet_struct tasklet; struct tasklet_struct tasklet;
u32 save_cfg; u32 save_cfg;
......
...@@ -2748,7 +2748,7 @@ static int __init coh901318_probe(struct platform_device *pdev) ...@@ -2748,7 +2748,7 @@ static int __init coh901318_probe(struct platform_device *pdev)
return err; return err;
} }
static int __exit coh901318_remove(struct platform_device *pdev) static int coh901318_remove(struct platform_device *pdev)
{ {
struct coh901318_base *base = platform_get_drvdata(pdev); struct coh901318_base *base = platform_get_drvdata(pdev);
...@@ -2760,7 +2760,7 @@ static int __exit coh901318_remove(struct platform_device *pdev) ...@@ -2760,7 +2760,7 @@ static int __exit coh901318_remove(struct platform_device *pdev)
static struct platform_driver coh901318_driver = { static struct platform_driver coh901318_driver = {
.remove = __exit_p(coh901318_remove), .remove = coh901318_remove,
.driver = { .driver = {
.name = "coh901318", .name = "coh901318",
}, },
......
...@@ -62,6 +62,8 @@ ...@@ -62,6 +62,8 @@
#include <linux/rculist.h> #include <linux/rculist.h>
#include <linux/idr.h> #include <linux/idr.h>
#include <linux/slab.h> #include <linux/slab.h>
#include <linux/acpi.h>
#include <linux/acpi_dma.h>
#include <linux/of_dma.h> #include <linux/of_dma.h>
static DEFINE_MUTEX(dma_list_mutex); static DEFINE_MUTEX(dma_list_mutex);
...@@ -174,7 +176,8 @@ static struct class dma_devclass = { ...@@ -174,7 +176,8 @@ static struct class dma_devclass = {
#define dma_device_satisfies_mask(device, mask) \ #define dma_device_satisfies_mask(device, mask) \
__dma_device_satisfies_mask((device), &(mask)) __dma_device_satisfies_mask((device), &(mask))
static int static int
__dma_device_satisfies_mask(struct dma_device *device, dma_cap_mask_t *want) __dma_device_satisfies_mask(struct dma_device *device,
const dma_cap_mask_t *want)
{ {
dma_cap_mask_t has; dma_cap_mask_t has;
...@@ -463,7 +466,8 @@ static void dma_channel_rebalance(void) ...@@ -463,7 +466,8 @@ static void dma_channel_rebalance(void)
} }
} }
static struct dma_chan *private_candidate(dma_cap_mask_t *mask, struct dma_device *dev, static struct dma_chan *private_candidate(const dma_cap_mask_t *mask,
struct dma_device *dev,
dma_filter_fn fn, void *fn_param) dma_filter_fn fn, void *fn_param)
{ {
struct dma_chan *chan; struct dma_chan *chan;
...@@ -505,7 +509,8 @@ static struct dma_chan *private_candidate(dma_cap_mask_t *mask, struct dma_devic ...@@ -505,7 +509,8 @@ static struct dma_chan *private_candidate(dma_cap_mask_t *mask, struct dma_devic
* @fn: optional callback to disposition available channels * @fn: optional callback to disposition available channels
* @fn_param: opaque parameter to pass to dma_filter_fn * @fn_param: opaque parameter to pass to dma_filter_fn
*/ */
struct dma_chan *__dma_request_channel(dma_cap_mask_t *mask, dma_filter_fn fn, void *fn_param) struct dma_chan *__dma_request_channel(const dma_cap_mask_t *mask,
dma_filter_fn fn, void *fn_param)
{ {
struct dma_device *device, *_d; struct dma_device *device, *_d;
struct dma_chan *chan = NULL; struct dma_chan *chan = NULL;
...@@ -555,12 +560,16 @@ EXPORT_SYMBOL_GPL(__dma_request_channel); ...@@ -555,12 +560,16 @@ EXPORT_SYMBOL_GPL(__dma_request_channel);
* @dev: pointer to client device structure * @dev: pointer to client device structure
* @name: slave channel name * @name: slave channel name
*/ */
struct dma_chan *dma_request_slave_channel(struct device *dev, char *name) struct dma_chan *dma_request_slave_channel(struct device *dev, const char *name)
{ {
/* If device-tree is present get slave info from here */ /* If device-tree is present get slave info from here */
if (dev->of_node) if (dev->of_node)
return of_dma_request_slave_channel(dev->of_node, name); return of_dma_request_slave_channel(dev->of_node, name);
/* If device was enumerated by ACPI get slave info from here */
if (ACPI_HANDLE(dev))
return acpi_dma_request_slave_chan_by_name(dev, name);
return NULL; return NULL;
} }
EXPORT_SYMBOL_GPL(dma_request_slave_channel); EXPORT_SYMBOL_GPL(dma_request_slave_channel);
......
This diff is collapsed.
This diff is collapsed.
...@@ -212,8 +212,11 @@ struct dw_dma_chan { ...@@ -212,8 +212,11 @@ struct dw_dma_chan {
/* hardware configuration */ /* hardware configuration */
unsigned int block_size; unsigned int block_size;
bool nollp; bool nollp;
/* custom slave configuration */
unsigned int request_line; unsigned int request_line;
struct dw_dma_slave slave; unsigned char src_master;
unsigned char dst_master;
/* configuration passed via DMA_SLAVE_CONFIG */ /* configuration passed via DMA_SLAVE_CONFIG */
struct dma_slave_config dma_sconfig; struct dma_slave_config dma_sconfig;
...@@ -247,7 +250,6 @@ struct dw_dma { ...@@ -247,7 +250,6 @@ struct dw_dma {
/* hardware configuration */ /* hardware configuration */
unsigned char nr_masters; unsigned char nr_masters;
unsigned char data_width[4]; unsigned char data_width[4];
unsigned int request_line_base;
struct dw_dma_chan chan[0]; struct dw_dma_chan chan[0];
}; };
......
...@@ -859,8 +859,7 @@ static struct dma_async_tx_descriptor *imxdma_prep_dma_cyclic( ...@@ -859,8 +859,7 @@ static struct dma_async_tx_descriptor *imxdma_prep_dma_cyclic(
desc = list_first_entry(&imxdmac->ld_free, struct imxdma_desc, node); desc = list_first_entry(&imxdmac->ld_free, struct imxdma_desc, node);
if (imxdmac->sg_list) kfree(imxdmac->sg_list);
kfree(imxdmac->sg_list);
imxdmac->sg_list = kcalloc(periods + 1, imxdmac->sg_list = kcalloc(periods + 1,
sizeof(struct scatterlist), GFP_KERNEL); sizeof(struct scatterlist), GFP_KERNEL);
...@@ -1145,7 +1144,7 @@ static int __init imxdma_probe(struct platform_device *pdev) ...@@ -1145,7 +1144,7 @@ static int __init imxdma_probe(struct platform_device *pdev)
return ret; return ret;
} }
static int __exit imxdma_remove(struct platform_device *pdev) static int imxdma_remove(struct platform_device *pdev)
{ {
struct imxdma_engine *imxdma = platform_get_drvdata(pdev); struct imxdma_engine *imxdma = platform_get_drvdata(pdev);
...@@ -1162,7 +1161,7 @@ static struct platform_driver imxdma_driver = { ...@@ -1162,7 +1161,7 @@ static struct platform_driver imxdma_driver = {
.name = "imx-dma", .name = "imx-dma",
}, },
.id_table = imx_dma_devtype, .id_table = imx_dma_devtype,
.remove = __exit_p(imxdma_remove), .remove = imxdma_remove,
}; };
static int __init imxdma_module_init(void) static int __init imxdma_module_init(void)
......
...@@ -1462,7 +1462,7 @@ static int __init sdma_probe(struct platform_device *pdev) ...@@ -1462,7 +1462,7 @@ static int __init sdma_probe(struct platform_device *pdev)
return ret; return ret;
} }
static int __exit sdma_remove(struct platform_device *pdev) static int sdma_remove(struct platform_device *pdev)
{ {
return -EBUSY; return -EBUSY;
} }
...@@ -1473,7 +1473,7 @@ static struct platform_driver sdma_driver = { ...@@ -1473,7 +1473,7 @@ static struct platform_driver sdma_driver = {
.of_match_table = sdma_dt_ids, .of_match_table = sdma_dt_ids,
}, },
.id_table = sdma_devtypes, .id_table = sdma_devtypes,
.remove = __exit_p(sdma_remove), .remove = sdma_remove,
}; };
static int __init sdma_module_init(void) static int __init sdma_module_init(void)
......
...@@ -892,7 +892,7 @@ MODULE_PARM_DESC(ioat_interrupt_style, ...@@ -892,7 +892,7 @@ MODULE_PARM_DESC(ioat_interrupt_style,
* ioat_dma_setup_interrupts - setup interrupt handler * ioat_dma_setup_interrupts - setup interrupt handler
* @device: ioat device * @device: ioat device
*/ */
static int ioat_dma_setup_interrupts(struct ioatdma_device *device) int ioat_dma_setup_interrupts(struct ioatdma_device *device)
{ {
struct ioat_chan_common *chan; struct ioat_chan_common *chan;
struct pci_dev *pdev = device->pdev; struct pci_dev *pdev = device->pdev;
...@@ -941,6 +941,7 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device) ...@@ -941,6 +941,7 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device)
} }
} }
intrctrl |= IOAT_INTRCTRL_MSIX_VECTOR_CONTROL; intrctrl |= IOAT_INTRCTRL_MSIX_VECTOR_CONTROL;
device->irq_mode = IOAT_MSIX;
goto done; goto done;
msix_single_vector: msix_single_vector:
...@@ -956,6 +957,7 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device) ...@@ -956,6 +957,7 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device)
pci_disable_msix(pdev); pci_disable_msix(pdev);
goto msi; goto msi;
} }
device->irq_mode = IOAT_MSIX_SINGLE;
goto done; goto done;
msi: msi:
...@@ -969,6 +971,7 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device) ...@@ -969,6 +971,7 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device)
pci_disable_msi(pdev); pci_disable_msi(pdev);
goto intx; goto intx;
} }
device->irq_mode = IOAT_MSIX;
goto done; goto done;
intx: intx:
...@@ -977,6 +980,7 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device) ...@@ -977,6 +980,7 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device)
if (err) if (err)
goto err_no_irq; goto err_no_irq;
device->irq_mode = IOAT_INTX;
done: done:
if (device->intr_quirk) if (device->intr_quirk)
device->intr_quirk(device); device->intr_quirk(device);
...@@ -987,9 +991,11 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device) ...@@ -987,9 +991,11 @@ static int ioat_dma_setup_interrupts(struct ioatdma_device *device)
err_no_irq: err_no_irq:
/* Disable all interrupt generation */ /* Disable all interrupt generation */
writeb(0, device->reg_base + IOAT_INTRCTRL_OFFSET); writeb(0, device->reg_base + IOAT_INTRCTRL_OFFSET);
device->irq_mode = IOAT_NOIRQ;
dev_err(dev, "no usable interrupts\n"); dev_err(dev, "no usable interrupts\n");
return err; return err;
} }
EXPORT_SYMBOL(ioat_dma_setup_interrupts);
static void ioat_disable_interrupts(struct ioatdma_device *device) static void ioat_disable_interrupts(struct ioatdma_device *device)
{ {
......
...@@ -39,6 +39,7 @@ ...@@ -39,6 +39,7 @@
#define to_ioat_desc(lh) container_of(lh, struct ioat_desc_sw, node) #define to_ioat_desc(lh) container_of(lh, struct ioat_desc_sw, node)
#define tx_to_ioat_desc(tx) container_of(tx, struct ioat_desc_sw, txd) #define tx_to_ioat_desc(tx) container_of(tx, struct ioat_desc_sw, txd)
#define to_dev(ioat_chan) (&(ioat_chan)->device->pdev->dev) #define to_dev(ioat_chan) (&(ioat_chan)->device->pdev->dev)
#define to_pdev(ioat_chan) ((ioat_chan)->device->pdev)
#define chan_num(ch) ((int)((ch)->reg_base - (ch)->device->reg_base) / 0x80) #define chan_num(ch) ((int)((ch)->reg_base - (ch)->device->reg_base) / 0x80)
...@@ -48,6 +49,14 @@ ...@@ -48,6 +49,14 @@
*/ */
#define NULL_DESC_BUFFER_SIZE 1 #define NULL_DESC_BUFFER_SIZE 1
enum ioat_irq_mode {
IOAT_NOIRQ = 0,
IOAT_MSIX,
IOAT_MSIX_SINGLE,
IOAT_MSI,
IOAT_INTX
};
/** /**
* struct ioatdma_device - internal representation of a IOAT device * struct ioatdma_device - internal representation of a IOAT device
* @pdev: PCI-Express device * @pdev: PCI-Express device
...@@ -72,11 +81,16 @@ struct ioatdma_device { ...@@ -72,11 +81,16 @@ struct ioatdma_device {
void __iomem *reg_base; void __iomem *reg_base;
struct pci_pool *dma_pool; struct pci_pool *dma_pool;
struct pci_pool *completion_pool; struct pci_pool *completion_pool;
#define MAX_SED_POOLS 5
struct dma_pool *sed_hw_pool[MAX_SED_POOLS];
struct kmem_cache *sed_pool;
struct dma_device common; struct dma_device common;
u8 version; u8 version;
struct msix_entry msix_entries[4]; struct msix_entry msix_entries[4];
struct ioat_chan_common *idx[4]; struct ioat_chan_common *idx[4];
struct dca_provider *dca; struct dca_provider *dca;
enum ioat_irq_mode irq_mode;
u32 cap;
void (*intr_quirk)(struct ioatdma_device *device); void (*intr_quirk)(struct ioatdma_device *device);
int (*enumerate_channels)(struct ioatdma_device *device); int (*enumerate_channels)(struct ioatdma_device *device);
int (*reset_hw)(struct ioat_chan_common *chan); int (*reset_hw)(struct ioat_chan_common *chan);
...@@ -131,6 +145,20 @@ struct ioat_dma_chan { ...@@ -131,6 +145,20 @@ struct ioat_dma_chan {
u16 active; u16 active;
}; };
/**
* struct ioat_sed_ent - wrapper around super extended hardware descriptor
* @hw: hardware SED
* @sed_dma: dma address for the SED
* @list: list member
* @parent: point to the dma descriptor that's the parent
*/
struct ioat_sed_ent {
struct ioat_sed_raw_descriptor *hw;
dma_addr_t dma;
struct ioat_ring_ent *parent;
unsigned int hw_pool;
};
static inline struct ioat_chan_common *to_chan_common(struct dma_chan *c) static inline struct ioat_chan_common *to_chan_common(struct dma_chan *c)
{ {
return container_of(c, struct ioat_chan_common, common); return container_of(c, struct ioat_chan_common, common);
...@@ -179,7 +207,7 @@ __dump_desc_dbg(struct ioat_chan_common *chan, struct ioat_dma_descriptor *hw, ...@@ -179,7 +207,7 @@ __dump_desc_dbg(struct ioat_chan_common *chan, struct ioat_dma_descriptor *hw,
struct device *dev = to_dev(chan); struct device *dev = to_dev(chan);
dev_dbg(dev, "desc[%d]: (%#llx->%#llx) cookie: %d flags: %#x" dev_dbg(dev, "desc[%d]: (%#llx->%#llx) cookie: %d flags: %#x"
" ctl: %#x (op: %d int_en: %d compl: %d)\n", id, " ctl: %#10.8x (op: %#x int_en: %d compl: %d)\n", id,
(unsigned long long) tx->phys, (unsigned long long) tx->phys,
(unsigned long long) hw->next, tx->cookie, tx->flags, (unsigned long long) hw->next, tx->cookie, tx->flags,
hw->ctl, hw->ctl_f.op, hw->ctl_f.int_en, hw->ctl_f.compl_write); hw->ctl, hw->ctl_f.op, hw->ctl_f.int_en, hw->ctl_f.compl_write);
...@@ -201,7 +229,7 @@ ioat_chan_by_index(struct ioatdma_device *device, int index) ...@@ -201,7 +229,7 @@ ioat_chan_by_index(struct ioatdma_device *device, int index)
return device->idx[index]; return device->idx[index];
} }
static inline u64 ioat_chansts(struct ioat_chan_common *chan) static inline u64 ioat_chansts_32(struct ioat_chan_common *chan)
{ {
u8 ver = chan->device->version; u8 ver = chan->device->version;
u64 status; u64 status;
...@@ -218,6 +246,26 @@ static inline u64 ioat_chansts(struct ioat_chan_common *chan) ...@@ -218,6 +246,26 @@ static inline u64 ioat_chansts(struct ioat_chan_common *chan)
return status; return status;
} }
#if BITS_PER_LONG == 64
static inline u64 ioat_chansts(struct ioat_chan_common *chan)
{
u8 ver = chan->device->version;
u64 status;
/* With IOAT v3.3 the status register is 64bit. */
if (ver >= IOAT_VER_3_3)
status = readq(chan->reg_base + IOAT_CHANSTS_OFFSET(ver));
else
status = ioat_chansts_32(chan);
return status;
}
#else
#define ioat_chansts ioat_chansts_32
#endif
static inline void ioat_start(struct ioat_chan_common *chan) static inline void ioat_start(struct ioat_chan_common *chan)
{ {
u8 ver = chan->device->version; u8 ver = chan->device->version;
...@@ -321,6 +369,7 @@ bool ioat_cleanup_preamble(struct ioat_chan_common *chan, ...@@ -321,6 +369,7 @@ bool ioat_cleanup_preamble(struct ioat_chan_common *chan,
dma_addr_t *phys_complete); dma_addr_t *phys_complete);
void ioat_kobject_add(struct ioatdma_device *device, struct kobj_type *type); void ioat_kobject_add(struct ioatdma_device *device, struct kobj_type *type);
void ioat_kobject_del(struct ioatdma_device *device); void ioat_kobject_del(struct ioatdma_device *device);
int ioat_dma_setup_interrupts(struct ioatdma_device *device);
extern const struct sysfs_ops ioat_sysfs_ops; extern const struct sysfs_ops ioat_sysfs_ops;
extern struct ioat_sysfs_entry ioat_version_attr; extern struct ioat_sysfs_entry ioat_version_attr;
extern struct ioat_sysfs_entry ioat_cap_attr; extern struct ioat_sysfs_entry ioat_cap_attr;
......
...@@ -137,6 +137,7 @@ struct ioat_ring_ent { ...@@ -137,6 +137,7 @@ struct ioat_ring_ent {
#ifdef DEBUG #ifdef DEBUG
int id; int id;
#endif #endif
struct ioat_sed_ent *sed;
}; };
static inline struct ioat_ring_ent * static inline struct ioat_ring_ent *
...@@ -157,6 +158,7 @@ static inline void ioat2_set_chainaddr(struct ioat2_dma_chan *ioat, u64 addr) ...@@ -157,6 +158,7 @@ static inline void ioat2_set_chainaddr(struct ioat2_dma_chan *ioat, u64 addr)
int ioat2_dma_probe(struct ioatdma_device *dev, int dca); int ioat2_dma_probe(struct ioatdma_device *dev, int dca);
int ioat3_dma_probe(struct ioatdma_device *dev, int dca); int ioat3_dma_probe(struct ioatdma_device *dev, int dca);
void ioat3_dma_remove(struct ioatdma_device *dev);
struct dca_provider *ioat2_dca_init(struct pci_dev *pdev, void __iomem *iobase); struct dca_provider *ioat2_dca_init(struct pci_dev *pdev, void __iomem *iobase);
struct dca_provider *ioat3_dca_init(struct pci_dev *pdev, void __iomem *iobase); struct dca_provider *ioat3_dca_init(struct pci_dev *pdev, void __iomem *iobase);
int ioat2_check_space_lock(struct ioat2_dma_chan *ioat, int num_descs); int ioat2_check_space_lock(struct ioat2_dma_chan *ioat, int num_descs);
......
This diff is collapsed.
...@@ -30,11 +30,6 @@ ...@@ -30,11 +30,6 @@
#define IOAT_PCI_DID_SCNB 0x65FF #define IOAT_PCI_DID_SCNB 0x65FF
#define IOAT_PCI_DID_SNB 0x402F #define IOAT_PCI_DID_SNB 0x402F
#define IOAT_VER_1_2 0x12 /* Version 1.2 */
#define IOAT_VER_2_0 0x20 /* Version 2.0 */
#define IOAT_VER_3_0 0x30 /* Version 3.0 */
#define IOAT_VER_3_2 0x32 /* Version 3.2 */
#define PCI_DEVICE_ID_INTEL_IOAT_IVB0 0x0e20 #define PCI_DEVICE_ID_INTEL_IOAT_IVB0 0x0e20
#define PCI_DEVICE_ID_INTEL_IOAT_IVB1 0x0e21 #define PCI_DEVICE_ID_INTEL_IOAT_IVB1 0x0e21
#define PCI_DEVICE_ID_INTEL_IOAT_IVB2 0x0e22 #define PCI_DEVICE_ID_INTEL_IOAT_IVB2 0x0e22
...@@ -46,6 +41,29 @@ ...@@ -46,6 +41,29 @@
#define PCI_DEVICE_ID_INTEL_IOAT_IVB8 0x0e2e #define PCI_DEVICE_ID_INTEL_IOAT_IVB8 0x0e2e
#define PCI_DEVICE_ID_INTEL_IOAT_IVB9 0x0e2f #define PCI_DEVICE_ID_INTEL_IOAT_IVB9 0x0e2f
#define PCI_DEVICE_ID_INTEL_IOAT_HSW0 0x2f20
#define PCI_DEVICE_ID_INTEL_IOAT_HSW1 0x2f21
#define PCI_DEVICE_ID_INTEL_IOAT_HSW2 0x2f22
#define PCI_DEVICE_ID_INTEL_IOAT_HSW3 0x2f23
#define PCI_DEVICE_ID_INTEL_IOAT_HSW4 0x2f24
#define PCI_DEVICE_ID_INTEL_IOAT_HSW5 0x2f25
#define PCI_DEVICE_ID_INTEL_IOAT_HSW6 0x2f26
#define PCI_DEVICE_ID_INTEL_IOAT_HSW7 0x2f27
#define PCI_DEVICE_ID_INTEL_IOAT_HSW8 0x2f2e
#define PCI_DEVICE_ID_INTEL_IOAT_HSW9 0x2f2f
#define PCI_DEVICE_ID_INTEL_IOAT_BWD0 0x0C50
#define PCI_DEVICE_ID_INTEL_IOAT_BWD1 0x0C51
#define PCI_DEVICE_ID_INTEL_IOAT_BWD2 0x0C52
#define PCI_DEVICE_ID_INTEL_IOAT_BWD3 0x0C53
#define IOAT_VER_1_2 0x12 /* Version 1.2 */
#define IOAT_VER_2_0 0x20 /* Version 2.0 */
#define IOAT_VER_3_0 0x30 /* Version 3.0 */
#define IOAT_VER_3_2 0x32 /* Version 3.2 */
#define IOAT_VER_3_3 0x33 /* Version 3.3 */
int system_has_dca_enabled(struct pci_dev *pdev); int system_has_dca_enabled(struct pci_dev *pdev);
struct ioat_dma_descriptor { struct ioat_dma_descriptor {
...@@ -147,7 +165,17 @@ struct ioat_xor_ext_descriptor { ...@@ -147,7 +165,17 @@ struct ioat_xor_ext_descriptor {
}; };
struct ioat_pq_descriptor { struct ioat_pq_descriptor {
uint32_t size; union {
uint32_t size;
uint32_t dwbes;
struct {
unsigned int rsvd:25;
unsigned int p_val_err:1;
unsigned int q_val_err:1;
unsigned int rsvd1:4;
unsigned int wbes:1;
} dwbes_f;
};
union { union {
uint32_t ctl; uint32_t ctl;
struct { struct {
...@@ -162,9 +190,14 @@ struct ioat_pq_descriptor { ...@@ -162,9 +190,14 @@ struct ioat_pq_descriptor {
unsigned int hint:1; unsigned int hint:1;
unsigned int p_disable:1; unsigned int p_disable:1;
unsigned int q_disable:1; unsigned int q_disable:1;
unsigned int rsvd:11; unsigned int rsvd2:2;
unsigned int wb_en:1;
unsigned int prl_en:1;
unsigned int rsvd3:7;
#define IOAT_OP_PQ 0x89 #define IOAT_OP_PQ 0x89
#define IOAT_OP_PQ_VAL 0x8a #define IOAT_OP_PQ_VAL 0x8a
#define IOAT_OP_PQ_16S 0xa0
#define IOAT_OP_PQ_VAL_16S 0xa1
unsigned int op:8; unsigned int op:8;
} ctl_f; } ctl_f;
}; };
...@@ -172,7 +205,10 @@ struct ioat_pq_descriptor { ...@@ -172,7 +205,10 @@ struct ioat_pq_descriptor {
uint64_t p_addr; uint64_t p_addr;
uint64_t next; uint64_t next;
uint64_t src_addr2; uint64_t src_addr2;
uint64_t src_addr3; union {
uint64_t src_addr3;
uint64_t sed_addr;
};
uint8_t coef[8]; uint8_t coef[8];
uint64_t q_addr; uint64_t q_addr;
}; };
...@@ -221,4 +257,40 @@ struct ioat_pq_update_descriptor { ...@@ -221,4 +257,40 @@ struct ioat_pq_update_descriptor {
struct ioat_raw_descriptor { struct ioat_raw_descriptor {
uint64_t field[8]; uint64_t field[8];
}; };
struct ioat_pq16a_descriptor {
uint8_t coef[8];
uint64_t src_addr3;
uint64_t src_addr4;
uint64_t src_addr5;
uint64_t src_addr6;
uint64_t src_addr7;
uint64_t src_addr8;
uint64_t src_addr9;
};
struct ioat_pq16b_descriptor {
uint64_t src_addr10;
uint64_t src_addr11;
uint64_t src_addr12;
uint64_t src_addr13;
uint64_t src_addr14;
uint64_t src_addr15;
uint64_t src_addr16;
uint64_t rsvd;
};
union ioat_sed_pq_descriptor {
struct ioat_pq16a_descriptor a;
struct ioat_pq16b_descriptor b;
};
#define SED_SIZE 64
struct ioat_sed_raw_descriptor {
uint64_t a[8];
uint64_t b[8];
uint64_t c[8];
};
#endif #endif
...@@ -94,6 +94,23 @@ static struct pci_device_id ioat_pci_tbl[] = { ...@@ -94,6 +94,23 @@ static struct pci_device_id ioat_pci_tbl[] = {
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_IVB8) }, { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_IVB8) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_IVB9) }, { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_IVB9) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW0) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW1) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW2) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW3) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW4) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW5) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW6) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW7) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW8) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW9) },
/* I/OAT v3.3 platforms */
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD0) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD1) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD2) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD3) },
{ 0, } { 0, }
}; };
MODULE_DEVICE_TABLE(pci, ioat_pci_tbl); MODULE_DEVICE_TABLE(pci, ioat_pci_tbl);
...@@ -190,6 +207,9 @@ static void ioat_remove(struct pci_dev *pdev) ...@@ -190,6 +207,9 @@ static void ioat_remove(struct pci_dev *pdev)
if (!device) if (!device)
return; return;
if (device->version >= IOAT_VER_3_0)
ioat3_dma_remove(device);
dev_err(&pdev->dev, "Removing dma and dca services\n"); dev_err(&pdev->dev, "Removing dma and dca services\n");
if (device->dca) { if (device->dca) {
unregister_dca_provider(device->dca, &pdev->dev); unregister_dca_provider(device->dca, &pdev->dev);
......
...@@ -79,6 +79,8 @@ ...@@ -79,6 +79,8 @@
#define IOAT_CAP_APIC 0x00000080 #define IOAT_CAP_APIC 0x00000080
#define IOAT_CAP_XOR 0x00000100 #define IOAT_CAP_XOR 0x00000100
#define IOAT_CAP_PQ 0x00000200 #define IOAT_CAP_PQ 0x00000200
#define IOAT_CAP_DWBES 0x00002000
#define IOAT_CAP_RAID16SS 0x00020000
#define IOAT_CHANNEL_MMIO_SIZE 0x80 /* Each Channel MMIO space is this size */ #define IOAT_CHANNEL_MMIO_SIZE 0x80 /* Each Channel MMIO space is this size */
...@@ -93,6 +95,8 @@ ...@@ -93,6 +95,8 @@
#define IOAT_CHANCTRL_ERR_COMPLETION_EN 0x0004 #define IOAT_CHANCTRL_ERR_COMPLETION_EN 0x0004
#define IOAT_CHANCTRL_INT_REARM 0x0001 #define IOAT_CHANCTRL_INT_REARM 0x0001
#define IOAT_CHANCTRL_RUN (IOAT_CHANCTRL_INT_REARM |\ #define IOAT_CHANCTRL_RUN (IOAT_CHANCTRL_INT_REARM |\
IOAT_CHANCTRL_ERR_INT_EN |\
IOAT_CHANCTRL_ERR_COMPLETION_EN |\
IOAT_CHANCTRL_ANY_ERR_ABORT_EN) IOAT_CHANCTRL_ANY_ERR_ABORT_EN)
#define IOAT_DMA_COMP_OFFSET 0x02 /* 16-bit DMA channel compatibility */ #define IOAT_DMA_COMP_OFFSET 0x02 /* 16-bit DMA channel compatibility */
......
...@@ -1642,7 +1642,7 @@ static int __init ipu_idmac_init(struct ipu *ipu) ...@@ -1642,7 +1642,7 @@ static int __init ipu_idmac_init(struct ipu *ipu)
return dma_async_device_register(&idmac->dma); return dma_async_device_register(&idmac->dma);
} }
static void __exit ipu_idmac_exit(struct ipu *ipu) static void ipu_idmac_exit(struct ipu *ipu)
{ {
int i; int i;
struct idmac *idmac = &ipu->idmac; struct idmac *idmac = &ipu->idmac;
...@@ -1756,7 +1756,7 @@ static int __init ipu_probe(struct platform_device *pdev) ...@@ -1756,7 +1756,7 @@ static int __init ipu_probe(struct platform_device *pdev)
return ret; return ret;
} }
static int __exit ipu_remove(struct platform_device *pdev) static int ipu_remove(struct platform_device *pdev)
{ {
struct ipu *ipu = platform_get_drvdata(pdev); struct ipu *ipu = platform_get_drvdata(pdev);
...@@ -1781,7 +1781,7 @@ static struct platform_driver ipu_platform_driver = { ...@@ -1781,7 +1781,7 @@ static struct platform_driver ipu_platform_driver = {
.name = "ipu-core", .name = "ipu-core",
.owner = THIS_MODULE, .owner = THIS_MODULE,
}, },
.remove = __exit_p(ipu_remove), .remove = ipu_remove,
}; };
static int __init ipu_init(void) static int __init ipu_init(void)
......
...@@ -13,43 +13,31 @@ ...@@ -13,43 +13,31 @@
#include <linux/device.h> #include <linux/device.h>
#include <linux/err.h> #include <linux/err.h>
#include <linux/module.h> #include <linux/module.h>
#include <linux/rculist.h> #include <linux/mutex.h>
#include <linux/slab.h> #include <linux/slab.h>
#include <linux/of.h> #include <linux/of.h>
#include <linux/of_dma.h> #include <linux/of_dma.h>
static LIST_HEAD(of_dma_list); static LIST_HEAD(of_dma_list);
static DEFINE_SPINLOCK(of_dma_lock); static DEFINE_MUTEX(of_dma_lock);
/** /**
* of_dma_get_controller - Get a DMA controller in DT DMA helpers list * of_dma_find_controller - Get a DMA controller in DT DMA helpers list
* @dma_spec: pointer to DMA specifier as found in the device tree * @dma_spec: pointer to DMA specifier as found in the device tree
* *
* Finds a DMA controller with matching device node and number for dma cells * Finds a DMA controller with matching device node and number for dma cells
* in a list of registered DMA controllers. If a match is found the use_count * in a list of registered DMA controllers. If a match is found a valid pointer
* variable is increased and a valid pointer to the DMA data stored is retuned. * to the DMA data stored is retuned. A NULL pointer is returned if no match is
* A NULL pointer is returned if no match is found. * found.
*/ */
static struct of_dma *of_dma_get_controller(struct of_phandle_args *dma_spec) static struct of_dma *of_dma_find_controller(struct of_phandle_args *dma_spec)
{ {
struct of_dma *ofdma; struct of_dma *ofdma;
spin_lock(&of_dma_lock);
if (list_empty(&of_dma_list)) {
spin_unlock(&of_dma_lock);
return NULL;
}
list_for_each_entry(ofdma, &of_dma_list, of_dma_controllers) list_for_each_entry(ofdma, &of_dma_list, of_dma_controllers)
if ((ofdma->of_node == dma_spec->np) && if ((ofdma->of_node == dma_spec->np) &&
(ofdma->of_dma_nbcells == dma_spec->args_count)) { (ofdma->of_dma_nbcells == dma_spec->args_count))
ofdma->use_count++;
spin_unlock(&of_dma_lock);
return ofdma; return ofdma;
}
spin_unlock(&of_dma_lock);
pr_debug("%s: can't find DMA controller %s\n", __func__, pr_debug("%s: can't find DMA controller %s\n", __func__,
dma_spec->np->full_name); dma_spec->np->full_name);
...@@ -57,22 +45,6 @@ static struct of_dma *of_dma_get_controller(struct of_phandle_args *dma_spec) ...@@ -57,22 +45,6 @@ static struct of_dma *of_dma_get_controller(struct of_phandle_args *dma_spec)
return NULL; return NULL;
} }
/**
* of_dma_put_controller - Decrement use count for a registered DMA controller
* @of_dma: pointer to DMA controller data
*
* Decrements the use_count variable in the DMA data structure. This function
* should be called only when a valid pointer is returned from
* of_dma_get_controller() and no further accesses to data referenced by that
* pointer are needed.
*/
static void of_dma_put_controller(struct of_dma *ofdma)
{
spin_lock(&of_dma_lock);
ofdma->use_count--;
spin_unlock(&of_dma_lock);
}
/** /**
* of_dma_controller_register - Register a DMA controller to DT DMA helpers * of_dma_controller_register - Register a DMA controller to DT DMA helpers
* @np: device node of DMA controller * @np: device node of DMA controller
...@@ -93,6 +65,7 @@ int of_dma_controller_register(struct device_node *np, ...@@ -93,6 +65,7 @@ int of_dma_controller_register(struct device_node *np,
{ {
struct of_dma *ofdma; struct of_dma *ofdma;
int nbcells; int nbcells;
const __be32 *prop;
if (!np || !of_dma_xlate) { if (!np || !of_dma_xlate) {
pr_err("%s: not enough information provided\n", __func__); pr_err("%s: not enough information provided\n", __func__);
...@@ -103,8 +76,11 @@ int of_dma_controller_register(struct device_node *np, ...@@ -103,8 +76,11 @@ int of_dma_controller_register(struct device_node *np,
if (!ofdma) if (!ofdma)
return -ENOMEM; return -ENOMEM;
nbcells = be32_to_cpup(of_get_property(np, "#dma-cells", NULL)); prop = of_get_property(np, "#dma-cells", NULL);
if (!nbcells) { if (prop)
nbcells = be32_to_cpup(prop);
if (!prop || !nbcells) {
pr_err("%s: #dma-cells property is missing or invalid\n", pr_err("%s: #dma-cells property is missing or invalid\n",
__func__); __func__);
kfree(ofdma); kfree(ofdma);
...@@ -115,12 +91,11 @@ int of_dma_controller_register(struct device_node *np, ...@@ -115,12 +91,11 @@ int of_dma_controller_register(struct device_node *np,
ofdma->of_dma_nbcells = nbcells; ofdma->of_dma_nbcells = nbcells;
ofdma->of_dma_xlate = of_dma_xlate; ofdma->of_dma_xlate = of_dma_xlate;
ofdma->of_dma_data = data; ofdma->of_dma_data = data;
ofdma->use_count = 0;
/* Now queue of_dma controller structure in list */ /* Now queue of_dma controller structure in list */
spin_lock(&of_dma_lock); mutex_lock(&of_dma_lock);
list_add_tail(&ofdma->of_dma_controllers, &of_dma_list); list_add_tail(&ofdma->of_dma_controllers, &of_dma_list);
spin_unlock(&of_dma_lock); mutex_unlock(&of_dma_lock);
return 0; return 0;
} }
...@@ -132,32 +107,20 @@ EXPORT_SYMBOL_GPL(of_dma_controller_register); ...@@ -132,32 +107,20 @@ EXPORT_SYMBOL_GPL(of_dma_controller_register);
* *
* Memory allocated by of_dma_controller_register() is freed here. * Memory allocated by of_dma_controller_register() is freed here.
*/ */
int of_dma_controller_free(struct device_node *np) void of_dma_controller_free(struct device_node *np)
{ {
struct of_dma *ofdma; struct of_dma *ofdma;
spin_lock(&of_dma_lock); mutex_lock(&of_dma_lock);
if (list_empty(&of_dma_list)) {
spin_unlock(&of_dma_lock);
return -ENODEV;
}
list_for_each_entry(ofdma, &of_dma_list, of_dma_controllers) list_for_each_entry(ofdma, &of_dma_list, of_dma_controllers)
if (ofdma->of_node == np) { if (ofdma->of_node == np) {
if (ofdma->use_count) {
spin_unlock(&of_dma_lock);
return -EBUSY;
}
list_del(&ofdma->of_dma_controllers); list_del(&ofdma->of_dma_controllers);
spin_unlock(&of_dma_lock);
kfree(ofdma); kfree(ofdma);
return 0; break;
} }
spin_unlock(&of_dma_lock); mutex_unlock(&of_dma_lock);
return -ENODEV;
} }
EXPORT_SYMBOL_GPL(of_dma_controller_free); EXPORT_SYMBOL_GPL(of_dma_controller_free);
...@@ -172,8 +135,8 @@ EXPORT_SYMBOL_GPL(of_dma_controller_free); ...@@ -172,8 +135,8 @@ EXPORT_SYMBOL_GPL(of_dma_controller_free);
* specifiers, matches the name provided. Returns 0 if the name matches and * specifiers, matches the name provided. Returns 0 if the name matches and
* a valid pointer to the DMA specifier is found. Otherwise returns -ENODEV. * a valid pointer to the DMA specifier is found. Otherwise returns -ENODEV.
*/ */
static int of_dma_match_channel(struct device_node *np, char *name, int index, static int of_dma_match_channel(struct device_node *np, const char *name,
struct of_phandle_args *dma_spec) int index, struct of_phandle_args *dma_spec)
{ {
const char *s; const char *s;
...@@ -198,7 +161,7 @@ static int of_dma_match_channel(struct device_node *np, char *name, int index, ...@@ -198,7 +161,7 @@ static int of_dma_match_channel(struct device_node *np, char *name, int index,
* Returns pointer to appropriate dma channel on success or NULL on error. * Returns pointer to appropriate dma channel on success or NULL on error.
*/ */
struct dma_chan *of_dma_request_slave_channel(struct device_node *np, struct dma_chan *of_dma_request_slave_channel(struct device_node *np,
char *name) const char *name)
{ {
struct of_phandle_args dma_spec; struct of_phandle_args dma_spec;
struct of_dma *ofdma; struct of_dma *ofdma;
...@@ -220,14 +183,15 @@ struct dma_chan *of_dma_request_slave_channel(struct device_node *np, ...@@ -220,14 +183,15 @@ struct dma_chan *of_dma_request_slave_channel(struct device_node *np,
if (of_dma_match_channel(np, name, i, &dma_spec)) if (of_dma_match_channel(np, name, i, &dma_spec))
continue; continue;
ofdma = of_dma_get_controller(&dma_spec); mutex_lock(&of_dma_lock);
ofdma = of_dma_find_controller(&dma_spec);
if (!ofdma)
continue;
chan = ofdma->of_dma_xlate(&dma_spec, ofdma); if (ofdma)
chan = ofdma->of_dma_xlate(&dma_spec, ofdma);
else
chan = NULL;
of_dma_put_controller(ofdma); mutex_unlock(&of_dma_lock);
of_node_put(dma_spec.np); of_node_put(dma_spec.np);
......
...@@ -16,6 +16,8 @@ ...@@ -16,6 +16,8 @@
#include <linux/platform_device.h> #include <linux/platform_device.h>
#include <linux/slab.h> #include <linux/slab.h>
#include <linux/spinlock.h> #include <linux/spinlock.h>
#include <linux/of_dma.h>
#include <linux/of_device.h>
#include "virt-dma.h" #include "virt-dma.h"
...@@ -67,6 +69,10 @@ static const unsigned es_bytes[] = { ...@@ -67,6 +69,10 @@ static const unsigned es_bytes[] = {
[OMAP_DMA_DATA_TYPE_S32] = 4, [OMAP_DMA_DATA_TYPE_S32] = 4,
}; };
static struct of_dma_filter_info omap_dma_info = {
.filter_fn = omap_dma_filter_fn,
};
static inline struct omap_dmadev *to_omap_dma_dev(struct dma_device *d) static inline struct omap_dmadev *to_omap_dma_dev(struct dma_device *d)
{ {
return container_of(d, struct omap_dmadev, ddev); return container_of(d, struct omap_dmadev, ddev);
...@@ -629,8 +635,22 @@ static int omap_dma_probe(struct platform_device *pdev) ...@@ -629,8 +635,22 @@ static int omap_dma_probe(struct platform_device *pdev)
pr_warn("OMAP-DMA: failed to register slave DMA engine device: %d\n", pr_warn("OMAP-DMA: failed to register slave DMA engine device: %d\n",
rc); rc);
omap_dma_free(od); omap_dma_free(od);
} else { return rc;
platform_set_drvdata(pdev, od); }
platform_set_drvdata(pdev, od);
if (pdev->dev.of_node) {
omap_dma_info.dma_cap = od->ddev.cap_mask;
/* Device-tree DMA controller registration */
rc = of_dma_controller_register(pdev->dev.of_node,
of_dma_simple_xlate, &omap_dma_info);
if (rc) {
pr_warn("OMAP-DMA: failed to register DMA controller\n");
dma_async_device_unregister(&od->ddev);
omap_dma_free(od);
}
} }
dev_info(&pdev->dev, "OMAP DMA engine driver\n"); dev_info(&pdev->dev, "OMAP DMA engine driver\n");
...@@ -642,18 +662,32 @@ static int omap_dma_remove(struct platform_device *pdev) ...@@ -642,18 +662,32 @@ static int omap_dma_remove(struct platform_device *pdev)
{ {
struct omap_dmadev *od = platform_get_drvdata(pdev); struct omap_dmadev *od = platform_get_drvdata(pdev);
if (pdev->dev.of_node)
of_dma_controller_free(pdev->dev.of_node);
dma_async_device_unregister(&od->ddev); dma_async_device_unregister(&od->ddev);
omap_dma_free(od); omap_dma_free(od);
return 0; return 0;
} }
static const struct of_device_id omap_dma_match[] = {
{ .compatible = "ti,omap2420-sdma", },
{ .compatible = "ti,omap2430-sdma", },
{ .compatible = "ti,omap3430-sdma", },
{ .compatible = "ti,omap3630-sdma", },
{ .compatible = "ti,omap4430-sdma", },
{},
};
MODULE_DEVICE_TABLE(of, omap_dma_match);
static struct platform_driver omap_dma_driver = { static struct platform_driver omap_dma_driver = {
.probe = omap_dma_probe, .probe = omap_dma_probe,
.remove = omap_dma_remove, .remove = omap_dma_remove,
.driver = { .driver = {
.name = "omap-dma-engine", .name = "omap-dma-engine",
.owner = THIS_MODULE, .owner = THIS_MODULE,
.of_match_table = of_match_ptr(omap_dma_match),
}, },
}; };
......
...@@ -476,7 +476,7 @@ static struct pch_dma_desc *pdc_desc_get(struct pch_dma_chan *pd_chan) ...@@ -476,7 +476,7 @@ static struct pch_dma_desc *pdc_desc_get(struct pch_dma_chan *pd_chan)
dev_dbg(chan2dev(&pd_chan->chan), "scanned %d descriptors\n", i); dev_dbg(chan2dev(&pd_chan->chan), "scanned %d descriptors\n", i);
if (!ret) { if (!ret) {
ret = pdc_alloc_desc(&pd_chan->chan, GFP_NOIO); ret = pdc_alloc_desc(&pd_chan->chan, GFP_ATOMIC);
if (ret) { if (ret) {
spin_lock(&pd_chan->lock); spin_lock(&pd_chan->lock);
pd_chan->descs_allocated++; pd_chan->descs_allocated++;
......
...@@ -26,6 +26,7 @@ ...@@ -26,6 +26,7 @@
#include <linux/scatterlist.h> #include <linux/scatterlist.h>
#include <linux/of.h> #include <linux/of.h>
#include <linux/of_dma.h> #include <linux/of_dma.h>
#include <linux/err.h>
#include "dmaengine.h" #include "dmaengine.h"
#define PL330_MAX_CHAN 8 #define PL330_MAX_CHAN 8
...@@ -2288,13 +2289,12 @@ static inline void fill_queue(struct dma_pl330_chan *pch) ...@@ -2288,13 +2289,12 @@ static inline void fill_queue(struct dma_pl330_chan *pch)
/* If already submitted */ /* If already submitted */
if (desc->status == BUSY) if (desc->status == BUSY)
break; continue;
ret = pl330_submit_req(pch->pl330_chid, ret = pl330_submit_req(pch->pl330_chid,
&desc->req); &desc->req);
if (!ret) { if (!ret) {
desc->status = BUSY; desc->status = BUSY;
break;
} else if (ret == -EAGAIN) { } else if (ret == -EAGAIN) {
/* QFull or DMAC Dying */ /* QFull or DMAC Dying */
break; break;
...@@ -2904,9 +2904,9 @@ pl330_probe(struct amba_device *adev, const struct amba_id *id) ...@@ -2904,9 +2904,9 @@ pl330_probe(struct amba_device *adev, const struct amba_id *id)
pi->mcbufsz = pdat ? pdat->mcbuf_sz : 0; pi->mcbufsz = pdat ? pdat->mcbuf_sz : 0;
res = &adev->res; res = &adev->res;
pi->base = devm_request_and_ioremap(&adev->dev, res); pi->base = devm_ioremap_resource(&adev->dev, res);
if (!pi->base) if (IS_ERR(pi->base))
return -ENXIO; return PTR_ERR(pi->base);
amba_set_drvdata(adev, pdmac); amba_set_drvdata(adev, pdmac);
......
#
# DMA engine configuration for sh
#
config SH_DMAE_BASE
bool "Renesas SuperH DMA Engine support"
depends on (SUPERH && SH_DMA) || (ARM && ARCH_SHMOBILE)
depends on !SH_DMA_API
default y
select DMA_ENGINE
help
Enable support for the Renesas SuperH DMA controllers.
config SH_DMAE
tristate "Renesas SuperH DMAC support"
depends on SH_DMAE_BASE
help
Enable support for the Renesas SuperH DMA controllers.
config SUDMAC
tristate "Renesas SUDMAC support"
depends on SH_DMAE_BASE
help
Enable support for the Renesas SUDMAC controllers.
obj-$(CONFIG_SH_DMAE) += shdma-base.o obj-$(CONFIG_SH_DMAE_BASE) += shdma-base.o
obj-$(CONFIG_SH_DMAE) += shdma.o obj-$(CONFIG_SH_DMAE) += shdma.o
obj-$(CONFIG_SUDMAC) += sudmac.o
This diff is collapsed.
...@@ -16,6 +16,7 @@ ...@@ -16,6 +16,7 @@
#include <linux/of_address.h> #include <linux/of_address.h>
#include <linux/of_device.h> #include <linux/of_device.h>
#include <linux/of_platform.h> #include <linux/of_platform.h>
#include <linux/clk.h>
#include <linux/sirfsoc_dma.h> #include <linux/sirfsoc_dma.h>
#include "dmaengine.h" #include "dmaengine.h"
...@@ -78,6 +79,7 @@ struct sirfsoc_dma { ...@@ -78,6 +79,7 @@ struct sirfsoc_dma {
struct sirfsoc_dma_chan channels[SIRFSOC_DMA_CHANNELS]; struct sirfsoc_dma_chan channels[SIRFSOC_DMA_CHANNELS];
void __iomem *base; void __iomem *base;
int irq; int irq;
struct clk *clk;
bool is_marco; bool is_marco;
}; };
...@@ -639,6 +641,12 @@ static int sirfsoc_dma_probe(struct platform_device *op) ...@@ -639,6 +641,12 @@ static int sirfsoc_dma_probe(struct platform_device *op)
return -EINVAL; return -EINVAL;
} }
sdma->clk = devm_clk_get(dev, NULL);
if (IS_ERR(sdma->clk)) {
dev_err(dev, "failed to get a clock.\n");
return PTR_ERR(sdma->clk);
}
ret = of_address_to_resource(dn, 0, &res); ret = of_address_to_resource(dn, 0, &res);
if (ret) { if (ret) {
dev_err(dev, "Error parsing memory region!\n"); dev_err(dev, "Error parsing memory region!\n");
...@@ -698,6 +706,8 @@ static int sirfsoc_dma_probe(struct platform_device *op) ...@@ -698,6 +706,8 @@ static int sirfsoc_dma_probe(struct platform_device *op)
tasklet_init(&sdma->tasklet, sirfsoc_dma_tasklet, (unsigned long)sdma); tasklet_init(&sdma->tasklet, sirfsoc_dma_tasklet, (unsigned long)sdma);
clk_prepare_enable(sdma->clk);
/* Register DMA engine */ /* Register DMA engine */
dev_set_drvdata(dev, sdma); dev_set_drvdata(dev, sdma);
ret = dma_async_device_register(dma); ret = dma_async_device_register(dma);
...@@ -720,6 +730,7 @@ static int sirfsoc_dma_remove(struct platform_device *op) ...@@ -720,6 +730,7 @@ static int sirfsoc_dma_remove(struct platform_device *op)
struct device *dev = &op->dev; struct device *dev = &op->dev;
struct sirfsoc_dma *sdma = dev_get_drvdata(dev); struct sirfsoc_dma *sdma = dev_get_drvdata(dev);
clk_disable_unprepare(sdma->clk);
dma_async_device_unregister(&sdma->dma); dma_async_device_unregister(&sdma->dma);
free_irq(sdma->irq, sdma); free_irq(sdma->irq, sdma);
irq_dispose_mapping(sdma->irq); irq_dispose_mapping(sdma->irq);
...@@ -742,7 +753,18 @@ static struct platform_driver sirfsoc_dma_driver = { ...@@ -742,7 +753,18 @@ static struct platform_driver sirfsoc_dma_driver = {
}, },
}; };
module_platform_driver(sirfsoc_dma_driver); static __init int sirfsoc_dma_init(void)
{
return platform_driver_register(&sirfsoc_dma_driver);
}
static void __exit sirfsoc_dma_exit(void)
{
platform_driver_unregister(&sirfsoc_dma_driver);
}
subsys_initcall(sirfsoc_dma_init);
module_exit(sirfsoc_dma_exit);
MODULE_AUTHOR("Rongjun Ying <rongjun.ying@csr.com>, " MODULE_AUTHOR("Rongjun Ying <rongjun.ying@csr.com>, "
"Barry Song <baohua.song@csr.com>"); "Barry Song <baohua.song@csr.com>");
......
...@@ -30,6 +30,7 @@ ...@@ -30,6 +30,7 @@
#include <linux/of.h> #include <linux/of.h>
#include <linux/of_device.h> #include <linux/of_device.h>
#include <linux/platform_device.h> #include <linux/platform_device.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h> #include <linux/pm_runtime.h>
#include <linux/slab.h> #include <linux/slab.h>
#include <linux/clk/tegra.h> #include <linux/clk/tegra.h>
...@@ -199,6 +200,7 @@ struct tegra_dma_channel { ...@@ -199,6 +200,7 @@ struct tegra_dma_channel {
/* Channel-slave specific configuration */ /* Channel-slave specific configuration */
struct dma_slave_config dma_sconfig; struct dma_slave_config dma_sconfig;
struct tegra_dma_channel_regs channel_reg;
}; };
/* tegra_dma: Tegra DMA specific information */ /* tegra_dma: Tegra DMA specific information */
...@@ -1213,7 +1215,6 @@ static const struct tegra_dma_chip_data tegra20_dma_chip_data = { ...@@ -1213,7 +1215,6 @@ static const struct tegra_dma_chip_data tegra20_dma_chip_data = {
.support_channel_pause = false, .support_channel_pause = false,
}; };
#if defined(CONFIG_OF)
/* Tegra30 specific DMA controller information */ /* Tegra30 specific DMA controller information */
static const struct tegra_dma_chip_data tegra30_dma_chip_data = { static const struct tegra_dma_chip_data tegra30_dma_chip_data = {
.nr_channels = 32, .nr_channels = 32,
...@@ -1243,7 +1244,6 @@ static const struct of_device_id tegra_dma_of_match[] = { ...@@ -1243,7 +1244,6 @@ static const struct of_device_id tegra_dma_of_match[] = {
}, },
}; };
MODULE_DEVICE_TABLE(of, tegra_dma_of_match); MODULE_DEVICE_TABLE(of, tegra_dma_of_match);
#endif
static int tegra_dma_probe(struct platform_device *pdev) static int tegra_dma_probe(struct platform_device *pdev)
{ {
...@@ -1252,20 +1252,14 @@ static int tegra_dma_probe(struct platform_device *pdev) ...@@ -1252,20 +1252,14 @@ static int tegra_dma_probe(struct platform_device *pdev)
int ret; int ret;
int i; int i;
const struct tegra_dma_chip_data *cdata = NULL; const struct tegra_dma_chip_data *cdata = NULL;
const struct of_device_id *match;
if (pdev->dev.of_node) { match = of_match_device(tegra_dma_of_match, &pdev->dev);
const struct of_device_id *match; if (!match) {
match = of_match_device(of_match_ptr(tegra_dma_of_match), dev_err(&pdev->dev, "Error: No device match found\n");
&pdev->dev); return -ENODEV;
if (!match) {
dev_err(&pdev->dev, "Error: No device match found\n");
return -ENODEV;
}
cdata = match->data;
} else {
/* If no device tree then fallback to tegra20 */
cdata = &tegra20_dma_chip_data;
} }
cdata = match->data;
tdma = devm_kzalloc(&pdev->dev, sizeof(*tdma) + cdata->nr_channels * tdma = devm_kzalloc(&pdev->dev, sizeof(*tdma) + cdata->nr_channels *
sizeof(struct tegra_dma_channel), GFP_KERNEL); sizeof(struct tegra_dma_channel), GFP_KERNEL);
...@@ -1448,11 +1442,74 @@ static int tegra_dma_runtime_resume(struct device *dev) ...@@ -1448,11 +1442,74 @@ static int tegra_dma_runtime_resume(struct device *dev)
return 0; return 0;
} }
#ifdef CONFIG_PM_SLEEP
static int tegra_dma_pm_suspend(struct device *dev)
{
struct tegra_dma *tdma = dev_get_drvdata(dev);
int i;
int ret;
/* Enable clock before accessing register */
ret = tegra_dma_runtime_resume(dev);
if (ret < 0)
return ret;
tdma->reg_gen = tdma_read(tdma, TEGRA_APBDMA_GENERAL);
for (i = 0; i < tdma->chip_data->nr_channels; i++) {
struct tegra_dma_channel *tdc = &tdma->channels[i];
struct tegra_dma_channel_regs *ch_reg = &tdc->channel_reg;
ch_reg->csr = tdc_read(tdc, TEGRA_APBDMA_CHAN_CSR);
ch_reg->ahb_ptr = tdc_read(tdc, TEGRA_APBDMA_CHAN_AHBPTR);
ch_reg->apb_ptr = tdc_read(tdc, TEGRA_APBDMA_CHAN_APBPTR);
ch_reg->ahb_seq = tdc_read(tdc, TEGRA_APBDMA_CHAN_AHBSEQ);
ch_reg->apb_seq = tdc_read(tdc, TEGRA_APBDMA_CHAN_APBSEQ);
}
/* Disable clock */
tegra_dma_runtime_suspend(dev);
return 0;
}
static int tegra_dma_pm_resume(struct device *dev)
{
struct tegra_dma *tdma = dev_get_drvdata(dev);
int i;
int ret;
/* Enable clock before accessing register */
ret = tegra_dma_runtime_resume(dev);
if (ret < 0)
return ret;
tdma_write(tdma, TEGRA_APBDMA_GENERAL, tdma->reg_gen);
tdma_write(tdma, TEGRA_APBDMA_CONTROL, 0);
tdma_write(tdma, TEGRA_APBDMA_IRQ_MASK_SET, 0xFFFFFFFFul);
for (i = 0; i < tdma->chip_data->nr_channels; i++) {
struct tegra_dma_channel *tdc = &tdma->channels[i];
struct tegra_dma_channel_regs *ch_reg = &tdc->channel_reg;
tdc_write(tdc, TEGRA_APBDMA_CHAN_APBSEQ, ch_reg->apb_seq);
tdc_write(tdc, TEGRA_APBDMA_CHAN_APBPTR, ch_reg->apb_ptr);
tdc_write(tdc, TEGRA_APBDMA_CHAN_AHBSEQ, ch_reg->ahb_seq);
tdc_write(tdc, TEGRA_APBDMA_CHAN_AHBPTR, ch_reg->ahb_ptr);
tdc_write(tdc, TEGRA_APBDMA_CHAN_CSR,
(ch_reg->csr & ~TEGRA_APBDMA_CSR_ENB));
}
/* Disable clock */
tegra_dma_runtime_suspend(dev);
return 0;
}
#endif
static const struct dev_pm_ops tegra_dma_dev_pm_ops = { static const struct dev_pm_ops tegra_dma_dev_pm_ops = {
#ifdef CONFIG_PM_RUNTIME #ifdef CONFIG_PM_RUNTIME
.runtime_suspend = tegra_dma_runtime_suspend, .runtime_suspend = tegra_dma_runtime_suspend,
.runtime_resume = tegra_dma_runtime_resume, .runtime_resume = tegra_dma_runtime_resume,
#endif #endif
SET_SYSTEM_SLEEP_PM_OPS(tegra_dma_pm_suspend, tegra_dma_pm_resume)
}; };
static struct platform_driver tegra_dmac_driver = { static struct platform_driver tegra_dmac_driver = {
...@@ -1460,7 +1517,7 @@ static struct platform_driver tegra_dmac_driver = { ...@@ -1460,7 +1517,7 @@ static struct platform_driver tegra_dmac_driver = {
.name = "tegra-apbdma", .name = "tegra-apbdma",
.owner = THIS_MODULE, .owner = THIS_MODULE,
.pm = &tegra_dma_dev_pm_ops, .pm = &tegra_dma_dev_pm_ops,
.of_match_table = of_match_ptr(tegra_dma_of_match), .of_match_table = tegra_dma_of_match,
}, },
.probe = tegra_dma_probe, .probe = tegra_dma_probe,
.remove = tegra_dma_remove, .remove = tegra_dma_remove,
......
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment