Commit dd967117 authored by Linus Torvalds's avatar Linus Torvalds

Merge tag 'iommu-updates-v4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:

 - big-endian support and preparation for defered probing for the Exynos
   IOMMU driver

 - simplifications in iommu-group id handling

 - support for Mediatek generation one IOMMU hardware

 - conversion of the AMD IOMMU driver to use the generic IOVA allocator.
   This driver now also benefits from the recent scalability
   improvements in the IOVA code.

 - preparations to use generic DMA mapping code in the Rockchip IOMMU
   driver

 - device tree adaption and conversion to use generic page-table code
   for the MSM IOMMU driver

 - an iova_to_phys optimization in the ARM-SMMU driver to greatly
   improve page-table teardown performance with VFIO

 - various other small fixes and conversions

* tag 'iommu-updates-v4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (59 commits)
  iommu/amd: Initialize dma-ops domains with 3-level page-table
  iommu/amd: Update Alias-DTE in update_device_table()
  iommu/vt-d: Return error code in domain_context_mapping_one()
  iommu/amd: Use container_of to get dma_ops_domain
  iommu/amd: Flush iova queue before releasing dma_ops_domain
  iommu/amd: Handle IOMMU_DOMAIN_DMA in ops->domain_free call-back
  iommu/amd: Use dev_data->domain in get_domain()
  iommu/amd: Optimize map_sg and unmap_sg
  iommu/amd: Introduce dir2prot() helper
  iommu/amd: Implement timeout to flush unmap queues
  iommu/amd: Implement flush queue
  iommu/amd: Allow NULL pointer parameter for domain_flush_complete()
  iommu/amd: Set up data structures for flush queue
  iommu/amd: Remove align-parameter from __map_single()
  iommu/amd: Remove other remains of old address allocator
  iommu/amd: Make use of the generic IOVA allocator
  iommu/amd: Remove special mapping code for dma_ops path
  iommu/amd: Pass gfp-flags to iommu_map_page()
  iommu/amd: Implement apply_dm_region call-back
  iommu/amd: Create a list of reserved iova addresses
  ...
parents 77d9ada2 f360d324
* ARM SMMUv3 Architecture Implementation * ARM SMMUv3 Architecture Implementation
The SMMUv3 architecture is a significant deparature from previous The SMMUv3 architecture is a significant departure from previous
revisions, replacing the MMIO register interface with in-memory command revisions, replacing the MMIO register interface with in-memory command
and event queues and adding support for the ATS and PRI components of and event queues and adding support for the ATS and PRI components of
the PCIe specification. the PCIe specification.
......
* Mediatek IOMMU Architecture Implementation * Mediatek IOMMU Architecture Implementation
Some Mediatek SOCs contain a Multimedia Memory Management Unit (M4U) which Some Mediatek SOCs contain a Multimedia Memory Management Unit (M4U), and
uses the ARM Short-Descriptor translation table format for address translation. this M4U have two generations of HW architecture. Generation one uses flat
pagetable, and only supports 4K size page mapping. Generation two uses the
ARM Short-Descriptor translation table format for address translation.
About the M4U Hardware Block Diagram, please check below: About the M4U Hardware Block Diagram, please check below:
...@@ -36,7 +38,9 @@ in each larb. Take a example, There are many ports like MC, PP, VLD in the ...@@ -36,7 +38,9 @@ in each larb. Take a example, There are many ports like MC, PP, VLD in the
video decode local arbiter, all these ports are according to the video HW. video decode local arbiter, all these ports are according to the video HW.
Required properties: Required properties:
- compatible : must be "mediatek,mt8173-m4u". - compatible : must be one of the following string:
"mediatek,mt2701-m4u" for mt2701 which uses generation one m4u HW.
"mediatek,mt8173-m4u" for mt8173 which uses generation two m4u HW.
- reg : m4u register base and size. - reg : m4u register base and size.
- interrupts : the interrupt of m4u. - interrupts : the interrupt of m4u.
- clocks : must contain one entry for each clock-names. - clocks : must contain one entry for each clock-names.
...@@ -46,7 +50,8 @@ Required properties: ...@@ -46,7 +50,8 @@ Required properties:
according to the local arbiter index, like larb0, larb1, larb2... according to the local arbiter index, like larb0, larb1, larb2...
- iommu-cells : must be 1. This is the mtk_m4u_id according to the HW. - iommu-cells : must be 1. This is the mtk_m4u_id according to the HW.
Specifies the mtk_m4u_id as defined in Specifies the mtk_m4u_id as defined in
dt-binding/memory/mt8173-larb-port.h. dt-binding/memory/mt2701-larb-port.h for mt2701 and
dt-binding/memory/mt8173-larb-port.h for mt8173
Example: Example:
iommu: iommu@10205000 { iommu: iommu@10205000 {
......
* QCOM IOMMU
The MSM IOMMU is an implementation compatible with the ARM VMSA short
descriptor page tables. It provides address translation for bus masters outside
of the CPU, each connected to the IOMMU through a port called micro-TLB.
Required Properties:
- compatible: Must contain "qcom,apq8064-iommu".
- reg: Base address and size of the IOMMU registers.
- interrupts: Specifiers for the MMU fault interrupts. For instances that
support secure mode two interrupts must be specified, for non-secure and
secure mode, in that order. For instances that don't support secure mode a
single interrupt must be specified.
- #iommu-cells: The number of cells needed to specify the stream id. This
is always 1.
- qcom,ncb: The total number of context banks in the IOMMU.
- clocks : List of clocks to be used during SMMU register access. See
Documentation/devicetree/bindings/clock/clock-bindings.txt
for information about the format. For each clock specified
here, there must be a corresponding entry in clock-names
(see below).
- clock-names : List of clock names corresponding to the clocks specified in
the "clocks" property (above).
Should be "smmu_pclk" for specifying the interface clock
required for iommu's register accesses.
Should be "smmu_clk" for specifying the functional clock
required by iommu for bus accesses.
Each bus master connected to an IOMMU must reference the IOMMU in its device
node with the following property:
- iommus: A reference to the IOMMU in multiple cells. The first cell is a
phandle to the IOMMU and the second cell is the stream id.
A single master device can be connected to more than one iommu
and multiple contexts in each of the iommu. So multiple entries
are required to list all the iommus and the stream ids that the
master is connected to.
Example: mdp iommu and its bus master
mdp_port0: iommu@7500000 {
compatible = "qcom,apq8064-iommu";
#iommu-cells = <1>;
clock-names =
"smmu_pclk",
"smmu_clk";
clocks =
<&mmcc SMMU_AHB_CLK>,
<&mmcc MDP_AXI_CLK>;
reg = <0x07500000 0x100000>;
interrupts =
<GIC_SPI 63 0>,
<GIC_SPI 64 0>;
qcom,ncb = <2>;
};
mdp: qcom,mdp@5100000 {
compatible = "qcom,mdp";
...
iommus = <&mdp_port0 0
&mdp_port0 2>;
};
...@@ -2,16 +2,31 @@ SMI (Smart Multimedia Interface) Common ...@@ -2,16 +2,31 @@ SMI (Smart Multimedia Interface) Common
The hardware block diagram please check bindings/iommu/mediatek,iommu.txt The hardware block diagram please check bindings/iommu/mediatek,iommu.txt
Mediatek SMI have two generations of HW architecture, mt8173 uses the second
generation of SMI HW while mt2701 uses the first generation HW of SMI.
There's slight differences between the two SMI, for generation 2, the
register which control the iommu port is at each larb's register base. But
for generation 1, the register is at smi ao base(smi always on register
base). Besides that, the smi async clock should be prepared and enabled for
SMI generation 1 to transform the smi clock into emi clock domain, but that is
not needed for SMI generation 2.
Required properties: Required properties:
- compatible : must be "mediatek,mt8173-smi-common" - compatible : must be one of :
"mediatek,mt2701-smi-common"
"mediatek,mt8173-smi-common"
- reg : the register and size of the SMI block. - reg : the register and size of the SMI block.
- power-domains : a phandle to the power domain of this local arbiter. - power-domains : a phandle to the power domain of this local arbiter.
- clocks : Must contain an entry for each entry in clock-names. - clocks : Must contain an entry for each entry in clock-names.
- clock-names : must contain 2 entries, as follows: - clock-names : must contain 3 entries for generation 1 smi HW and 2 entries
for generation 2 smi HW as follows:
- "apb" : Advanced Peripheral Bus clock, It's the clock for setting - "apb" : Advanced Peripheral Bus clock, It's the clock for setting
the register. the register.
- "smi" : It's the clock for transfer data and command. - "smi" : It's the clock for transfer data and command.
They may be the same if both source clocks are the same. They may be the same if both source clocks are the same.
- "async" : asynchronous clock, it help transform the smi clock into the emi
clock domain, this clock is only needed by generation 1 smi HW.
Example: Example:
smi_common: smi@14022000 { smi_common: smi@14022000 {
......
...@@ -3,7 +3,9 @@ SMI (Smart Multimedia Interface) Local Arbiter ...@@ -3,7 +3,9 @@ SMI (Smart Multimedia Interface) Local Arbiter
The hardware block diagram please check bindings/iommu/mediatek,iommu.txt The hardware block diagram please check bindings/iommu/mediatek,iommu.txt
Required properties: Required properties:
- compatible : must be "mediatek,mt8173-smi-larb" - compatible : must be one of :
"mediatek,mt8173-smi-larb"
"mediatek,mt2701-smi-larb"
- reg : the register and size of this local arbiter. - reg : the register and size of this local arbiter.
- mediatek,smi : a phandle to the smi_common node. - mediatek,smi : a phandle to the smi_common node.
- power-domains : a phandle to the power domain of this local arbiter. - power-domains : a phandle to the power domain of this local arbiter.
......
...@@ -6223,6 +6223,7 @@ M: Joerg Roedel <joro@8bytes.org> ...@@ -6223,6 +6223,7 @@ M: Joerg Roedel <joro@8bytes.org>
L: iommu@lists.linux-foundation.org L: iommu@lists.linux-foundation.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git T: git git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git
S: Maintained S: Maintained
F: Documentation/devicetree/bindings/iommu/
F: drivers/iommu/ F: drivers/iommu/
IP MASQUERADING IP MASQUERADING
......
...@@ -89,8 +89,8 @@ config MSM_IOMMU ...@@ -89,8 +89,8 @@ config MSM_IOMMU
bool "MSM IOMMU Support" bool "MSM IOMMU Support"
depends on ARM depends on ARM
depends on ARCH_MSM8X60 || ARCH_MSM8960 || COMPILE_TEST depends on ARCH_MSM8X60 || ARCH_MSM8960 || COMPILE_TEST
depends on BROKEN
select IOMMU_API select IOMMU_API
select IOMMU_IO_PGTABLE_ARMV7S
help help
Support for the IOMMUs found on certain Qualcomm SOCs. Support for the IOMMUs found on certain Qualcomm SOCs.
These IOMMUs allow virtualization of the address space used by most These IOMMUs allow virtualization of the address space used by most
...@@ -111,6 +111,7 @@ config AMD_IOMMU ...@@ -111,6 +111,7 @@ config AMD_IOMMU
select PCI_PRI select PCI_PRI
select PCI_PASID select PCI_PASID
select IOMMU_API select IOMMU_API
select IOMMU_IOVA
depends on X86_64 && PCI && ACPI depends on X86_64 && PCI && ACPI
---help--- ---help---
With this option you can enable support for AMD IOMMU hardware in With this option you can enable support for AMD IOMMU hardware in
...@@ -343,4 +344,22 @@ config MTK_IOMMU ...@@ -343,4 +344,22 @@ config MTK_IOMMU
If unsure, say N here. If unsure, say N here.
config MTK_IOMMU_V1
bool "MTK IOMMU Version 1 (M4U gen1) Support"
depends on ARM
depends on ARCH_MEDIATEK || COMPILE_TEST
select ARM_DMA_USE_IOMMU
select IOMMU_API
select MEMORY
select MTK_SMI
select COMMON_CLK_MT2701_MMSYS
select COMMON_CLK_MT2701_IMGSYS
select COMMON_CLK_MT2701_VDECSYS
help
Support for the M4U on certain Mediatek SoCs. M4U generation 1 HW is
Multimedia Memory Managememt Unit. This option enables remapping of
DMA memory accesses for the multimedia subsystem.
if unsure, say N here.
endif # IOMMU_SUPPORT endif # IOMMU_SUPPORT
...@@ -7,7 +7,7 @@ obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o ...@@ -7,7 +7,7 @@ obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o
obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o
obj-$(CONFIG_IOMMU_IOVA) += iova.o obj-$(CONFIG_IOMMU_IOVA) += iova.o
obj-$(CONFIG_OF_IOMMU) += of_iommu.o obj-$(CONFIG_OF_IOMMU) += of_iommu.o
obj-$(CONFIG_MSM_IOMMU) += msm_iommu.o msm_iommu_dev.o obj-$(CONFIG_MSM_IOMMU) += msm_iommu.o
obj-$(CONFIG_AMD_IOMMU) += amd_iommu.o amd_iommu_init.o obj-$(CONFIG_AMD_IOMMU) += amd_iommu.o amd_iommu_init.o
obj-$(CONFIG_AMD_IOMMU_V2) += amd_iommu_v2.o obj-$(CONFIG_AMD_IOMMU_V2) += amd_iommu_v2.o
obj-$(CONFIG_ARM_SMMU) += arm-smmu.o obj-$(CONFIG_ARM_SMMU) += arm-smmu.o
...@@ -18,6 +18,7 @@ obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o ...@@ -18,6 +18,7 @@ obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o
obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
obj-$(CONFIG_IRQ_REMAP) += intel_irq_remapping.o irq_remapping.o obj-$(CONFIG_IRQ_REMAP) += intel_irq_remapping.o irq_remapping.o
obj-$(CONFIG_MTK_IOMMU) += mtk_iommu.o obj-$(CONFIG_MTK_IOMMU) += mtk_iommu.o
obj-$(CONFIG_MTK_IOMMU_V1) += mtk_iommu_v1.o
obj-$(CONFIG_OMAP_IOMMU) += omap-iommu.o obj-$(CONFIG_OMAP_IOMMU) += omap-iommu.o
obj-$(CONFIG_OMAP_IOMMU_DEBUG) += omap-iommu-debug.o obj-$(CONFIG_OMAP_IOMMU_DEBUG) += omap-iommu-debug.o
obj-$(CONFIG_ROCKCHIP_IOMMU) += rockchip-iommu.o obj-$(CONFIG_ROCKCHIP_IOMMU) += rockchip-iommu.o
......
This diff is collapsed.
...@@ -421,7 +421,6 @@ struct protection_domain { ...@@ -421,7 +421,6 @@ struct protection_domain {
bool updated; /* complete domain flush required */ bool updated; /* complete domain flush required */
unsigned dev_cnt; /* devices assigned to this domain */ unsigned dev_cnt; /* devices assigned to this domain */
unsigned dev_iommu[MAX_IOMMUS]; /* per-IOMMU reference count */ unsigned dev_iommu[MAX_IOMMUS]; /* per-IOMMU reference count */
void *priv; /* private data */
}; };
/* /*
......
...@@ -960,7 +960,7 @@ static int __init amd_iommu_v2_init(void) ...@@ -960,7 +960,7 @@ static int __init amd_iommu_v2_init(void)
spin_lock_init(&state_lock); spin_lock_init(&state_lock);
ret = -ENOMEM; ret = -ENOMEM;
iommu_wq = create_workqueue("amd_iommu_v2"); iommu_wq = alloc_workqueue("amd_iommu_v2", WQ_MEM_RECLAIM, 0);
if (iommu_wq == NULL) if (iommu_wq == NULL)
goto out; goto out;
......
...@@ -2687,6 +2687,8 @@ static int __init arm_smmu_init(void) ...@@ -2687,6 +2687,8 @@ static int __init arm_smmu_init(void)
if (ret) if (ret)
return ret; return ret;
pci_request_acs();
return bus_set_iommu(&pci_bus_type, &arm_smmu_ops); return bus_set_iommu(&pci_bus_type, &arm_smmu_ops);
} }
......
...@@ -987,8 +987,8 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, ...@@ -987,8 +987,8 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,
* handler seeing a half-initialised domain state. * handler seeing a half-initialised domain state.
*/ */
irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx];
ret = request_irq(irq, arm_smmu_context_fault, IRQF_SHARED, ret = devm_request_irq(smmu->dev, irq, arm_smmu_context_fault,
"arm-smmu-context-fault", domain); IRQF_SHARED, "arm-smmu-context-fault", domain);
if (ret < 0) { if (ret < 0) {
dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n", dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n",
cfg->irptndx, irq); cfg->irptndx, irq);
...@@ -1028,7 +1028,7 @@ static void arm_smmu_destroy_domain_context(struct iommu_domain *domain) ...@@ -1028,7 +1028,7 @@ static void arm_smmu_destroy_domain_context(struct iommu_domain *domain)
if (cfg->irptndx != INVALID_IRPTNDX) { if (cfg->irptndx != INVALID_IRPTNDX) {
irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx];
free_irq(irq, domain); devm_free_irq(smmu->dev, irq, domain);
} }
free_io_pgtable_ops(smmu_domain->pgtbl_ops); free_io_pgtable_ops(smmu_domain->pgtbl_ops);
...@@ -1986,15 +1986,15 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev) ...@@ -1986,15 +1986,15 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
} }
for (i = 0; i < smmu->num_global_irqs; ++i) { for (i = 0; i < smmu->num_global_irqs; ++i) {
err = request_irq(smmu->irqs[i], err = devm_request_irq(smmu->dev, smmu->irqs[i],
arm_smmu_global_fault, arm_smmu_global_fault,
IRQF_SHARED, IRQF_SHARED,
"arm-smmu global fault", "arm-smmu global fault",
smmu); smmu);
if (err) { if (err) {
dev_err(dev, "failed to request global IRQ %d (%u)\n", dev_err(dev, "failed to request global IRQ %d (%u)\n",
i, smmu->irqs[i]); i, smmu->irqs[i]);
goto out_free_irqs; goto out_put_masters;
} }
} }
...@@ -2006,10 +2006,6 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev) ...@@ -2006,10 +2006,6 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
arm_smmu_device_reset(smmu); arm_smmu_device_reset(smmu);
return 0; return 0;
out_free_irqs:
while (i--)
free_irq(smmu->irqs[i], smmu);
out_put_masters: out_put_masters:
for (node = rb_first(&smmu->masters); node; node = rb_next(node)) { for (node = rb_first(&smmu->masters); node; node = rb_next(node)) {
struct arm_smmu_master *master struct arm_smmu_master *master
...@@ -2050,7 +2046,7 @@ static int arm_smmu_device_remove(struct platform_device *pdev) ...@@ -2050,7 +2046,7 @@ static int arm_smmu_device_remove(struct platform_device *pdev)
dev_err(dev, "removing device with active domains!\n"); dev_err(dev, "removing device with active domains!\n");
for (i = 0; i < smmu->num_global_irqs; ++i) for (i = 0; i < smmu->num_global_irqs; ++i)
free_irq(smmu->irqs[i], smmu); devm_free_irq(smmu->dev, smmu->irqs[i], smmu);
/* Turn the thing off */ /* Turn the thing off */
writel(sCR0_CLIENTPD, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0); writel(sCR0_CLIENTPD, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0);
...@@ -2096,8 +2092,10 @@ static int __init arm_smmu_init(void) ...@@ -2096,8 +2092,10 @@ static int __init arm_smmu_init(void)
#endif #endif
#ifdef CONFIG_PCI #ifdef CONFIG_PCI
if (!iommu_present(&pci_bus_type)) if (!iommu_present(&pci_bus_type)) {
pci_request_acs();
bus_set_iommu(&pci_bus_type, &arm_smmu_ops); bus_set_iommu(&pci_bus_type, &arm_smmu_ops);
}
#endif #endif
return 0; return 0;
......
...@@ -241,8 +241,20 @@ int dmar_insert_dev_scope(struct dmar_pci_notify_info *info, ...@@ -241,8 +241,20 @@ int dmar_insert_dev_scope(struct dmar_pci_notify_info *info,
if (!dmar_match_pci_path(info, scope->bus, path, level)) if (!dmar_match_pci_path(info, scope->bus, path, level))
continue; continue;
if ((scope->entry_type == ACPI_DMAR_SCOPE_TYPE_ENDPOINT) ^ /*
(info->dev->hdr_type == PCI_HEADER_TYPE_NORMAL)) { * We expect devices with endpoint scope to have normal PCI
* headers, and devices with bridge scope to have bridge PCI
* headers. However PCI NTB devices may be listed in the
* DMAR table with bridge scope, even though they have a
* normal PCI header. NTB devices are identified by class
* "BRIDGE_OTHER" (0680h) - we don't declare a socpe mismatch
* for this special case.
*/
if ((scope->entry_type == ACPI_DMAR_SCOPE_TYPE_ENDPOINT &&
info->dev->hdr_type != PCI_HEADER_TYPE_NORMAL) ||
(scope->entry_type == ACPI_DMAR_SCOPE_TYPE_BRIDGE &&
(info->dev->hdr_type == PCI_HEADER_TYPE_NORMAL &&
info->dev->class >> 8 != PCI_CLASS_BRIDGE_OTHER))) {
pr_warn("Device scope type does not match for %s\n", pr_warn("Device scope type does not match for %s\n",
pci_name(info->dev)); pci_name(info->dev));
return -EINVAL; return -EINVAL;
...@@ -1155,8 +1167,6 @@ static int qi_check_fault(struct intel_iommu *iommu, int index) ...@@ -1155,8 +1167,6 @@ static int qi_check_fault(struct intel_iommu *iommu, int index)
(unsigned long long)qi->desc[index].high); (unsigned long long)qi->desc[index].high);
memcpy(&qi->desc[index], &qi->desc[wait_index], memcpy(&qi->desc[index], &qi->desc[wait_index],
sizeof(struct qi_desc)); sizeof(struct qi_desc));
__iommu_flush_cache(iommu, &qi->desc[index],
sizeof(struct qi_desc));
writel(DMA_FSTS_IQE, iommu->reg + DMAR_FSTS_REG); writel(DMA_FSTS_IQE, iommu->reg + DMAR_FSTS_REG);
return -EINVAL; return -EINVAL;
} }
...@@ -1231,9 +1241,6 @@ int qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu) ...@@ -1231,9 +1241,6 @@ int qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu)
hw[wait_index] = wait_desc; hw[wait_index] = wait_desc;
__iommu_flush_cache(iommu, &hw[index], sizeof(struct qi_desc));
__iommu_flush_cache(iommu, &hw[wait_index], sizeof(struct qi_desc));
qi->free_head = (qi->free_head + 2) % QI_LENGTH; qi->free_head = (qi->free_head + 2) % QI_LENGTH;
qi->free_cnt -= 2; qi->free_cnt -= 2;
......
...@@ -54,6 +54,10 @@ typedef u32 sysmmu_pte_t; ...@@ -54,6 +54,10 @@ typedef u32 sysmmu_pte_t;
#define lv2ent_small(pent) ((*(pent) & 2) == 2) #define lv2ent_small(pent) ((*(pent) & 2) == 2)
#define lv2ent_large(pent) ((*(pent) & 3) == 1) #define lv2ent_large(pent) ((*(pent) & 3) == 1)
#ifdef CONFIG_BIG_ENDIAN
#warning "revisit driver if we can enable big-endian ptes"
#endif
/* /*
* v1.x - v3.x SYSMMU supports 32bit physical and 32bit virtual address spaces * v1.x - v3.x SYSMMU supports 32bit physical and 32bit virtual address spaces
* v5.0 introduced support for 36bit physical address space by shifting * v5.0 introduced support for 36bit physical address space by shifting
...@@ -322,14 +326,27 @@ static void __sysmmu_set_ptbase(struct sysmmu_drvdata *data, phys_addr_t pgd) ...@@ -322,14 +326,27 @@ static void __sysmmu_set_ptbase(struct sysmmu_drvdata *data, phys_addr_t pgd)
__sysmmu_tlb_invalidate(data); __sysmmu_tlb_invalidate(data);
} }
static void __sysmmu_enable_clocks(struct sysmmu_drvdata *data)
{
BUG_ON(clk_prepare_enable(data->clk_master));
BUG_ON(clk_prepare_enable(data->clk));
BUG_ON(clk_prepare_enable(data->pclk));
BUG_ON(clk_prepare_enable(data->aclk));
}
static void __sysmmu_disable_clocks(struct sysmmu_drvdata *data)
{
clk_disable_unprepare(data->aclk);
clk_disable_unprepare(data->pclk);
clk_disable_unprepare(data->clk);
clk_disable_unprepare(data->clk_master);
}
static void __sysmmu_get_version(struct sysmmu_drvdata *data) static void __sysmmu_get_version(struct sysmmu_drvdata *data)
{ {
u32 ver; u32 ver;
clk_enable(data->clk_master); __sysmmu_enable_clocks(data);
clk_enable(data->clk);
clk_enable(data->pclk);
clk_enable(data->aclk);
ver = readl(data->sfrbase + REG_MMU_VERSION); ver = readl(data->sfrbase + REG_MMU_VERSION);
...@@ -342,10 +359,7 @@ static void __sysmmu_get_version(struct sysmmu_drvdata *data) ...@@ -342,10 +359,7 @@ static void __sysmmu_get_version(struct sysmmu_drvdata *data)
dev_dbg(data->sysmmu, "hardware version: %d.%d\n", dev_dbg(data->sysmmu, "hardware version: %d.%d\n",
MMU_MAJ_VER(data->version), MMU_MIN_VER(data->version)); MMU_MAJ_VER(data->version), MMU_MIN_VER(data->version));
clk_disable(data->aclk); __sysmmu_disable_clocks(data);
clk_disable(data->pclk);
clk_disable(data->clk);
clk_disable(data->clk_master);
} }
static void show_fault_information(struct sysmmu_drvdata *data, static void show_fault_information(struct sysmmu_drvdata *data,
...@@ -427,10 +441,7 @@ static void __sysmmu_disable_nocount(struct sysmmu_drvdata *data) ...@@ -427,10 +441,7 @@ static void __sysmmu_disable_nocount(struct sysmmu_drvdata *data)
writel(CTRL_DISABLE, data->sfrbase + REG_MMU_CTRL); writel(CTRL_DISABLE, data->sfrbase + REG_MMU_CTRL);
writel(0, data->sfrbase + REG_MMU_CFG); writel(0, data->sfrbase + REG_MMU_CFG);
clk_disable(data->aclk); __sysmmu_disable_clocks(data);
clk_disable(data->pclk);
clk_disable(data->clk);
clk_disable(data->clk_master);
} }
static bool __sysmmu_disable(struct sysmmu_drvdata *data) static bool __sysmmu_disable(struct sysmmu_drvdata *data)
...@@ -475,10 +486,7 @@ static void __sysmmu_init_config(struct sysmmu_drvdata *data) ...@@ -475,10 +486,7 @@ static void __sysmmu_init_config(struct sysmmu_drvdata *data)
static void __sysmmu_enable_nocount(struct sysmmu_drvdata *data) static void __sysmmu_enable_nocount(struct sysmmu_drvdata *data)
{ {
clk_enable(data->clk_master); __sysmmu_enable_clocks(data);
clk_enable(data->clk);
clk_enable(data->pclk);
clk_enable(data->aclk);
writel(CTRL_BLOCK, data->sfrbase + REG_MMU_CTRL); writel(CTRL_BLOCK, data->sfrbase + REG_MMU_CTRL);
...@@ -488,6 +496,12 @@ static void __sysmmu_enable_nocount(struct sysmmu_drvdata *data) ...@@ -488,6 +496,12 @@ static void __sysmmu_enable_nocount(struct sysmmu_drvdata *data)
writel(CTRL_ENABLE, data->sfrbase + REG_MMU_CTRL); writel(CTRL_ENABLE, data->sfrbase + REG_MMU_CTRL);
/*
* SYSMMU driver keeps master's clock enabled only for the short
* time, while accessing the registers. For performing address
* translation during DMA transaction it relies on the client
* driver to enable it.
*/
clk_disable(data->clk_master); clk_disable(data->clk_master);
} }
...@@ -524,16 +538,15 @@ static void sysmmu_tlb_invalidate_flpdcache(struct sysmmu_drvdata *data, ...@@ -524,16 +538,15 @@ static void sysmmu_tlb_invalidate_flpdcache(struct sysmmu_drvdata *data,
{ {
unsigned long flags; unsigned long flags;
clk_enable(data->clk_master);
spin_lock_irqsave(&data->lock, flags); spin_lock_irqsave(&data->lock, flags);
if (is_sysmmu_active(data)) { if (is_sysmmu_active(data) && data->version >= MAKE_MMU_VER(3, 3)) {
if (data->version >= MAKE_MMU_VER(3, 3)) clk_enable(data->clk_master);
__sysmmu_tlb_invalidate_entry(data, iova, 1); __sysmmu_tlb_invalidate_entry(data, iova, 1);
clk_disable(data->clk_master);
} }
spin_unlock_irqrestore(&data->lock, flags); spin_unlock_irqrestore(&data->lock, flags);
clk_disable(data->clk_master);
} }
static void sysmmu_tlb_invalidate_entry(struct sysmmu_drvdata *data, static void sysmmu_tlb_invalidate_entry(struct sysmmu_drvdata *data,
...@@ -572,6 +585,8 @@ static void sysmmu_tlb_invalidate_entry(struct sysmmu_drvdata *data, ...@@ -572,6 +585,8 @@ static void sysmmu_tlb_invalidate_entry(struct sysmmu_drvdata *data,
spin_unlock_irqrestore(&data->lock, flags); spin_unlock_irqrestore(&data->lock, flags);
} }
static struct iommu_ops exynos_iommu_ops;
static int __init exynos_sysmmu_probe(struct platform_device *pdev) static int __init exynos_sysmmu_probe(struct platform_device *pdev)
{ {
int irq, ret; int irq, ret;
...@@ -602,37 +617,22 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev) ...@@ -602,37 +617,22 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
} }
data->clk = devm_clk_get(dev, "sysmmu"); data->clk = devm_clk_get(dev, "sysmmu");
if (!IS_ERR(data->clk)) { if (PTR_ERR(data->clk) == -ENOENT)
ret = clk_prepare(data->clk);
if (ret) {
dev_err(dev, "Failed to prepare clk\n");
return ret;
}
} else {
data->clk = NULL; data->clk = NULL;
} else if (IS_ERR(data->clk))
return PTR_ERR(data->clk);
data->aclk = devm_clk_get(dev, "aclk"); data->aclk = devm_clk_get(dev, "aclk");
if (!IS_ERR(data->aclk)) { if (PTR_ERR(data->aclk) == -ENOENT)
ret = clk_prepare(data->aclk);
if (ret) {
dev_err(dev, "Failed to prepare aclk\n");
return ret;
}
} else {
data->aclk = NULL; data->aclk = NULL;
} else if (IS_ERR(data->aclk))
return PTR_ERR(data->aclk);
data->pclk = devm_clk_get(dev, "pclk"); data->pclk = devm_clk_get(dev, "pclk");
if (!IS_ERR(data->pclk)) { if (PTR_ERR(data->pclk) == -ENOENT)
ret = clk_prepare(data->pclk);
if (ret) {
dev_err(dev, "Failed to prepare pclk\n");
return ret;
}
} else {
data->pclk = NULL; data->pclk = NULL;
} else if (IS_ERR(data->pclk))
return PTR_ERR(data->pclk);
if (!data->clk && (!data->aclk || !data->pclk)) { if (!data->clk && (!data->aclk || !data->pclk)) {
dev_err(dev, "Failed to get device clock(s)!\n"); dev_err(dev, "Failed to get device clock(s)!\n");
...@@ -640,15 +640,10 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev) ...@@ -640,15 +640,10 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
} }
data->clk_master = devm_clk_get(dev, "master"); data->clk_master = devm_clk_get(dev, "master");
if (!IS_ERR(data->clk_master)) { if (PTR_ERR(data->clk_master) == -ENOENT)
ret = clk_prepare(data->clk_master);
if (ret) {
dev_err(dev, "Failed to prepare master's clk\n");
return ret;
}
} else {
data->clk_master = NULL; data->clk_master = NULL;
} else if (IS_ERR(data->clk_master))
return PTR_ERR(data->clk_master);
data->sysmmu = dev; data->sysmmu = dev;
spin_lock_init(&data->lock); spin_lock_init(&data->lock);
...@@ -665,6 +660,8 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev) ...@@ -665,6 +660,8 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
pm_runtime_enable(dev); pm_runtime_enable(dev);
of_iommu_set_ops(dev->of_node, &exynos_iommu_ops);
return 0; return 0;
} }
...@@ -709,6 +706,7 @@ static struct platform_driver exynos_sysmmu_driver __refdata = { ...@@ -709,6 +706,7 @@ static struct platform_driver exynos_sysmmu_driver __refdata = {
.name = "exynos-sysmmu", .name = "exynos-sysmmu",
.of_match_table = sysmmu_of_match, .of_match_table = sysmmu_of_match,
.pm = &sysmmu_pm_ops, .pm = &sysmmu_pm_ops,
.suppress_bind_attrs = true,
} }
}; };
...@@ -716,7 +714,7 @@ static inline void update_pte(sysmmu_pte_t *ent, sysmmu_pte_t val) ...@@ -716,7 +714,7 @@ static inline void update_pte(sysmmu_pte_t *ent, sysmmu_pte_t val)
{ {
dma_sync_single_for_cpu(dma_dev, virt_to_phys(ent), sizeof(*ent), dma_sync_single_for_cpu(dma_dev, virt_to_phys(ent), sizeof(*ent),
DMA_TO_DEVICE); DMA_TO_DEVICE);
*ent = val; *ent = cpu_to_le32(val);
dma_sync_single_for_device(dma_dev, virt_to_phys(ent), sizeof(*ent), dma_sync_single_for_device(dma_dev, virt_to_phys(ent), sizeof(*ent),
DMA_TO_DEVICE); DMA_TO_DEVICE);
} }
...@@ -1357,7 +1355,6 @@ static int __init exynos_iommu_of_setup(struct device_node *np) ...@@ -1357,7 +1355,6 @@ static int __init exynos_iommu_of_setup(struct device_node *np)
if (!dma_dev) if (!dma_dev)
dma_dev = &pdev->dev; dma_dev = &pdev->dev;
of_iommu_set_ops(np, &exynos_iommu_ops);
return 0; return 0;
} }
......
...@@ -1672,7 +1672,7 @@ static int iommu_init_domains(struct intel_iommu *iommu) ...@@ -1672,7 +1672,7 @@ static int iommu_init_domains(struct intel_iommu *iommu)
return -ENOMEM; return -ENOMEM;
} }
size = ((ndomains >> 8) + 1) * sizeof(struct dmar_domain **); size = (ALIGN(ndomains, 256) >> 8) * sizeof(struct dmar_domain **);
iommu->domains = kzalloc(size, GFP_KERNEL); iommu->domains = kzalloc(size, GFP_KERNEL);
if (iommu->domains) { if (iommu->domains) {
...@@ -1737,7 +1737,7 @@ static void disable_dmar_iommu(struct intel_iommu *iommu) ...@@ -1737,7 +1737,7 @@ static void disable_dmar_iommu(struct intel_iommu *iommu)
static void free_dmar_iommu(struct intel_iommu *iommu) static void free_dmar_iommu(struct intel_iommu *iommu)
{ {
if ((iommu->domains) && (iommu->domain_ids)) { if ((iommu->domains) && (iommu->domain_ids)) {
int elems = (cap_ndoms(iommu->cap) >> 8) + 1; int elems = ALIGN(cap_ndoms(iommu->cap), 256) >> 8;
int i; int i;
for (i = 0; i < elems; i++) for (i = 0; i < elems; i++)
...@@ -2076,7 +2076,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain, ...@@ -2076,7 +2076,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
spin_unlock(&iommu->lock); spin_unlock(&iommu->lock);
spin_unlock_irqrestore(&device_domain_lock, flags); spin_unlock_irqrestore(&device_domain_lock, flags);
return 0; return ret;
} }
struct domain_context_mapping_data { struct domain_context_mapping_data {
......
...@@ -576,7 +576,7 @@ static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops, ...@@ -576,7 +576,7 @@ static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops,
return 0; return 0;
found_translation: found_translation:
iova &= (ARM_LPAE_GRANULE(data) - 1); iova &= (ARM_LPAE_BLOCK_SIZE(lvl, data) - 1);
return ((phys_addr_t)iopte_to_pfn(pte,data) << data->pg_shift) | iova; return ((phys_addr_t)iopte_to_pfn(pte,data) << data->pg_shift) | iova;
} }
......
...@@ -34,8 +34,7 @@ ...@@ -34,8 +34,7 @@
#include <trace/events/iommu.h> #include <trace/events/iommu.h>
static struct kset *iommu_group_kset; static struct kset *iommu_group_kset;
static struct ida iommu_group_ida; static DEFINE_IDA(iommu_group_ida);
static struct mutex iommu_group_mutex;
struct iommu_callback_data { struct iommu_callback_data {
const struct iommu_ops *ops; const struct iommu_ops *ops;
...@@ -144,9 +143,7 @@ static void iommu_group_release(struct kobject *kobj) ...@@ -144,9 +143,7 @@ static void iommu_group_release(struct kobject *kobj)
if (group->iommu_data_release) if (group->iommu_data_release)
group->iommu_data_release(group->iommu_data); group->iommu_data_release(group->iommu_data);
mutex_lock(&iommu_group_mutex); ida_simple_remove(&iommu_group_ida, group->id);
ida_remove(&iommu_group_ida, group->id);
mutex_unlock(&iommu_group_mutex);
if (group->default_domain) if (group->default_domain)
iommu_domain_free(group->default_domain); iommu_domain_free(group->default_domain);
...@@ -186,26 +183,17 @@ struct iommu_group *iommu_group_alloc(void) ...@@ -186,26 +183,17 @@ struct iommu_group *iommu_group_alloc(void)
INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->devices);
BLOCKING_INIT_NOTIFIER_HEAD(&group->notifier); BLOCKING_INIT_NOTIFIER_HEAD(&group->notifier);
mutex_lock(&iommu_group_mutex); ret = ida_simple_get(&iommu_group_ida, 0, 0, GFP_KERNEL);
if (ret < 0) {
again:
if (unlikely(0 == ida_pre_get(&iommu_group_ida, GFP_KERNEL))) {
kfree(group); kfree(group);
mutex_unlock(&iommu_group_mutex); return ERR_PTR(ret);
return ERR_PTR(-ENOMEM);
} }
group->id = ret;
if (-EAGAIN == ida_get_new(&iommu_group_ida, &group->id))
goto again;
mutex_unlock(&iommu_group_mutex);
ret = kobject_init_and_add(&group->kobj, &iommu_group_ktype, ret = kobject_init_and_add(&group->kobj, &iommu_group_ktype,
NULL, "%d", group->id); NULL, "%d", group->id);
if (ret) { if (ret) {
mutex_lock(&iommu_group_mutex); ida_simple_remove(&iommu_group_ida, group->id);
ida_remove(&iommu_group_ida, group->id);
mutex_unlock(&iommu_group_mutex);
kfree(group); kfree(group);
return ERR_PTR(ret); return ERR_PTR(ret);
} }
...@@ -348,6 +336,9 @@ static int iommu_group_create_direct_mappings(struct iommu_group *group, ...@@ -348,6 +336,9 @@ static int iommu_group_create_direct_mappings(struct iommu_group *group,
list_for_each_entry(entry, &mappings, list) { list_for_each_entry(entry, &mappings, list) {
dma_addr_t start, end, addr; dma_addr_t start, end, addr;
if (domain->ops->apply_dm_region)
domain->ops->apply_dm_region(dev, domain, entry);
start = ALIGN(entry->start, pg_size); start = ALIGN(entry->start, pg_size);
end = ALIGN(entry->start + entry->length, pg_size); end = ALIGN(entry->start + entry->length, pg_size);
...@@ -1483,9 +1474,6 @@ static int __init iommu_init(void) ...@@ -1483,9 +1474,6 @@ static int __init iommu_init(void)
{ {
iommu_group_kset = kset_create_and_add("iommu_groups", iommu_group_kset = kset_create_and_add("iommu_groups",
NULL, kernel_kobj); NULL, kernel_kobj);
ida_init(&iommu_group_ida);
mutex_init(&iommu_group_mutex);
BUG_ON(!iommu_group_kset); BUG_ON(!iommu_group_kset);
return 0; return 0;
......
This diff is collapsed.
...@@ -42,74 +42,53 @@ ...@@ -42,74 +42,53 @@
*/ */
#define MAX_NUM_MIDS 32 #define MAX_NUM_MIDS 32
/* Maximum number of context banks that can be present in IOMMU */
#define IOMMU_MAX_CBS 128
/** /**
* struct msm_iommu_dev - a single IOMMU hardware instance * struct msm_iommu_dev - a single IOMMU hardware instance
* name Human-readable name given to this IOMMU HW instance
* ncb Number of context banks present on this IOMMU HW instance * ncb Number of context banks present on this IOMMU HW instance
* dev: IOMMU device
* irq: Interrupt number
* clk: The bus clock for this IOMMU hardware instance
* pclk: The clock for the IOMMU bus interconnect
* dev_node: list head in qcom_iommu_device_list
* dom_node: list head for domain
* ctx_list: list of 'struct msm_iommu_ctx_dev'
* context_map: Bitmap to track allocated context banks
*/ */
struct msm_iommu_dev { struct msm_iommu_dev {
const char *name; void __iomem *base;
int ncb; int ncb;
struct device *dev;
int irq;
struct clk *clk;
struct clk *pclk;
struct list_head dev_node;
struct list_head dom_node;
struct list_head ctx_list;
DECLARE_BITMAP(context_map, IOMMU_MAX_CBS);
}; };
/** /**
* struct msm_iommu_ctx_dev - an IOMMU context bank instance * struct msm_iommu_ctx_dev - an IOMMU context bank instance
* name Human-readable name given to this context bank * of_node node ptr of client device
* num Index of this context bank within the hardware * num Index of this context bank within the hardware
* mids List of Machine IDs that are to be mapped into this context * mids List of Machine IDs that are to be mapped into this context
* bank, terminated by -1. The MID is a set of signals on the * bank, terminated by -1. The MID is a set of signals on the
* AXI bus that identifies the function associated with a specific * AXI bus that identifies the function associated with a specific
* memory request. (See ARM spec). * memory request. (See ARM spec).
* num_mids Total number of mids
* node list head in ctx_list
*/ */
struct msm_iommu_ctx_dev { struct msm_iommu_ctx_dev {
const char *name; struct device_node *of_node;
int num; int num;
int mids[MAX_NUM_MIDS]; int mids[MAX_NUM_MIDS];
int num_mids;
struct list_head list;
}; };
/**
* struct msm_iommu_drvdata - A single IOMMU hardware instance
* @base: IOMMU config port base address (VA)
* @ncb The number of contexts on this IOMMU
* @irq: Interrupt number
* @clk: The bus clock for this IOMMU hardware instance
* @pclk: The clock for the IOMMU bus interconnect
*
* A msm_iommu_drvdata holds the global driver data about a single piece
* of an IOMMU hardware instance.
*/
struct msm_iommu_drvdata {
void __iomem *base;
int irq;
int ncb;
struct clk *clk;
struct clk *pclk;
};
/**
* struct msm_iommu_ctx_drvdata - an IOMMU context bank instance
* @num: Hardware context number of this context
* @pdev: Platform device associated wit this HW instance
* @attached_elm: List element for domains to track which devices are
* attached to them
*
* A msm_iommu_ctx_drvdata holds the driver data for a single context bank
* within each IOMMU hardware instance
*/
struct msm_iommu_ctx_drvdata {
int num;
struct platform_device *pdev;
struct list_head attached_elm;
};
/*
* Look up an IOMMU context device by its context name. NULL if none found.
* Useful for testing and drivers that do not yet fully have IOMMU stuff in
* their platform devices.
*/
struct device *msm_iommu_get_ctx(const char *ctx_name);
/* /*
* Interrupt handler for the IOMMU context fault interrupt. Hooking the * Interrupt handler for the IOMMU context fault interrupt. Hooking the
* interrupt is not supported in the API yet, but this will print an error * interrupt is not supported in the API yet, but this will print an error
......
/* Copyright (c) 2010-2011, Code Aurora Forum. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
* only version 2 as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
* 02110-1301, USA.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/iommu.h>
#include <linux/interrupt.h>
#include <linux/err.h>
#include <linux/slab.h>
#include "msm_iommu_hw-8xxx.h"
#include "msm_iommu.h"
struct iommu_ctx_iter_data {
/* input */
const char *name;
/* output */
struct device *dev;
};
static struct platform_device *msm_iommu_root_dev;
static int each_iommu_ctx(struct device *dev, void *data)
{
struct iommu_ctx_iter_data *res = data;
struct msm_iommu_ctx_dev *c = dev->platform_data;
if (!res || !c || !c->name || !res->name)
return -EINVAL;
if (!strcmp(res->name, c->name)) {
res->dev = dev;
return 1;
}
return 0;
}
static int each_iommu(struct device *dev, void *data)
{
return device_for_each_child(dev, data, each_iommu_ctx);
}
struct device *msm_iommu_get_ctx(const char *ctx_name)
{
struct iommu_ctx_iter_data r;
int found;
if (!msm_iommu_root_dev) {
pr_err("No root IOMMU device.\n");
goto fail;
}
r.name = ctx_name;
found = device_for_each_child(&msm_iommu_root_dev->dev, &r, each_iommu);
if (!found) {
pr_err("Could not find context <%s>\n", ctx_name);
goto fail;
}
return r.dev;
fail:
return NULL;
}
EXPORT_SYMBOL(msm_iommu_get_ctx);
static void msm_iommu_reset(void __iomem *base, int ncb)
{
int ctx;
SET_RPUE(base, 0);
SET_RPUEIE(base, 0);
SET_ESRRESTORE(base, 0);
SET_TBE(base, 0);
SET_CR(base, 0);
SET_SPDMBE(base, 0);
SET_TESTBUSCR(base, 0);
SET_TLBRSW(base, 0);
SET_GLOBAL_TLBIALL(base, 0);
SET_RPU_ACR(base, 0);
SET_TLBLKCRWE(base, 1);
for (ctx = 0; ctx < ncb; ctx++) {
SET_BPRCOSH(base, ctx, 0);
SET_BPRCISH(base, ctx, 0);
SET_BPRCNSH(base, ctx, 0);
SET_BPSHCFG(base, ctx, 0);
SET_BPMTCFG(base, ctx, 0);
SET_ACTLR(base, ctx, 0);
SET_SCTLR(base, ctx, 0);
SET_FSRRESTORE(base, ctx, 0);
SET_TTBR0(base, ctx, 0);
SET_TTBR1(base, ctx, 0);
SET_TTBCR(base, ctx, 0);
SET_BFBCR(base, ctx, 0);
SET_PAR(base, ctx, 0);
SET_FAR(base, ctx, 0);
SET_CTX_TLBIALL(base, ctx, 0);
SET_TLBFLPTER(base, ctx, 0);
SET_TLBSLPTER(base, ctx, 0);
SET_TLBLKCR(base, ctx, 0);
SET_PRRR(base, ctx, 0);
SET_NMRR(base, ctx, 0);
SET_CONTEXTIDR(base, ctx, 0);
}
}
static int msm_iommu_probe(struct platform_device *pdev)
{
struct resource *r;
struct clk *iommu_clk;
struct clk *iommu_pclk;
struct msm_iommu_drvdata *drvdata;
struct msm_iommu_dev *iommu_dev = dev_get_platdata(&pdev->dev);
void __iomem *regs_base;
int ret, irq, par;
if (pdev->id == -1) {
msm_iommu_root_dev = pdev;
return 0;
}
drvdata = kzalloc(sizeof(*drvdata), GFP_KERNEL);
if (!drvdata) {
ret = -ENOMEM;
goto fail;
}
if (!iommu_dev) {
ret = -ENODEV;
goto fail;
}
iommu_pclk = clk_get(NULL, "smmu_pclk");
if (IS_ERR(iommu_pclk)) {
ret = -ENODEV;
goto fail;
}
ret = clk_prepare_enable(iommu_pclk);
if (ret)
goto fail_enable;
iommu_clk = clk_get(&pdev->dev, "iommu_clk");
if (!IS_ERR(iommu_clk)) {
if (clk_get_rate(iommu_clk) == 0)
clk_set_rate(iommu_clk, 1);
ret = clk_prepare_enable(iommu_clk);
if (ret) {
clk_put(iommu_clk);
goto fail_pclk;
}
} else
iommu_clk = NULL;
r = platform_get_resource_byname(pdev, IORESOURCE_MEM, "physbase");
regs_base = devm_ioremap_resource(&pdev->dev, r);
if (IS_ERR(regs_base)) {
ret = PTR_ERR(regs_base);
goto fail_clk;
}
irq = platform_get_irq_byname(pdev, "secure_irq");
if (irq < 0) {
ret = -ENODEV;
goto fail_clk;
}
msm_iommu_reset(regs_base, iommu_dev->ncb);
SET_M(regs_base, 0, 1);
SET_PAR(regs_base, 0, 0);
SET_V2PCFG(regs_base, 0, 1);
SET_V2PPR(regs_base, 0, 0);
par = GET_PAR(regs_base, 0);
SET_V2PCFG(regs_base, 0, 0);
SET_M(regs_base, 0, 0);
if (!par) {
pr_err("%s: Invalid PAR value detected\n", iommu_dev->name);
ret = -ENODEV;
goto fail_clk;
}
ret = request_irq(irq, msm_iommu_fault_handler, 0,
"msm_iommu_secure_irpt_handler", drvdata);
if (ret) {
pr_err("Request IRQ %d failed with ret=%d\n", irq, ret);
goto fail_clk;
}
drvdata->pclk = iommu_pclk;
drvdata->clk = iommu_clk;
drvdata->base = regs_base;
drvdata->irq = irq;
drvdata->ncb = iommu_dev->ncb;
pr_info("device %s mapped at %p, irq %d with %d ctx banks\n",
iommu_dev->name, regs_base, irq, iommu_dev->ncb);
platform_set_drvdata(pdev, drvdata);
clk_disable(iommu_clk);
clk_disable(iommu_pclk);
return 0;
fail_clk:
if (iommu_clk) {
clk_disable(iommu_clk);
clk_put(iommu_clk);
}
fail_pclk:
clk_disable_unprepare(iommu_pclk);
fail_enable:
clk_put(iommu_pclk);
fail:
kfree(drvdata);
return ret;
}
static int msm_iommu_remove(struct platform_device *pdev)
{
struct msm_iommu_drvdata *drv = NULL;
drv = platform_get_drvdata(pdev);
if (drv) {
if (drv->clk) {
clk_unprepare(drv->clk);
clk_put(drv->clk);
}
clk_unprepare(drv->pclk);
clk_put(drv->pclk);
memset(drv, 0, sizeof(*drv));
kfree(drv);
}
return 0;
}
static int msm_iommu_ctx_probe(struct platform_device *pdev)
{
struct msm_iommu_ctx_dev *c = dev_get_platdata(&pdev->dev);
struct msm_iommu_drvdata *drvdata;
struct msm_iommu_ctx_drvdata *ctx_drvdata;
int i, ret;
if (!c || !pdev->dev.parent)
return -EINVAL;
drvdata = dev_get_drvdata(pdev->dev.parent);
if (!drvdata)
return -ENODEV;
ctx_drvdata = kzalloc(sizeof(*ctx_drvdata), GFP_KERNEL);
if (!ctx_drvdata)
return -ENOMEM;
ctx_drvdata->num = c->num;
ctx_drvdata->pdev = pdev;
INIT_LIST_HEAD(&ctx_drvdata->attached_elm);
platform_set_drvdata(pdev, ctx_drvdata);
ret = clk_prepare_enable(drvdata->pclk);
if (ret)
goto fail;
if (drvdata->clk) {
ret = clk_prepare_enable(drvdata->clk);
if (ret) {
clk_disable_unprepare(drvdata->pclk);
goto fail;
}
}
/* Program the M2V tables for this context */
for (i = 0; i < MAX_NUM_MIDS; i++) {
int mid = c->mids[i];
if (mid == -1)
break;
SET_M2VCBR_N(drvdata->base, mid, 0);
SET_CBACR_N(drvdata->base, c->num, 0);
/* Set VMID = 0 */
SET_VMID(drvdata->base, mid, 0);
/* Set the context number for that MID to this context */
SET_CBNDX(drvdata->base, mid, c->num);
/* Set MID associated with this context bank to 0*/
SET_CBVMID(drvdata->base, c->num, 0);
/* Set the ASID for TLB tagging for this context */
SET_CONTEXTIDR_ASID(drvdata->base, c->num, c->num);
/* Set security bit override to be Non-secure */
SET_NSCFG(drvdata->base, mid, 3);
}
clk_disable(drvdata->clk);
clk_disable(drvdata->pclk);
dev_info(&pdev->dev, "context %s using bank %d\n", c->name, c->num);
return 0;
fail:
kfree(ctx_drvdata);
return ret;
}
static int msm_iommu_ctx_remove(struct platform_device *pdev)
{
struct msm_iommu_ctx_drvdata *drv = NULL;
drv = platform_get_drvdata(pdev);
if (drv) {
memset(drv, 0, sizeof(struct msm_iommu_ctx_drvdata));
kfree(drv);
}
return 0;
}
static struct platform_driver msm_iommu_driver = {
.driver = {
.name = "msm_iommu",
},
.probe = msm_iommu_probe,
.remove = msm_iommu_remove,
};
static struct platform_driver msm_iommu_ctx_driver = {
.driver = {
.name = "msm_iommu_ctx",
},
.probe = msm_iommu_ctx_probe,
.remove = msm_iommu_ctx_remove,
};
static struct platform_driver * const drivers[] = {
&msm_iommu_driver,
&msm_iommu_ctx_driver,
};
static int __init msm_iommu_driver_init(void)
{
return platform_register_drivers(drivers, ARRAY_SIZE(drivers));
}
static void __exit msm_iommu_driver_exit(void)
{
platform_unregister_drivers(drivers, ARRAY_SIZE(drivers));
}
subsys_initcall(msm_iommu_driver_init);
module_exit(msm_iommu_driver_exit);
MODULE_LICENSE("GPL v2");
MODULE_AUTHOR("Stepan Moskovchenko <stepanm@codeaurora.org>");
...@@ -34,7 +34,7 @@ ...@@ -34,7 +34,7 @@
#include <dt-bindings/memory/mt8173-larb-port.h> #include <dt-bindings/memory/mt8173-larb-port.h>
#include <soc/mediatek/smi.h> #include <soc/mediatek/smi.h>
#include "io-pgtable.h" #include "mtk_iommu.h"
#define REG_MMU_PT_BASE_ADDR 0x000 #define REG_MMU_PT_BASE_ADDR 0x000
...@@ -93,20 +93,6 @@ ...@@ -93,20 +93,6 @@
#define MTK_PROTECT_PA_ALIGN 128 #define MTK_PROTECT_PA_ALIGN 128
struct mtk_iommu_suspend_reg {
u32 standard_axi_mode;
u32 dcm_dis;
u32 ctrl_reg;
u32 int_control0;
u32 int_main_control;
};
struct mtk_iommu_client_priv {
struct list_head client;
unsigned int mtk_m4u_id;
struct device *m4udev;
};
struct mtk_iommu_domain { struct mtk_iommu_domain {
spinlock_t pgtlock; /* lock for page table */ spinlock_t pgtlock; /* lock for page table */
...@@ -116,19 +102,6 @@ struct mtk_iommu_domain { ...@@ -116,19 +102,6 @@ struct mtk_iommu_domain {
struct iommu_domain domain; struct iommu_domain domain;
}; };
struct mtk_iommu_data {
void __iomem *base;
int irq;
struct device *dev;
struct clk *bclk;
phys_addr_t protect_base; /* protect memory base */
struct mtk_iommu_suspend_reg reg;
struct mtk_iommu_domain *m4u_dom;
struct iommu_group *m4u_group;
struct mtk_smi_iommu smi_imu; /* SMI larb iommu info */
bool enable_4GB;
};
static struct iommu_ops mtk_iommu_ops; static struct iommu_ops mtk_iommu_ops;
static struct mtk_iommu_domain *to_mtk_domain(struct iommu_domain *dom) static struct mtk_iommu_domain *to_mtk_domain(struct iommu_domain *dom)
...@@ -455,7 +428,6 @@ static int mtk_iommu_of_xlate(struct device *dev, struct of_phandle_args *args) ...@@ -455,7 +428,6 @@ static int mtk_iommu_of_xlate(struct device *dev, struct of_phandle_args *args)
if (!dev->archdata.iommu) { if (!dev->archdata.iommu) {
/* Get the m4u device */ /* Get the m4u device */
m4updev = of_find_device_by_node(args->np); m4updev = of_find_device_by_node(args->np);
of_node_put(args->np);
if (WARN_ON(!m4updev)) if (WARN_ON(!m4updev))
return -EINVAL; return -EINVAL;
...@@ -552,25 +524,6 @@ static int mtk_iommu_hw_init(const struct mtk_iommu_data *data) ...@@ -552,25 +524,6 @@ static int mtk_iommu_hw_init(const struct mtk_iommu_data *data)
return 0; return 0;
} }
static int compare_of(struct device *dev, void *data)
{
return dev->of_node == data;
}
static int mtk_iommu_bind(struct device *dev)
{
struct mtk_iommu_data *data = dev_get_drvdata(dev);
return component_bind_all(dev, &data->smi_imu);
}
static void mtk_iommu_unbind(struct device *dev)
{
struct mtk_iommu_data *data = dev_get_drvdata(dev);
component_unbind_all(dev, &data->smi_imu);
}
static const struct component_master_ops mtk_iommu_com_ops = { static const struct component_master_ops mtk_iommu_com_ops = {
.bind = mtk_iommu_bind, .bind = mtk_iommu_bind,
.unbind = mtk_iommu_unbind, .unbind = mtk_iommu_unbind,
......
/*
* Copyright (c) 2015-2016 MediaTek Inc.
* Author: Honghui Zhang <honghui.zhang@mediatek.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*/
#ifndef _MTK_IOMMU_H_
#define _MTK_IOMMU_H_
#include <linux/clk.h>
#include <linux/component.h>
#include <linux/device.h>
#include <linux/io.h>
#include <linux/iommu.h>
#include <linux/list.h>
#include <linux/spinlock.h>
#include <soc/mediatek/smi.h>
#include "io-pgtable.h"
struct mtk_iommu_suspend_reg {
u32 standard_axi_mode;
u32 dcm_dis;
u32 ctrl_reg;
u32 int_control0;
u32 int_main_control;
};
struct mtk_iommu_client_priv {
struct list_head client;
unsigned int mtk_m4u_id;
struct device *m4udev;
};
struct mtk_iommu_domain;
struct mtk_iommu_data {
void __iomem *base;
int irq;
struct device *dev;
struct clk *bclk;
phys_addr_t protect_base; /* protect memory base */
struct mtk_iommu_suspend_reg reg;
struct mtk_iommu_domain *m4u_dom;
struct iommu_group *m4u_group;
struct mtk_smi_iommu smi_imu; /* SMI larb iommu info */
bool enable_4GB;
};
static int compare_of(struct device *dev, void *data)
{
return dev->of_node == data;
}
static int mtk_iommu_bind(struct device *dev)
{
struct mtk_iommu_data *data = dev_get_drvdata(dev);
return component_bind_all(dev, &data->smi_imu);
}
static void mtk_iommu_unbind(struct device *dev)
{
struct mtk_iommu_data *data = dev_get_drvdata(dev);
component_unbind_all(dev, &data->smi_imu);
}
#endif
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
/*
* Copyright (c) 2015 MediaTek Inc.
* Author: Honghui Zhang <honghui.zhang@mediatek.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*/
#ifndef _MT2701_LARB_PORT_H_
#define _MT2701_LARB_PORT_H_
/*
* Mediatek m4u generation 1 such as mt2701 has flat m4u port numbers,
* the first port's id for larb[N] would be the last port's id of larb[N - 1]
* plus one while larb[0]'s first port number is 0. The definition of
* MT2701_M4U_ID_LARBx is following HW register spec.
* But m4u generation 2 like mt8173 have different port number, it use fixed
* offset for each larb, the first port's id for larb[N] would be (N * 32).
*/
#define LARB0_PORT_OFFSET 0
#define LARB1_PORT_OFFSET 11
#define LARB2_PORT_OFFSET 21
#define LARB3_PORT_OFFSET 43
#define MT2701_M4U_ID_LARB0(port) ((port) + LARB0_PORT_OFFSET)
#define MT2701_M4U_ID_LARB1(port) ((port) + LARB1_PORT_OFFSET)
#define MT2701_M4U_ID_LARB2(port) ((port) + LARB2_PORT_OFFSET)
/* Port define for larb0 */
#define MT2701_M4U_PORT_DISP_OVL_0 MT2701_M4U_ID_LARB0(0)
#define MT2701_M4U_PORT_DISP_RDMA1 MT2701_M4U_ID_LARB0(1)
#define MT2701_M4U_PORT_DISP_RDMA MT2701_M4U_ID_LARB0(2)
#define MT2701_M4U_PORT_DISP_WDMA MT2701_M4U_ID_LARB0(3)
#define MT2701_M4U_PORT_MM_CMDQ MT2701_M4U_ID_LARB0(4)
#define MT2701_M4U_PORT_MDP_RDMA MT2701_M4U_ID_LARB0(5)
#define MT2701_M4U_PORT_MDP_WDMA MT2701_M4U_ID_LARB0(6)
#define MT2701_M4U_PORT_MDP_ROTO MT2701_M4U_ID_LARB0(7)
#define MT2701_M4U_PORT_MDP_ROTCO MT2701_M4U_ID_LARB0(8)
#define MT2701_M4U_PORT_MDP_ROTVO MT2701_M4U_ID_LARB0(9)
#define MT2701_M4U_PORT_MDP_RDMA1 MT2701_M4U_ID_LARB0(10)
/* Port define for larb1 */
#define MT2701_M4U_PORT_VDEC_MC_EXT MT2701_M4U_ID_LARB1(0)
#define MT2701_M4U_PORT_VDEC_PP_EXT MT2701_M4U_ID_LARB1(1)
#define MT2701_M4U_PORT_VDEC_PPWRAP_EXT MT2701_M4U_ID_LARB1(2)
#define MT2701_M4U_PORT_VDEC_AVC_MV_EXT MT2701_M4U_ID_LARB1(3)
#define MT2701_M4U_PORT_VDEC_PRED_RD_EXT MT2701_M4U_ID_LARB1(4)
#define MT2701_M4U_PORT_VDEC_PRED_WR_EXT MT2701_M4U_ID_LARB1(5)
#define MT2701_M4U_PORT_VDEC_VLD_EXT MT2701_M4U_ID_LARB1(6)
#define MT2701_M4U_PORT_VDEC_VLD2_EXT MT2701_M4U_ID_LARB1(7)
#define MT2701_M4U_PORT_VDEC_TILE_EXT MT2701_M4U_ID_LARB1(8)
#define MT2701_M4U_PORT_VDEC_IMG_RESZ_EXT MT2701_M4U_ID_LARB1(9)
/* Port define for larb2 */
#define MT2701_M4U_PORT_VENC_RCPU MT2701_M4U_ID_LARB2(0)
#define MT2701_M4U_PORT_VENC_REC_FRM MT2701_M4U_ID_LARB2(1)
#define MT2701_M4U_PORT_VENC_BSDMA MT2701_M4U_ID_LARB2(2)
#define MT2701_M4U_PORT_JPGENC_RDMA MT2701_M4U_ID_LARB2(3)
#define MT2701_M4U_PORT_VENC_LT_RCPU MT2701_M4U_ID_LARB2(4)
#define MT2701_M4U_PORT_VENC_LT_REC_FRM MT2701_M4U_ID_LARB2(5)
#define MT2701_M4U_PORT_VENC_LT_BSDMA MT2701_M4U_ID_LARB2(6)
#define MT2701_M4U_PORT_JPGDEC_BSDMA MT2701_M4U_ID_LARB2(7)
#define MT2701_M4U_PORT_VENC_SV_COMV MT2701_M4U_ID_LARB2(8)
#define MT2701_M4U_PORT_VENC_RD_COMV MT2701_M4U_ID_LARB2(9)
#define MT2701_M4U_PORT_JPGENC_BSDMA MT2701_M4U_ID_LARB2(10)
#define MT2701_M4U_PORT_VENC_CUR_LUMA MT2701_M4U_ID_LARB2(11)
#define MT2701_M4U_PORT_VENC_CUR_CHROMA MT2701_M4U_ID_LARB2(12)
#define MT2701_M4U_PORT_VENC_REF_LUMA MT2701_M4U_ID_LARB2(13)
#define MT2701_M4U_PORT_VENC_REF_CHROMA MT2701_M4U_ID_LARB2(14)
#define MT2701_M4U_PORT_IMG_RESZ MT2701_M4U_ID_LARB2(15)
#define MT2701_M4U_PORT_VENC_LT_SV_COMV MT2701_M4U_ID_LARB2(16)
#define MT2701_M4U_PORT_VENC_LT_RD_COMV MT2701_M4U_ID_LARB2(17)
#define MT2701_M4U_PORT_VENC_LT_CUR_LUMA MT2701_M4U_ID_LARB2(18)
#define MT2701_M4U_PORT_VENC_LT_CUR_CHROMA MT2701_M4U_ID_LARB2(19)
#define MT2701_M4U_PORT_VENC_LT_REF_LUMA MT2701_M4U_ID_LARB2(20)
#define MT2701_M4U_PORT_VENC_LT_REF_CHROMA MT2701_M4U_ID_LARB2(21)
#define MT2701_M4U_PORT_JPGDEC_WDMA MT2701_M4U_ID_LARB2(22)
#endif
...@@ -152,6 +152,7 @@ struct iommu_dm_region { ...@@ -152,6 +152,7 @@ struct iommu_dm_region {
* @domain_set_attr: Change domain attributes * @domain_set_attr: Change domain attributes
* @get_dm_regions: Request list of direct mapping requirements for a device * @get_dm_regions: Request list of direct mapping requirements for a device
* @put_dm_regions: Free list of direct mapping requirements for a device * @put_dm_regions: Free list of direct mapping requirements for a device
* @apply_dm_region: Temporary helper call-back for iova reserved ranges
* @domain_window_enable: Configure and enable a particular window for a domain * @domain_window_enable: Configure and enable a particular window for a domain
* @domain_window_disable: Disable a particular window for a domain * @domain_window_disable: Disable a particular window for a domain
* @domain_set_windows: Set the number of windows for a domain * @domain_set_windows: Set the number of windows for a domain
...@@ -186,6 +187,8 @@ struct iommu_ops { ...@@ -186,6 +187,8 @@ struct iommu_ops {
/* Request/Free a list of direct mapping requirements for a device */ /* Request/Free a list of direct mapping requirements for a device */
void (*get_dm_regions)(struct device *dev, struct list_head *list); void (*get_dm_regions)(struct device *dev, struct list_head *list);
void (*put_dm_regions)(struct device *dev, struct list_head *list); void (*put_dm_regions)(struct device *dev, struct list_head *list);
void (*apply_dm_region)(struct device *dev, struct iommu_domain *domain,
struct iommu_dm_region *region);
/* Window handling functions */ /* Window handling functions */
int (*domain_window_enable)(struct iommu_domain *domain, u32 wnd_nr, int (*domain_window_enable)(struct iommu_domain *domain, u32 wnd_nr,
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment